Python 2 series to be retired by April 2020

The CPython core development community is urging users to migrate to Python 3 as it will be the only version that will be updated for bugs and security vulnerabilities.

After nearly 20 years of development on the Python 2 series, the last major version 2.7 will be released in April 2020, and then all development will cease for Python 2. Users are urged to migrate to Python 3 to benefit from its many improvements, as well as to avoid potential security vulnerabilities in Python 2.x after April 2020. This move will free limited resources for the CPthyon core developer community for other important work.

The final Python 2.7 maintenance release was originally planned for 2015. However, it was delayed 5 years to give people adequate time to migrate and to work closely with vendors and redistributors to ensure that supported Python 3 migration options were available. Part of the reason for this delay was because the stricter text model in Python 3 was forcing the resolution of non-trivial Unicode handling issues in the reference interpreter and standard library, and in migrated libraries and applications

Python 3 is a noticeable improvement to Python. There is ground-up support for Unicode and internationalization. It better expresses common idioms and patterns, which in code makes it easier to read and reason about. Improvements in concurrency, fault handling, testing, and debugging provide developers with the opportunity to create more robust and secure applications.

Going forward, Python 3 will be the only major version of CPython that is actively maintained for bugs and security issues.

More information:

• Press release: https://www.python.org/psf/press-release/pr20191220/
• Simple FAQ: https://www.python.org/doc/sunset-python-2/
• Comprehensive FAQ: http://python-notes.curiousefficiency.org/en/latest/python3/questions_and_answers.html
• Developer porting guide: https://portingguide.readthedocs.io

 

Clarification Update January 8, 2020

Effective January 1, 2020, no new bug reports, fixes, or changes will be made to Python 2.

After Python 2.7.17 (October 19, 2019) was released, some additional changes accumulated before the end of 2019 when the core development team froze the 2.7 branch. As a final service to the community, python-dev will bundle those fixes -- and only those fixes -- into a final 2.7.18 release. The release date for 2.7.18 will be in April 2020 because that allows time for the release managers to complete a release candidate and final release.

Mozilla and Chan Zuckerberg Initiative to support pip

The Python Software Foundation is receiving $407,000 USD to support work on pip in 2020. Thank you to Mozilla (through its Mozilla Open Source Support Awards) and to the Chan Zuckerberg Initiative for this funding! This foundational, transformational work will release Python developers and users to concentrate on the tools they're making and using, instead of troubleshooting dependency conflicts. Specifically, CZI and Mozilla are funding user experience, communications/publicity, and testing work (including developing robust testing infrastructure) as well as core feature development and review.

 

What we're doing with the money

Computers need to know the right order to install pieces of software ("to install x, you need to install y first"). So, when programmers share software, we have to precisely describe those installation prerequisites, and our installation tools need to navigate tricky situations where they're getting conflicting instructions. This project will make pip, a package installer for Python, better at handling that tricky logic, and easier for people to use and troubleshoot.

Millions of people and organizations use tools written in Python, and nearly the entire ecosystem of Python software projects depends on pip. Our project will help everyone more easily install software, diagnose and fix problems, and maintain infrastructure.

The Chan Zuckerberg Initiative funding is, in particular, aimed at improving Essential Open Source Software for Science. Scientists use many tools written in Python -- many of CZI's awardees in this round are written in Python -- but, also, researchers often want to write tools and share them with each other to help science advance faster. Our work will include research and improvements so the installation process will be easier to use and understand, which will enable researchers to build better applications and compose complex toolchains more easily.

We've laid out a detailed three-phase work plan on our pip 2020 Donor-funded Roadmap wiki page. To summarize:

• Mozilla is awarding PSF a Mozilla Open Source Support Award (Track I: Foundational Technology) for $207,000, which is paying for 5 months of:
• Python development work: Reviewing and responding to open issues and pull requests, refactoring build logic, collaborating with downstream projects and users about config flags and transition schedules, working on the dependency resolver itself and fixing bugs.
• Initial user experience research and design work: Reading existing bug reports and posts about Python package management confusion, interviewing users and running user tests, developing user journey maps and workflows, and working with maintainers to write documentation and help messages and to design resolver user experience.
• CZI is giving PSF an Essential Open Source Software for Science grant for $200,000, which is paying for:
• 12 months of Python development, test infrastructure, and project maintenance: Triaging bugs and reviewing pull requests, writing test cases, testing lead developers' work, building test infrastructure, investigating and fixing bugs, and writing the raw material for documentation to help future maintainers onboard better.
• 4 months of Phase III user experience research and design work: Training maintainers in UX design, doing further user tests on the new pip, developing a checklist for developing new features, and making templates for commands, error messages, output, documentation, and config files.
• Travel for initial developer onboarding and for some contributors to attend PyCon North America.
• And both CZI and Mozilla are paying for project management (planning, testing, editing, coordinating, communicating with stakeholders, announcing, reporting to funders, and getting obstacles out of everyone's way) and PSF administrative work (recruiting and overseeing contractors, project oversight, and financial processing).

The Packaging Working Group is currently selecting contractors for the programming parts of this work. The other two contractors have already been selected:

• Simply Secure is "an educational nonprofit 501(c)3 that supports practitioners in designing ethically-informed, values-driven technology that protects human rights." As experts in open source user experience, Simply Secure will bring UX research and design skills to the command-line experience of Python package management.
• Changeset Consulting, LLC, providing project management, is a returning contractor, having previously worked on the PyPI rewrite launch and improvements to PyPI's accessibility, security, and internationalization. Changeset lead Sumana Harihareswara was also the lead grantwriter for these funding proposals.

 

Why this and why now?

We're partway through a next-generation rewrite of pip's dependency resolver. The project ran into massive technical debt, but the refactoring is nearly finished and prototype functionality is in alpha now.

We need to finish the resolver because many other packaging improvements are blocked on it, it'll fix many dependency issues for Python users, and it'll fix installation problems for conda, certbot, WebSocket, and many other projects. And we need to improve pip's UX by providing better error messages and prompts, logs, output, and reporting, consistently across features, to fit users' mental models better, make hairy problems easier to untangle, and reduce unintended data loss.

The Packaging Working Group looks for potential improvements in Python packaging and distribution that are well-scoped, have community consensus, and could be expedited through funding. In the past three years, the Packaging WG has received funding for several improvements to PyPI -- $170,000 from Mozilla, $80,000 from OTF, and $100,000 from Facebook -- and is seeking to help other packaging tools. In June, pip maintainers and Packaging Working Group members discussed the importance and difficulty of rolling out the new resolver. We worked together to write and submit proposals to Mozilla and the Chan Zuckerberg Initiative.

 

What's next?

This work will start by early January 2020. Day-to-day work will mostly happen in pip's GitHub repository and the Python developers' Zulip livechat. You can check for regular reports at the Python Insider blog and the Packaging category of Python's Discourse developer forum, archived at the Packaging WG's wiki page. And we'll publicize calls for volunteers, especially for user interviews and tests, on this blog, on community mailing lists, and on Twitter.

The Packaging WG will continue to seek funding for future improvements in pip, manylinux, setuptools, the Python Packaging User Guide, PyPI, etc.

 

Thanks

This award continues our relationship with Mozilla, which supported Python packaging tools with a Mozilla Open Source Support Award in 2017 for Warehouse. Thank you, Mozilla! (MOSS has a number of types of awards, which are open to different sorts of open source/free software projects. If your project is looking for financial support, do check the MOSS website to see if you qualify.)

This is new funding from the Chan Zuckerberg Initiative. This project is being made possible in part by a grant from the Chan Zuckerberg Initiative DAF, an advised fund of Silicon Valley Community Foundation. Thank you, CZI! (If your free software/open source project is used by biology researchers, check the Essential Open Source Software for Science Request for Applications and consider applying for the next round).

Thank you to the pip and PyPA maintainers, to the PSF and the Packaging WG, and to all the contributors and volunteers who work on or use Python packaging tools.

Giving Tuesday 2019

For the first time the PSF is participating in Giving Tuesday! This event is held annually the Tuesday after Thanksgiving - this year on December 3rd, 2019. The global celebration runs for 24 hours and begins at midnight local time.

Please donate on December 3rd and help us meet our goal of $10,000!

Donations support sprints, meetups, community events, Python documentation, fiscal sponsorships, software development, and community projects. Your contributions help fund the critical tools you use every day.

What if everyone around the world, gave together, on one day? Please consider supporting the PSF on Giving Tuesday, December 3rd, 2019. 

Your donations have IMPACT

----------------------------------------

Our Annual Report will show just a few ways your support has made a difference, thanks to the generous support from our partners and friends. Some highlights are below:

• Over $137,200 was awarded in financial aid to 143 PyCon attendees in 2019.
• $324,000 was paid in grants to recipients in 51 different countries.
• Donations and fundraisers resulted in $489,152 of revenue. This represents 15% of our total 2018 revenue. PSF and PyCon sponsors contributed over $1,071K in revenue!

How your donation dollars are spent:

• $99 pays for 6 months of Python meetup subscriptions
• $60 a month ($2.00 a day) pays for one workshop, impacting over 250 people
• $.50 a day ($15 a month) pays for a meetup.com subscription for one Python group
• $1 a day ($30 a month) supports a regional workshop, impacting over 200 people.
• The PSF meetup.com network currently supports 68 groups with 89,000 members in 16 countries. It costs $.60 per member per month to support these worldwide meetups. 

Comments from grant recipients:

"The PSF Fiscal Sponsorship allows us to focus on building community, while they handle our non-profit status, accounting, and back office." - Eric Holscher, PyCascades Conference Organizer

"The PSF was North Bay Python's first sponsor. Their early financial support for our mission helped kickstart what has become one of the most well-regarded regional conferences in our community." - Christopher Neugebauer, Conference Organizer

"Without the support of the PSF, our events would not have been possible. Many of our attendees are now working or interning as Python or Django Developers." - Jeel Mehta, Django Girls Bhavnagar, India Conference Volunteer

"The PSF grant allowed us to run an all day workshop for women. More organizations should apply for a PSF grant so they can enable and motivate more people, especially minorities, to start their great journey into programming." - Women in Technology, Peru

From the team at the PSF!

______________________________