The grant-funded team working on improvements to pip in 2020 has now passed the halfway mark. Here's an update on where are so far and what's next.
Funding and Timeline Status
The plan that we proposed last year said that, by now, we would have finished Foundational work (Phase I) and Resolver work (Phase II), and the team would be doing Maintenance and Sustainability work (Phase III). Please see the timeline for user experience work and the timelines for development work.
We are behind where we had planned to be in the work roadmap. This is partially because the COVID-19 pandemic disrupted our work, but also because of policy and architecture decisions the project needed, and because foundational user experience research work has taken more time than we originally allotted. Thus, we have finished the Phase I and Phase II sections of the development work, and are approximately 75% of the way through the Phase I and Phase II user experience work. See below for accomplishments so far.
Funding: we predicted that we would be approximately 80% of the way through our one-year project budget (since the second half of the year has a slower work pace, primarily focusing on maintaining and deepening the work we started in the first half). We are now approximately 71% of the way through the budget, which gives us flexibility for the second half of the project.
Accomplishments
- Multiple pip releases (changelog) since we started in January, including 332 pull requests merged since we kicked off (out of 2637 merged since 2011), and including improvements to the resolver library resolvelib: 22 pull requests merged since we kicked off
- pip's new dependency resolver is about to go into beta. We released pip 20.1 in April which included an alpha version of the new resolver (hidden behind an optional "--unstable-feature=resolver" flag, but usable). This month we will release pip 20.2, which will include a robust beta of the new resolver (hidden behind an optional "--use-feature=2020-resolver" flag) that we will encourage users to test.
- User experience data-gathering included:
- Recruited over 200 Python community members to be part of the UX research "panel" of respondents (sign up if you are interested)
- Administered 5 surveys to gather feedback about issues with the pip resolver and dependency management
- Interviewed and/or did user tests with over 30 maintainers and users so far
- UX findings and resulting improvements included:
- Documentation improvements especially regarding pip internals and architecture
Next steps
Phase III development work commences next month. We will continue to improve the pip dependency resolver in response to testers' feedback. This will help us prepare to release pip 20.3, with the new resolver on by default, in October. We'll also review and respond to code contributions and new issues, to keep up with the pip code and issue review queue, help new contributors develop into continuing contributors, and help existing contributors grow into co-maintainers.
And our user experience work will also enter Phase III, deepening and expanding foundational research in Python packaging. We will recruit more users for interviews and surveys, develop user journey maps & workflows, work with maintainers to write documentation and help messages, develop templates for UI bugs, commands, error messages, output, documentation, and configuration files, and teach pip maintainers UX practices.
For more info or to contribute:
We run this project as transparently as possible, so you can:
- read and participate in GitHub issues
- contact us in the Discourse forum or Zulip chat
- read our meeting notes
Thank you
Thanks to our contractors on this project: Nicole Harris, Bernard Tyers, and Georgia Bullen of Simply Secure; Pradyun Gedam; Ilan Schnell; Paul F. Moore of Atos; Tzu-ping Chung; Sumana Harihareswara of Changeset Consulting.
This award continues our relationship with Mozilla, which supported Python packaging tools with a Mozilla Open Source Support Award in 2017 for Warehouse. Thank you, Mozilla! (MOSS has a number of types of awards, which are open to different sorts of open source/free software projects. If your project is looking for financial support, do check the MOSS website to see if you qualify.)
This is new funding from the Chan Zuckerberg Initiative. This project is being made possible in part by a grant from the Chan Zuckerberg Initiative DAF, an advised fund of Silicon Valley Community Foundation. Thank you, CZI! (If your free software/open source project is used by biology researchers, check the Essential Open Source Software for Science Request for Applications and consider applying for the next round).
Thank you to the pip and PyPA maintainers, to the PSF and the Packaging WG, and to all the contributors and volunteers who work on or use Python packaging tools.