Monday, July 13, 2020

Pip team midyear report

The grant-funded team working on improvements to pip in 2020 has now passed the halfway mark. Here's an update on where are so far and what's next.

Funding and Timeline Status

The plan that we proposed last year said that, by now, we would have finished Foundational work (Phase I) and Resolver work (Phase II), and the team would be doing Maintenance and Sustainability work (Phase III). Please see the timeline for user experience work and the timelines for development work.

We are behind where we had planned to be in the work roadmap. This is partially because the COVID-19 pandemic disrupted our work, but also because of policy and architecture decisions the project needed, and because foundational user experience research work has taken more time than we originally allotted. Thus, we have finished the Phase I and Phase II sections of the development work, and are approximately 75% of the way through the Phase I and Phase II user experience work. See below for accomplishments so far.

Funding: we predicted that we would be approximately 80% of the way through our one-year project budget (since the second half of the year has a slower work pace, primarily focusing on maintaining and deepening the work we started in the first half). We are now approximately 71% of the way through the budget, which gives us flexibility for the second half of the project.

Accomplishments

  • pip's new dependency resolver is about to go into beta. We released pip 20.1 in April which included an alpha version of the new resolver (hidden behind an optional "--unstable-feature=resolver" flag, but usable). This month we will release pip 20.2, which will include a robust beta of the new resolver (hidden behind an optional "--use-feature=2020-resolver" flag) that we will encourage users to test.
  • User experience data-gathering included:
    • Administered 5 surveys to gather feedback about issues with the pip resolver and dependency management
    • Interviewed and/or did user tests with over 30 maintainers and users so far
  • UX findings and resulting improvements included:

Next steps

Phase III development work commences next month. We will continue to improve the pip dependency resolver in response to testers' feedback. This will help us prepare to release pip 20.3, with the new resolver on by default, in October. We'll also review and respond to code contributions and new issues, to keep up with the pip code and issue review queue, help new contributors develop into continuing contributors, and help existing contributors grow into co-maintainers.

And our user experience work will also enter Phase III, deepening and expanding foundational research in Python packaging. We will recruit more users for interviews and surveys, develop user journey maps & workflows, work with maintainers to write documentation and help messages, develop templates for UI bugs, commands, error messages, output, documentation, and configuration files, and teach pip maintainers UX practices.

For more info or to contribute:

We run this project as transparently as possible, so you can:

Thank you

Thanks to our contractors on this project: Nicole Harris, Bernard Tyers, and Georgia Bullen of Simply Secure; Pradyun Gedam; Ilan Schnell; Paul F. Moore of Atos; Tzu-ping Chung; Sumana Harihareswara of Changeset Consulting.
This award continues our relationship with Mozilla, which supported Python packaging tools with a Mozilla Open Source Support Award in 2017 for Warehouse. Thank you, Mozilla! (MOSS has a number of types of awards, which are open to different sorts of open source/free software projects. If your project is looking for financial support, do check the MOSS website to see if you qualify.)

This is new funding from the Chan Zuckerberg Initiative. This project is being made possible in part by a grant from the Chan Zuckerberg Initiative DAF, an advised fund of Silicon Valley Community Foundation. Thank you, CZI! (If your free software/open source project is used by biology researchers, check the Essential Open Source Software for Science Request for Applications and consider applying for the next round).

Thank you to the pip and PyPA maintainers, to the PSF and the Packaging WG, and to all the contributors and volunteers who work on or use Python packaging tools.

Wednesday, July 01, 2020

Announcing the PSF Project Funding Working Group


For the past 3 years, the PSF has been working on grant funded projects to improve our internal systems and platforms. This work has been done with the Packaging Working Group, and focused on our packaging ecosystem of PyPI and pip. We have been able to show that applying directed funding to open source projects has the ability to dramatically increase the speed of development, and move our community forward in a much more sustained way than relying solely on volunteer effort.

Along with the external grant funding of PSF projects, we have also committed PSF funds in the past to improve developments of community projects. This shows that the experience of directed funding is applicable to our community projects, as well as our own. An example here is the BeeWare project that was given funding via our Education Grants last year:



Another wonderful example has been a number of scientific Python projects that have raised large amounts of grant funding, mostly through NumFocus. They have been a large inspiration for our focus on grant funding as an important source of revenue for open source projects. The scientific open source community has been immeasurably improved by this funding, and we hope to expand this opportunity to the entire Python community.

Helping the community get funding


The PSF has created the Project Funding Working Group to help our community seek similar funding for their own projects. We hope to expand the amount of money going into the Python community as a whole, by providing resources and advice to projects who are interested in seeking funding from external sources.

Our charter starts with our intended purpose:

This Working Group researches, and advises Python community volunteers on applying for external grants and similar funding to advance the mission of the PSF, which includes, but is not limited to, things such as advancing the Python core, Python-related infrastructure, key Python projects, and Python education and awareness.
You can read the entire charter for more information about the vision for the group that we intend to build over the medium and long term.

Resources

In the short term, the first resource that we have put together is a list of potential funders that are applicable to our community. It’s on GitHub, and we welcome contributions to the list if you know of additional sources of funding. The other initial resource we are able to provide is advice, so if you have any questions about funding, you can email us at project-funding-wg@python.org, and we will do our best to help. We can advise you on picking tasks to propose, making a budget, writing a proposal, and more.
We are excited about the possibilities for the Python community when we see more funding being applied to our mission. There is a lot of amazing open source software out there being built by volunteers, and we hope that giving them additional resources will create even more impact for our mission of advancing the Python community. 
-- Eric Holscher, co-chair, Project Funding Working Group