As the Python Packaging Project Manager, my work has mostly concentrated on improving user experience on PyPI. As the Python Packaging ecosystem grows from strength to strength, one of the initiatives I have undertaken is to define a cohesive strategy for Python Packaging.
My Mission
Python Packaging is a diverse landscape dotted with many tools that cater to generic and niche user requirements. As a community, something that is poorly understood is where the community wants to be in 5 years. Understanding where we want to be is important to allow us to identify key goals for the future and how we will reach them, and to ensure we place users at the heart of everything we do. By defining key deliverables driven by community consensus, securing funding becomes easier as we can demonstrate that the community vision will drive innovation and enable better user experience and adoption of Packaging tools.
User Survey
In order to understand what end users are looking for when they use Packaging tools, Nicole Harris and I worked with the community to finalize the survey questions. Nicole developed the final set of survey questions. The survey results are summarized in this document.
From the survey responses, it was clear that while Packaging tools were improving with time, most users found the landscape too complex to navigate. Based on the survey responses, the strategy discussion was condensed to these topics-
Development of a single Packaging tool or a more unified experience
Better support for Packaging users
Phasing out legacy systems
Supporting each other in the community
Encouraging long-term contribution
To take this discussion further, I invited maintainers and contributors of any Packaging tool to answer this question:
“Can we reduce the number of tools and bring about some form of unification? Can we do anything else to reduce the complexity?”
After an interesting and lively discussion, we still do not have a clear consensus of what a solution should look like, but we do have a way forward.
Since we are at the beginning of this conversation, we don’t know what unification means yet. As Pradyun Gedam suggested, there are many options as to what it could look like-
Unification of PyPI/conda models
Unification of the consumer-facing tooling
Unification of the publisher-facing tooling
Unification of the workflow setups/tooling
Unification/Consistency in the deployment processes
Unification/Consistency in “Python” installation/management experience
Unification of the interface of tools
Based on these criteria, there is (some) consensus in driving forward unification of workflow setups/tooling and unification of interface of tools.
The next major question was, if we do go ahead with unification of specific elements, which elements of Packaging would offer the best solution. To provide a unified UI/UX for end users, some of the solutions that were suggested were
Reusable libraries
Expanding pip to include more functionalities
Recommending existing tools such as hatch, poetry etc.
This discussion also raised a few interesting concerns. Understandably, there is concern over the time and effort that volunteers have devoted in developing tools and whether all this effort will come to naught if we recommend a specific tool. As the discussion continues, there is a large question over the part that PyPA plays and whether PyPA would be willing to take a stand on recommending a specific tool. One concern that keeps coming up is that historically there has been poor communication between PyPA, Packaging tools and end users and if anything will change this time.
The way forward
In order to flesh out the details of the actual solution, I recommend the following steps:
Identify the tasks that a unified solution has to accomplish
Identify the best way to deliver the solution. This could be a new tool, expanding an existing tool or developing standards/libraries.
Submit PEPs to ensure community governance and consensus
Ensure there is buy-in from end users for proposed solution(s)
Define development roadmaps, milestones, key deliverables and timeline
Generate funding to support development
Develop and deliver
Communicate, communicate, communicate
This will be a long and intensive process. But the time and effort invested in this project will be worth it as it will result in innovation and better user experience.
We will be discussing four more big questions over the next three months that will be used to help us plan the future of Packaging at Python. As we continue the strategy discussions, I invite everyone to participate in the discussion threads on Discuss.
We are grateful to Bloomberg for generously providing the funding for this role and making this valuable work possible.
