Tuesday, September 25, 2018

CPython Core Developer Sprint 2018

This September, twenty-nine core committers arrived at Microsoft’s main campus in Redmond, Washington, USA for the now-annual core developer sprints. These are an opportunity for the core team to have focused discussions, in-depth conversations and work free from interruption for five days. By the end of the week, thirty-one core developers went home tired but satisfied.

The major sponsor of the sprints this year was the Python Software Foundation. Microsoft provided the venue and some events during the week, and Facebook, Microsoft, Google, LinkedIn, Bloomberg, and a number of small companies covered their employees’ expenses.

Major Achievements

With over thirty people working for a week, there were many achievements and improvements made for Python 3.8 and earlier versions. Some of the highlights include:

  • The asyncio documentation was completely rewritten
  • Most of PEP 572 (assignment expressions) was implemented and tested
  • Recursive tracebacks and syntax errors in f-strings were improved to provide better feedback
  • The Automerge bot was enabled on the CPython GitHub repository, helping reduce the amount of manual work done by core committers
  • importlib_metadata was improved, helping to standardize the way Python exposes information about installed packages
  • The zipimport module was rewritten in Python code, which will enable new development and improvements for importing modules directly from ZIP files
  • Over 45 contributions (besides our own) were reviewed and merged, and over 80 issues were closed in the CPython repository, with many others in related projects such as Buildbot, Roundup, blurb, and our GitHub bots


Governance Discussions

With our BDFL of Python retiring earlier this year, we spent some time discussing how to approach the future of Python. More than any specific outcomes, the discussions were very fruitful and helped many of us see how similar our positions are to each other.

While no decisions have been made, all current proposals intend to keep the PEP process, and limit the new leaders’ responsibilities to arbitrating controversial decisions.

Final proposals are due by the end of September. See PEP 8000 for an overview of the process and links to related PEPs.

New Core Developers

Those who read the first paragraph carefully will have noticed that more committers left the sprints than arrived. This happened because we appointed two new core developers during the week. Congratulations to Emily Morehouse and Lisa Roach!


From Raymond Hettinger’s announcement post:
Emily is the Director of Engineering at Cuttlesoft. She has previously attended two Language Summits and three core development sprints at PyCon. Since July, Emily has worked with Guido's guidance to implement PEP 572, Assignment Expressions.  She has also worked with Eric Snow to dive into CPython's runtime as well as subinterpreters.  This year at PyCon she gave a talk on Python's AST.
Lisa has a background in network engineering and supported the Cisco sale engineer team to develop high quality Python product demonstrations.  Later she moved to the Facebook security team.  This is her third core developer sprint.  She and Guido are co-authors of PEP 526, Syntax for Variable Annotations. Last year, she worked with Eric Smith on PEP 557, Data Classes.

Other Blogs

Other attendees have posted their own blogs describing their experiences at the sprints. (This list may be updated over time as more are published.)


Thank you!

A huge thanks to all the participants who attended, the various companies who sponsored parts of the event, and the PSF for covering the majority of travel expenses. Thanks also to those contributors who were unable to make it this year. Hopefully next year we can include even more core contributors.

Attendees: Brett Cannon, Kushal Das, Ned Deily, Steve Dower, Ethan Furman, Larry Hastings, Christian Heimes, Raymond Hettinger, Łukasz Langa, Ezio Melotti, Emily Morehouse, Benjamin Peterson, Davin Potts, Lisa Roach, Pablo Galindo Salgado, Neil Schemenauer, Yury Selivanov, Eric V. Smith, Gregory P. Smith, Nathaniel Smith, Eric Snow, Victor Stinner, Andrew Svetlov, Guido van Rossum, Dino Viehland, Petr Viktorin, Zachary Ware, Barry Warsaw, Mariatta Wijaya, Carol Willing

Written by: Steve Dower

Tuesday, August 28, 2018

Redesigning the Python Package Index

In April this year a new version of the Python Package Index (PyPI) was released, an effort made possible by a generous award from the Mozilla Open Source Support program.

A major change in PyPI is the new user interface - something that had not seen any sizeable updates for over ten years. Understandably there have been questions about what’s next for the new UI, which as the designer of Warehouse (the project powering PyPI), I would like to address.


The PyPI homepage

The new design


PyPI’s new design is a reflection of the Python community - modern, welcoming and inclusive.

The design emphasises inclusion by providing extensive help documentation, user friendly interfaces, accessible font sizes and a high-contrast color scheme. The entire site works across all resolutions, meaning that you can now use PyPI on any device.

These features are no accident, as the goal of the redesign is to make PyPI a success for as many users as possible. This a big challenge as over 15 million people from 236 different countries visit PyPI.org each year (Aug 2017 - July 2018).

While the new design is informed by usability standards and user experience best practices, it’s now time to take PyPI to the next level: informing design improvements by engaging in community research and user testing.

Next steps


The first area we are looking to improve is the project's detail page (view example), which is - as the name suggests - the page where the details of a particular project can be viewed.

This is by far the most visited page type on PyPI. In July 2018 alone, 76.59% of page visits were to a project detail page, or 3,594,956 visits from a total page visit count of 4,693,561. The majority of users arrive on these pages direct from Google or other sources, bypassing the PyPI home and search pages.

Given these numbers, even a small improvement in the efficiency of this page would return great results for the community. The question is: how should we decide what to change? What one user might think of an improvement, others may consider a regression.

Conducting user research


In an effort to better understand what our users want from the aforementioned page, we ran a design research exercise (full writeup) asking Python community members to rank the importance of different pieces of information on the page. 1,926 people participated in the exercise. These were the results:




From this research, we can ascertain that many users highly value the project description, information about the required version of Python, and links to the project online. In contrast, few users value the trove classifiers, list of maintainers, or instructions on how to see statistics about the project.

Such insights are essential in driving the redesign in a way that prioritises important information for the largest number of users.

Running user tests


In conjunction with this research we are also establishing a user testing program, where PyPI users will give their feedback to the team via a remote video call; answering specific usage questions or completing certain tasks on the site.

Our goal is to run these sessions with a diverse group of users, accounting for the different people, places and ways that Python is used, while establishing major “pain points” with the current design. From this, we can open issues in the issue tracker to address problems, as has happened before with previous rounds of user tests conducted on the project management interfaces.

User tests can also be used to test new design concepts, compare the performance between old design vs new proposals, and ascertain if the proposed design solutions are truely performant.

Help us help PyPI!


So - how can you help us take PyPI to the next level?

If you’d like to participate in user tests, please register your interest. Depending on your profile and availability, we will be in touch to organise a testing session.

If you have a background in design/research or are interested in facilitating user tests, please contact me on nicole@pypi.org. All help is welcome!

We appreciate new contributors to the Warehouse project, with reserved issues for new contributors and love donations to the Python Packaging Working Group as these help us build a more sustainable model for Python packaging.

Finally, if you are interested in participating in future rounds of user research, please follow PyPI on Twitter or keep an eye on PyPI.org for future announcements!