Monday, March 26, 2018

Warehouse: All New PyPI is now in beta


  1. Context
  2. Migrating
    1. New PyPI Features
    2. Deprecations
  3. Future Plans
  4. Security
  5. Please test!
    1. Workflows
  6. IRC/Twitter livechat hours
  7. Contact us

The all new Python Package Index is now in beta at pypi.org. We predict the full switch will happen in April 2018 (roadmap), so here's a heads-up about why we're switching, what's changed, and what to expect. To get an email when the new site replaces the old one, please sign up for the low-traffic PyPI announcements email list.

    Context


    The legacy PyPI site at pypi.python.org started in the early 2000s, before modern web frameworks. The legacy codebase has made it hard to maintain and even harder to develop new features, even as past maintainers put in tremendous effort to continuously reduce outages.

    The new PyPI at pypi.org (codebase: Warehouse) looks more modern, and is up-to-date under the hood too. A modern web framework (Pyramid), 100% backend test coverage, and a Docker-based development environment make it easier for current and new developers to maintain and run it and add features.

    Thanks to Mozilla's Open Source Support funding, we have designed and added new features, overhauled infrastructure, and worked towards redirecting traffic to the new site and shutting down the old one. The full switch will include redirecting browser and pip install traffic from the old site; then, sometime in late April or early May, the legacy site will be entirely shut down.


    Migrating


    You may not need to change anything right away. Thanks to redirects, your sites, services, and tools will probably be able to seamlessly switch to the new site.

    Users: On Windows and Linux, no change is necessary as long as your version of OpenSSL supports TLSv1.2. pip install should work as normal. macOS/OS X users running version 10.12 or older need to upgrade to the latest pip (9.0.3) to connect to PyPI securely:


        curl https://bootstrap.pypa.io/get-pip.py | python

    Package maintainers: If you use setup.py upload to upload releases, we recommend you switch to Twine. New PyPI takes the same username/password as legacy PyPI did. If you have problems, follow the packagers' migration guide.

    API users: follow the API users' migration guide.

    If you're affected by a deprecation (below), you should adapt and migrate by early April 2018. For help, come to a livechat or contact us. Sign up for the low-traffic PyPI announcements email listto get a heads-up when we have a more precise date.


    New PyPI Features

    • mobile-responsive UI  
    • chronological release history for each project (example)  
    • easy-to-read project activity journal for project maintainers  
    • better search and filtering  
    • support for multiple project URLs (e.g., for a homepage and a repo)  
    • support for Markdown READMEs for source distributions (soon: wheels too)  
    • user-visible Gravatars and email addresses for maintainers  
    • no need to "register" a project before initial upload  
    • better accessibility (and more work to come)  
    • newer backend infrastructure, supporting new features and a more scalable PyPI  


    Deprecations


    Things that already have gone away (sometimes for policy or spam-fighting reasons) include:

    Things that will go away once legacy PyPI shuts down:

    Late 2018 or later:


    Future plans


    See our issue tracker. Includes:

    For updates, please sign up for the low-traffic PyPI announcements email list.

    Security


    If you find any potential security vulnerabilities, please follow our published security policy. Please don't report security issues in Warehouse via GitHub, IRC, or mailing lists. Instead, please directly email the security team.


    Please test!


    The point of the beta is to find and fix bugs. Please help us. Most of these workflows you can test on pypi.org, using the same login as you use on pypi.python.org (legacy PyPI). For testing destructive actions, like removing an owner, deleting a project, or deleting a release, please use test.pypi.org.


    Workflows


    Package users:
    • Register/confirm a new user  
    • Login/logout  
    • Reset password  
    • Search for projects  
    • pip install a package  
    • Download release files via browser  
    • Call JSON, RSS, Simple, and XML-RPC APIs  

    Project maintainers:
    • Add/remove a maintainer  
    • Add/remove an owner  
    • Transition ownership  
    • Remove a project  
    • Remove a release  
    • View journals for a project  
    • View journals for a release  
    • Upload a new release (source distribution and wheel; upgrade your versions of twine and setuptools first)  
    • Confirm display of project description, release history, download files, project links, maintainers, tags, and classifiers (example)  


    IRC/Twitter livechat hours


    Warehouse developers will be in IRC, in #pypa-dev on Freenode, and on Twitter (#newpypi), available to talk about problems you run into, or about how to hack on Warehouse:
    1. Tuesday, March 27th, 9am-10am PDT, noon-1pm EDT, 18:00-19:00 CEST, 9:30pm-10:30pm India, 16:00-17:00 UTC  
    2. Friday, March 30th, 10-11am EDT, 16:00-17:00 CEST, 7:30pm-8:30pm India, 14:00-15:00 UTC  
    3. Tuesday, April 3rd, 8am-9am PDT, 11am-noon EDT, 17:00-18:00 CEST, 8:30pm-9:30pm India, 15:00-16:00 UTC  
    4. Thursday, April 5th, 5pm-6pm PDT, 8pm-9pm EDT, (April 5th) 8am-9am Manila, (April 5th) 10am-11am Melbourne, (April 5th) 0:00-1:00 UTC  

    Feel free to drop in! (By participating, you agree to abide by the PyPA Code of Conduct.)


    Contact us



    (By participating, you agree to abide by the PyPA Code of Conduct.)

    Thank you for using PyPI!

    Tobacco Warehouse in Louisville, Kentucky 1906

    Monday, February 26, 2018

    Python package maintainers, help test the new PyPI!


    Warehouse is a next-generation Python Package Repository which will replace the existing code base that currently powers PyPI. (See the source code on GitHub, and roadmap for details.) We are seeking maintainers of projects on PyPI to test it and send us bug reports, and we're seeking new contributors to Warehouse.

    Since Warehouse must be a reimplementation of the existing PyPI, please focus initially on any differences, missing features, or incorrect behavior that is exhibited on pypi.org that affect your workflows for account management and package maintainership. We'll be soliciting feedback on other concerns soon! Feedback on user experience, accessibility, and overall ease of use are welcome. Go to the pre-production deployment at pypi.org and try it out!

    Background


    Last year, the PSF's Packaging Working Group successfully applied for a grant from Mozilla's Open Source Support program. Mozilla awarded USD 170,000 to get PyPI upgraded. The team's been working since early December—see our progress reports and roadmap—and pypi.org now has the essential features that package maintainers, and most other users, need.


    Guidelines for Particpation


    By participating, you agree to abide by the PyPA Code of Conduct.
    You should sign up for the PyPI Announcement Mailing List for updates.

    Things to test


    Most of these you can test on pypi.org, using the same login as you use on pypi.python.org (legacy PyPI). For testing destructive actions, like removing an owner, deleting a project, or deleting a release, please use test.pypi.org, which has an entirely separate package index.

    Workflows


    • Add/Remove Maintainer
    • Add/Remove Owner
    • Transition Ownership
    • User Registration and Confirmation
    • Login/Logout
    • Password Reset
    • Remove a project
    • Remove a release
    • View Journals for a Project
    • View Journals for a Release
    Warehouse has handled new projects and new release uploads since last summer, so those workflows are not the most important to test now. The list of workflows above are the ones we request you to exercise.

    Security


    If you find any potential security vulnerabilities, please follow our published security policy. Please don't report security issues in Warehouse via GitHub, IRC, or mailing lists. Instead, please directly email one or more of our maintainers.

    IRC livechat hours


    Warehouse developers will be in IRC, in #pypa-dev on Freenode, and available to talk about problems you run into, or about how to hack on Warehouse:
    1. Tuesday Feb 27th: 1700 UTC / noon-1pm EST
    2. Tuesday Feb 27th: 2300 UTC / 6pm-7pm EST
    3. Thursday March 1st: 1700 UTC / noon-1pm EST
    4. Thursday March 1st: 2300 UTC / 6pm-7pm EST
    Feel free to drop in!

    Notice


    We're working hard on nearly every aspect of the Warehouse codebase to get it ready for production deployment and we are shipping features nearly every day, so check back and maybe even try using pypi.org for your maintainer activities full time. Due to the rate of change some errors, downtime, and outright broken features may occur. We have some automated reporting of the scenarios in place, but let us know!

    Reminder: Sign up for the PyPI Announcement Mailing List to be kept in the loop as we continue this process.

    Dive into the code


    We've improved Warehouse's developer experience substantially in the last few months. We have several open "good first contribution" issues and a guide to getting started. One of our maintainers, Ernest W. Durbin III, wants to help you dive in and to give you Warehouse stickers, and has 30-minute 1:1 slots available to mentor you. We'll also be running a sprint at PyCon North America in May!

    Contact us



    Thank you for testing Warehouse! You're helping us launch sooner and future users of PyPI will appreciate it.

    — The PyPI Team



    Photo © Mark Hunter