Thursday, July 30, 2020

Upgrade to pip 20.2, plus, changes coming in 20.3

On behalf of the Python Packaging Authority, I am pleased to announce that we have just released pip 20.2, a new version of pip. You can install it by running python -m pip install --upgrade pip.

The highlights for this release are:

- The beta of the next-generation dependency resolver is available -- please test
- Faster installations from wheel files
- Improved handling of wheels containing non-ASCII file contents
- Faster pip list using parallelized network operations
- Installed packages now contain metadata about whether they were directly requested by the user (PEP 376’s REQUESTED file)

The new dependency resolver is off by default because it is in beta and not yet ready for everyday use. The new dependency resolver is significantly stricter and more consistent when it receives incompatible instructions, and reduces support for certain kinds of constraints files, so some workarounds and workflows may break. Please test it with the --use-feature=2020-resolver flag. Please see our guide on how to test and migrate, how to report issues, and context for the change.

Please report bugs through the resolver testing survey.

Thanks to all who tested the alpha of the new resolver in pip 20.1 for feedback that helped us get it to the beta stage.

We are preparing to change the default dependency resolution behavior and make the new resolver the default in pip 20.3 (in October 2020).

This release also partially optimizes pip’s network usage during installation (as part of a Google Summer of Code project by McSinyx). Please test it with pip install --use-feature=2020-resolver --use-feature=fast-deps and report bugs to the issue tracker. This functionality is still experimental and not ready for everyday use.

You can find more details (including deprecations and removals) in the changelog.

As with all pip releases, a significant amount of the work was contributed by pip’s user community. Huge thanks to all who have contributed, whether through code, documentation, issue reports and/or discussion. Your help keeps pip improving, and is hugely appreciated. Specific thanks go to Mozilla (through its Mozilla Open Source Support Awards) and to the Chan Zuckerberg Initiative DAF, an advised fund of Silicon Valley Community Foundation, for their funding that enabled substantial work on the new resolver.

Tuesday, July 21, 2020

Python 3.8.5 released as a security hotfix. 3.9.0b5, the last beta before 3.9.0, also available

This is a combined release of Python 3.8.5 and 3.9.0b5. Both are significant but for different reasons. Let’s dig in!

Security content in 3.8.5

We decided to release 3.8.5 ahead of schedule due to a number of security-related fixes. All details can be found in the change log but the gist is:

CVE-2019-20907: infinite loop in a maliciously created .tar file
BPO-41288: segmentation fault during unpickling of objects using a crafted NEWOBJ_EX opcode
BPO-39603: HTTP headers could be injected through a maliciously crafted method parameter in http.client
the original fix for CVE-2020-15801 caused a regression in 3.8.4 (see: BPO-41304)

A small number of other urgent regression fixes and quality-of-life improvements are also present in the release. Get the release here:

https://www.python.org/downloads/release/python-385/

Maintenance releases for the 3.8 series will continue at the regular bi-monthly calendar, with 3.8.6 planned for mid-September 2020.

The last beta of Python 3.9.0 now also available

Python 3.9 is still in development. This release, 3.9.0b5, is the last of five planned beta release previews. Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. You can get 3.9.0b5 here:

https://www.python.org/downloads/release/python-390b5/

The next pre-release, the first release candidate of Python 3.9.0, will be 3.9.0rc1. It is currently scheduled for 2020-08-10.

Call to action

We strongly encourage maintainers of third-party Python projects to test with 3.9 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (2020-08-10). Our goal is have no ABI changes after beta 5 and as few code changes as possible after 3.9.0rc1, the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.9 as possible during the beta phase.
Please keep in mind that this is a preview release and its use is not recommended for production environments.

A reminder for core developers

To help make Python 3.9.0 the best possible release, our Development Cycle section of the Python Developer’s Guide documents that:

A branch preparing for an RC release can only have bugfixes applied that have been reviewed by other core developers. Generally, these issues must be severe enough (e.g. crashes) that they deserve fixing before the final release. All other issues should be deferred to the next development cycle, since stability is the strongest concern at this point.
You cannot skip the peer review during an RC, no matter how small! Even if it is a simple copy-and-paste change, everything requires peer review from a core developer.

Major new features of the 3.9 series, compared to 3.8

Some of the new major new features and changes in Python 3.9 are:

PEP 584, Union Operators in dict
PEP 585, Type Hinting Generics In Standard Collections
PEP 593, Flexible function and variable annotations
PEP 602, Python adopts a stable annual release cadence
PEP 615, Support for the IANA Time Zone Database in the Standard Library
PEP 616, String methods to remove prefixes and suffixes
PEP 617, New PEG parser for CPython
BPO 38379, garbage collection does not block on resurrected objects;
BPO 38692, os.pidfd_open added that allows process management without races and signals;
BPO 39926, Unicode support updated to version 13.0.0;
BPO 1635741, when Python is initialized multiple times in the same process, it does not leak memory anymore;
A number of Python builtins (range, tuple, set, frozenset, list, dict) are now sped up using PEP 590 vectorcall;
A number of Python modules (_abc, audioop, _bz2, _codecs, _contextvars, _crypt, _functools, _json, _locale, operator, resource, time, _weakref) now use multiphase initialization as defined by PEP 489;
A number of standard library modules (audioop, ast, grp, _hashlib, pwd, _posixsubprocess, random, select, struct, termios, zlib) are now using the stable ABI defined by PEP 384.
(Hey, fellow core developer, if a feature you find important is missing from this list, let Łukasz know.)

We hope you enjoy the new releases!

Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself or through organization contributions to the Python Software Foundation.

Your friendly release team,
Ned Deily @nad
Steve Dower @steve.dower
Łukasz Langa @ambv

Monday, July 13, 2020

Python 3.8.4 is now available

Python 3.8.4 is the fourth maintenance release of Python 3.8. Go get it here:

https://www.python.org/downloads/release/python-384/

Maintenance releases for the 3.8 series will continue at regular bi-monthly intervals, with 3.8.5 planned for mid-September 2020.

What’s new?

The Python 3.8 series is the newest feature release of the Python language, and it contains many new features and optimizations. See the “What’s New in Python 3.8” document for more information about features included in the 3.8 series.

This is the first bugfix release that is considerably smaller than the previous three. There’s almost 20% fewer changes at 162 commits than the average of previous three bugfix releases. Detailed information about all changes made in version 3.8.4 specifically can be found in its change log. Note that compared to 3.8.3, version 3.8.4 also contains the changes introduced in 3.8.4rc1.

We hope you enjoy Python 3.8!

Friday, July 3, 2020

Python 3.9.0b4 is now ready for testing

On behalf of the entire Python development community, and the currently serving Python release team in particular, I’m pleased to announce the release of Python 3.9.0b4. Get it here:

https://www.python.org/downloads/release/python-390b4/

This is a beta preview of Python 3.9

Python 3.9 is still in development. This release, 3.9.0b4, is the fourth of five planned beta release previews.

Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release.

Call to action

We strongly encourage maintainers of third-party Python projects to test with 3.9 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (2020-08-10). Our goal is have no ABI changes after beta 5 and as few code changes as possible after 3.9.0rc1, the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.9 as possible during the beta phase.

Please keep in mind that this is a preview release and its use is not recommended for production environments.

Major new features of the 3.9 series, compared to 3.8

Some of the new major new features and changes in Python 3.9 are:

PEP 584, Union Operators in dict
PEP 585, Type Hinting Generics In Standard Collections
PEP 593, Flexible function and variable annotations
PEP 602, Python adopts a stable annual release cadence
PEP 615, Support for the IANA Time Zone Database in the Standard Library
PEP 616, String methods to remove prefixes and suffixes
PEP 617, New PEG parser for CPython
BPO 38379, garbage collection does not block on resurrected objects;
BPO 38692, os.pidfd_open added that allows process management without races and signals;
BPO 39926, Unicode support updated to version 13.0.0;
BPO 1635741, when Python is initialized multiple times in the same process, it does not leak memory anymore;
A number of Python builtins (range, tuple, set, frozenset, list, dict) are now sped up using PEP 590 vectorcall;
A number of Python modules (_abc, audioop, _bz2, _codecs, _contextvars, _crypt, _functools, _json, _locale, operator, resource, time, _weakref) now use multiphase initialization as defined by PEP 489;
A number of standard library modules (audioop, ast, grp, _hashlib, pwd, _posixsubprocess, random, select, struct, termios, zlib) are now using the stable ABI defined by PEP 384.
(Hey, fellow core developer, if a feature you find important is missing from this list, let Łukasz know.)

The next pre-release, the fifth beta release of Python 3.9, will be 3.9.0b5. It is currently scheduled for 2020-07-20.

More resources

Your friendly release team,
Ned Deily @nad
Steve Dower @steve.dower
Łukasz Langa @ambv