June Releases - 2.6.7, 2.7.2, 3.1.4

June is a big month for Python releases, with an update coming out of all active branches.

2.6.7

A new source-only release of Python 2.6.7 is available, providing fixes to three security issues. Now that the 2.6 line is in security-mode, these releases will happen on an as-needed basis until October 2013 in source-only form. If you require binary installers, you should consider an upgrade to 2.7 or 3.2.

2.6.7 is the first release to contain a fix to the previously covered urllib vulnerability. Additionally, an smtpd DoS vulnerability (Issue #9129) and SimpleHTTPServer.list_directory XSS vulnerability (Issue #11442) are fixed.

2.7.2

The last minor version of the 2.x line, 2.7, received over 150 bug fixes since 2.7.1 in November 2010. 2.7.2 source and binary installers are available as of June 12, which include the security fixes mentioned in 2.6.7.

A number of crashes are fixed: a situation when Python incorrectly used non-Python managed memory while it was being modified by another thread, when deleting __abstractmethods__ from a class, accessing a memory-mapped file past its length, and several others.

A fix to getpass corrects a regression in regards to CTRL-C and CTRL-Z handling. multiprocessing received a number of fixes, including treating Windows services like frozen executables and a correction to a race condition when terminating multiprocessing.Pool workers. mmap was fixed to work with file sizes and offsets larger than 4 GB even on 32-bit builds, and a TypeError is now raised rather than segfaulting when trying to write to a non-writeable map.

For a full list of changes, see the 2.7.2 news file.

3.1.4

3.1.4 is the last bug-fix release of the 3.1.x line, sending 3.1 into security-mode as the 3.2 line carries on. 3.1.4 contains over 100 bug fixes since the 3.1.3 release in November 2010. As with 2.7.2, binary installers are available as of June 12, and 3.1.4 is the first 3.x release to contain the security fixes listed in 2.6.7.

3.1.4 corrects some problems with __dir__ lookups on objects, dates past 2038 in the Windows implementation of os.stat and os.utime, and a number of 64-bit cleanups. The io library saw a number of changes in returning None when nothing was read and raising appropriate exceptions in other spots. ctypes callback arguments were fixed on 64-bit Windows and a crash was also remedied.

For a full list of changes, see the 3.1.4 news file.

3.2.1

3.2.1 is currently in the release candidate phase, with one round already completed and a second release candidate expected soon. We would greatly appreciate 3.2 users trying out the release candidates to ensure we cover any issues you may be seeing. If you have any bugs to report, please file them on bugs.python.org.

New faulthandler module in Python 3.3 helps debugging

When a user reports that your program crashes or hangs, sometimes you can only help to try and collect more information and outline a scenario to reproduce the situation. Even with a reliable user scenario, as a developer you are often unable to reproduce the situation due to environment differences, e.g., operating system and compiler. If you are lucky, the user will be able to install debug tools, but most of time you will have to wait until another person is able to obtain more information from the same situation.

Fatal Errors

A new module introduced in Python 3.3 should help this problem: faulthandler. faulthandler provides the ability to dump the Python traceback on a fatal error such as a segmentation fault, division by zero, abort, or bus error. You can enable it inside your application using faulthandler.enable(), by providing the -X faulthandler option to the Python executable, or with the PYTHONFAULTHANDLER=1 environment variable. Output example:

Fatal Python error: Segmentation fault

Current thread 0x00007f7babc6b700:
  File "Lib/test/crashers/gc_inspection.py", line 29 in g
  File "Lib/test/crashers/gc_inspection.py", line 32 in <module>
Segmentation fault

Timeout

faulthandler can also dump the traceback after a timeout using faulthandler.dump_tracebacks_later(timeout). Call it again to restart the timer or call faulthandler.cancel_dump_tracebacks_later() to stop the timer. Output example:

Timeout (0:01:00)!
Current thread 0x00007f987d459700:
  File "Lib/test/crashers/infinite_loop_re.py", line 20 in <module>

Use the repeat=True option to dump the traceback each timeout seconds, or exit=True to immediatly exit the program in an unsafe fashion, e.g. don't flush files.

User Signal

If you have access to the host on which the program is running, you can use faulthandler.register(signal) to install a signal handler to dump the traceback when signal is received. On UNIX, for example, you can use the SIGUSR1 signal: kill -USR1 <pid> will dump the current traceback. This feature is not available on Windows. Output example:

Current thread 0x00007fdc3da74700:
  File "Lib/test/crashers/infinite_loop_re.py", line 19 in <module>

Another possibility is to explicitly call faulthandler.dump_traceback() in your program.

Security Issues and the Output File

faulthandler is disabled by default for security reasons, mainly because it stores the file descriptor of sys.stderr and writes the tracebacks into this file descriptor. If sys.stderr is closed and the file descriptor is reused, the file descriptor may be a socket, a pipe, a critical file or something else. By default, faulthandler writes the tracebacks to sys.stderr, but you can specify another file. For more information, see the faulthandler documentation.

Third-party Module for Older Python Versions

faulthandler is also maintained as a third-party module for Python 2.5 through 3.2 on PyPI. The major difference between the Python 3.3 module and the third-party module is the implementation of dump_tracebacks_later(): Python 3.3 uses a thread with a timeout on a lock, whereas the third party uses SIGALRM and alarm().

The lock timeout, which is a new feature of Python 3.3, has a microsecond resolution. The alarm() timer used on older versions has a resolution of one second, and the SIGALRM signal may interrupt the current system call which will fail with an EINTR error.

Early Success

The new faulthandler module has already helped with tracking down race conditions in our buildbots. We hope that it will also help you in your programs.

The Python Core Mentorship Program

Jesse Noller recently announced the formation of the Python Core Mentorship program. The idea behind the program is to help programmers, including students and developers from other projects, connect with experienced contributors who serve as mentors to ease them into Python Core development.

Contributors Wanted

The mentors will help people regardless of experience level by bringing them up to speed, answering questions, and giving guidance as needed in a non-confrontational and welcoming way. The contributors will receive guidance through the entire contribution process, including discussions on the related mailing lists, the bug tracker, Mercurial, code reviews, and much more.

Early Success

The program already has been successful, and the participants have actively committed a number of patches. There have also been several constructive discussions on the mailing list, helping guide people in the right direction for a variety of issues.

Code of Conduct

The program has a code of conduct explained on the website that aims to assuage concerns many new contributors have when interacting with experienced developers and mailing lists on contribution in general. Jesse and the other mentors hope that this program can act as a model for other projects long-term, not just benefiting Python-Core. They also want the program to help increase the overall diversity of the contributors to Python.

Signing Up

The program is run via the mailing list and has a clear, concise website devoted to it. If you would like to join to ask questions and begin on the path of core contribution, or even if you are an experienced developer (even experienced in Python-Core) looking to ask questions you're worried about asking on other lists, this is an excellent opportunity to jump in, ask and get your feet wet!

Portuguese, German, Korean, and Traditional Chinese Translations

The Python Insider translation project is continuing to grow! Today we are launching Portuguese, German, Korean, and Traditional Chinese versions of the blog. The translators have already started publishing the backlog of posts. As with the other translations, these parallel editions may lag slightly behind the original posts on Python Insider.

Romanian and Simplified Chinese Translations

The Python Insider team is very excited to announce two new blogs today. Translators for Romanian and Simplified Chinese have joined the Translation Project, and have already started publishing the backlog of posts. As with the other translations, these parallel editions may lag slightly behind the original posts on Python Insider.

Jython Migrates to Mercurial

Jython has finally migrated from Subversion to Mercurial. This has been a long time coming: unfortunately we had a difficult Subversion repo that took some effort to cleanly convert to a different revision control system.

The new official Jython repo is now hosted @

http://hg.python.org/jython

with a BitBucket Mirror for easy forking.

There's also a larger read-only repo with ongoing feature branches (converted to Mercurial Bookmarks) hosted at http://hg.python.org/jython-fullhistory

Mercurial makes it even easier to contribute to Jython, pull up a fork and come help us build Jython 2.6!

Python 3.3 to Drop Support for OS/2, Windows 2000, and VMS

Every so often there comes a time to prune the list of supported operating systems to match the usage landscape. On top of that, the pool of contributing developers on a platform also holds significance, as there needs to be someone around to complete development tasks in order to have a quality release. Other factors, such as the age of an operating system and its hinderance on future development work, also weigh on the list.

Victor Stinner recently proposed dropping OS/2 and VMS support for CPython, a year after his original question on OS/2 support. Victor's original inquiry came around the time of his seemingly non-stop Unicode efforts, specifically for an issue with os.execvpe() supporting environment variables via the PEP 383 surrogateescape handler. OS/2 and VMS currently have no representation on the development team and receive no testing during the release process.

The process of writing this post got me thinking about a previous discussion about removing Windows 2000, which seemed to fall to the wayside. Systems setting COMSPEC to command.com were also supposed to be on the chopping block back then. As of now, both have joined OS/2 and VMS. Windows 2000 is up for removal in order to make development work easier, removing the need to account for legacy APIs on an operating system which hit end-of-life in 2010.

In order to begin removing support for those systems, Victor and I started by updating PEP 11.

PEP 11

This PEP outlines the operating systems that are no longer supported and explains the process of adding a system to that list.

Once it is decided that an operating system can start the process of removal, it is formally announced as unsupported. This announcement traditionally goes for the in-development version, so dropping support of OS/2, Windows 2000, and VMS begins with Python 3.3.

The first stage is fairly hands off, more of a raising of the white flag. It's a signal that there's no one around to maintain the code and ensure a quality release. Changes to compilation and installation may be made to alert users on those platforms that the platform is unsupported. A note will go into the "What's New" document listing the newly unsupported platforms.

After a release cycle of being unsupported, the version afterwards becomes fair game for removal of code. In this case, code can be removed in 3.4. There probably won't be a wholesale removal of that code, but developers that come across it in their normal work may remove any #ifdef blocks, configure sections, or out-of-date code.

What You Can Do

If you are a user of OS/2 or VMS, there are a few things you can do to save your platform.

Become a Maintainer

Nothing says support better than an active developer. Andrew MacIntyre has been the OS/2 maintainer for some time now, and he stated during Victor's first OS/2 query that OS/2 is behind on Unicode support, so that's certainly an area that needs focus. VMS appears to have some amount of external support via http://www.vmspython.org, but as discussed in issue 11918, someone needs to step up to allow the continued VMS support upstream.

If you are interested in taking over for either platform, see the developer's guide for the current development proccesses.

Contribute a build slave

With an active developer, a platform stands a better chance of survival. With a build slave, a platform stands an even better chance, not only at survival but also at quality.

Python uses Buildbot for continuous integration, and build slaves are currently provided for Linux, Mac, Windows, and Open Indiana (Solaris), for various versions, architectures, and configurations. Being able to donate a machine to the build fleet for OS/2 or VMS would allow those platforms to receive the same attention that more mainstream platforms receive.

If you can donate either time or hardware to help keep OS/2 and VMS alive, contact the python-dev mailing list to coordinate your efforts.