A Python Launcher For Windows

Mark Hammond (author of pywin32 and long-time supporter of Python on Windows) has written PEP 397, which describes a new launcher for Python on Windows. Vinay Sanjip (author of the standard library logging module) has recently created an implementation of the launcher, downloadable from https://bitbucket.org/vinay.sajip/pylauncher/downloads

The launcher allows Python scripts (.py and .pyw files) on Windows to specify the version of Python which should be used, allowing simultaneous use of Python 2 and 3.

Windows users should consider downloading the launcher and testing it, to help the Python developers iron out any remaining issues. The launcher is packaged as a standalone application, and will support currently available versions of Python. The intention is that once the launcher is finalised, it will be included as part of Python 3.3 (although it will remain available as a standalone download for users of earlier versions).

Two versions of the launcher are available - launcher.msi which installs in the Program Files directory, and launchsys.msi which installs in Windows' System32 directory. (There are also 64-bit versions for 64-bit versions of Windows).

Some Details About the Launcher

The full specification of the behaviour of the launcher is given in PEP 397. To summarise the basic principles:

• The launcher supplies two executables - py.exe (the console version) and pyw.exe (the GUI version).
• The launcher is registered as the handler for .py (console) and .pyw (GUI) file extensions.
• When executing a script, the launcher looks for a Unix-style #! (shebang) line in the script. It recognises executable names python (system default python), python2 (default Python 2 release) and python3 (default Python 3 release). The precise details can easily be customised on a per-user or per-machine basis.
• When used standalone, the py.exe command launches the Python interactive interpreter. Command line switches are supported, so that py -2 launches Python 2, py -3 launches Python 3, and py launches the default version.

Simple Usage Instructions

When it is installed, the launcher associates itself with .py and .pyw scripts. Unless you do anything else, scripts will be run using the default Python on the machine, so you will see no change. One thing you might like to do, if you use the console a lot, is to add .py to your PATHEXT variable so that scripts don't get executed in a separate console.

To specify that a script must use Python 2, simply add:

#!/usr/bin/env python2

as the first line of the script. (This is a Unix-compatible form. If you don't need Unix compatibility, #!python2 will do).

If on the other hand, you want to specify that a script must use Python 3, add:

#!/usr/bin/env python3

as the first line.

You can also start the Python interpreter using any of the following commands:

# Default version of Python
py
# Python 2
py -2
# Python 3
py -3

For this to work, the py.exe executable must be on your path. This is automatic with the launchsys version of the installer, but the install directory (C:\Program Files\Python Launcher) must be added manually to PATH with launcher.msi.

Further Reading

The following email threads on python-dev cover some of the key discussions:

• Mark's initial announcement of the draft PEP: http://mail.python.org/pipermail/python-dev/2011-March/109509.html
• The second draft of the PEP: http://mail.python.org/pipermail/python-dev/2011-March/109786.html
• Vinay's initial query about a C implementation of the launcher: http://mail.python.org/pipermail/python-dev/2011-June/112145.html
• Vinay's announcement of his C implementation: http://mail.python.org/pipermail/python-dev/2011-July/112184.html
• Vinay's call for testers: http://mail.python.org/pipermail/python-dev/2011-July/112251.html

CPython 3.2.1 Released

On behalf of the python-dev team, release manager Georg Brandl has announced the final release of CPython 3.2.1. Windows installers and tarballs are available as of July 10, so please consider upgrading to this release.

The What's New document lists all of the new features in 3.2, and the Misc/NEWS file in the source lists each bug fixed.

If you find any issues with this release or any other, please report them to http://bugs.python.org/.

3.2.1 Release Candidate 2 Released

Following up a big month of releases in June, the second release candidate of the 3.2.1 line is now ready. Since the first release candidate on May 15, over 40 issues have been fixed. We encourage everyone to test their projects with this candidate to get one last look before the final release of 3.2.1.

What's fixed?

I/O

#1195 spent a few years witout a fix, but a simple addition to clear errors before calling fgets solves the problem of interrupting sys.stdin.read() with CTRL-D inside of input(). The io system saw a cleanup in #12175 with the readall method with None being the return value on a read() which returns None, and a ValueError is now raised when a file can't be opened.

Although this isn't new for RC2, #11272 is an important 3.2.1 fix to input() on Windows - the fixing of a trailing \r. The issue has been reported many times over and affects a many people (distutils upload command anyone?), so hopefully 3.2.1 does the trick for you.

Windows

3.2.0 brought a new feature for Windows: os.symlink support. With that feature came #12084, os.stat was improperly evaluating Windows symlinks, so the inner workings of the various stat functions were corrected.

A user noticed that os.path.isdir was slow, and the fact that it relied on os.stat contributed to that, especially when evaluating symlinks (which are generally twice as slow as regular files). While os.path.isdir isn't anyone's performance bottleneck, it's called numerous times on interpreter startup so changing it in #11583 to use GetFileAttributes gives a tiny speedup to build on.

subprocess

Creating a Popen object with unexpected arguments was causing an AttributeError, but that was reported in #12085 and was fixed by the reporter. Due to a change in 3.2.0, Popen wasn't correctly handling empty environment variables, specifically the env argument. #12383 was created for the issue and was promptly fixed.

...and more!

For a full list of changes through 3.2.1 RC2, check out the change log and download it now!

As always, please report any issues you find to http://bugs.python.org. We appreciate your help in making great Python releases.

June Releases - 2.6.7, 2.7.2, 3.1.4

June is a big month for Python releases, with an update coming out of all active branches.

2.6.7

A new source-only release of Python 2.6.7 is available, providing fixes to three security issues. Now that the 2.6 line is in security-mode, these releases will happen on an as-needed basis until October 2013 in source-only form. If you require binary installers, you should consider an upgrade to 2.7 or 3.2.

2.6.7 is the first release to contain a fix to the previously covered urllib vulnerability. Additionally, an smtpd DoS vulnerability (Issue #9129) and SimpleHTTPServer.list_directory XSS vulnerability (Issue #11442) are fixed.

2.7.2

The last minor version of the 2.x line, 2.7, received over 150 bug fixes since 2.7.1 in November 2010. 2.7.2 source and binary installers are available as of June 12, which include the security fixes mentioned in 2.6.7.

A number of crashes are fixed: a situation when Python incorrectly used non-Python managed memory while it was being modified by another thread, when deleting __abstractmethods__ from a class, accessing a memory-mapped file past its length, and several others.

A fix to getpass corrects a regression in regards to CTRL-C and CTRL-Z handling. multiprocessing received a number of fixes, including treating Windows services like frozen executables and a correction to a race condition when terminating multiprocessing.Pool workers. mmap was fixed to work with file sizes and offsets larger than 4 GB even on 32-bit builds, and a TypeError is now raised rather than segfaulting when trying to write to a non-writeable map.

For a full list of changes, see the 2.7.2 news file.

3.1.4

3.1.4 is the last bug-fix release of the 3.1.x line, sending 3.1 into security-mode as the 3.2 line carries on. 3.1.4 contains over 100 bug fixes since the 3.1.3 release in November 2010. As with 2.7.2, binary installers are available as of June 12, and 3.1.4 is the first 3.x release to contain the security fixes listed in 2.6.7.

3.1.4 corrects some problems with __dir__ lookups on objects, dates past 2038 in the Windows implementation of os.stat and os.utime, and a number of 64-bit cleanups. The io library saw a number of changes in returning None when nothing was read and raising appropriate exceptions in other spots. ctypes callback arguments were fixed on 64-bit Windows and a crash was also remedied.

For a full list of changes, see the 3.1.4 news file.

3.2.1

3.2.1 is currently in the release candidate phase, with one round already completed and a second release candidate expected soon. We would greatly appreciate 3.2 users trying out the release candidates to ensure we cover any issues you may be seeing. If you have any bugs to report, please file them on bugs.python.org.

New faulthandler module in Python 3.3 helps debugging

When a user reports that your program crashes or hangs, sometimes you can only help to try and collect more information and outline a scenario to reproduce the situation. Even with a reliable user scenario, as a developer you are often unable to reproduce the situation due to environment differences, e.g., operating system and compiler. If you are lucky, the user will be able to install debug tools, but most of time you will have to wait until another person is able to obtain more information from the same situation.

Fatal Errors

A new module introduced in Python 3.3 should help this problem: faulthandler. faulthandler provides the ability to dump the Python traceback on a fatal error such as a segmentation fault, division by zero, abort, or bus error. You can enable it inside your application using faulthandler.enable(), by providing the -X faulthandler option to the Python executable, or with the PYTHONFAULTHANDLER=1 environment variable. Output example:

Fatal Python error: Segmentation fault

Current thread 0x00007f7babc6b700:
  File "Lib/test/crashers/gc_inspection.py", line 29 in g
  File "Lib/test/crashers/gc_inspection.py", line 32 in <module>
Segmentation fault

Timeout

faulthandler can also dump the traceback after a timeout using faulthandler.dump_tracebacks_later(timeout). Call it again to restart the timer or call faulthandler.cancel_dump_tracebacks_later() to stop the timer. Output example:

Timeout (0:01:00)!
Current thread 0x00007f987d459700:
  File "Lib/test/crashers/infinite_loop_re.py", line 20 in <module>

Use the repeat=True option to dump the traceback each timeout seconds, or exit=True to immediatly exit the program in an unsafe fashion, e.g. don't flush files.

User Signal

If you have access to the host on which the program is running, you can use faulthandler.register(signal) to install a signal handler to dump the traceback when signal is received. On UNIX, for example, you can use the SIGUSR1 signal: kill -USR1 <pid> will dump the current traceback. This feature is not available on Windows. Output example:

Current thread 0x00007fdc3da74700:
  File "Lib/test/crashers/infinite_loop_re.py", line 19 in <module>

Another possibility is to explicitly call faulthandler.dump_traceback() in your program.

Security Issues and the Output File

faulthandler is disabled by default for security reasons, mainly because it stores the file descriptor of sys.stderr and writes the tracebacks into this file descriptor. If sys.stderr is closed and the file descriptor is reused, the file descriptor may be a socket, a pipe, a critical file or something else. By default, faulthandler writes the tracebacks to sys.stderr, but you can specify another file. For more information, see the faulthandler documentation.

Third-party Module for Older Python Versions

faulthandler is also maintained as a third-party module for Python 2.5 through 3.2 on PyPI. The major difference between the Python 3.3 module and the third-party module is the implementation of dump_tracebacks_later(): Python 3.3 uses a thread with a timeout on a lock, whereas the third party uses SIGALRM and alarm().

The lock timeout, which is a new feature of Python 3.3, has a microsecond resolution. The alarm() timer used on older versions has a resolution of one second, and the SIGALRM signal may interrupt the current system call which will fail with an EINTR error.

Early Success

The new faulthandler module has already helped with tracking down race conditions in our buildbots. We hope that it will also help you in your programs.

The Python Core Mentorship Program

Jesse Noller recently announced the formation of the Python Core Mentorship program. The idea behind the program is to help programmers, including students and developers from other projects, connect with experienced contributors who serve as mentors to ease them into Python Core development.

Contributors Wanted

The mentors will help people regardless of experience level by bringing them up to speed, answering questions, and giving guidance as needed in a non-confrontational and welcoming way. The contributors will receive guidance through the entire contribution process, including discussions on the related mailing lists, the bug tracker, Mercurial, code reviews, and much more.

Early Success

The program already has been successful, and the participants have actively committed a number of patches. There have also been several constructive discussions on the mailing list, helping guide people in the right direction for a variety of issues.

Code of Conduct

The program has a code of conduct explained on the website that aims to assuage concerns many new contributors have when interacting with experienced developers and mailing lists on contribution in general. Jesse and the other mentors hope that this program can act as a model for other projects long-term, not just benefiting Python-Core. They also want the program to help increase the overall diversity of the contributors to Python.

Signing Up

The program is run via the mailing list and has a clear, concise website devoted to it. If you would like to join to ask questions and begin on the path of core contribution, or even if you are an experienced developer (even experienced in Python-Core) looking to ask questions you're worried about asking on other lists, this is an excellent opportunity to jump in, ask and get your feet wet!

Portuguese, German, Korean, and Traditional Chinese Translations

The Python Insider translation project is continuing to grow! Today we are launching Portuguese, German, Korean, and Traditional Chinese versions of the blog. The translators have already started publishing the backlog of posts. As with the other translations, these parallel editions may lag slightly behind the original posts on Python Insider.