Meet the Team: Brian Curtin

This post is part of the "Meet the Team" series of posts, which is meant to give a brief introduction to the Python core development team.

Name:	Brian Curtin
Location:	Chicago, IL
Home Page:	http://blog.briancurtin.com/

How long have you been using Python?

On a day to day basis going on 6 years. Prior to that I used it occasionally for a class in college and also at a summer internship.

How long have you been a core committer?

Just over a year. March 24 marked my first year with the group.

How did you get started as a core developer? Do you remember your first commit?

I got started after noticing a documentation bug while writing an extension module at work, then I submitted a simple patch and Georg Brandl committed it almost immediately. After having that quick success and a fresh source checkout, I wanted to dive in and learn more about the modules I was using and ended up writing a patch to add context manager support to zipfile.

The first few commits I made were documentation fixes in order to keep it simple early on. My first code commit was to add a few features and expand test coverage in the winreg module.

Which parts of Python are you working on now?

As one of the few Windows users involved in CPython development, I try to keep an eye on whatever issues Windows users are having. Due to that, I've had a chance to work on a bunch of the standard library, including modules I hadn't used. I haven't done much with the interpreter itself, but I'm looking to change that.

What do you do with Python when you aren't doing core development work?

I build a variety of test tools for a trading database which is written in C++. There's an extension module for the data API so we can easily write regression tests, performance tests, and we're always trying to build more.

What do you do when you aren't programming?

I'm a huge baseball fan. I umpire college baseball in the spring, various leagues in the summer, and mix in watching and going to Chicago Cubs games.

Meet the Team: Nick Coghlan

This post is part of the "Meet the Team" series of posts, which is meant to give a brief introduction to the Python core development team.

Name:	Nick Coghlan
Location:	Brisbane, Australia
Home Page:	http://www.boredomandlaziness.org

How long have you been using Python?

First encountered 1.5.2 around 1999 when our lecturer used it for a networking course. Started using 2.2 professionally for automated testing around 2002 and never looked back.

How long have you been a core committer?

Guido gave me access in 2005 to update PEP 343 (primarily ditching the context method)

How did you get started as a core developer? Do you remember your first commit?

As far as contributing patches goes, I had 3 months off in 2004 and spent a bunch of it working with Raymond and Facundo on the decimal module, primarily running the telco benchmarks and finding ways to speed up the code. A few of the stranger hacks in the decimal module (like the fast path for checking for special cases and the use of strings when converting tuples of digits to integers) stem from that time.

My actual first commit would have been to PEP 343, and then after that probably to the AST compiler branch as we finished it up for inclusion in 2.5.

Which parts of Python are you working on now?

runpy, functools and contextlib are the main things that tend to end up in my inbox. I also keep an eye on what Brett and Victor are doing with import, what Raymond is doing with collections and itertools, and anything that happens to the compiler. I'm also fascinated by the cultural side of things.

What do you do with Python when you aren't doing core development work?

Not a great deal, actually. The Python stuff at work generally just ticks away doing its thing, so there isn't a lot of call to hack on it at the moment. I do want to do something to tidy up my digital music library, but the scripts for that are just a hack job at the moment.

What do you do when you aren't programming?

Tae kwon do, computer gaming, soccer, reading, etc, etc...

New Blog Design

If you read Python Insider through a feed reader, you may not have seen the new page design Marcin Wojtczuk created for us. It looks great while maintaining a lightweight feel, and we couldn't be happier with the results.

Thank you for your time and efforts, Marcin!

urllib Security Vulnerability Fixed

Guido van Rossum recently pushed a fix for CVE-2011-1521, a security issue in Python's URL libraries. While security issues are rare, it's a good opportunity to let the community in on the process behind reporting, handling, and fixing these issues as they arise.

Reporting an Issue

If you've found a security issue within CPython, the first thing we ask is that you keep the details of the issue private. After determining that you have found a legitimate security issue, generating a succinct but detailed report is key to transferring your knowledge to the core developers.

A good report clearly explains how the relevant parts of the system are affected by the issue. If the issue occurs on a specific platform or due to a dependency, that's helpful to know as well. The affected versions are useful to know, and it's likely that the vulernability will be tested for all active versions as well. Lastly, if you have a test case that shows the issue, be sure to include it. Your report should be sent to the security@python.org group.

Niels Heinen of the Google Security Team recently submitted a good report. He discovered an issue with HTTP 302 redirection handling in the standard library urllib and urllib2 modules. What he found was that a server could redirect requests to inappropriate schemes, leading to situations which could compromise data or systems. In his initial report, Neils explains two scenarios where these redirections could expose problems.

First, since urllib/urllib2 supplies a handler for the file:// URL scheme, a redirection to file:///etc/passwd could expose password data. Neils also explained that redirection to a system device like file:///dev/zero could lead to exhaustion of resources leading to a denial of service.

Handling a Report

Due to the sensitive nature of security reports, the security@python.org list is maintained by a small group of trusted developers who analyze and act on reports as soon as possible. If you wish to keep your transmissions to the list encrypted, see the security news page for OpenPGP details.

If the group determines that there is in fact a security issue, a public bug report may be made with an accompanying patch. In this case, Guido van Rossum made the issue public in issue #11662, complete with an initial patch.

Fixing the Issue

What Guido's patch does is restrict redirection to http://, https://, and ftp:// URL schemes. FTP redirection was deemed acceptable, and it's actually a common redirection: download mirroring systems sometimes redirect requests to geographically convenient FTP servers.

For Python 2.x, FancyURLopener's redirect_internal method now raises an IOError when redirection to an inappropriate scheme is requested. HTTPRedirectHandler's http_error_302 does the same, only raising HTTPError. In Python 3, urllib.request received the same fixes. Included with the patch are two tests which exercise redirection to both valid and invalid schemes.

As for users receiving the fix, the final security release of Python 2.5 will be occurring soon. While there are no scheduled dates for the next patch releases of the maintenance branches - 2.6, 2.7, 3.1, and 3.2 - all received the code to fix the vulnerability.

Meet the Team: Tarek Ziadé

This post is part of the "Meet the Team" series of posts, which is meant to give a brief introduction to the Python core development team.

Name:	Tarek Ziadé
Location:	Turcey near Dijon, Burgundy, France
Home Page:	http://ziade.org

How long have you been using Python?

Around ten years

How long have you been a core committer?

Since December 21 2008

How did you get started as a core developer? Do you remember your first commit?

I started as a core developer in order to maintain Distutils and make it evolve.

My first commit as a core developer was a fix for small bug in a distutils feature I proposed before I became a commiter. That feature was added the week before in Python. It's the ability to configure Distutils' register and upload commands to work with several pypi-like servers.

I committed with my brand new rights on Wed, 24 Dec 2008, which happens to be my birthday, and also the 17th anniversary of the 0.9.4 release of Python.

Which parts of Python are you working on now?

In the stdlib: sysconfig, distutils, packaging (to be added in 3.3), shutil, pkgutil, and occasionally in other modules

What do you do with Python when you aren't doing core development work?

I work at Mozilla in the Service team, where I build web services using Python

What do you do when you aren't programming?

I read comics/graphic novels, write books, play with my kids, drink wines with my wife, and try to renovate my 1848's house.

Formalizing the AST Change Control Policy

Python exposes an abstract syntax tree (AST) representing the compiled form of Python source code in the AST module. The AST module allows user code to inspect and manipulate of the AST representation, in between parsing of source and compilation of bytecode.

Although the meaning of Python code is defined by the language reference, the AST module is a CPython implementation detail, and is not required to be implemented in other Python implementations.

Compatibility of the AST

As part of work to rewrite the CPython peephole optimizer to work on the AST (rather than on the raw bytecode, as is currently the case), Eugene Toder needed to make some changes to the structure of the AST. As a CPython implementation detail, it wasn't immediately clear what backward compatibility policies applied to the AST. So, Eugene asked the question on python-dev. Was it necessary, when changing the AST, to ensure backward compatibility?

The general consensus was that compatibility is not required. The AST module exposes a constant, ast.__version__, which provides a means for user code to vary its behaviour depending on the version of the AST it encounters. This was viewed as sufficient compatibility for an implementation-specific module.

Other Python Implementations

In actual fact, both Jython and IronPython maintainers pointed out that their respective implementations either had a compatible AST module, or intended to provide one. Even so, they did not feel that this meant that the AST should be frozen, and were happy that as long as the ast.__version__ constant changed, the AST could be modified in incompatible ways.

One point that was raised is that a full suite of tests in test_ast.py would help other implementations to ensure that their AST representations were compatible with CPython. Increasing the coverage of test_ast.py would make a good project for someone who wanted to get involved with Python internals!

What Will Happen Next?

The patch which started the discussion is not yet included in CPython. So possibly, nothing will happen. However, if it does get committed, the AST will change in an incompatible way. The ast.__version__ constant will change to reflect this, so user code will know, but changes will be needed. More generally, this will be the way AST changes will be handled in future.

The Python developers are interested in how widely the AST is used, and how much impact this policy will have. If any readers have code that will be affected by the change, they are encouraged to participate in the discussion on python-dev.

Thomas Heller Steps Down as ctypes Maintainer

The Python development community owes a big thanks to long-time ctypes maintainer Thomas Heller. Earlier this month, Thomas announced his departure from the CPython project, the home of his ctypes library since Python 2.5.

I had a chance to talk with Thomas and he filled me in on his history with Python and his ctypes and py2exe projects.

Python

Back in 1999, Thomas came into Mark Lutz's Programming Python while looking for a resource to learn Python and became fascinated with the language right away. He was in the process of replacing Scheme as the extension language for a large C program he had written for Windows.

As for how he got involved in the development team, his first contribution to CPython (and open source in general), was a small Windows-related patch to distutils. His interest in distutils ultimately led him to the creation of the bdist_wininst command to create point-and-click Windows installers. From there, Greg Ward invited him to the python-dev group where he eventually received commit access.

py2exe

Like many Windows users, he had the need to deploy shrink-wrapped Python applications as a single executable file. Early approaches to the problem came from Python luminaries Fredrik Lundh's squeeze and Christian Tismer's sqfreeze, and Thomas contributed several patches to Gordon McMillan's Installer project.

His interest in distutils led Thomas to consider porting Installer to an extension to the packaging library. However, he ended up rewriting the source in order to make use of the existing distutils framework. In the end, he chose the simple yet descriptive name py2exe for the project.

ctypes

The idea for ctypes came from a need to go beyond what pywin32 provided at the time. Additionally, his work with Scheme required an interface to Windows APIs much like his Python work did, so he wanted to keep his project going.

ctypes saw its first public release in 2003 around the release of Python 2.3, after Thomas received numerous requests to publish the project. He mentioned what used to be his small personal project on his Starship page, but it grew into a widely used library in no time.

He originally started the project on Windows but quickly heard calls for a Linux port, which the community helped him complete. With the Linux port came the introduction of libffi to the project, which he also began using on Windows to replace its lower-level implementation.

2006 marked a 1.0 release for ctypes, which corresponded with the library's acceptance into the standard library in Python 2.5. After years of hard work and numerous releases per year, ctypes was now bundled with Python and available by default to a much wider audience.

It took a lot of people to get ctypes to where it is today, and Thomas wants to thank everyone involved, especially Robin Becker. Robin was instrumental in the early phases of the project and contributed both knowledge and encouragement.

A New ctypes Maintainer

After all of the hard work Thomas put in over the years, we would hate to see the project come to a stand still. If you have C experience and time to help out the Python project, the community would greatly appreciate your effort. Check out the new developer guide and search the bug tracker for more information.

Updated: Fixed some links.