Category Archives: Languages

Python argparse

Command-line processing

As of Python 2.7, the preferred command-line parsing library is argparse. The pattern is straightforward – create an ArgumentParser, add all the command-line options, then call it and extract arguments.

import argparse
def commandline():
  parser = argparse.ArgumentParser(
    description='Your command-line description',

  parser.add_argument('path', nargs='*', help='help string')
  parser.add_argument('--user', action='append', help='help string')
  parser.add_argument('--verbose', action='count', default=0, help='help string')
  parser.add_argument('--quiet', action='store_true', help='help string')

  args = parser.parser_args()

  print '  path   =    %s' % ',\n              '.join(args.path)
  print '  user   =    %s' % ',\n              '.join(args.user)
  print '  verbose= %d' % args.verbose
  print '  quiet = %s' % 'True' if args.quiet else 'False'


sub/one sub/three --user=sub/two --verbose --quiet --verbose

this produces

path    = sub/one
user    = sub/two
verbose = 2
quiet   = True

Arguments are positional or optional. In the example above, all arguments not attached to one of the listed options is gathered up into the ‘path’ positional argument.

More reading



Gitstats, source at

I think this could be turned into something pretty cool with a little bit of work. As-is, it uses gnuplot to do the visualization, so that makes it a little hard to use on Windows. It would be nicer if it used matplotlib or the Python ggplot2, or the approach that IPython Notebook took.

So, speaking of “the future of visualization in Python”, here are some links.

And for fun, to emulate XKCD

Cog – interesting Python-based code generator

I like the approach, which is that you can (1) do generated code for any language and (2) edit the generated code with some freedom, because there are markers to separate generated code from non-generated code.

Cog uses Python – it runs any Python code inside the file, replacing the whole file with the output from the generation. I’ll have to try it out in conjunction with SCons.

Extending Python import

I want Python’s import to look something like this

import google.protobuf.message from ('Python Google Protocol Buffers', '2.5.0',

The extra information is hints – hints to the import system to make sure that it found the correct google package, hints to a search/download system on where to look for it if it’s not already present locally. This would basically replace pip. I don’t want to need to install packages. I don’t mind installing packages, you can prebake a system, but it should be an option for efficiency’s sake, not something that everyone must do.

Go has taken a big step in this direction, but it’s a little inflexible. It will download a package from a remote repository, but you have to specify the path, and it’s to a specific repository. This has long-term implications, all your code will need updating if the repository moves. Also, this is not very distributed, and you can’t package things up easily for bundle distribution. Every first-time run of a Go program is going to hit the remote repositories – packages are cached in your local program, but not even system-wide.

What I want is very loose coupling. If I have modules preinstalled on my system and they are the right modules, they get used. If not, my import casts a net to find it, and this search process can be enhanced as desired. For example, a company might want to keep a local cache server of packages so that most searches can be satisfied locally, only going out to other networks for unknown packages. Or an organization might push a giant set of likely-defined modules to every system.

I also want as much flexibility as possible. I like the Google search approach – no schema that has to absolutely be followed, it’s all hints, hints to a search engine. But the hints are specific enough that you’ll find exactly what you are looking for.

Also, I want something that automatically handles architecture differences, platform differences, compiler/interpreter differences, and so on. If I run my code in Python 2.7, I want it to use a Python 2.7-compatible library (e.g. perhaps already precompiled for my exact version of Python), but if I run it in Python 3.3, I might expect a different Python 3.3-savvy library to be used. Or if I’m running on a Windows 7 64-bit machine, then I want any native code to be specific to my architecture and operating system.


surrogate is a library used to create stubs for non-existing modules in sys.modules. Its aim is for unit testing, but it’s an example of manipulating sys.modules.

PEP 302 — New Import Hooks is the original specification for import hooks, kept around because the current system uses it for documentation on how to make custom importers.

The current system is described in The import system and importlib in the Python 3 documentation; also see sys.meta_path and related in the sys module reference. I don’t yet know how much of this is in Python 2.7. However, this still doesn’t let me get extra hint information to my custom importer.

The sum of all knowledge of Python packaging is contained herein: Python Packaging User Guide. OK, maybe not quite, but this is from the Python Packaging Authority, a group working on packaging and installing distributions in Python.

Bento is another interesting Python package system.

Some related module-system PEPs