I have a need to unload modules, or rather packages. It turns out to be a little messy, due to how import places stubs into
sys.modules for lookups it knows will always fail. Or, rather, in order to properly unload modules, you have to know how modules were loaded in the first place.
Let’s say we have a package called
package, i.e. a directory
package/, with a
__init__.py file inside it, along with the module files for
Let’s assume that __init__.py uses some other Python modules.
import os import sys ... do stuff
What do you think will be in
sys.modules after you
You’ll see this
... package <module 'package' from 'package\__init__.pyc'> package.os None package.sys None ...
The first entry is the code loaded for
package. However, what are those next two entries? For example, the package.os entry records the fact that an import of os inside package is a redirection to a top-level os package, not a module inside package. This is an optimization for future imports, and nothing more. Furthermore, these exist only if PEP 328 is not in effect (explicit relative imports).
The basic drill for unloading a module is to remove any related entries from sys.modules, and then to hope you can remove all variables pointing inside the loaded module. Typically, this will be the symbol created from the import. So, all things being equal, you do this to import
and this to “un-import”
del sys.modules["package"] del package
and now the next
import package that you do will cause Python to search for your package module and load it (import is a no-op on an already-loaded module).
You may or may not need to remove the stub entries. Presumably you are unloading and reloading in order to load a different version of the module (in my case, I’m bootstrapping to a newer version). If so, then it really depends on whether or not you overload names that might be in the search path. If you don’t ever do this, then don’t worry about it.
But it’s not really that simple. One of your modules might include others of your modules. For me, when I import package.cache (one of the modules inside package), I get this
package <module 'package' from 'package\__init__.pyc'> package.__future__ None package.cache <module 'package.cache' from 'package\cache.pyc'> package.errno None package.httplib None package.lockfile <module 'package.lockfile' from 'package\lockfile.pyc'> package.os None package.package None package.shutil None package.socket None package.stat None package.sys None package.tempfile None package.threading None package.time None package.urllib None package.util <module 'package.util' from 'package\util.pyc'> package.zipfile None
My package.cache module in turn uses package.lockfile and package.util, and those import lots of standard modules, all of which get stubs. So to unload everything, I really need to remove everything starting with package. That might look something like this
for k in [x for x in sys.modules.keys() if x.startswith('package.')]: del sys.modules[k] del sys.modules['package'] del package
which would remove everything inside package, and package, and then remove the local dictionary that contains package’s variables. My Python is probably not idiomatic enough yet, the use of that list comprehension seems strained.
For some reason, the Win32 Python doesn’t implement hardlinks and symlinks, even though it’s there in the operating system as of Windows Vista. Here’s a simple version that’s a demonstration (a real version should memoize the lookup for the two functions, and probably handle Unicode, maybe some more error checking etc etc).
import os import platform import sys source = sys.argv dest = sys.argv type = sys.argv def CreateHardLink(src, dst): import ctypes flags = 1 if source is not None and os.path.isdir(src) else 0 if not ctypes.windll.kernel32.CreateHardLinkA(dst, src, flags): raise OSError def CreateSymbolicLink(src, dst): import ctypes flags = 1 if source is not None and os.path.isdir(src) else 0 if not ctypes.windll.kernel32.CreateSymbolicLinkA(dst, src, flags): raise OSError if platform.system() == 'Windows': print 'hi there' os.link = CreateHardLink os.symlink = CreateSymbolicLink if type == 'link': os.link(source, dest) elif type == 'symlink': os.symlink(source, dest) else: raise Exception('what?')
I like it. The idea of writing a web site directly in Python, rather than in some framework with a clumsily evolved template language, is very appealing.
There’s Win32 versions floating around the net too.
It’s not a make-or-break thing, but sometimes you want to make console output more readable.
And sometimes you want to know if you’re connected to a terminal or are redirecting to a file.
I like Python because you can do dynamic methods and attributes. This makes for more readable code compared to (mis)-using dictionaries to do this.
This is a great thread about Twisted’s Deferred object, used to manage callbacks.
Patterns for clear and yet robust async programming are very important. This was a great write-up by Guido van Rossum explaining Deferred, and inferring its design decisions.
The actual Deferred documentation is here: http://twistedmatrix.com/documents/12.3.0/core/howto/defer.html