Sean Gillies Blog

Hugh Cayless has written an OpenURL image layer for OpenLayers that pulls imagery from Djakota. I'm eager to see it in action. I've heard other library folks talking about doing this kind of thing with "GeoPDF"; my hope (I'm not speaking for Hugh or UNC) is that they'll take a look at this kind of non-proprietary solution before they do.

Christopher Schmidt explains the traditional approach to wrapping functions and methods, one I use regularly; Python's built-in property function, as a decorator, produces read-only properties, but can provide read-write property access when used traditionally.

Are decorators merely cosmetic? I'm of the opinion that some syntaxes are better than others. You're likely to agree that:

>>> 1 + 2
3

is more concise, readable, and intuitive than

>>> int(1).__add__(2)
3

but may not agree that Python's decorators are a syntactic improvement. PEP 318 was hotly debated, but is final; decorators are in, and they'll be expanded in 3.0.

The motivation for decorators is compelling:

The current method of applying a transformation to a function or method places the actual transformation after the function body. For large functions this separates a key component of the function's behavior from the definition of the rest of the function's external interface. For example:

def foo(self):
    perform method operation
foo = classmethod(foo)

This becomes less readable with longer methods. It also seems less than pythonic to name the function three times for what is conceptually a single declaration. A solution to this problem is to move the transformation of the method closer to the method's own declaration. The intent of the new syntax is to replace:

def foo(cls):
    pass
foo = synchronized(lock)(foo)
foo = classmethod(foo)

with an alternative that places the decoration in the function's declaration:

@classmethod
@synchronized(lock)
def foo(cls):
    pass

Even if calling code isn't exactly broken, wrapping a function more than likely changes the function's signature in some way; keeping all signature specification (such as it is in Python) at the head of a function is a good thing and requires some syntax like that of PEP 318. GIS programmers who've come to Python in the past several years via ArcGIS should get with @. If you can't or won't, that's fine too; there's another way, as Christopher shows.

On "prettier code": all else being equal, prettier code is more readable code. It's code that can teach, that can be more easily modified by others. In some ways, better code.

One downside of the decorator syntax: ability to test decorators in a doctest eludes me. The following:

def noisy(func):
    """
    >>> @noisy
    >>> print foo()
    Blah, blah, blah
    1
    """
    def wrapper(*args):
        print "Blah, blah, blah"
        return func(*args)
    return wrapper

@noisy
def foo():
    return 1

fails:

Exception raised:
    Traceback (most recent call last):
    ...
       @noisy

    ^
     SyntaxError: unexpected EOF while parsing

Could be ignorance on my part.

Last month I blogged about Python logging and how to avoid using print statements in geoprocessing code. But your crufty old code isn't going to rewrite itself, and you're overworked already. An efficient fix would be optimal, and I've got one that only requires a little time to learn how to use Python decorators.

Say you have a module and function that does some geoprocessing work and prints various messages along the way. Something like this:

def work():
    print "Starting some work."
    print "Doing some work ..."
    print "Finished the work ..."

if __name__ == "__main__":
    work()

which, when run, produces output in your terminal:

seang$ python work.py
Starting some work.
Doing some work ...
Finished the work ...

Now, your function is much more gnarly than work(), and rewriting it will only sap your goodwill toward its author. You'd think it would be possible to somehow wrap the work() function, catching those print statements and redirecting them to a logger – while not breaking code that calls work() – all in a reusable fashion. And it is possible, using a decorator like the 'logprints' class in the code below:

import logging
from StringIO import StringIO
import sys

logging.basicConfig(
    level=logging.DEBUG,
    format='%(asctime)s %(levelname)s %(message)s',
    filename='work.log',
    filemode='w'
    )

class logprints(object):

    def __init__(self, func):
        # Called when function is decorated
        self.func = func

    def __call__(self, *args, **kwargs):
        # Called when decorated function is called

        # save reference to stdout
        saved = sys.stdout

        # make a string buffer and redirect stdout
        net = StringIO()
        sys.stdout = net

        # call original function
        retval = self.func(*args, **kwargs)

        # restore stdout
        sys.stdout = saved

        # read captured lines and log them
        net.seek(0)
        for line in net.readlines():
            logging.info(line.rstrip())

        # return original function's return value(s)
        return retval

@logprints
def work():
    print "Starting some work."
    print "Doing some work ..."
    print "Finished the work ..."

if __name__ == "__main__":
    work()

The statement "@logprints" is interpreted as "decorate the immediately following function with the 'logprints' class." On import of this module, the method logprints.__init__() is called with 'work' as the sole argument. Afterwards, when work() is interpreted, logprints.__call__() is called. That method acts as a proxy for the original, now decorated, function. Here is the print capturing and logging decorator in action:

seang$ python work2.py
seang$ cat work.log
2008-12-30 12:14:44,044 INFO Starting some work.
2008-12-30 12:14:44,044 INFO Doing some work ...
2008-12-30 12:14:44,044 INFO Finished the work ...

Yes, you could have redirected the output of the original script in the terminal, but remember that Python's logging module sets you up to do much more.

I've recently learned how to use parameterized decorators by following the examples in Bruce Eckel's article. I'm using one to deprecate functions in Shapely:

import warnings

class deprecated(object):

    """Mark a function deprecated.
    """

    def __init__(self, version="'unknown'"):
        self.version = version
        self.msg_tmpl = "Call to deprecated function '%s', to be removed in version %s"

    def __call__(self, func):
        def wrapping(*args, **kwargs):
            warnings.warn(self.msg_tmpl % (func.__name__, self.version),
                          DeprecationWarning,
                          stacklevel=2
                          )

            return func(*args, **kwargs)
        wrapping.__name__ = func.__name__
        wrapping.__doc__ = func.__doc__
        wrapping.__dict__.update(func.__dict__)
        return wrapping

Marking a function deprecated like:

>>> from shapely.deprecation import deprecated
>>> @deprecated(version="1.1")
... def foo():
...     return None
...

causes a warning to be emitted when the function is called:

>>> foo()
/Users/seang/code/gispy-lab/bin/labpy:1:
DeprecationWarning: Call to deprecated function 'foo',
to be removed in version 1.1

Deprecation-marking decorators are a great solution (which I first saw used, in a different form, in Zope 3). Why would you want to rewrite a function that's going away in the next software version?

Decorators can also be chained. In Shapely I've factored the check for non-nullness of GEOS geometries into a decorator and chain it with the built-in property decorator:

@property
@exceptNull
def geoms(self):
    return GeometrySequence(self, LineString)

To this effect:

>>> from shapely.geometry import MultiPoint
>>> m = MultiPoint()
>>> m.geoms
Traceback (most recent call last):
...
ValueError: Null geometry supports no operations

The exception is raised by the 'exceptNull' decorator.

Not much specifically about GIS here, I'll admit, but GIS programming in Python is, or should be, just Python programming. Feel free to comment if you see any interesting applications of decorators.

My work is done. Or, at least, the part of my work not involved with deprogramming OGC web services cult members. And the part of my work not involved with tooting my own horn. For example, check out this blog post from 2005 (2005!) on emailing Python script errors. Prescient, huh? Too bad I didn't write "or send a message not exceeding 140 characters -- a 'tweet', so to speak -- to your 'followers'." instead of "or ping your enterprise's paging system." If I had done that, you'd never hear the end of it.

I've spent this short week tuning up my new laptop's development environment, and a side effect of this work is a new build system for replicable, isolated Python, GIS, and image/raster processing environments. Ichpage replaces Gdawg on my machine. It supplies:

• GDAL (osgeo.gdal, etc)
• geojson
• geopy
• keytree
• lxml
• Numpy
• PIL
• PyProj
• Rtree
• Shapely

and their various library dependencies (libgdal, libgeos_c, libspatialindex, libxml2, libxslt). To get started, clone or get the tarball, cd into the directory, and execute:

$ virtualenv .
$ source ./bin/activate
(ichpage)$ python bootstrap.py
(ichpage)$ buildout
(ichpage)$ . ./setenv
(ichpage)$ labpy
>>> from osgeo import gdal
>>> from shapely.geometry import Point
...

To do tasks include linking the GDAL utilities into the environment's bin directory, adding WorldMill, perhaps adding matplotlib. For now, it's a way for me to manage C libs while I develop Shapely and Rtree, and perhaps useful to other geospatial Python developers.

Check out this interesting article about the reanimation of an orphaned plant database and its associated ArcIMS instance. The analysis of the issues is sound. I disagree, of course, with their conclusion that ArcIMS is something worth learning and deploying in 2008, and this raises in my mind another issue that the authors did not identify: is not the project's data and its provenance the thing that is most important to preserve? Must the interface cruft around it be preserved in anything other than an archived form, if at all? The ArcIMS user interface and the species database browser are no kind of programmable web APIs; it's unlikely any other application would be broken by a switch to some free web mapping framework or modern search interface.

Now there's a question: switch to what? If this story is just the beginning, and bigger boxes of used, discarded, but potentially useful first-generation web/GIS projects end up in the laps of librarians, a turnkey (and open source, naturally) ArcIMS to MapServer/MapGuide migration tool might be a handy thing. I wouldn't be surprised if such a thing existed. Its authors might want to consider pitching it to GIS librarians in higher education.

Geojson 1.0.1 fixes a bug in serialization of features with no geometry.

While I was fiddling with a related blog post, word came out about Obama's appointment of John Holdren to the post of White House science advisor. Holdren's Boston Globe op-ed from the summer is now a must read:

The few climate-change "skeptics" with any sort of scientific credentials continue to receive attention in the media out of all proportion to their numbers, their qualifications, or the merit of their arguments. And this muddying of the waters of public discourse is being magnified by the parroting of these arguments by a larger population of amateur skeptics with no scientific credentials at all.

I'm not sure, but I think he might be talking about you, John Christy and Joe Francica. It looks like we're going to start "counting the bears" in earnest. If Francica was taken aback by Obama's comment on the rancid state of Interior, I fear he may need someone to grab him a fainting couch for this.