Py2exe: Adding pkg_resources required by dependencies

by mandel on April 3rd, 2011

One of the things we wanted to achieve for the Windows port of Ubuntu One was to deploy to the users systems .exe files rather than requiring them to have python and all the different dependencies installed in their machine. There are different reasons we wanted to do this, but this post is not related to that. The goal of this post is to explain what to do when you are using py2exe and you depend on a package such as lazr.restfulclient.

Why lazr.restfulclient?

There are different reasons why I’m using lazr.restfulclient as an example:

  • It is a dependency we do have on Ubuntu One, and therefore I already have done the work with it.
  • It uses two features of setuptools that do not play well with py2exe:
    • It uses namespaced packages.
    • I uses pkg_resources to load resources used for the client.

Working around the use of namedspaced packages

This is actually a fairly easy thing to solve and it is well documented in the py2exe wiki, nevertheless I’d like to show it in this post so that the inclusion of the lazr.restfulclient is complete.

The main issue with namedspaced packages is that you have to tell the module finder from py2exe where to find those packages, which in our example are lazr.authentication, lazr.restfulclient and lazr.uri. A way to do that would be the following:

import lazr
    import as modulefinder
except ImportError:
    import modulefinder
for p in lazr.__path__:
        modulefinder.AddPackagePath(__name__, p)

Adding the lazr resources

This is a more problematic issue to solve since we have to work around a limitation found in py2exe. The lazr.restfulcient tries to load a resource from the py2exe but as the zipfile is reserved for compiled files, and therefore the module fails. In py2exe there is no way to state that those resource files have to be copied to the which would mean that an error is raised at runtime when trying to use the lib but not at build time.

The best way (if not the only one) to solve this is to extend the py2exe command to copy the resource files to the folders that are zipped before they are embedded, that way pkg_resource will be able to load the file with no problems.

import os
import glob
import lazr.restfulclient
from py2exe.build_exe import py2exe as build_exe
class LazrMediaCollector(build_exe):
    """Extension that copies lazr missing data."""
    def copy_extensions(self, extensions):
        """Copy the missing extensions."""
        build_exe.copy_extensions(self, extensions)
        # Create the media subdir where the
        # Python files are collected.
        media = os.path.join('lazr', 'restfulclient')
        full = os.path.join(self.collect_dir, media)
        if not os.path.exists(full):
        # Copy the media files to the collection dir.
        # Also add the copied file to the list of compiled
        # files so it will be included in zipfile.
        for f in glob.glob(lazr.restfulclient.__path__[0] + '/*.txt'):
            name = os.path.basename(f)
            self.copy_file(f, os.path.join(full, name))
            self.compiled_files.append(os.path.join(media, name))

In order to use the above command class to perform the compilation you simply have to tell setup tools which command class to use.

cmdclass = {'py2exe' : LazrMediaCollector}

With the above done, you can use the usual ‘python install py2exe’. Now, the question for the Internet, can this be done with Pyinstaller?