British Atmospheric Data Centre Python Eggs

British Atmospheric Data Centre Overview Part I: Installers  What are Eggs?  What Eggs can do for me?  Package installation with Eggs Part II: Developers  Component overview  Package compatibility  Developing with Eggs Part III: Demo

British Atmospheric Data Centre What are Python Eggs? Eggs = + JARs CPAN + Backwards Compatibility

British Atmospheric Data Centre Eggs ≈ JARs.egg zip archive Drop into sys.path pkg_resources API What about  Dependencies?  Scripts?  C Extensions? EGG-INFO package_1 package_2 code data _____________.egg

British Atmospheric Data Centre Eggs ≈ CPAN $ easy_install MyPackage>= find MyPackage v download MyPackage v download MyPackage dependencies build all install all... $ python >>> import MyPackage >>>

British Atmospheric Data Centre Python Cheeseshop British Atmospheric Data Centre

easy_install examples $ easy_install MyPackage $ easy_install MyPackage==3.2 $ easy_install -f MyPackage $ easy_install --upgrade MyPackage $ easy_install –d temp_eggs MyPackage-prealpha $ easy_install $SROOT/MyPackage/trunk#egg=MyPackage-dev $ easy_install -m MyPackage==3.2a1

British Atmospheric Data Centre Installing setuptools recipe 1: globally Ensure distutils is available Ensure you can write python installation $ wget $ python ez_setup.py

British Atmospheric Data Centre Installing setuptools recipe 2: using PYTHONPATH Ensure distutils is available In ~/.profile export PYTHONPATH=$HOME/lib/python export PATH=$HOME/bin:$PATH In ~/.pydistutilsrc [install] install_lib = ~/lib/python install_scripts = ~/bin $ wget $ python ez_setup.py

British Atmospheric Data Centre Key Advantages Self contained Try new software easily  install / test / throw away Track development  select version  download from SVN Supports plug-in architectures

British Atmospheric Data Centre Part II Developers

British Atmospheric Data Centre Egg components > pkg_resources dependencies resources metadata namespaces > setuptools build eggs > distutils > easy_install find download install upgrade > setuptools > ez_setup.py

British Atmospheric Data Centre Distutils $ python setup.py install $ python setup.py install –prefix=$HOME $ python setup.py build $ sudo python setup.py install $ python setup.py build –build-opt install

British Atmospheric Data Centre Distutils Problems  Insufficient documentation  Easier to shortcut than use properly Consequences  Inconsistent setup.py interface  Monolithic frameworks

British Atmospheric Data Centre CDAT

British Atmospheric Data Centre cdat-lite / vcs-lite $ URL= $ easy_install –f $URL vcs-lite

British Atmospheric Data Centre Matplotlib

British Atmospheric Data Centre Matplotlib easy_install numpy easy_install matplotlib easy_install basemap easy_install –f $SVN basemap 

British Atmospheric Data Centre M2Crypto $ easy_install M2Crypto No way to use the –openssl option! setup.py's key mistakes  Parses the –openssl argument outside distutils.  Sets include_dirs even if using default location.

British Atmospheric Data Centre It will probably work if you Stick to the distutils framework!  including argument parsing install locations use __file__ or __name__ to find resources

British Atmospheric Data Centre Setuptools: preliminaries import ez_setup ez_setup.use_setuptools() from setuptools import setup, find_packages setup( name = 'MyPackage', version = '1.0',... packages = find_packages('lib', exclude=["test_*"]), package_dir = {'' : 'lib'},... )

British Atmospheric Data Centre Setuptools: dependencies setup(... install_requires = [ 'MyDep1>=0.3', 'MyDep2==1.2rc2', 'MyDep3==dev' ], dependency_links = [ ' ' ],... )

British Atmospheric Data Centre Setuptools: resources setup(... zip_safe = False, include_package_data = True, exclude_package_data = {'': ['README.txt']},... )

British Atmospheric Data Centre Setuptools: scripts setup(... entry_points = { 'console_scripts': [ 'foo = my_package.some_module:main_func', 'bar = other_module:some_func', ], 'gui_scripts': [ 'baz = my_package_gui.start_func', ]},... )

British Atmospheric Data Centre Setuptools: and more namespace packages entry points extras SVN hooks unit tests

British Atmospheric Data Centre pkg_resources API #udunits_name=sys.prefix+'/lib/python'+version+'/site- packages/unidata/udunits.dat' import pkg_resources udunits_name = pkg_resources.resource_filename('unidata', 'udunits.dat') import pkg_resources pkg_resources.require('ZSI==2.0rc1')

British Atmospheric Data Centre Using Eggs in NDG Stripping out CDAT components  cdat-lite, vcs-lite Splitting the ndg package into multiple eggs  namespace packages Managing multiple versions  ZSI Shared code in ndg_util.egg Component bundles

British Atmospheric Data Centre Part III: Demo A WSGI compliant OpeNDAP / WMS server in exactly 0 lines of python

British Atmospheric Data Centre #!/bin/sh # Log commands to stdout set -x # Setup staging area rm -fr eggs myserver demo.log mkdir eggs export PYTHONPATH=$PWD/eggs export PATH=$PYTHONPATH:$PATH DAPSVN= # Bootstrap setuptools python ez_setup.py -d eggs >>demo.log 2>&1 # Install pydap prerequisites easy_install -d eggs -f \ numpy Imaging >>demo.log 2>&1 # Install pydap and plugins easy_install -d eggs \ dap[server] dap.plugins.netcdf dap.responses.wms \ $DAPSVN/responses/html >>demo.log 2>&1 # Create the server project paster create -t dap_server myserver project=My-Server >>demo.log 2>&1 # Move data to server directory cp *.nc myserver/data >>demo.log 2>&1 # Start the server paster serve myserver/server.ini Demo script British Atmospheric Data Centre

[server:main] use = egg:Paste#http # Change to to make public host = port = 8080 [filter-app:main] use = egg:Paste#httpexceptions next = cascade [composit:cascade] use = egg:Paste#cascade app1 = static app2 = pydap catch = 404 [app:static] use = egg:Paste#static document_root = %(here)s/data [app:pydap] use = egg:dap name = My-Server root = %(here)s/data verbose = 0 template = %(here)s/template Demo paste.deploy configuration British Atmospheric Data Centre

The End

British Atmospheric Data Centre References   