🌐 Mailman 3 November 2007 - Pythonmac-SIG - python.org

sys.path on Leopard
by Brian Granger Nov. 3, 2007

Nov. 3, 2007

Hi, I have been playing around with python on Leopard today. Overall, I am very pleased, but I just ran into a problem that will affect a large number of users. In Leopard, Apple includes a number of python packages in: ls /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python CoreGraphics easy_install.pyc setuptools-0.6c7-py2.5.egg-info OpenSSL fetchmailconf.py site.py PyObjC fetchmailconf.pyc site.pyc PyObjC.pth fetchmailconf.pyo site.pyo PyRSS2Gen-1.0.0-py2.5.egg-info libsvn svn PyRSS2Gen.py macholib twisted PyRSS2Gen.pyc macholib-1.2.1.dev-py2.5.egg-info wx Twisted-2.4.0-py2.5.egg-info modulegraph wx-2.8-mac-unicode Twisted_Words-0.4.0-py2.5.egg-info modulegraph-0.7.2.dev-py2.5.egg-info wxPython Twisted_Xish-0.4.0-py2.5.egg-info numpy wxPython_common-2.8.4.0-py2.5.egg-info altgraph numpy-1.0.1-py2.5.egg-info wxaddons altgraph-0.6.8.dev-py2.5.egg-info pkg_resources.py wxaddons-2.8.4.0-py2.5.egg-info bdist_mpkg pkg_resources.pyc wxversion.py bdist_mpkg-0.4.3.dev-py2.5.egg-info py2app wxversion.pyc bonjour py2app-0.4.1.dev-py2.5.egg-info xattr bonjour_py-0.2-py2.5.egg-info pyOpenSSL-0.6-py2.5.egg-info xattr-0.5-py2.5.egg-info dateutil python_dateutil-1.2-py2.5.egg-info zope easy_install.py setuptools zope.interface-3.3.0-py2.5.egg-info At first I was very excited to see that Apple is including things like setuptools, numpy, twisted...... But then I saw that some of the versions are quite old. Naturally, I grabbed the latest numpy version from svn and did a python setup.py install. Everything built just fine, but when I did an import numpy, I got Apples version: >>> import numpy >>> print numpy.__file__ /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python/numpy/__init__.pyc After looking at sys.path, it became clear what the problem is: Running python setup.py install on Leopard causes packages to be installed in the usual: /Library/Python/2.5/site-packages But, Apple put this directory _after_ /System/Library/Frameworks/Python.framework/Versions/2.5/Extras/lib/python in sys.path. This, even if a user installs a newer version of one of these packages, the builtin python will always use Apple's older version. Obviously, I can set PYTHONPATH to get /Library/Python/2.5/site-packages earlier in the sys.path list, but this is going to bite many people. Is this a bug that I should report to Apple, or is there a better work around? Also, this information needs to be posted somewhere so we don't see this question a billion times on various lists over the lifetime of Leopard. Cheers, Brian

4 5

Round 2 with Leopard+Python
by Brian Granger Nov. 3, 2007

Nov. 3, 2007

Hi, In the process of working through the issues with sys.path on Leopard, I have found another potential Leopard bug that is particularly nasty. In Tiger, sudo preserves environment variables: $ export FOO=/tmp $ python -c "import os; print os.environ['FOO']" /tmp $ sudo python -c "import os; print os.environ['FOO']" /tmp But, in Leopard, sudo does not perserve environment variables: $ export FOO=/tmp $ python -c "import os; print os.environ['FOO']" /tmp $ sudo python -c "import os; print os.environ['FOO']" Password: Traceback (most recent call last): File "<string>", line 1, in <module> File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/UserDict.py", line 22, in __getitem__ raise KeyError(key) KeyError: 'FOO' This is a big problem. First, if you have set PYTHONPATH to point sys.path at the site-packages in /Library, this setting will be lost when you do: sudo python setup.py install On another package. I encountered this in building pytables, which requires numpy >= 1.0.3. I had installed numpy 1.0.4, and set my PYTHONPATH to point to it. But, the pytables setup.py script failts because PYTHONPATH is lost and it only sees the older (1.0.1) builtin numpy. Second, some setup.py scripts use environment variables to determine how things are built, find other dependencies, etc. Currently, this will fail on Leopard if such packages are installed into locations that require sudo. I haven't tried it yet, but I expect that this will also hold true for other python installations. The behavior also shows up with ruby on Leopard. The solution currently is to install all packages to locations that don't require sudo to write to. I will file a bug report, but until the bug is fixed, we should explore putting a note on the numpy/scipy site - and even possibly on the python.org site to describe the problem and its workaround. Brian

4 5

Re: [Pythonmac-SIG] [Numpy-discussion] Problem with numpy on Leopard
by Brian Granger Nov. 3, 2007

Nov. 3, 2007

On 11/1/07, Bill Janssen <janssen(a)parc.com> wrote: > > > It's not entirely silly. This has been the advice given to app > > > developers on this list and the PyObjC list for years now. It's nice > > > to have a better system Python for quick scripts, but it's still the > > > System Python. It's Apples, for their stuff that uses Python. And it > > > is specific to OS 10.5. If you use it to build your app, your app is > > > pretty much guaranteed not to work when someone tries to run it on > > > 10.6, etc. > > > > At some level I agree - I haven't used the system python on Tiger for > > a long time now. But, that is not how Apple "sells" the python on > > their system. > > This is really interesting. For my apps, I use the system Python on > Tiger, and expect to do it again with Leopard. If I have to have a > specific version of an extension that's needed, I make sure the > directory where I keep it is on the sys.path of my application before > the standard directories. That seems to be about all that's required. > I'm not sure what all the FUD about the system Python is... There is little fear, uncertainty or doubt about this. The problem is that this represents a huge change of behavior for users making the transition to Leopard. On Tiger, a user could download any basically any python pacakge and do "python setup.py install" and things would work. Now under Leopard, this becomes "python setup.py install" + muck with PYTHONPATH or .pth files. I personally have no problem doing this, but many of the casual python users with whom I work will be tripped up by this....and they will come and ask me what to do....over and over again.... Brian

2 1

System python woes
by Dav Clark Nov. 2, 2007

Nov. 2, 2007

I've seen a couple of posts now about difficulty using the system python in Leopard. I wonder if anyone else has used the solution offered by Phillip Eby here: http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-pyth… Basically, this wraps your system (or any) python and allows you to install libraries into that wrapped python - including ones that would override the system defaults. I suppose this strategy could even be "wrapped up" in a distributed application bundle of some sort. I've used this solution for my own development purposes, and it's worked fine. I'd be curious to hear other's experiences or ideas. Dav

2 1

Re: [Pythonmac-SIG] Pythonmac-SIG Digest, Vol 55, Issue 2
by Noah Gift Nov. 2, 2007

Nov. 2, 2007

> > > > ---------- Forwarded message ---------- > From: Christopher Barker <Chris.Barker(a)noaa.gov> > To: Discussion of Numerical Python <numpy-discussion(a)scipy.org>, > Pythonmac-Sig <pythonmac-sig(a)python.org> > Date: Thu, 01 Nov 2007 16:12:02 -0700 > Subject: Re: [Pythonmac-SIG] [Numpy-discussion] Problem with numpy on > Leopard > Robert Kern wrote: > > The way I see it, python is a very good, robust, general purpose tool. > It's a great thing for OS suppliers to provide python for system and > users use -- I'd love to see it treated as other system libraries and > utilities are, but to do that, there must be a versioning system for > packages -- so Apple can use the version they installed, and a user can > install and upgrade along side it, and not break anything. I was just thinking the same thing. It is awesome that Apple is getting really big into Python, but it presents a small worry about how the system version of Python and the fancy bridge, and the inevitable rapid development of these third party modules, and Python itself. For example, try upgrading upgrading, say, mod_python on Cent OS 4.4. Good luck with that...

1 0

pyobjc 2.0 has landed in the repository
by Ronald Oussoren Nov. 2, 2007

Nov. 2, 2007

Folks, I have commited the version of pyobjc that's included in Leopard into the public repository. This version is mostly useful on Leopard right now, as it doesn't include complete metadata files but relies on the metadata files on the system. Those are present in Leopard, but not on Tiger (because we made those up during the development of Leopard). There is also a copy of PyOpenGL there that builds cleanly on OSX (both Leopard and Tiger). There are currently three URLs for this version: * http://svn.red-bean.com/pyobjc/branches/pyobjc2 This is the trunk for PyObjC 2.x development * http://svn.red-bean.com/pyobjc/branches/pyobjc-20x-branch This is a branch for pyobjc 2.0.x, pure bugfix releases * http://svn.red-bean.com/pyobjc/tags/r200-leopard The tag for pyobjc 2.0 I have some plans for future development, but will probably focus on development with rather than on PyObjC for a while. Ronald

3 2