🌐 Mailman 3 December 2004 - Python-list - python.org

Re: Keyword arguments - strange behaviour?
by Steve Holden Jan. 5, 2005

Jan. 5, 2005

harold fellermann wrote: > Hi, > > I cannot see any strange behavior. this code works exacly as you and I > suspect: > > >>> def otherfunction(x) : > .... return x > .... > >>> def function(arg=otherfunction(5)) : > .... return arg > .... > >>> function(3) > 3 > >>> function() > 5 > > Or is this not what you excepted? > Channelling the effbot, I think he was asking what namespace context you expected the expression "arg=otherfunction(x)" to be evaluated in when it's used at the time of a function call to dynamically create a new default value for arg. regards Steve -- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/ Holden Web LLC +1 703 861 4237 +1 800 494 3119

4 5

Re: what is lambda used for in real code?
by Steven Bethard Jan. 5, 2005

Jan. 5, 2005

Adam DePrince wrote: > Lets not forget the "real reason" for lambda ... the elegance of > orthogonality. Why treat functions differently than any other object? > > We can operate on every other class without having to involve the > namespace, why should functions be any different? Yup. I think in most of the examples that I didn't know how to rewrite, this was basically the issue. On the other hand, I do think that lambdas get overused, as indicated by the number of examples I *was* able to rewrite.[1] Still, I have to admit that in some cases (especially those involving reduce), I wish the coder had named the function -- it would have given me a little bit more documentation as to what the code was trying to do. On the other hand, in other cases, like when a function is a keyword argument to another function (e.g. inspect.py's "def formatargspec..." example) using a def statement and naming the function would be redundant. Steve [1] Note that this isn't entirely fair to the examples, some of which were written before list comprehensions, generator expressions and itemgetter/attrgetter.

2 1

Speed ain't bad
by Bulba! Jan. 4, 2005

Jan. 4, 2005

One of the posters inspired me to do profiling on my newbie script (pasted below). After measurements I have found that the speed of Python, at least in the area where my script works, is surprisingly high. This is the experiment: a script recreates the folder hierarchy somewhere else and stores there the compressed versions of files from source hierarchy (the script is doing additional backups of the disk of file server at the company where I work onto other disks, with compression for sake of saving space). The data was: 468 MB, 15057 files, 1568 folders (machine: win2k, python v2.3.3) The time that WinRAR v3.20 (with ZIP format and normal compression set) needed to compress all that was 119 seconds. The Python script time (running under profiler) was, drumroll... 198 seconds. Note that the Python script had to laboriously recreate the tree of 1568 folders and create over 15 thousand compressed files, so it had more work to do actually than WinRAR did. The size of compressed data was basically the same, about 207 MB. I find it very encouraging that in the real world area of application a newbie script written in the very high-level language can have the performance that is not that far from the performance of "shrinkwrap" pro archiver (WinRAR is excellent archiver, both when it comes to compression as well as speed). I do realize that this is mainly the result of all the "underlying infrastructure" of Python. Great work, guys. Congrats. The only thing I'm missing in this picture is knowledge if my script could be further optimised (not that I actually need better performance, I'm just curious what possible solutions could be). Any takers among the experienced guys? Profiling results: >>> p3.sort_stats('cumulative').print_stats(40) Fri Dec 31 01:04:14 2004 p3.tmp 580543 function calls (568607 primitive calls) in 198.124 CPU seconds Ordered by: cumulative time List reduced from 69 to 40 due to restriction <40> ncalls tottime percall cumtime percall filename:lineno(function) 1 0.013 0.013 198.124 198.124 profile:0(z3()) 1 0.000 0.000 198.110 198.110 <string>:1(?) 1 0.000 0.000 198.110 198.110 <interactive input>:1(z3) 1 1.513 1.513 198.110 198.110 zmtree3.py:26(zmtree) 15057 14.504 0.001 186.961 0.012 zmtree3.py:7(zf) 15057 147.582 0.010 148.778 0.010 C:\Python23\lib\zipfile.py:388(write) 15057 12.156 0.001 12.156 0.001 C:\Python23\lib\zipfile.py:182(__init__) 32002 7.957 0.000 8.542 0.000 C:\PYTHON23\Lib\ntpath.py:266(isdir) 13826/1890 2.550 0.000 8.143 0.004 C:\Python23\lib\os.py:206(walk) 30114 3.164 0.000 3.164 0.000 C:\Python23\lib\zipfile.py:483(close) 60228 1.753 0.000 2.149 0.000 C:\PYTHON23\Lib\ntpath.py:157(split) 45171 0.538 0.000 2.116 0.000 C:\PYTHON23\Lib\ntpath.py:197(basename) 15057 1.285 0.000 1.917 0.000 C:\PYTHON23\Lib\ntpath.py:467(abspath) 33890 0.688 0.000 1.419 0.000 C:\PYTHON23\Lib\ntpath.py:58(join) 109175 0.783 0.000 0.783 0.000 C:\PYTHON23\Lib\ntpath.py:115(splitdrive) 15057 0.196 0.000 0.768 0.000 C:\PYTHON23\Lib\ntpath.py:204(dirname) 33890 0.433 0.000 0.731 0.000 C:\PYTHON23\Lib\ntpath.py:50(isabs) 15057 0.544 0.000 0.632 0.000 C:\PYTHON23\Lib\ntpath.py:438(normpath) 32002 0.431 0.000 0.585 0.000 C:\PYTHON23\Lib\stat.py:45(S_ISDIR) 15057 0.555 0.000 0.555 0.000 C:\Python23\lib\zipfile.py:149(FileHeader) 15057 0.483 0.000 0.483 0.000 C:\Python23\lib\zipfile.py:116(__init__) 151 0.002 0.000 0.435 0.003 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\framework\winout.py:171(write) 151 0.002 0.000 0.432 0.003 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\framework\winout.py:489(write) 151 0.013 0.000 0.430 0.003 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\framework\winout.py:461(HandleOutput) 76 0.087 0.001 0.405 0.005 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\framework\winout.py:430(QueueFlush) 15057 0.239 0.000 0.340 0.000 C:\Python23\lib\zipfile.py:479(__del__) 15057 0.157 0.000 0.157 0.000 C:\Python23\lib\zipfile.py:371(_writecheck) 32002 0.154 0.000 0.154 0.000 C:\PYTHON23\Lib\stat.py:29(S_IFMT) 76 0.007 0.000 0.146 0.002 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\framework\winout.py:262(dowrite) 76 0.007 0.000 0.137 0.002 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\scintilla\formatter.py:221(OnStyleNeeded) 76 0.011 0.000 0.118 0.002 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\framework\interact.py:197(Colorize) 76 0.110 0.001 0.112 0.001 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\scintilla\control.py:69(SCIInsertText) 76 0.079 0.001 0.081 0.001 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\scintilla\control.py:333(GetTextRange) 76 0.018 0.000 0.020 0.000 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\scintilla\control.py:296(SetSel) 76 0.006 0.000 0.018 0.000 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\scintilla\document.py:149(__call__) 227 0.003 0.000 0.012 0.000 C:\Python23\lib\Queue.py:172(get_nowait) 76 0.007 0.000 0.011 0.000 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\framework\interact.py:114(ColorizeInteractiveCode) 532 0.011 0.000 0.011 0.000 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\scintilla\control.py:330(GetTextLength) 76 0.001 0.000 0.010 0.000 C:\PYTHON23\lib\site-packages\Pythonwin\pywin\scintilla\view.py:256(OnBraceMatch) 1888 0.009 0.000 0.009 0.000 C:\PYTHON23\Lib\ntpath.py:245(islink) --- Script: #!/usr/bin/python import os import sys from zipfile import ZipFile, ZIP_DEFLATED def zf(sfpath, targetdir): if (sys.platform[:3] == 'win'): tgfpath=sfpath[2:] else: tgfpath=sfpath zfdir=os.path.dirname(os.path.abspath(targetdir) + tgfpath) zfpath=zfdir + os.path.sep + os.path.basename(tgfpath) + '.zip' if(not os.path.isdir(zfdir)): os.makedirs(zfdir) archive=ZipFile(zfpath, 'w', ZIP_DEFLATED) sfile=open(sfpath,'rb') zfname=os.path.basename(tgfpath) archive.write(sfpath, os.path.basename(zfpath), ZIP_DEFLATED) archive.close() ssize=os.stat(sfpath).st_size zsize=os.stat(zfpath).st_size return (ssize,zsize) def zmtree(sdir,tdir): n=0 ssize=0 zsize=0 sys.stdout.write('\n ') for root, dirs, files in os.walk(sdir): for file in files: res=zf(os.path.join(root,file),tdir) ssize+=res[0] zsize+=res[1] n=n+1 #sys.stdout.write('.') if (n % 200 == 0): print " %.2fM (%.2fM)" % (ssize/1048576.0, zsize/1048576.0) #sys.stdout.write(' ') return (n, ssize, zsize) if __name__=="__main__": if len(sys.argv) == 3: if(os.path.isdir(sys.argv[1]) and os.path.isdir(sys.argv[2])): (n,ssize,zsize)=zmtree(os.path.abspath(sys.argv[1]),os.path.abspath(sys.argv[2])) print "\n\n Summary:\n Number of files compressed: %d\n Total size of original files: %.2fM\n \ Total size of compressed files: %.2fM" % (n, ssize/1048576.0, zsize/1048576.0) sys.exit(0) else: print "Incorrect arguments." if (not os.path.isdir(sys.argv[1])): print sys.argv[1] + " is not directory." if (not os.path.isdir(sys.argv[2])): print sys.argv[2] + " is not directory." print "\n Usage:\n " + sys.argv[0] + " source-directory target-directory" -- It's a man's life in a Python Programming Association.

7 9

Parallelization with Python: which, where, how?
by Mathias Jan. 4, 2005

Jan. 4, 2005

Dear NG, I have a (pretty much) "emberassingly parallel" problem and look for the right toolbox to parallelize it over a cluster of homogenous linux workstations. I don't need automatic loop-parallelization or the like since I prefer to prepare the work packets "by hand". I simply need - to specify a list of clients - a means of sending a work packet to a free client and receiving the result (hopefully automatically without need to login to each one) - optionally a timeout mechanism if a client doesn't respond - optionally help for debugging of remote clients So far I've seen scipy's COW (cluster of workstation) package, but couldn't find documentation or even examples for it (and the small example in the code crashes...). I've noticed PYRO as well, but didn't look too far yet. Can someone recommend a parallelization approach? Are there examples or documentation? Has someone got experience with stability and efficiency? Thanks a lot, Mathias

6 5

Optional Static Typing
by bearophileHUGS＠lycos.com Jan. 4, 2005

Jan. 4, 2005

Adding Optional Static Typing to Python looks like a quite complex thing, but useful too: http://www.artima.com/weblogs/viewpost.jsp?thread=85551 I have just a couple of notes: Boo (http://boo.codehaus.org/) is a different language, but I like its "as" instead of ":" and "->", to have: def min(a as iterable(T)) as T: Instead of: def min(a: iterable(T)) -> T: Sometimes it can be useful to mix parts with static typing and parts without it in the same module (because dynamic typing is very useful sometimes), but some other times you want to be sure to have a full typed and checked module. So maybe a kind of compiler directive (like the # used to define encoding) can be used to define a module fully typed and fully checked by a pychecker-like. (A module with static typing can be very useful for a "true" python compiler too.) Bear hugs, bearophile

24 35

hidding the "console" window + system global shortcut keys
by Gabriel Cosentino de Barros Jan. 4, 2005

Jan. 4, 2005

Hi, it's a broad topic because they all poped in my head at the same time while writting the alpha version of a desktop-helper app. The app must be the less obstrusive possible. some history: It's mainly for win32. I started writting this to compensate the lack of posix tools and pipes on win32. in *NIX i could use wc, grep, sort, et al to deal with files(?). In windows, i mainly depend on my text editor. I strugle with no 'grep' or no 'cut' or no easy way to pass data between programs other then copy paste or saving the file. The actual method of saving, opening a w32 port of bash, and then doing the usual *NIX way is too intrusive in windows because you have to leave you "ambient", do things, go back to your previous "ambient". So, the program was to sit in the taskbar or in the systray. and by some mouse clicks or global key combinations transform the selected text or the clipboard content with a easy-to-input posix like syntax mimic by python functions. I haven't even scrateched the surface for all that. but for that pre-alpha i *really* want to mainly do those things: 1. Give a painfull death to the "console window" i will need console output only in debuging 2. Set global key biddings. #1 is really superfulous, i know, but #2 is really driving me nuts. the key bidings would be idealy in a multiplataform way, since i use* W32, Irix, BeOS and OSX. in that order :) Thanks for any hints Gabriel * I also use Plan9 but it has then times more 'pipes' suport in the GUI then any *NIX has in CLI hehe and yep, no linux/bsdish desktop here. the CLI enviroment fulfill perfectly my needs. X only gets in the way

3 2

Dr. Dobb's Python-URL! - weekly Python news and links (Dec 30)
by Cameron Laird Jan. 4, 2005

Jan. 4, 2005

QOTW: "I found the discussion of unicode, in any python book I have, insufficient." -- Thomas Heller "If you develop on a Mac, ... Objective-C could come in handy. . . . PyObjC makes mixing the two languages dead easy and more convenient than indoor plumbing." -- Robert Kern Among other activities, the PSF aggregates donors with dollars destined to do good Python works, and developers expert in obscure corners of Pythonia. http://groups-beta.google.com/group/comp.lang.python.announce/browse_thread… http://groups-beta.google.com/group/comp.lang.python.announce/browse_thread… Yippee! The martellibot promises to explain Unicode for Pythoneers. http://groups-beta.google.com/group/comp.lang.python/msg/6015a5a05c206712 The glorious SciPy project supports *multiple* worthwhile Wikis. http://www.scipy.org/wikis Good style in Python does not generally include "in-place" operations on lists. Several cleaner idioms are possible. http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/c… Assume you're comfortable with tuples' semantics, immutability, and so on. Do you correctly understand the basics of their syntax, though? This is another opportunity to think about Unicode, by the way. http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/9… Robert Kern, Paul Rubin, Mike Meyer, Alex Martelli, and others provide disproportionately high-quality advice (and tangents!) on the subject of languages which complement Python. http://groups-beta.google.com/group/comp.lang.python/browse_thread/thread/b… ======================================================================== Everything Python-related you want is probably one or two clicks away in these pages: Python.org's Python Language Website is the traditional center of Pythonia http://www.python.org Notice especially the master FAQ http://www.python.org/doc/FAQ.html PythonWare complements the digest you're reading with the marvelous daily python url http://www.pythonware.com/daily Mygale is a news-gathering webcrawler that specializes in (new) World-Wide Web articles related to Python. http://www.awaretek.com/nowak/mygale.html While cosmetically similar, Mygale and the Daily Python-URL are utterly different in their technologies and generally in their results. comp.lang.python.announce announces new Python software. Be sure to scan this newsgroup weekly. http://groups.google.com/groups?oi=djq&as_ugroup=comp.lang.python.announce Brett Cannon continues the marvelous tradition established by Andrew Kuchling and Michael Hudson of intelligently summarizing action on the python-dev mailing list once every other week. http://www.python.org/dev/summary/ The Python Package Index catalogues packages. http://www.python.org/pypi/ The somewhat older Vaults of Parnassus ambitiously collects references to all sorts of Python resources. http://www.vex.net/~x/parnassus/ Much of Python's real work takes place on Special-Interest Group mailing lists http://www.python.org/sigs/ The Python Business Forum "further[s] the interests of companies that base their business on ... Python." http://www.python-in-business.org Python Success Stories--from air-traffic control to on-line match-making--can inspire you or decision-makers to whom you're subject with a vision of what the language makes practical. http://www.pythonology.com/success The Python Software Foundation (PSF) has replaced the Python Consortium as an independent nexus of activity. It has official responsibility for Python's development and maintenance. http://www.python.org/psf/ Among the ways you can support PSF is with a donation. http://www.python.org/psf/donate.html Kurt B. Kaiser publishes a weekly report on faults and patches. http://www.google.com/groups?as_usubject=weekly%20python%20patch Cetus collects Python hyperlinks. http://www.cetus-links.org/oo_python.html Python FAQTS http://python.faqts.com/ The Cookbook is a collaborative effort to capture useful and interesting recipes. http://aspn.activestate.com/ASPN/Cookbook/Python Among several Python-oriented RSS/RDF feeds available are http://www.python.org/channews.rdf http://bootleg-rss.g-blog.net/pythonware_com_daily.pcgi http://python.de/backend.php For more, see http://www.syndic8.com/feedlist.php?ShowMatch=python&ShowStatus=all The old Python "To-Do List" now lives principally in a SourceForge reincarnation. http://sourceforge.net/tracker/?atid=355470&group_id=5470&func=browse http://python.sourceforge.net/peps/pep-0042.html The online Python Journal is posted at pythonjournal.cognizor.com. editor(a)pythonjournal.com and editor(a)pythonjournal.cognizor.com welcome submission of material that helps people's understanding of Python use, and offer Web presentation of your work. deli.cio.us presents an intriguing approach to reference commentary. It already aggregates quite a bit of Python intelligence. http://del.icio.us/tag/python *Py: the Journal of the Python Language* http://www.pyzine.com Archive probing tricks of the trade: http://groups.google.com/groups?oi=djq&as_ugroup=comp.lang.python&num=100 http://groups.google.com/groups?meta=site%3Dgroups%26group%3Dcomp.lang.pyth… Previous - (U)se the (R)esource, (L)uke! - messages are listed here: http://www.ddj.com/topics/pythonurl/ http://purl.org/thecliff/python/url.html (dormant) or http://groups.google.com/groups?oi=djq&as_q=+Python-URL!&as_ugroup=comp.lan… Suggestions/corrections for next week's posting are always welcome. E-mail to <Python-URL(a)phaseit.net> should get through. To receive a new issue of this posting in e-mail each Monday morning (approximately), ask <claird(a)phaseit.net> to subscribe. Mention "Python-URL!". -- The Python-URL! Team-- Dr. Dobb's Journal (http://www.ddj.com) is pleased to participate in and sponsor the "Python-URL!" project.

3 2

Re: input record sepArator (equivalent of "$|" of perl)
by John Machin Jan. 4, 2005

Jan. 4, 2005

Nick Coghlan wrote: [snip] > delimeter. Hey, Terry, another varmint over here!

11 12

Zope newsgroups? Need help with POSKeyError
by Bob Horvath Jan. 4, 2005

Jan. 4, 2005

I have a Zope/Plone combination that I have been having POSKeyErrors with for a while now. Are there any newsgroups that deal with Zope?

3 3

Re: Howto Extract PNG from binary file @ 0x80?
by Aaron Jan. 4, 2005

Jan. 4, 2005

On Sat, 11 Dec 2004 10:53:19 +0100, Fredrik Lundh wrote: > "flamesrock" <mishpaha(a)telus.net> wrote: > >> As a newbie to the language, I have no idea where to start..please bare >> with me.. >> >> The simcity 4 savegame file has a png image stored at the hex location >> 0x80. What I want to extract it and create a file with the .png >> extension in that directory. >> >> Can somebody explain with a snippet of code how I would accomplish >> this? I've looked around but the information is too vague and I don't >> know how to start. > > there are *many* ways to ignore the first 128 bytes when you read a file > (you can seek to the right location, you can read 128 bytes and throw them > a way, you can read one byte 128 times and throw each one of them away, > you can read all data and remove the first 128 bytes, etc). > > here's a snippet that understands the structure of the PNG, and stops copying > when it reaches the end of the PNG: > > import struct > > def pngcopy(infile, outfile): > > # copy header > header = infile.read(8) > if header != "\211PNG\r\n\032\n": > raise IOError("not a valid PNG file") > outfile.write(header) > > # copy chunks, until IEND > while 1: > chunk = infile.read(8) > size, cid = struct.unpack("!l4s", chunk) > outfile.write(chunk) > outfile.write(infile.read(size)) > outfile.write(infile.read(4)) # checksum > if cid == "IEND": > break > > to use this, open the input file (the simcity file) and the output file > (the png file you want to create) in binary mode, use the "seek" > method to move to the right place in the simcity file, and call "pngcopy" > with the two file objects. > > infile = open("mysimcityfile", "rb") > infile.seek(0x80) > outfile = open("myimage.png", "wb") > pngcopy(infile, outfile) > outfile.close() > infile.close() > > hope this helps! > > </F> Thanks! And what a pleasant surprise! I just noticed you authored the book I've been studying - Python Standard Library. Its awesome ;) Oh- and Sorry for the late response. Google groups wouldn't allow me to followup. I'm using pan now.

3 5