🌐 Mailman 3 January 2000 - Python-list - python.org

performance question when using embedding/extending
by Eibl Christian Object Vision Jan. 31, 2000

Jan. 31, 2000

hi, i need to call a python function from my c++ app and this python function in turn accesses an instanciated c++ object (i pass the pointer to the object as a parameter). in the python function i have approx. 20-30 calls to the object (simple member accessing). the goal: provide a scripting interface for computation of certain parameters depending on the state of a changing object... when i'm doing a simple performance test (just calling the function 100.000 times in c++) it turns out that on my 300MHZ PII the function is called 32.000 times per second if i don't access any attributes via python and it goes down to 900 times per second if i place 30 access calls in the python function. i "quantified" the application an see that almost all the cpu-time is used in "PyEval_CallObject". the application i'm building needs at least 10-times the performance it has now... my question: - is this just the (slow) way it is ? - is there a faster way to use embedding/extending (details see below) ? - is python not suitable for this task (the old system was also an interpreter, but specifically written for this task) ? the details:: i'm using the following code in my app to call a python function in a module: <snip> _pythonCaller_p->runWithObject( "i", &finalResult, somePtr_p, "_SomeClass_p" ) </snip> the "runWithObject" method is defined as follows: NOTE: i'm using SWIG with shadow classes to generate python interfaces for my C++ classes! <snip> bool PythonCaller::runWithObject( char* resultFormat, void* result_p, void* pyObj_p, char* pyType) { PyObject* func_p = 0; PyObject* args_p = 0; PyObject* resu_p = 0; if( _doReload ) { func_p = loadAttribute( _moduleName, _functionName ); } else { func_p = _pyFunction; } if( func_p==0 ) { return false; } char aTmp[256]; SWIG_MakePtr( aTmp, pyObj_p, pyType ); args_p = Py_BuildValue( "(s)", aTmp ); if( args_p==0 ) { return false; } resu_p = PyEval_CallObject( func_p, args_p ); if( resu_p==0 ) PyErr_Print(); if( _doReload ) { Py_DECREF(func_p); } Py_DECREF(args_p); </snip> the python function called with this code is defined as follows: import Mc import Mcc def calcAttrFinalCoding(mp) : mpi = Mc.MailpiecePtr( mp ) mpc = mpi.fetch() if mpc.getPbrInfoStatus() == Mcc.barcodeOk : return Mcc.attrCodBc if mpi.getBvInfoStatus() == Mcc.barcodeOk : return Mcc.attrCodBc if mpi.getPbrInfoStatus() == Mcc.barcodeOk : return Mcc.attrCodBc if mpi.getBvInfoStatus() == Mcc.barcodeOk : return Mcc.attrCodBc if mpi.getPbrInfoStatus() == Mcc.barcodeOk : return Mcc.attrCodBc return 1 thanx for any help! Chris -- Christian Eibl Object Vision Software AG, München Christian.Eibl(a)ovsag.de | www.ovsag.de mobile: +49.171.7623844 or +43.676.3060031 pgp-fingerprint: D681 0321 30BE FA99 AD03 FBE7 38C5 E788 2846 25DB

2 1

C++-Extension
by Tilo Schwarz Jan. 31, 2000

Jan. 31, 2000

Hi all, the Extending and Embedding the Python Interpreter reference says, that (1.11) "If the main program (the Python interpreter) is compiled and linked by the C compiler, global or static objects with constructors cannot be used. This is not a problem if the main program is linked by the C++ compiler" Does that mean, C++-Extension are not "portable" - means, that I need a C++-linked python interpreter to load my C++-Extension? Not portable, because on a different platform I don't know if there is a C++-linked python interpreter and some code might use global or static objects with constructors. Am I right? (Hopefully not) I'm asking, because I get the following error message: > python Python 1.5.2 (#1, Jul 23 1999, 06:38:16) [GCC egcs-2.91.66 19990314/Linux (egcs- on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> import _CircleDetector Traceback (innermost last): File "<stdin>", line 1, in ? ImportError: ./_CircleDetectormodule.so: undefined symbol: cerr >>> I'm not sure about cerr, I don't use it, but some STL-code... Thanks for any help! Bye Tilo -------------------- Tilo Schwarz, DaimlerChrysler, Research Center Ulm, FT3/AB Address: Wilhelm-Runge-Str.11, P.O. Box 23 60, 89013 Ulm, Germany Phone: +49 731 505 2376 Email: tilo.schwarz(a)daimlerchrysler.com

2 1

Python1.5.2/Zope 2.1.3 on MacOS X Server notes
by Jeffrey P Shell Jan. 31, 2000

Jan. 31, 2000

These are some quick notes from building Python 1.5.2 with threads and shared libraries on MacOS X Server 1.02. As a note, The MacNN Xclave (xclave.macnn.com) reports on configuring Python with --with-host=powerpc-apple-netbsd . I didn't try this option with what I did below, but it couldn't hurt to try on older versions of Rhapsody and newer developr releases of MacOS X who aren't picked up by the current configure scripts. This document, and future updates to it, can be found at http://www.zope.org/Members/jshell/buildingZopeOnMacOSXServer --- Building Zope 2.1.3 on MacOS X Server Last time I told this story, I was having problems getting dynamic libraries (.so files) to work with Python without configuring with the --with-next-framework option, which didn't compile to the results I wanted. Below are the steps to building Python 1.5.2 with threads and dynamic libraries on MacOS X Server and to building Zope 2.1.3. Python:: First - configuring ./configure --with-threads --with-dyld Next - MAKE make LDFLAGS="-undefined suppress" make install The LDFLAGS="-undefined suppress" option is passed into the linker and will suppress errors about undefined references. Without this setting, dynamic libraries will not get past the linker stage. Make install installs Python into the configured area (you should be root to do this). Zope:: A change has to be made in one of the files used for building/configuring Zope that will hopefully be ironed out in the near future. For now, do the following In inst/do.py, change line 135 from do('make') to do('make LDFLAGS="-undefined suppress"') (We'll find a way of fixing this in the future so this shouldn't have to be edited. There should be a Python configuration option when building for setting these LDFLAGS into the default Makefile.pre.in) Then, for reasons currently unknown to me, cPickle issues have to be resolved. The Zope build process makes two cPickle.so files. These files appear to be identical and cause a conflict with each other at runtime due to multiple symbol definitions, and they cause a conflict with the cPickle.so that is built with Python. Lastly: cd lib/python/ rm cPickle.so cd lib/python/ZODB/ mv cPickle.so /usr/local/lib/python1.5/lib-dynload/ You should be root to do the last step of moving the Zope cPickle.so into the default Python area. This is still too much to do, in my opinion, but it's the best way I've found yet. Any suggestions about how to get the LDFLAGS option set at Python configuration time so that the generated Makefiles that Zope uses will have them set, let me know. I tried setting LDFLAGS as an environment variable before running configure, but it uses $LDFLAGS at the beggining of configuration to test the C compilor, and since the '-undefined suppress' option is a linker option, an error is returned and configuration stops. Or, any suggestions or advice on how to set Python up to resolve multiple definitions (due to multiple cPickle.so files) automatically will be greatly appreciated. The cPickle in Zope is newer than the one in Python 1.5.2, I believe. On my to-do list now (but pretty far down the list, sadly) is to make this into a MacOS X Server package similar to other Zope releases (meaning, the Zope package will be a full release with its own Python). The timeframe of this is uncertain. jeffrey(a)Digicool.com

2 1

RE: Whitespace delimiters suck
by Mike Fletcher Jan. 31, 2000

Jan. 31, 2000

Using voice dictation on my machine in PythonWin, this sequence becomes: "class test colon new-line py-def test underscore function open-paren close-paren colon new line print open-quote foxtrot oscar oscar close-quote new-line" and then I hit the backspace twice. Of course, I don't actually use the (very awkward, I agree) navigation features (or even the character-by-character spelling for that matter). Using the voice dictation for basic input, using plain-English spellings, and creating a few simple macros (e.g. py-def (def), py-equals (==), triple-quote (\No-Space'''\No-Space), doublex (wx\No-Space \cap)) and redefining underscore to have no spacing around it, while in an auto-indenting editor, gives you a decent programming environment (though certainly not perfect). You still need your hands, but you can let them hang at your side most of the time. Compared to the pain of typing, a considerable improvement, but still a long way from perfect. This is using Dragon NaturallySpeaking and a little application for customising its vocabulary, incidentally. Enjoy, Mike -----Original Message----- From: Eric S. Johansson [mailto:esj@harvee.billerica.ma.us] Sent: Monday, January 31, 2000 12:42 PM To: python-list(a)python.org Subject: Re: Whitespace delimiters suck ... which is spoken: "class test colon new-line indent delta echo foxtrot test no-space underscore nospace function nospace left- paren right-paren indent new-line print open-quote foxtrot oscar oscar new-line dedent new-line dedent" ...

1 0

where is bbpy?
by niliinsk＠reed.edu Jan. 31, 2000

Jan. 31, 2000

Howdy. The Python FAQ section 7.10 ( http://www.python.org/doc/FAQ.html#7.10 ) claims that there is a Python plug-in for BBedit in the Mac distribution called BBPy. However, there is no sign of it in the distribution or anywhere outside of the FAQ. Where has it gone? Has anyone seen it? Cheers, Noah -- Noah Iliinsky niliinsk at reed dot edu I use PGP. "If we decided that community came first, how would we use our tools differently?" -Howard Rheingold, Wired 7.01 pg. 163

2 1

Date and Time
by tiddlerdeja＠my-deja.com Jan. 31, 2000

Jan. 31, 2000

I'm using Python and COM. One of my 3rd party COM objects returns a Date time thing like this: <PyTime:"11/09/99 11:53:23 AM"> I'd like to create a Python class that returns the Date/Time like this. How would I best mimick this? Also what is PyTime? Is it related to the Python/COM interface? Any help greatly appreciated. Thanks. Sent via Deja.com http://www.deja.com/ Before you buy.

4 4

Multidimensional Dictionaries: Possible?
by w6uv＠hotmail.com Jan. 31, 2000

Jan. 31, 2000

Are multidimensional dictionaries possible in Python? I'm sure they're possible, but I can't figure out the syntax. Here's how I do it in Perl: %dict = (); # create empty dictionary $dict{$key1}{$key2} = $string; # make an entry $newstring = ${dict{$key1}{$key2}}; # access an entry -- Jerry Gardner | "Bill Clinton has all the steely resolve w6uv(a)hotmail.com | of a kamikaze pilot on his 37th mission."

7 7

Newbie: How to create a global or static variable?
by e.e.sutton＠cummins.com Jan. 31, 2000

Jan. 31, 2000

I am trying to convert a simple Microsoft VBScript example script which manipulates an Excel spreadsheet via COM (excel.vbs) to Python for an apples-to-apples comparison of this particular script and how it looks in each language. I know can I have classes in Pyhton but I do not want to do that yet. 1 - How do I create a global variable in a script that is in scope for all the functions defined in the script? 2 - Can I create a static variable with an initial assignment in a function? I have not yet found the documentation I need to explain the scope of variables in Python. I am coming from a C/C++ background. Sent via Deja.com http://www.deja.com/ Before you buy.

2 1

IOError: (0,"Error") when closing "os.popen"ed file?
by rob＠hooft.net Jan. 31, 2000

Jan. 31, 2000

In my project tools, I have a routine "myopen" with the following code: compresstable={'.Z':'compress','.gz':'gzip','.bz2':'bzip2'} def myopen(filename,mode): """Open a file. Based on the filename extension, open it through different compression/decompression program pipes. parameters: filename : name of file to open. mode : 'r' or 'w' for read-only or write-only open. """ from interface import compress if mode=='r': for ext,prog in compresstable.items(): if filename[-len(ext):]==ext: return os.popen(prog+' -dc '+filename+' 2> /dev/null' ,'r') else: return open(filename,'r') elif mode=='w': for ext,prog in compresstable.items(): if filename[-len(ext):]==ext: return os.popen(prog+' > '+filename,'w') else: return open(filename,'w') raise IOError("unknown filemode?") Now I can use f=projtls.myopen("file.Z","w") to write compressed data to disk: myopen will open the file as: f=os.popen('compress > file.Z','w') This approach has worked until a few weeks ago..... Now we have a new fast computer running RedHat linux 6.1, and a 450MHz PIII processor. Suddenly, I started getting strange exceptions from "f.close()" statements for these "popen"ed files: IOError: (0,"Error") When such an error on close occurs, most of the time the last few kB of data are missing from the compressed output file. After hacking for a while, I managed to work around the problem by doing: time.sleep(0.1) f.close() But this is highly unsatisfactory..... Is there anyone that can give me a better suggestion? Regards, Rob Hooft -- ===== rob(a)hooft.net http://www.xs4all.nl/~hooft/rob/ ===== ===== R&D, Nonius BV, Delft http://www.nonius.nl/ ===== ===== PGPid 0xFA19277D ========================== Use Linux! =========

2 5

Riskless deletion of nested structures
by Christian Tismer Jan. 31, 2000

Jan. 31, 2000

Howdy, Please review! While implementing Stackless Python, a new problem arose: Nested structures like frame chains and tracebacks can now easily grow somuch that they cause a stack overflow on deallocation. To protect lists, tuples, frames, dicts and tracebacks against this, I wrote a stackless deallocator. At the moment, everything is done in trashcan.c . This gives a slight performance loss of 5% for pystone, most probably due to the double indirection and non-local code reference. It is yet a hack, since I'm grabbing the tp->dealloc pointers of these types and replace them by safe versions. This just in order to try out things quickly. Later I will change this and incorporate the stack checks into the affected modules, after I got some feedback on this. This patch applies to Stackless and standard Python as well: Deallocation of deeply nested structures will never again cause a stack overflow. Installation for the intermediate version: Insert a line _Py_trashcan_install(); at the end of Py_Initialize in pythonrun.c Please try it and check my code wether there is a better solution. cheers - chris -- Christian Tismer :^) <mailto:tismer@appliedbiometrics.com> Applied Biometrics GmbH : Have a break! Take a ride on Python's Düppelstr. 31 : *Starship* http://starship.python.net 12163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home

1 0