🌐 Mailman 3 October 2005 - Python-list - python.org

Re: Raw string fu
by Paul McGuire Nov. 21, 2005

Nov. 21, 2005

"Raw string fu"? A new martial art?

2 1

Re: Rename files with numbers
by dudufigueiredo＠gmail.com Nov. 18, 2005

Nov. 18, 2005

Ok, so the function simplifyed without loops: def renamer(folder, band): archive = #file to transform rest = archive[3:] print band + " -",rest.capitalize() obs: the file names came this way(with spaces or apostrophes) from the cd i imported.

4 3

Re: Python vs Ruby
by bruno modulix Nov. 16, 2005

Nov. 16, 2005

Michael Ekstrand wrote: > On Friday 21 October 2005 07:07, bruno modulix wrote: > >>>>Python is more like Java. >> >><troll> >>Err... Python is more like what Java would have been if Java was a >>smart dynamic hi-level object oriented language !-) >></troll> > > > +1. Python is easily applicable to most of the problem domain of Java, > but solves problems much more elegantly. It just isn't shipped embedded > in all leading browsers :-(. It's been a long time since I last saw a Java applet on a website. -- bruno desthuilliers python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for p in 'onurb(a)xiludom.gro'.split('@')])"

2 1

Newbie question: string replace
by usgog＠yahoo.com Nov. 16, 2005

Nov. 16, 2005

I have a config file with the following contents: service A = { params { dir = "c:\test", username = "test", password = "test" } } I want to find username and replace the value with another value. I don't know what the username value in advance though. How to do it in python?

9 15

Background process for ssh port forwarding
by Jesse Rosenthal Nov. 15, 2005

Nov. 15, 2005

Hello all, I'm writing a script which will backup data from my machine to a server using rsync. It checks to see if I am on the local network. If I am, it runs rsync over ssh to 192.168.2.6 using the pexpect module to log in. That's the easy part. Now, when I'm not on the local network, I first want to open up an ssh connection to do port forwarding, so something like this: def hostforward(): #This is based on the assumption that the passfile is the gnus #authinfo file, or has a similar format... f = open(PASS_FILE, "r") f_list = f.read().split(' ') f.close() #Now, we get the entry after "password" (be slicker to make it a #dictionary, but maybe wouldn't work as well). pass_index = f_list.index('password') + 1 forwardpass = f_list[pass_index] #now we connect command = 'ssh -l %s -L 2022:%s:22 %s' % \ (login, my_server, forwarding_server) connection = pexpect.spawn(command) connection.expect('.*assword:') connection.sendline(forwardpass) If I end this with 'connection.interact()', I will end up logged in to the forwarding server. But what I really want is to go on and run rsync to localhost port 2022, which will forward to my_server port 22. So, how can I put the ssh connection I set up in hostforward() in the background? I need to make sure that connection is made before I can run the rsync command. I've looked at threading, but that seems excessive. There must be an easier way. Whatever I do, though, I'll need to use pexpect to spawn the processes, since I'll need to log in to ssh servers with a password. Thanks for any help. --Jesse

4 5

MoinMoin - Can't create new pages
by ferg Nov. 11, 2005

Nov. 11, 2005

I've checked all the FAQs and help - spent a couple of hours on this now and it's really bugging me. I've got Moin 1.3.5 installed on IIS 5.0. I can view and edit pages without any problems, but cannot create new pages - either trough the GUI, or by creating a new WikiName and clicking on it. I'm stumped as to whether this is a Moin ACL problem (played around a lot with this, but to no avail) or perhaps an IIS config problem (though the Web Account has full control to the data directory). The Wiki is an internal intranet one for our development group, so I don't really care if everyone is an admin, I just need to be able to create new pages anyway. Here is my wikiconfig.py >>>> FILE START >>>> # -*- coding: iso-8859-1 -*- # IMPORTANT! This encoding (charset) setting MUST be correct! If you live in a # western country and you don't know that you use utf-8, you probably want to # use iso-8859-1 (or some other iso charset). If you use utf-8 (a Unicode # encoding) you MUST use: coding: utf-8 # That setting must match the encoding your editor uses when you modify the # settings below. If it does not, special non-ASCII chars will be wrong. """ MoinMoin - Configuration for a single wiki If you run a single wiki only, you can omit the farmconfig.py config file and just use wikiconfig.py - it will be used for every request we get in that case. Note that there are more config options than you'll find in the version of this file that is installed by default; see the module MoinMoin.multiconfig for a full list of names and their default values. Also, the URL http://moinmoin.wikiwikiweb.de/HelpOnConfiguration has a list of config options. @copyright: 2000-2005 by Juergen Hermann <jh(a)web.de> @license: GNU GPL, see COPYING for details. """ from MoinMoin.multiconfig import DefaultConfig class Config(DefaultConfig): # Wiki identity ---------------------------------------------------- # Site name, used by default for wiki name-logo [Unicode] sitename = u'ISS Wiki' # Wiki logo. You can use an image, text or both. [Unicode] # Example: u'<img src="/wiki/mywiki.png" alt="My Wiki">My Wiki' # For no logo or text, use '' logo_string = sitename # The interwiki name used in interwiki links interwikiname = None # Critical setup --------------------------------------------------- # Misconfiguration here will render your wiki unusable. Check that # all directories are accessible by the web server or moin server. # If you encounter problems, try to set data_dir and data_underlay_dir # to absolute paths. # Where your mutable wiki pages are. You want to make regular # backups of this directory. data_dir = './data/' # Where read-only system and help page are. You might want to share # this directory between several wikis. When you update MoinMoin, # you can safely replace the underlay directory with a new one. This # directory is part of MoinMoin distribution, you don't have to # backup it. data_underlay_dir = './underlay/' # This must be '/wiki' for twisted and standalone. For CGI, it should # match your Apache Alias setting. url_prefix = '/wiki' # Security ---------------------------------------------------------- # Security critical actions (disabled by default) # Uncomment to enable options you like. #allowed_actions = ['DeletePage', 'AttachFile', 'RenamePage'] # Enable acl (0 to disable) acl_enabled = 1 # IMPORTANT: grant yourself admin rights! replace YourName with # your user name. See HelpOnAccessControlLists for more help. # All acl_rights_xxx options must use unicode [Unicode] acl_rights_before = u"arkprod\it_32564:read,write,delete,revert,admin" # Link spam protection for public wikis (Uncomment to enable) # Needs a reliable internet connection. #from MoinMoin.util.antispam import SecurityPolicy # Mail -------------------------------------------------------------- # Configure to enable subscribing to pages (disabled by default) # or sending forgotten passwords. # SMTP server, e.g. "mail.provider.com" (empty or None to disable mail) mail_smarthost = "" # The return address, e.g "My Wiki <noreply(a)mywiki.org>" mail_from = "" # "user pwd" if you need to use SMTP AUTH mail_login = "" # User interface ---------------------------------------------------- # Add your wikis important pages at the end. It is not recommended to # remove the default links. Leave room for user links - don't use # more than 6 short items. # You MUST use Unicode strings here, but you need not use localized # page names for system and help pages, those will be used automatically # according to the user selected language. [Unicode] navi_bar = [ # Will use page_front_page, (default FrontPage) u'%(page_front_page)s', u'RecentChanges', u'FindPage', u'HelpContents', ] # The default theme anonymous or new users get theme_default = 'modern' # Language options -------------------------------------------------- # See http://moinmoin.wikiwikiweb.de/ConfigMarket for configuration in # YOUR language that other people contributed. # The main wiki language, set the direction of the wiki pages default_lang = 'en' # You must use Unicode strings here [Unicode] page_category_regex = u'^Category[A-Z]' page_dict_regex = u'[a-z]Dict$' page_form_regex = u'[a-z]Form$' page_group_regex = u'[a-z]Group$' page_template_regex = u'[a-z]Template$' # Content options --------------------------------------------------- # Show users hostnames in RecentChanges show_hosts = 1 # Enumerate headlines? show_section_numbers = 0 # Charts size, require gdchart (Set to None to disable). chart_options = {'width': 600, 'height': 300} >>>>> FILE END >>>>>>

2 4

Automatic binding of **kwargs to variables
by lbolognini＠gmail.com Nov. 9, 2005

Nov. 9, 2005

Hi all, I have a very long list of parameters coming from a web form to my method foo(self, **kwargs) I would like to avoid manually binding the variables to the values coming through the **kwargs dictionary, just to keep the code cleaner, I'd like to bind them automatically I was adviced against doing it by adding stuff to locals such as: locals()["x"] = 5 since the Python Docs say: Warning: The contents of this dictionary should not be modified; changes may not affect the values of local variables used by the interpreter. Is there a way to do this? Maybe object.__setattr__() or something that involves metaclasses? As I might go touching something that I shouldn't (like locals()) I'd like some advice from you before deciding which way to go. Thanks, Lorenzo

12 25

Print to printer
by avnit Nov. 7, 2005

Nov. 7, 2005

I can't seem to figure out how to print with my printer using python. I'm using Mac OSX 10.4. I was thinking maybe something with applescript. Does anyone know?

3 7

python gc performance in large apps
by Robby Dermody Nov. 7, 2005

Nov. 7, 2005

Hey guys (thus begins a book of a post :), I'm in the process of writing a commercial VoIP call monitoring and recording application suite in python and pyrex. Basically, this software sits in a VoIP callcenter-type environment (complete with agent phones and VoIP servers), sniffs voice data off of the network, and allows users to listen into calls. It can record calls as well. The project is about a year and 3 months in the making and lately the codebase has stabilized enough to where it can be used by some of our clients. The entire project has about 37,000 lines of python and pyrex code (along with 1-2K lines of unrelated java code). Now, some disjointed rambling about the architecture of this software. This software has two long-running server-type components. One component, the "director" application, is written in pure python and makes use of the twisted, nevow, and kinterbasdb libraries (which I realize link to some C extensions). The other component, the "harvester", is a mixture of python and pyrex, and makes use of the twisted library, along with using the C libs libpcap and glib on the pyrex end. Basically, the director is the "master" component. A single director process interacts with users of the system through a web and/or pygtk client application interface and can coordinate 1 to n harvesters spread about the world. The harvester is the "heavy lifter" component that sniffs the network traffic and sifts out the voice and signalling data. It then updates the director of call status changes, and can provide users of the system access to the data. It records the data to disk as well. The scalibility of this thing is really cool: given a single director sitting somewhere coordinating the list of agents, multiple harvester can be placed anywhere there is voice traffic. A user that logs into the director can end up seeing the activity of all of these seperate voice networks presented like a single giant mesh. Overall, I have been very pleased with python and the 3rd party libraries that I use (twisted, nevow, kinterbasdb and pygtk). It is a joy to program with, and I think the python community has done a fine job. However, as I have been running the software lately and profiling its memory usage, the one and only Big Problem I have seen is that of the memory usage. Ideally, the server application(s) should be able to run indefinitely, but from the results I'm seeing I will end up exhausting the memory on a 2 GB machine in 2 to 3 days of heavy load. Now normally I would not raise up an issue like this on this list, but based on the conversations held on this list lately, and the work done by Evan Jones (http://evanjones.ca/python-memory.html), I am led to believe that this memory usage -- while partially due to some probably leaks in my program -- is largely due to the current python gc. I have some graphs I made to show the extent of this memory usage growth: http://public.robbyd.fastmail.fm/iq-graph1.gif http://public.robbyd.fastmail.fm/iq-graph-director-rss.gif http://public.robbyd.fastmail.fm/iq-graph-harv-rss.gif The preceding three diagrams are the result of running the 1 director process and 1 harvester process on the same machine for about 48 hours. This is the most basic configuration of this software. I was running this application through /usr/bin/python (CPython) on a Debian 'testing' box running Linux 2.4 with 2GB of memory and Python version 2.3.5. During that time, I gathered the resident and virtual memory size of each component at 120 second intervals. I then imported this data into MINITAB and did some plots. The first one is a graph of the resident (RSS) and virtual memory usage of the two applications. The second one is a zoomed in graph of the director's resident memory usage (complete with a best fit quadratic), and the 3rd one is a zoomed in graph of the harvester's resident memory usage. To give you an idea of the network load these apps were undergoing during this sampling time, by the time 48 hours had passed, the harvester had gathered and parsed about 900 million packets. During the day there will be 50-70 agents talking. This number goes to 10-30 at night. In the diagrams above, one can see the night-day separation clearly. At night, the memory usage growth seemed to all but stop, but with the increased call volume of the day, it started shooting off again. When I first started gathering this data, I was hoping for a logarithmic curve, but at least after 48 hours, it looks like the usage increase is almost linear. (Although logarithmic may still be the case after it exceeds a gig or two of used memory. :) I'm not sure if this is something that I should expect from the current gc, and when it would stop. Now, as I stated above, I am certain that at least some of this increased memory usage is due to either un-collectable objects in the python code, or memory leaks in the pyrex code (where I make some use of malloc/free). I am working on finding and removing these issues, but from what I've seen with the help of gc UNCOLLECTABLE traces, there are not many un-collectable reference issues at least. Yes, there are some but definitely not enough to justify growth like I am seeing. The pyrex side should not be leaking too much, I'm very good about freeing what I allocate in pyrex/C land. I will be running that linked to a memory leak finding library in the next few days. Past the code reviews I've done, what makes me think that I don't have any *wild* leaks going on at least with the pyrex code is that I am seeing the same type of growth patterns in both apps, and I don't use any pyrex with the director. Yes, the harvester is consuming much more memory, but it also does the majority of the heavy lifting. I am alright with the app not freeing all the memory it can between high and low activity times, but what puzzles me is how the memory usage just keeps on growing and growing. Will it ever stop? What I would like to know if others on this list have had similar problems with python's gc in long running, larger python applications. Am I crazy or is this a real problem with python's gc itself? If it's a python gc issue, then it's my opinion that we will need to enhance the gc before python can really gain leverage as a language suitable for "enterprise-class" applications. I have surprised many other programmers that I'm writing an application like this in python/pyrex that works just as well and even more efficiently than the C/C++/Java competitors. The only thing I have left to show is that the app lasts as long between restarts. ;) Robby

5 5

Using graphviz to visualize trace.py output, anybody?
by svenn.are＠bjerkem.de Nov. 5, 2005

Nov. 5, 2005

Hi, has anybody thought of / already used graphviz to convert the output of trace.py into a graph? I looked at PyUMLGraph, but 1. PyUMLGraph does not successfully create a dot file, and 2. I don't really want a UML representation but a compact representation of program execution. Maybe there is some other tool that I am not aware of which can create this kind of trace. I use eclipse with pydev plugin on MacOS 10.3.9 Regards, Svenn

6 8