🌐 Mailman 3 June 2010 - Python-list - python.org

Lockless algorithms in python (Nothing to do with GIL)
by Zac Burns July 6, 2010

July 6, 2010

In my experience it is far more expensive to allocate a lock in python then it is the types that use them. Here are some examples: >>> timeit.timeit('Lock()', 'from threading import Lock') 1.4449114807669048 >>> timeit.timeit('dict()') 0.2821554294221187 >>> timeit.timeit('list()') 0.17358153222312467 Granted, I know these types do not allocate a lock - but a similarly defined user type would need to allocate a lock to guarantee thread-safety. I have not done a great deal of research into lockless algorithms thusfar, but would it be worthwhile to investigate implementing some of them in python to optimize performance critical sections of code? Many lockless algorithms that I have looked at thusfar require a "Compare and Swap" operation. Does python have an equivalent to this? Is python high level enough that it has something better than this or it simply doesn't need it? -- Zachary Burns (407)590-4814 Aim - Zac256FL Production Engineer Zindagi Games

3 5

Re: Python as a scripting language. Alternative to bash script?
by Mithrandir July 5, 2010

July 5, 2010

Michael Torrie <torriem(a)gmail.com> wrote in news:mailman.2313.1277759925.32709.python-list@python.org: > On 06/28/2010 02:06 PM, Mithrandir wrote: >> I can't see Python as an alt. to bash. (As I recall) Python is much >> more object-oriented than bash, but also there are many commands >> (such as apt- get, etc.) that would need Python equivs. However, I >> can see Python being used as a scripting alt. to C. > > OO is a plus for most things, in my book. As for "commands" they have > *nothing* to do with Bash. apt-get is not a Bash command. By your > logic tcsh or zsh would not be an alternate to bash, but in fact they > are. > > I use python for shell scripting quite often now. Anytime one of my > own Bash scripts exceeds 100 lines, I know it's time to switch it to > python. > Please read that link I posted a while back on how you can use > generators in python to replace many of the things that piping to > external commands did in Bash. > > There certainly are a few tasks that Bash is best at (chaining > commands together through pipes), but often Python already has support > for many of the things I'd use external commands and pipes in Bash > for. Bash is designed for working down at the level of files, > directories, and processes, but Python works pretty well too, if you > make some abstraction modules like my runcmd module that I use > extensively. > You both are correct. :) (I wrote that before my first cup of coffee, so my wording was way off. That, and I'm new to Python.) :) I think that Python "could" be a alternative to bash and have some advantages, but it's a long way off from being fully implemented.

2 1

Why Is Escaping Data Considered So Magical?
by Lawrence D'Oliveiro July 4, 2010

July 4, 2010

Just been reading this article <http://www.theregister.co.uk/2010/06/23/xxs_sql_injection_attacks_testing_r…> which says that a lot of security holes are arising these days because everybody is concentrating on unit testing of their own particular components, with less attention being devoted to overall integration testing. Fair enough. But it’s disconcerting to see some of the advice being offered in the reader comments, like “force everyone to use stored procedures”, or “force everyone to use prepared/parametrized statements”, “never construct ad-hoc SQL queries” and the like. I construct ad-hoc queries all the time. It really isn’t that hard to do safely. All you have to do is read the documentation—for example, <http://dev.mysql.com/doc/refman/5.0/en/string-syntax.html>—and then write a routine that takes arbitrary data and turns it into a valid string literal, like this <http://www.codecodex.com/wiki/Useful_MySQL_Routines#Quoting>. I’ve done this sort of thing for MySQL, for HTML and JavaScript (in both Python and JavaScript itself), and for Bash. It’s not hard to verify you’ve done it correctly. It lets you easily create table-updating code like the following, which makes it so easy to update the code to track changes in the database structure: sql.cursor.execute \ ( "update items set " + ", ".join ( tuple ( "%(name)s = %(value)s" % { "name" : field[0], "value" : SQLString(Params.getvalue ( "%s[%s]" % (field[1], urllib.quote(modify_id)) )) } for field in ( ("class_name", "modify_class"), ("make", "modify_make"), ("model", "modify_model"), ("details", "modify_details"), ("serial_nr", "modify_serial"), ("inventory_nr", "modify_invent"), ("when_purchased", "modify_when_purchased"), ... you get the idea ... ("location_name", "modify_location"), ("comment", "modify_comment"), ) ) + ( "last_modified = %d" % int(time.time()), ) ) + " where inventory_nr = %s" % SQLString(modify_id) )

19 54

CONTROLLED DEMOLITION INC explosive-charge placement technician Tom Sullivan 911 TESTIMONIAL Video
by nanothermite911fbibustards July 3, 2010

July 3, 2010

http://www.ae911truth.org/newsletter/2010/06/index.php#cdi Explosive Evidence at WTC Cited by Former CDI Employee News - News Releases By AE911Truth Written by Darcy Wearing and Richard Gage, AIA Thursday, 24 June 2010 18:55 Having had the privilege of speaking with Tom Sullivan, an actual explosive-charge placement technician, we have some new insights to pass along as to how controlled demolition works, where it started, and the effect that 9/11 had on the demolition industry. Sullivan gained his experience as an employee of the leading firm in this field, Controlled Demolition, Inc. (CDI). Sullivan stresses though “I do not in anyway represent CDI and what I have to say is based on my own experience and training,” Sullivan attended high school with Doug Loizeaux of the Loizeaux family. The Loizeaux family, through the father Jack, independently started the whole controlled-demolition industry and turned it into a highly profitable business. Sullivan, before he became connected to CDI, was an independent photographer during his early years in Maryland. He would be sent to CD sites and take still pictures of the jobs. He became infatuated with the CD industry. The time came when he would do both, being the placer of the “cutter charges” on the primary joints, and photographing the jobs for promoting the business. Soon he would switch to full-time employee status of CDI -- as verified by AE911Truth’s verification team. "It was very interesting, but also very hard work, long hours, especially in the cold weather," Sullivan reflects. He stated that the days began early, around 6 a.m., and they would work until the sun was down. Sullivan had the experience of preparing a building by placing the cutter charges throughout the primary joints, and then, of course, watching it all come down. Read the rest with photos: http://www.ae911truth.org/newsletter/2010/06/index.php#cdi

8 14

Re: Why Is Escaping Data Considered So Magical?
by Carl Banks July 3, 2010

July 3, 2010

On Jun 27, 9:54 pm, Kushal Kumaran <kushal.kumaran+pyt...(a)gmail.com> wrote: > On Mon, Jun 28, 2010 at 2:00 AM, Jorgen Grahn <grahn+n...(a)snipabacken.se> wrote: > > On Sun, 2010-06-27, Lawrence D'Oliveiro wrote: > >> In message <roy-854954.20435125062...(a)news.panix.com>, Roy Smith wrote: > > >>> I recently fixed a bug in some production code. The programmer was > >>> careful to use snprintf() to avoid buffer overflows. The only problem > >>> is, he wrote something along the lines of: > > >>> snprintf(buf, strlen(foo), foo); > > >> A long while ago I came up with this macro: > > >> #define Descr(v) &v, sizeof v > > >> making the correct version of the above become > > >> snprintf(Descr(buf), foo); > > > This is off-topic, but I believe snprintf() in C can *never* safely be > > the only thing you do to the buffer: you also have to NUL-terminate it > > manually in some corner cases. See the documentation. > > snprintf goes to great lengths to be safe, in fact. You might be > thinking of strncpy. Indeed, strncpy does not copy that final NUL if it's at or beyond the nth element. Probably the most mind-bogglingly stupid thing about the standard C library, which has lots of mind-boggling stupidity. Whenever I do an audit of someone's C code the first thing I do is search for strncpy and see if they set the nth character to 0. (They usually didn't.) Carl Banks

9 10

automate minesweeper with python
by Jay July 2, 2010

July 2, 2010

I would like to create a python script that plays the Windows game minesweeper. The python code logic and running minesweeper are not problems. However, "seeing" the 1-8 in the minesweeper map and clicking on squares is. I have no idea how to proceed.

9 13

MURDEROUS CRIMES of ODOUSLY RACIST FBI BUSTARDS !!!
by small Pox July 2, 2010

July 2, 2010

The MURDEROUS Bustards killed a man who was a SAINT . He ran a sunday soup and food place for the homeless and hungry. http://www.youtube.com/watch?v=Cnz5N9OubCQ http://www.youtube.com/watch?v=Cnz5N9OubCQ http://www.youtube.com/watch?v=Cnz5N9OubCQ http://www.youtube.com/watch?v=Cnz5N9OubCQ http://www.youtube.com/watch?v=Cnz5N9OubCQ

9 16

Python dynamic attribute creation
by WANG Cong July 2, 2010

July 2, 2010

Hi, list! I have a doubt about the design of dynamic attribute creation by assignments in Python. As we know, in Python, we are able to create a new attribute of a class dynamically by an assignment: >>> class test: pass ... >>> test.a = "hello" >>> test.a 'hello' >>> However, I still don't get the points why Python designs it like this. My points are: 1) Modifying a class attribute is metaprogramming, and this is modifying a class, i.e. adding a new attribute to it, thus this should belong to metaprogramming. (I know, strictly speaking, maybe my definition of "metaprogramming" here is incorrect, I _do_ welcome someone could correct me if I am really wrong, but that is not the main point here, please don't go off-topic.) 2) Metaprogramming should be distinguished with non-meta programming, like templates in C++, it is obvious to see if you are using template metaprogramming in C++. 3) Thus, allowing dynamic attribute creation by assignment _by default_ is not a good design for me. It is not obvious at all to see if I am doing metaprogramming at a first glance. 4) Also, this will _somewhat_ violate the OOP princples, in OOP, this is and should be implemented by inherence. 5) This could hide errors silently, for example, when I do: >>>test.typooooo = "blah" I am expecting Python will compllain that "Hey! You have a typo in the attribute name!". Also, this could make code worse to read, if I add a new attribute in one place, and add another one in the another place, and so on, what attributes the hell do I have finally?! I know someone will say you can change this by overriding the __set_attr__ function, like Recipe 6.3 in Python Cookbook. However, this is not default. In my points of view, a better design should be: 1) Disallow dynamic attribute creations by assignments _by default_, thus I expect an error when I do: >>> foo.new_attr = "blah" AttributeError: by default. 2) For people who want to add a new attribute at runtime, but not to override __set_attr__, he/she should switch to: >>> setattr(foo, "new_attr", "blah") This will be more like doing metaprogramming rather than non-meta programming, at least more obvious than using an assignment. 3) Allow users who don't like this to change by __set_attr__, of course. Someone argued with me that Python is a dynamic language, allowing this is natural. True, I do understand that attributes in Python are stored in an internal dictionary (__dict__), allowing assignments to an non-existing key is natural. However, this will be a little different when we talk about classes attributes, simple assignments could have side-effects, besides the traditional assignments effect, like in C, that is, creating a new attribute silently. So even from a view of beauty, this is not a good design. I hope someone could teach me more about why Python design it like it is. Any reply is more than welcome. Thanks for your time! -- Live like a child, think like the god.

21 51

Python/C++ timer intermittent bug
by Paul Grunau July 1, 2010

July 1, 2010

I have a problem with threading using the Python/C API. I have an extension that implements a timer, and the C++ timer callback function calls a Python function. The relevant code looks like this: static PyObject *timer_setmodname( PyObject *pSelf, PyObject *pArgs ) { char *b; PyArg_ParseTuple( pArgs, "s", &b ); mod = PyImport_ImportModule(b); if( mod == NULL ) { printf("Could not import %s\n",b); return Py_None; } modsetFlag = TRUE; return Py_None; } static PyObject *timer_setprocname( PyObject *pSelf, PyObject *pArgs ) { char *b; if( !modsetFlag )return Py_None; PyArg_ParseTuple( pArgs, "s", &b ); attr = PyObject_GetAttrString(mod,b); if( attr == NULL ) { printf("Could not import %s\n",b); return Py_None; } attrsetFlag = TRUE; return Py_None; } static void CALLBACK PeriodicTimer(UINT wTimerID, UINT msg, DWORD dwUser, DWORD dw1, DWORD dw2) { PyGILState_STATE pgs; pgs = PyGILState_Ensure(); if(attrsetFlag) { pres = PyObject_CallFunction(attr,NULL); if( pres == NULL )printf("CallFunction failed!\n"); } PyGILState_Release( pgs ); } The Python code that sets this up looks like this: fetimer.setmodname("Timeslice3") fetimer.setprocname("Timetester") print "\n Program Waiting for Time Slice" while True: time.sleep(0.010) and the module Timeslice3.py looks like this: #Timeslice3.py def Timetester(): pass When I run this stuff, it works fine for hundreds, often even thousands, of timer ticks (I've been testing with about thirty ticks per second, but it doesn't matter - it still crashes at ten or fewer ticks per second). Sometimes it runs for only a few seconds, sometimes for ten minutes or so. But it always eventually crashes Python. Usually it gives no error message. Sometimes, though, it does give an error message, but not always the same one. I've noted three that it has given in my testing so far: Fatal Python Error: This thread state must be current when releasing Fatal Python Error: PyThreadState_DeleteCurrent: no current tstate Fatal Python Error: PyEval_SaveThread: NULL tstate Can anybody help me make this code stable, so that it works all the time? I'm using Python 2.6.5 under Windows Vista, but it crashes under Windows XP as well.

2 1

Re: Python dynamic attribute creation
by Steven D'Aprano July 1, 2010

July 1, 2010

On Sat, 26 Jun 2010 10:10:39 -0700, Stephen Hansen wrote: > This characterization of adding attributes to an object as something > "else", some special kind of activity called "metaprogramming" I think I > reject outright, whereas I believe -- though I do not claim to speak for > him/her -- the OP's position is that using 'syntax' to add attributes is > actually a special case/activity. > > I consider it the standard, normal behavior. +1 [...] > In that situation, certainly: adding an attribute on the fly to that > formal definition seems entirely strange and special of an activity. But > that's only because you *chose* to *see* and *use* the object that way. > The "special"ness of the activity is entirely in your head, and to > Python, its an entirely normal event. Exactly. Things which other languages make scary and mysterious "metaprogramming" are, in Python, normal and ordinary programming. It doesn't seem to do us any harm. In fairness, the reason the other languages do so is probably so that the compiler can make optimizations leading to faster code, which is why CPython is a fairly plodding implementation. But that's due to the conservativeness of CPython: Unladen Swallow is faster, and PyPI is faster still, and the PyPI people expect to eventually be competitive with C for speed. > Python lets you associate significance with normal events so you can get > things done. But its just plodding along not really drinking whatever > kool-aid you are, though its happy you like your flavor and is entirely > content with letting you think its playing ball with you on that. Nice :) -- Steven

4 3