🌐 Mailman 3 June 2010 - Python-list - python.org

[ANN] optphart (alpha2)
by rantingrick June 29, 2010

June 29, 2010

""" I am pleased to announce optphart (alpha2)! ------------------- What is optphart? ------------------- optphart is the nemisis of the asinine interfaces and bulky libraries you may be accustomed to in the stdlib. All of which clog your scripts with wasted lines and your memory with complex interfaces far worse than any colon clogging putrifaction of junk food can hold a candle to. Do yourself a favor and give your scripts an enema by harnessing the simplistic elegance of the "optphart" module instead! ---------------------------- Whats new in this release: ---------------------------- - This release is less likely to blow chunks than the last. - Now you can pass single opts! Woo Hoo! """ def optphart(args): d = {'args':[], 'opts':[]} for arg in args: if arg.startswith('-'): try: key, value = arg.split('=', 1) except ValueError: d['opts'].append(arg[1:]) options = value.split(',') if len(options) > 1: d[key[1:]] = filter(None, options) else: d[key[1:]] = value else: d['args'].append(arg) return d if __name__ == '__main__': args = [ 'nakedArgumentOne', 'nakedArgumentTwo', '-key1=value1', '-key2=value2,', #trailing comma converts to array '-file=C:\\temp.txt', '-options1=Op1,Op2,Op3,Op4', '-options2=Op1,Op2,Op3,Op4,', '-help', '-regex', ] opts = optphart(args) #print opts.keys() for k,v in opts.items(): print '%s -> %s' %(k, v)

3 2

回复： I wander which is better? JSP or Python? And is there a place for JSP?
by rogerdai16 June 29, 2010

June 29, 2010

Oh,sorry.I was just to make a comparison between Python and JSP.Will Python take the place of JSP? 我的QQ空间 the Past 24 Hours. 昨天下午，毛概被点已经有所预感，没有郁闷的心情反倒是... ------------------ 原始邮件 ------------------ 发件人: "Chris Rebert"<clp2(a)rebertia.com>; 发送时间: 2010年6月28日(星期一) 中午1:09 收件人: "Roger"<rogerdai16(a)gmail.com>; 主题: Re: I wander which is better? JSP or Python? And is there a place for JSP? On Sun, Jun 27, 2010 at 9:49 PM, Roger <rogerdai16(a)gmail.com> wrote: > As I plan to study JSP, I find it extremly complicated and a part of > J2EE. > I did not attend to get the whole of J2EE. > I hope anybody can describe the future of JSP. > Is there a place for JSP? This is python-list/comp.lang.python; we discuss the **Python** programming language and related topics here. Your question is about **Java** and has nothing to do with Python. comp.lang.java.programmer is over there: http://groups.google.com/group/comp.lang.java.programmer/topics Regards, Chris -- http://blog.rebertia.com

3 2

os.system: string encoding
by Peter Kleiweg June 29, 2010

June 29, 2010

How do I set the string encoding for os.system to anything other then UTF-8? (peter) ~ echo $LANG nl_NL@euro (peter) ~ python3 Python 3.1.1 (r311:74480, Oct 2 2009, 11:50:52) >>> '\N{EURO SIGN}' '€' >>> import os >>> os.system('echo \N{EURO SIGN}') â?¬ 0 >>> -- Peter Kleiweg

3 4

Re: [OT] Re: Why Is Escaping Data Considered So Magical?
by Lawrence D'Oliveiro June 29, 2010

June 29, 2010

In message <mailman.2231.1277700501.32709.python-list(a)python.org>, Kushal Kumaran wrote: > On Sun, Jun 27, 2010 at 5:16 PM, Lawrence D'Oliveiro > <ldo(a)geek-central.gen.new_zealand> wrote: > >>In message <mailman.2184.1277626565.32709.python-list(a)python.org>, Kushal >> Kumaran wrote: >> >>> On Sun, Jun 27, 2010 at 9:47 AM, Lawrence D'Oliveiro >>> <ldo(a)geek-central.gen.new_zealand> wrote: >>> >>>> A long while ago I came up with this macro: >>>> >>>> #define Descr(v) &v, sizeof v >>>> >>>> making the correct version of the above become >>>> >>>> snprintf(Descr(buf), foo); >>> >>> Not quite right. If buf is a char array, as suggested by the use of >>> sizeof, then you're not passing a char* to snprintf. >> >> What am I passing, then? > > Here's what gcc tells me (I declared buf as char buf[512]): > sprintf.c:8: warning: passing argument 1 of ‘snprintf’ from > incompatible pointer type > /usr/include/stdio.h:363: note: expected ‘char * __restrict__’ but > argument is of type ‘char (*)[512]’ > > You just need to lose the & from the macro. Why does this work, then: ldo@theon:hack> cat test.c #include <stdio.h> int main(int argc, char ** argv) { char buf[512]; const int a = 2, b = 3; snprintf(&buf, sizeof buf, "%d + %d = %d\n", a, b, a + b); fprintf(stdout, buf); return 0; } /*main*/ ldo@theon:hack> ./test 2 + 3 = 5

2 1

Re: Lockless algorithms in python (Nothing to do with GIL)
by sturlamolden June 29, 2010

June 29, 2010

On 29 Jun, 05:11, Ryan Kelly <r...(a)rfk.id.au> wrote: > Very interesting idea. Will it work if accessed through ctypes? > > ticker = ctypes.c_int.in_dll(ctypes.pythonapi,"_Py_Ticker") > ticker.value = 0x7fffffff > > Or does ctypes muck with the GIL in a way that would break this idea? > >>> ctypes.pythonapi <PyDLL 'python dll', handle 1e000000 at 1ffead0> It's a PyDLL, so it should not mock with the GIL. from contextlib import contextmanager import ctypes _Py_Ticker = ctypes.c_int.in_dll(ctypes.pythonapi,"_Py_Ticker") @contextmanager def atomic(): tmp = _Py_Ticker.value _Py_Ticker.value = 0x7fffffff yield _Py_Ticker.value = tmp - 1 Now we can do with atomic(): # whatever pass Just make sure we don't call extension code that releases the GIL :)

1 0

optparse TypeError
by dirknbr June 29, 2010

June 29, 2010

I get an int object is not callable TypeError when I execute this. But I don't understand why. parser = optparse.OptionParser("usage: %lines [options] arg1") parser.add_option("-l", "--lines", dest="lines", default=10, type="int", help="number of lines") parser.add_option("-t", "--topbottom", dest="topbottom", default="T", type="str", help="T(op) or B(ottom)") (options, args) = parser.parse_args() if len(args) != 1: parser.error("incorrect number of arguments") lines=options.lines tb=options.topbottom Dirk lines(args[0],topbottom=tb,maxi=lines)

5 6

Re: Why are String Formatted Queries Considered So Magical?
by Peter H. Coffin June 29, 2010

June 29, 2010

On Mon, 28 Jun 2010 03:07:29 -0700, Dennis Lee Bieber wrote: > Coding for something like a DBTG network database did not allow for > easy changes in queries... What would be a simple join in SQL was > traversing a circular linked list in the DBTG database my college > taught. EG: loop get next "master" record; loop get next sub-record > [etc. until all needed data retrieved] until back to master; until back > to top of database. We'll also note that most of these you'd have to map out where each field in a record was by hand, any time you wanted to open the file. Often several times, because there would be multiple record layouts per file. -- 67. No matter how many shorts we have in the system, my guards will be instructed to treat every surveillance camera malfunction as a full-scale emergency. --Peter Anspach's list of things to do as an Evil Overlord

1 0

Re: Lockless algorithms in python (Nothing to do with GIL)
by sturlamolden June 29, 2010

June 29, 2010

> > Many lockless algorithms that I have looked at thusfar require a > > "Compare and Swap" operation. Does python have an equivalent to this? > > Is python high level enough that it has something better than this or > > it simply doesn't need it? Python does have a GIL, and contrary to the title, this has a lot to do with the GIL. Specifically, any set of operations protected by the GIL are atomic in the CPython interpreter. We can therefore get global synchronization 'for free', allowing 'lock free' data structures that avoid using threading.Lock, and replaces it with, well, nothing. :) How to do it (it takes tiny amounts of C or Cython): Take at look at page 14: http://www.dabeaz.com/python/UnderstandingGIL.pdf The critical part is the variable _Py_Ticker in Python's source file ceval.c, which is declared volatile int. As it is not declared static it is a global variable, and we can actually manipulate its value from a C extension: extern volatile int _Py_Ticker; // evil!!! Now imagine temporarily setting _Py_Ticker to some ridiculous high value like 0x7fffffff. That locks out all other threads, practically forever, until we restore _Py_Ticker back to it's original value. What this gives us is global synchronization for free! The GIL is there anyway, so it adds no additional overhead. By messing with _Py_Ticker we can force blocks of Python code to become atomic in the interpreter. With this hack we can therefore create data structures that are lock free relative to the CPython interpreter. It is evil, but very cheap compared to using threading.Lock. In fact it has exactly zero overhead: the GIL is there anyway, and it is already acquired! The safest placement of a _Py_Ticker hack is in a context manager. We have to make sure it gets restored to a sane value, even in presence of an exception. Enjoy :) P.S. If you get smart and think sys.setcheckinterval(sys.maxint) would do the trick, you are actually not that smart. It will lock our own thread out with a Poisson rate of 1./sys.getcheckinterval(). Thus we have to attack _Py_Ticker from C. P.P.S. And when we have taken control over the GIL, don't do anything that might release it from C (like reading from a file). We cannot afford to loose it :)

2 1

[python] how to ensure item in list or dict bind with "an uuid meaning" integer type ID?
by kee chen June 29, 2010

June 29, 2010

Dear all, I have 2 lists stored in 2 text files may have duplicated records, the raw data looks like this: lfruit lcountry ====== ========= orange japan pear china orange china apple american cherry india lemon china lemon japan strawberry korea banana thailand australia basically, what I want is: 1. all of the duplicated records need to be removed and 2. the unique items need bind with an unique integer ID, something like a PK in database, no sort needed. but before you give answer here, pls also read below. lfruit lcountry ====== ========= 1 orange 1 japan 2 pear 2 china 3 apple 3 american 4 cherry 4 india 5 lemon 5 taiwan 6 strawberry 6 korea 7 banana 7 thailand 8 australia Q1,the items in above lists may need to be added and deleted later, then how to make the list easy to extend and how to make sure the items have a sequenced, unique fixed, INTERGET type ID bind with those items? Here is why I want an INTEGER ID not hash or uuid: the "uuid4" is not working on my case because I want make that ID may transfer information in low cost in a MCU protocol style later, I means the INTEGER ID used here also as the binary stream position id in my protocol, take lfruit data here for example, a bin stream 0111100 can with the meaning of lfruit items exists or not. Also, a combination of 2 lists may needed later to generate new list or called matrix, also as above, an unique ID is also needed here: lcombination = [lfruit] * [lcountry] ============ 1 japan orange #(1,1) 2 japan pear #(1,2) 3 japan apple #(1,3) 4 japan cherry #(1,4) 5 japan lemon ... 6 japan strawberry ... 7 japan banana ... 8 china orange #(2,1) 9 china pear #(2,2) …… Q2, because the lcombination come from the extendable items in lists, then how to make sure the unique ID here also is always fixed and unique? BTW: my original plan is to use dict or list as the runtime data container and use sqlite as the storage also the assigee of the unique ID , however, base on answer from http://old.nabble.com/(python)-how-to-define-unchangeable-global-ID-in-a-ta… it may not just rely on sqlite ensure the unique ID assignee mechanism may works, then I asks help here, any answer or comment will be highly appricated! Thanks, KC

1 0

Re: Lockless algorithms in python (Nothing to do with GIL)
by sturlamolden June 29, 2010

June 29, 2010

> > Many lockless algorithms that I have looked at thusfar require a > > "Compare and Swap" operation. Does python have an equivalent to this? > > Is python high level enough that it has something better than this or > > it simply doesn't need it? Python does have a GIL, and contrary to the title, this has a lot to do with the GIL. Specifically, any set of operations protected by the GIL are atomic in the CPython interpreter. We can therefore get global synchronization 'for free', allowing 'lock free' data structures that avoid using threading.Lock, and replaces it with, well, nothing. :) How to do it (it takes tiny amounts of C or Cython): Take at look at page 14: http://www.dabeaz.com/python/UnderstandingGIL.pdf The critical part is the variable _Py_Ticker in Python's source file ceval.c, which is declared volatile int. As it is not declared static it is a global variable, and we can actually manipulate its value from a C extension: extern volatile int _Py_Ticker; // evil!!! Now imagine temporarily setting _Py_Ticker to some ridiculous high value like 0x7fffffff. That locks out all other threads, practically forever, until we restore _Py_Ticker back to it's original value. What this gives us is global synchronization for free! The GIL is there anyway, so it adds no additional overhead. By messing with _Py_Ticker we can force blocks of Python code to become atomic in the interpreter. With this hack we can therefore create data structures that are lock free relative to the CPython interpreter. It is evil, but very cheap compared to using threading.Lock. In fact it has exactly zero overhead: the GIL is there anyway, and it is already acquired! The safest placement of a _Py_Ticker hack is in a context manager. We have to make sure it gets restored to a sane value, even in presence of an exception. Enjoy :) P.S. If you get smart and think sys.setcheckinterval(sys.maxint) would do the trick, you are actually not that smart. It will lock our own thread out with a Poisson rate of 1./sys.getcheckinterval(). Thus we have to attack _Py_Ticker from C. P.P.S. And when we have taken control over the GIL, don't do anything that might release it from C (like reading from a file). We cannot afford to loose it :)

1 0