🌐 Mailman 3 July 2008 - Python-list - python.org

From D
by bearophileHUGS＠lycos.com Sept. 8, 2008

Sept. 8, 2008

There are various things I like about the D language that I think Python too may enjoy. Here are few bits (mostly syntactical ones): 1) (we have discussed part of this in the past) You can put underscores inside number literals, like 1_000_000, the compiler doesn't enforce the position of such underscores, so you can also put them like this: 1_00_000. You can put them in literals of decimals, binary, hex, etc. I think it's quite useful, because when in Python code I have a line like: for i in xrange(1000000): I need some time to count the zeros, because the lower levels of my visual systems can't count/group them quickly (perceptually). While in a syntax like: for i in xrange(1_000_000): my eyes help me group them at once. 2) Base 2 number literals, and base 2 "%b" printing with the writefln. Base-2 numbers are less common in Python code, but once in a while I use them. For example: import std.stdio; void main() { auto x = 0b0100_0011; writefln("%b", x); writefln("%.8b", x); writefln(x); } Prints: 1000011 01000011 67 3) All string literals are multi line. So you can write: a = "how are you"; There's no need for """ """. 4) With D I have created an xsplit() generator, and from my tests it's quite faster than the split(), expecially if the string/lines you want to split are few hundred chars long or more (it's not faster if you want to split very little strings). So I think Python can enjoy such string method too (you can probably simulate an xsplit with a regular expression, but the same is true for some other string methods too). Bye, bearophile

29 56

The Importance of Terminology's Quality
by xahlee＠gmail.com Sept. 2, 2008

Sept. 2, 2008

I'd like to introduce a blog post by Stephen Wolfram, on the design process of Mathematica. In particular, he touches on the importance of naming of functions. • Ten Thousand Hours of Design Reviews (2008 Jan 10) by Stephen Wolfram http://blog.wolfram.com/2008/01/10/ten-thousand-hours-of-design-reviews/ The issue is fitting here today, in our discussion of “closure” terminology recently, as well the jargons “lisp 1 vs lisp2” (multi- meaning space vs single-meaning space), “tail recursion”, “currying”, “lambda”, that perennially crop up here and elsewhere in computer language forums in wild misunderstanding and brouhaha. The functions in Mathematica, are usually very well-name, in contrast to most other computing languages. In particular, the naming in Mathematica, as Stephen Wolfram implied in his blog above, takes the perspective of naming by capturing the essense, or mathematical essence, of the keyword in question. (as opposed to, naming it according to convention, which often came from historical happenings) When a thing is well-named from the perspective of what it actually “mathematically” is, as opposed to historical developments, it avoids vast amount of potential confusion. Let me give a few example. • “lambda”, widely used as a keyword in functional languages, is named just “Function” in Mathematica. The “lambda” happend to be called so in the field of symbolic logic, is due to use of the greek letter lambda “λ” by happenstance. The word does not convey what it means. While, the name “Function”, stands for the mathematical concept of “function” as is. • Module, Block, in Mathematica is in lisp's various “let*”. The lisp's keywords “let”, is based on the English word “let”. That word is one of the English word with multitudes of meanings. If you look up its definition in a dictionary, you'll see that it means many disparate things. One of them, as in “let's go”, has the meaning of “permit; to cause to; allow”. This meaning is rather vague from a mathematical sense. Mathematica's choice of Module, Block, is based on the idea that it builds a self-contained segment of code. (however, the choice of Block as keyword here isn't perfect, since the word also has meanings like “obstruct; jam”) • Functions that takes elements out of list are variously named First, Rest, Last, Extract, Part, Take, Select, Cases, DeleteCases... as opposed to “car”, “cdr”, “filter”, “filter”, “pop”, “shift”, “unshift”, in lisps and perl and other langs. The above are some examples. The thing to note is that, Mathematica's choices are often such that the word stands for the meaning themselves in some logical and independent way as much as possible, without having dependent on a particular computer science field's context or history. One easy way to confirm this, is taking a keyword and ask a wide audience, who doesn't know about the language or even unfamiliar of computer programing, to guess what it means. The wide audience can be made up of mathematicians, scientists, engineers, programers, laymen. This general audience, are more likely to guess correctly what Mathematica's keyword is meant in the language, than the the name used in other computer languages who's naming choices goes by convention or context. (for example, Perl's naming heavily relies on unix culture (grep, pipe, hash...), while functional lang's namings are typically heavily based on the field of mathematical logic (e.g. lambda, currying, closure, monad, ...). Lisp's cons, car, cdr, are based on computer hardware (this particular naming, caused a major damage to the lisp language to this day). (Other examples: pop, shift are based on computer science jargon of “stack”. Grep is from Global Regular Expression Print, while Regular Expression is from theoretical computer science of Automata... The name regex has done major hidden damage to the computing industry, in the sense that if it have just called it “string patterns”, then a lot explanations, literatures, confusions, would have been avoided.)) (Note: Keywords or functions in Mathematica are not necessarily always best named. Nor are there always one absolute choice as best, as there are many other considerations, such as the force of wide existing convention, the context where the function are used, brevity, limitations of English language, different scientific context (e.g. math, physics, engineering), or even human preferences.) ---------------------------- Many of the issues regarding the importance and effects of terminology's quality, i've wrote about since about 2000. Here are the relevant essays: • Jargons of Info Tech Industry http://xahlee.org/UnixResource_dir/writ/jargons.html • The Jargon “Lisp1” vs “Lisp2” http://xahlee.org/emacs/lisp1_vs_lisp2.html • The Term Curring In Computer Science http://xahlee.org/UnixResource_dir/writ/currying.html • What Is Closure In A Programing Language http://xahlee.org/UnixResource_dir/writ/closure.html • What are OOP's Jargons and Complexities http://xahlee.org/Periodic_dosage_dir/t2/oop.html • Sun Microsystem's abuse of term “API” and “Interface” http://xahlee.org/java-a-day/interface.html • Math Terminology and Naming of Things http://xahlee.org/cmaci/notation/math_namings.html Xah xah(a)xahlee.org ∑ http://xahlee.org/ ☄

44 111

Native Code vs. Python code for modules
by koblas Aug. 31, 2008

Aug. 31, 2008

Ruby has been getting pummeled for the last year or more on the performance subject. They've been working hard at improving it. From my arm chair perspective Python is sitting on it's laurels and not taking this as seriously as it probably should. In general it's possible to make many comments that swirl around religion and approach, one of the things that I've noticed is that wile Python has a much more disciplined developer community they're not using this for the greater good. Specifically, in looking at a benchmark that was posted, python was an order of magnitude (20 secs vs. 1 sec) slower than ruby. In investigating the performance bottleneck is the random module. This got me to think, why does python have "pickle" and "cPickle"? Comes down to lowest common denominator, or the inability for somebody to write an optimized package that can mimic a python package. To that end why would somebody write big try catch blocks to see if modules exist and if they exist alias their names. Wouldn't it be better if there was a way that if I have an "interface compatible" native (aka C) module that has better performance that there could be a way that python would give it preference. e.g. import random(version=1.2, lang=c) or import random(version=1.2, lang=py) # use the python version by default or import random # use the latest version in the "fastest" code (C given preference) where there could be a nice set of "standard" key value pairs that could provide addtional hints as to what language and version of a library was to be used.

9 9

importing from .pyd
by moijes12 Aug. 29, 2008

Aug. 29, 2008

hi there is a .pyd file present in the same folder as the script abc.py by the name foo.pyd .I don't have foo.py .In the script abc.py I try import foo Error i get is ImportError: DLL load failed: The specified module could not be found. I have tried placing .pyd under C:\python25\DLLs but it didn't work.I have also given the path for foo.pyd in PYTHONPATH.but i still recieve the same error. Please suggest a solution moijes12

3 3

Function References
by squishywaffle＠gmail.com Aug. 28, 2008

Aug. 28, 2008

Greetings, I'm trying to wrap a function in a C library as a compiled C Python module. Everything is going great, but I've hit a snag. There's a function in the form of this: First the typedef: typedef void(*FPtr_DeviceMessageHandler) (const DeviceMessage, const char*); Then the actual function prototype: FPtr_DeviceMessageHandler RegisterDeviceMessageHandler(FPtr_DeviceMessageHandler); Whenever this USB device I'm using generates a message, it's then sent to that function whose reference you passed to RegisterDeviceMessageHandler(). Here's an example: void testfunc() { printf("test"); } ...on to main() RegisterDeviceMessageHandler(&testfunc) So I've defined a similar function on my C module to do just this, using the passed function reference to allow the device to send messages to a Python function of the user's choice. I'm just not sure how to do this, and the tutorials I've been digging through don't offer any help. I've found some functions that look promising, but can't figure out how to cast the function reference from Python into something the C RegisterDevice... function can handle. Here's what I've got: static PyObject * Py1_RegisterDeviceMessageHandler(PyObject *self, PyObject *args) { PyObject *handle, *parsed; if(!PyArg_ParseTuple(args, "O", handle)) { return NULL; } parsed = PyMethod_Function(handle); I1_RegisterDeviceMessageHandler(PyMethod_Function(handle)); Py_RETURN_TRUE; } // end Py1_RegisterDeviceMessageHandler() This fails since PyMethod_Function returns a PyObject. Is there a way to cast this to something generic? Casting to (void*) didn't seem to work. Thanks in advance!

6 15

Python Behind a Squid Corporate Proxy on Windows
by Larry Hale Aug. 21, 2008

Aug. 21, 2008

Greetings, Pythonistas! My employer has a Squid Proxy between my Python programs and The Internet. I've searched high-and-low, and can only find examples online of how to do basic authentication to connect TO an external Web Server, but not how to go THROUGH a (corporate) Proxy, thus my question here. I've tried all sorts of permutations based on my findings, but (obviously) nothing's working for me. I've come across conflicting information, including that urllib2 can't/won't do what I'm needing, though this statement was a couple years old. I'm running Python 2.5 on Windows XP SP2 64-bit. The Proxy is Squid (don't know what version, but can find out if this is relevant). urllib2 grabs the Proxy info fine: 'http': 'http://webfilter.xyz.local: 3128' (.getproxies()). There's the obvious necessity of telling "something" my ID/pass *somehow*. An example of one iteration: ###################### import urllib2 proxy_handler = urllib2.ProxyHandler( { "http": "http:// myusername:@webfilter.xyz.local:3128" } ) opener = urllib2.build_opener( proxy_handler ) urllib2.install_opener( opener ) response = opener.open( "http://python.org" ) print response ###################### I've tried situations using HTTPPasswordMgrWithDefaultRealm and its add_password method, also to no avail. (Again, THIS sort of thing seems related to the -external- Web Server, _not_ going *through* a Proxy. 'Course, I could be wrong. But if I am, I couldn't get this to work, either. ;) I've looked at what's happening with Wireshark; Py/my program/urllib2 makes the initial page request, Proxy replies HTTP Error 407: Proxy Authentication Required, and Py/my program/urllib2 simply stops at that, reporting the message/error. Please forgive me; I've been programming in Python for a few years now, but never had to deal with Web Proxies. I *have* tried to RTFMs, but the "documentation" for non-basic stuffs with urllib2 is... uh... well, I'm sure ye olde pro's know... :) Cheers, and *any* help would be appreciated! -Larry

3 6

Debugging of a long running process under Windows
by Propad Aug. 21, 2008

Aug. 21, 2008

Hello, I know this issue pops up once in a while, but I haven't found a good answer to it. I need to debug a long running application under windows. The application is a combined java/python framework for testing ECUs in the automotive industry. Basically, the Java GUI (Eclipse-based) starts test-cases written in Python and provides the console where the test-logs are seen. When there is a exception somewhere in the testcases (or the underlying functionallity, also written in Python), those are also logged, and then the framework usually continues with the next command in the same test case. I'd like to have a debugging facillity better than print statements. I imagine: a) something like a debugger poping up when I get an exception, or b) something debugger-like poping up when it reaches a command I entered something in the code, or c) Me pressing on a button and getting a debugger-like-thing that lets me look into the running, possibly halted code. I've done some little experiments with the code module, which looks nice but does not seem to get over the control from the java-part, and with popen2("cmd"), which seems not even to work if I start the code from a dosbox (the same console is keept), and same thing when strated by the Java-App. Just to add, using pdb (or pythonwin debugger) seems not to be an option, as it makes the test-runs much slower. Does somebody have an idea? It seems there used to be a python debugger called Pygdb, able to attach to a running application, but now it seems it disapeared (now there is a python debugger with the same name, linked to ViM). Thanx, Propad

3 2

Python regex question
by Tim van der Leeuw Aug. 15, 2008

Aug. 15, 2008

Hi, I'm trying to create a regular expression for matching some particular XML strings. I want to extract the contents of a particular XML tag, only if it follows one tag, but not follows another tag. Complicating this, is that there can be any number of other tags in between. So basically, my regular expression should have 3 parts: - first match - any random text, that should not contain string '<Xds' - second match I have a problem figuring out how to do the second part: a random bit of text, that should _not_ contain the substring '<Xds' ('<Xds' being the start of any tags which should not be in between my first and second match). Because of the variable length of the overal match, I cannot do this with a negative look-behind assertion, and a negative look-ahead assertion doesn't seem to work either. The regular expression that I have now is: r'(?s)<Xds\w*Policy>.*?<ref>(?P<pol_ref>\d+)</ref>' (hopefully without typos) Here '<Xds\w*Policy>' is my first match, and '<ref>(?P<pol_ref>\d+)</ref>' is my second match. In this expression, I want to change the generic '.*?', which matches everything, with something that matches every string that does not include the substring '<Xds'. I know that I could capture the text matched by '.*?' and manually check if it contains that string '<Xds', but that would be very hard to fit into the rest of the code, for a number of reasons. Does anyone have an idea how to do this within one regular expression? Regards, --Tim

3 2

interpreter vs. compiled
by castironpi Aug. 9, 2008

Aug. 9, 2008

I'm curious about some of the details of the internals of the Python interpreter: I understand C from a hardware perspective. x= y+ 1; Move value from place y into a register Add 1 to the value in the register Move the addition's result to place x The Python disassembly is baffling though. >>> y= 3 >>> dis.dis('x=y+1') 0 SETUP_LOOP 31037 (to 31040) 3 STORE_SLICE+3 4 <49> What are SETUP_LOOP and STORE_SLICE? What are these instructions?

16 37

Re: variable expansion with sqlite
by marc wyburn Aug. 8, 2008

Aug. 8, 2008

Hi and thanks, I was hoping to avoid having to weld qmarks together but I guess that's why people use things like SQL alchemy instead. It's a good lesson anyway. Thanks, Marc. On Jul 30, 2:24 pm, Tim Golden <m...(a)timgolden.me.uk> wrote: > Gerhard Häring wrote: > > My code would probably look very similar. Btw you don't need to use > > list() on an iterable to pass to executemany(). pysqlite's executemany() > > accepts anything iterable (so generators work fine, too). > > Thanks for that. My finger-memory told me to do that, possibly > because some *other* dbapi interface only accepts lists. Can't > quite remember. I'm usually all in favour of non-crystallised > iterators. > > > Also, with SQLite you can just skip data type definitions like > > VARCHAR(200). They're ignored anyway. > > Heh. Once again, finger memory forced me to put *something* > in there. I've been developing Enterprise databases for too > long :) > > TJG

2 1