🌐 Mailman 3 July 2010 - Python-list - python.org

Re: default behavior
by Steven D'Aprano July 29, 2010

July 29, 2010

On Thu, 29 Jul 2010 21:43:05 +0300, Nick Raptis wrote: > On 07/29/2010 09:12 PM, wheres pythonmonks wrote: >> How do I build an "int1" type that has a default value of 1? > You mean something like: > >>> x = int() > >>> x > 0 > >>> def myint(value=1): > ... return int(value) > ... > >>> myint() > 1 > >>> > >>> > That's ugly on so many levels.. Why do you think it's ugly? It's a function that returns an int, and it provides a default value which is different from the default value of the int constructor. It's a really simple function, and it has an equally simple implementation, and it's an obvious way to do it. Not the *one* obvious way, because subclassing int is equally obvious, but still obvious. -- Steven

1 0

Re: Newbie question regarding SSL and certificate verification
by John Nagle July 29, 2010

July 29, 2010

On 7/28/2010 6:26 PM, geremy condra wrote: > On Wed, Jul 28, 2010 at 4:41 PM, Jeffrey > Gaynor<jgaynor(a)ncsa.uiuc.edu> wrote: >> Hi, >> >> I am making a first large project in python and am having quite a >> bit of difficulty unscrambling various python versions and what >> they can/cannot do. To wit, I must communicate with certain >> services via https and am required to perform certificate >> verification on them. >> >> The problem is that I also have to do this under CentOS 5.5 which >> only uses python 2.4 as its default -- this is not negotiable. As >> near as I can tell from reading various posts, the https client >> does not do verification and there is no low-level SSL support to >> provide a workaround. Near as I can tell from reading, 2.6 does >> include this. Am I getting this right? Is there a simple way to do >> this? More to the point, I need to know pretty darn quick if this >> is impossible so we can try and plan for it. >> >> So the quick question: Has anyone done certificate verification >> using 2.4 and if so, how? >> >> Thanks! > > M2Crypto is the way to go here. I think there's an example on their > site. M2Crypto does that job quite well. Installing M2Crypto tends to be painful if you have to build it, though. See if you can find a pre- built version. You then need a "cacert.pem" file, with the root certificates you're going to trust. You can get one from http://curl.haxx.se/docs/caextract.html which converts Mozilla's format to a .pem file once a week. The actual Mozilla source file is at http://mxr.mozilla.org/mozilla/source/security/nss/lib/ckfw/builtins/certda… but M2Crypto needs it in .pem format. The new Python SSL module in 2.6 and later has a huge built-in security hole - it doesn't verify the domain against the certificate. As someone else put it, this means "you get to talk securely with your attacker." As long as the site or proxy has some valid SSL cert, any valid SSL cert copied from anywhere, the new Python SSL module will tell you everything is just fine. John Nagle

6 6

Re: default behavior
by John Nagle July 29, 2010

July 29, 2010

On 7/29/2010 11:12 AM, wheres pythonmonks wrote: > Why is the default value of an int zero? > >>>> x = int >>>> print x > <type 'int'> >>>> x() > 0 >>>> > > How do I build an "int1" type that has a default value of 1? >>> class int1(object) : ... def __init__(self) : ... self.val = 1 ... def __call__(self) : ... return(self.val) ... >>> x = int1() >>> x() 1 This isn't useful; you'd also have to define all the numeric operators for this type. And then there are mixed-type conversion issues. Inheriting from "int" is not too helpful, because you can't assign to the value of the base class. "self=1" won't do what you want. [Hopefully no speed penalty.] In your dreams. Although all numbers in CPython are "boxed", so there's more of a speed penalty with "int" itself than you might expect. There are some C libraries for handling large arrays if you really need to crunch numbers. John Nagle

2 1

Tabular Package: importing file
by Robert Faryabi July 29, 2010

July 29, 2010

Hi there; I'm using Tabular Package for manipulating tab-delimited data. There is a small problem that I cannot get my head around it. When I construct my tabarray from file, the black fields are replaced by "nan". Does any one knows how to just keep them as empty string (ie. ' ')? Thanks, -R

2 1

default behavior
by wheres pythonmonks July 29, 2010

July 29, 2010

Why is the default value of an int zero? >>> x = int >>> print x <type 'int'> >>> x() 0 >>> How do I build an "int1" type that has a default value of 1? [Hopefully no speed penalty.] I am thinking about applications with collections.defaultdict. What if I want to make a defaultdict of defaultdicts of lists? [I guess my Perl background is showing -- I miss auto-vivification.] W

2 1

Re: default behavior
by Paul Rubin July 29, 2010

July 29, 2010

wheres pythonmonks <wherespythonmonks(a)gmail.com> writes: > How do I build an "int1" type that has a default value of 1? > [Hopefully no speed penalty.] > I am thinking about applications with collections.defaultdict. You can supply an arbitary function to collections.defaultdict. It doesn't have to be a class. E.g. d = collections.defaultdict(lambda: 1) will do what you are asking.

2 1

Re: Ascii to Unicode.
by Ulrich Eckhardt July 29, 2010

July 29, 2010

Joe Goldthwaite wrote: > import unicodedata > > input = file('ascii.csv', 'rb') > output = file('unicode.csv','wb') > > for line in input.xreadlines(): > unicodestring = unicode(line, 'latin1') > output.write(unicodestring.encode('utf-8')) # This second encode > is what I was missing. Actually, I see two problems here: 1. "ascii.csv" is not an ASCII file but a Latin-1 encoded file, so there starts the first confusion. 2. "unicode.csv" is not a "Unicode" file, because Unicode is not a file format. Rather, it is a UTF-8 encoded file, which is one encoding of Unicode. This is the second confusion. > A number of you pointed out what I was doing wrong but I couldn't > understand it until I realized that the write operation didn't work until > it was using a properly encoded Unicode string. The write function wants bytes! Encoding a string in your favourite encoding yields bytes. > This still seems odd to me. I would have thought that the unicode > function would return a properly encoded byte stream that could then > simply be written to disk. No, unicode() takes a byte stream and decodes it according to the given encoding. You then get an internal representation of the string, a unicode object. This representation typically resembles UCS2 or UCS4, which are more suitable for internal manipulation than UTF-8. This object is a string btw, so typical stuff like concatenation etc are supported. However, the internal representation is a sequence of Unicode codepoints but not a guaranteed sequence of bytes which is what you want in a file. > Instead it seems like you have to re-encode the byte stream to some > kind of escaped Ascii before it can be written back out. As mentioned above, you have a string. For writing, that string needs to be transformed to bytes again. Note: You can also configure a file to read one encoding or write another. You then get unicode objects from the input which you can feed to the output. The important difference is that you only specify the encoding in one place and it will probably even be more performant. I'd have to search to find you the according library calls though, but starting point is http://docs.python.org. Good luck! Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

4 3

Multiprocessing taking too much time
by Shailendra July 29, 2010

July 29, 2010

Hi All, I have a following situation. ==================PSUDO CODE START================== class holds_big_array: big_array #has a big array def get_some_element(self, cond) # return some data from the array from the big array ==================PSUDO CODE END==================== I wanted to use multiprocessing module to parallelise calling "get_some_element". I used following kind of code ==================PSUDO CODE START================== pool = Pool(processes=2) holder =holds_big_array() #class instantiation def callback_f(result): do something with result loop many times pool.apply_async(holder.get_some_element,args,callback=callback_f) pool.close() pool.join() ==================PSUDO CODE END==================== Note: Had to do something to enable instance method being pickled... I tested this with less than realistic size of big_array . My parallel version works much slower than than the normal serial version (10-20 sec vs 7-8 min). I was wonder what could be the possible reason. Is it something to do that it is a instance method and some locking will make other process wait for the locks. Any idea how to trace where the program is spending time? Let me know if the information give is inadequate. Thanks in advance. Shailendra Vikas

2 1

Re: Newbie question regarding SSL and certificate verification
by John Nagle July 29, 2010

July 29, 2010

On 7/28/2010 10:23 PM, geremy condra wrote: > On Wed, Jul 28, 2010 at 10:08 PM, John Nagle<nagle(a)animats.com> wrote: >> On 7/28/2010 6:26 PM, geremy condra wrote: >>> >>> On Wed, Jul 28, 2010 at 4:41 PM, Jeffrey >>> Gaynor<jgaynor(a)ncsa.uiuc.edu> wrote: > >> The new Python SSL module in 2.6 and later has a huge built-in >> security hole - it doesn't verify the domain against the >> certificate. As someone else put it, this means "you get to >> talk securely with your attacker." As long as the site or proxy >> has some valid SSL cert, any valid SSL cert copied from anywhere, >> the new Python SSL module will tell you everything is just fine. >> >> John Nagle > > Did anything ever come of the discussion that you and Antoine had? > > Geremy Condra > > PS- the quote is due to Justin Samuel I had to write my own domain check. Did anyone re-open the bug report on that issue? John Nagle

1 0

write xml to txt encoding
by William Johnston July 29, 2010

July 29, 2010

Hello, I have a Python app that parses XML files and then writes to text files. However, the output text file is "sometimes" encoded in some Asian language. Here is my code: encoding = "iso-8859-1" clean_sent = nltk.clean_html(sent.text) clean_sent = clean_sent.encode(encoding, "ignore"); I also tried "UTF-8" encoding, but received the same results. Any suggestions? Thanks.

2 1