🌐 Mailman 3 September 2016 - Python-list - python.org

Re: Extend unicodedata with a name/pattern/regex search for character entity references?
by Veek. M Sept. 4, 2016

Sept. 4, 2016

Thomas 'PointedEars' Lahn wrote: > Veek. M wrote: > >> https://mail.python.org/pipermail//python-ideas/2014-October/029630.htm >> >> Wanted to know if the above link idea, > > … which is 404-compliant; the Internet Archive does not have it either > … > >> had been implemented > > Probably not. > >> and if there's a module that accepts a pattern like 'cap' and give >> you all the instances of unicode 'CAP' characters. > > I do not know any. > >> ⋂ \bigcap >> ⊓ \sqcap >> ∩ \cap >> ♑ \capricornus >> ⪸ \succapprox >> ⪷ \precapprox >> >> (above's from tex) >> >> I found two useful modules in this regard: unicode_tex, unicodedata >> but unicodedata is a builtin which does not do globs, regexs - so >> it's kind of limiting in nature. > > Quick hack: > > #-------------------------------------------------------------------- > from unicode_tex import unicode_to_tex_map > > for key, value \ > in filter(lambda item: "cap" in item[1], unicode_to_tex_map.items()): > print(key, value) > #-------------------------------------------------------------------- > > (Optimizations are welcome.) > > It is easy to come up with methods that take a globbing or a regular > expression (globbing expressions can be turned into regular > expressions easily) and returns, perhaps as a dictionary or list of > tuples, only the matching entries. > > Other than that I think you will have to turn the Unicode Character > Database (which is available via HTTP as one huge text file; see the > Python Tutorial on “Internet Access” for how to get it dynamically) > into whatever form suits you for querying it. > >> Would be nice if you could search html/xml character entity >> references as well. > > For what purpose? > > Your posting is lacking a real name in the “From” header field. > Ouch! Sorry for the bad link Thomas. The link is titled '[Python-ideas] Extend unicodedata with a name search' and I suspect this updated link data (http://code.activestate.com/lists/python-ideas/29504/) may work - if it doesn't you could google the title. I don't want to dump/replicate the existing Unicode data in module 'unicodedata'. Regarding purpose, well I need this for hexchat. I IRC a lot and often, I want to ask a question involving math symbols. I've written some python (included at the bottom) that translates: \help filter_word #into a list of symbols and names (serves as a memory jog). It also translates stuff like: A \cap B \epsilon C to A ∩ B ε C. but all this works with a subset of tex - it can't do complicated formula. I wanted to extend it further.. I don't think I shall be able to subscript integrals easily but I could make better use of the available unicode, which means making it more accessible (hence the pattern matching feature) - html/xml entities provide a new way of remembering stuff. --------------------------- Regarding the name (From field), my name *is* Veek.M though I tend to shorten it to Vek.M on Google (i think Veek was taken or some such thing). Just to be clear, my parents call me something closely related to Veek that is NOT Beek or Peek or Squeak or Sneak and my official name is something really weird. Identity theft being what it is, I probably am lying anyhow about all this, but it sounds funny so :p import hexchat import re, unicode_tex, unicodedata __module_name__ = 'Unicode' __module_version__ = '0.1' __module_description__ = 'Substitute \whatever with Unicode char in cmdline input' #re_repl = unicodedata.lookup('N-ARY UNION') def debug(*args): hexchat.prnt('#####{}#####'.format(*args)) def print_help(*args): hexchat.prnt('{}'.format(*args)) def send_message(word, word_eol, userdata): if not(word[0] == "65293"): return msg = hexchat.get_info('inputbox') if msg is None: return x = re.match(r'(^\\help)\s+(\w+)', msg) if x: filter = x.groups()[1] for key, value in unicode_tex.tex_to_unicode_map.items(): if filter in key: print_help(value + ' ' + key) hexchat.command("settext %s" % '') return tex_matches = re.findall(r'(\\\w+)', msg) for tex_word in tex_matches: repl = unicode_tex.tex_to_unicode_map.get(tex_word) if repl is None: repl = 'err' msg = re.sub(re.escape(tex_word), repl, msg) hexchat.command("settext %s" % msg) hexchat.hook_print('Key Press', send_message)

2 2

Re: Extend unicodedata with a name/pattern/regex search for character entity references?
by Jussi Piitulainen Sept. 4, 2016

Sept. 4, 2016

Chris Angelico <rosuav(a)gmail.com> writes: > On Sun, Sep 4, 2016 at 12:49 PM, Steve D'Aprano > <steve+python(a)pearwood.info> wrote: >> On Sun, 4 Sep 2016 12:19 pm, Chris Angelico wrote: >> >> [...] >>>> Please either comply, or give up your stupid and pointless obsession with >>>> trying to be the Internet Police for something that isn't even a real >>>> rule. >>> >>> His posts aren't making it across the news->list gateway any more. >>> Killfile him and move on... >> >> But but but... I couldn't do that. >> >> https://www.xkcd.com/386/ > > Ah, you got me. Can't force anyone to violate standards documents like > RFCs and XKCDs. > > ChrisA > who may or may not have recently pushed a commit to make something > XKCD 859 compliant... (-: (You were baiting for that.)

1 0

Re: Extend unicodedata with a name/pattern/regex search for character entity references?
by Steve D'Aprano Sept. 4, 2016

Sept. 4, 2016

On Sun, 4 Sep 2016 06:47 am, Thomas 'PointedEars' Lahn wrote: > Your posting is lacking a real name in the “From” header field. Thomas, if that is really your name, how do we know that: Thomas 'PointedEars' Lahn is a real name? Is sounds made up to me. I'm afraid that we're going to have to insist that you scan you birth certificate, drivers licence and other forms of ID and send them to us so that we can see proof that this is your real name. Please either comply, or give up your stupid and pointless obsession with trying to be the Internet Police for something that isn't even a real rule. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse.

2 1

Re: Strange behaviour with numbers in exponential notation
by Christian Gollwitzer Sept. 3, 2016

Sept. 3, 2016

Am 03.09.16 um 02:31 schrieb Marco Sulla: > On 2 September 2016 at 21:12, Christian Gollwitzer <auriocus(a)gmx.de> wrote: >> Am 02.09.16 um 19:24 schrieb Marco Sulla: >> Because Python has no long double type? > > Python no of course, but C++ yes, and CPython is written in C++. > However, I think the answer is here: > https://en.wikipedia.org/wiki/Long_double#Implementations > Briefly, long double is not implemented in all hardwares, or its > implementation is really different from machine to machine. Yes - 64 bit IEEE floats are today the universally available floating point data type across different machines. If you want arbitrary precision floats, you can have them in Python, too: https://pypi.python.org/pypi/bigfloat/ Of course this is a "software emulation" and is consequently slower then hardware floats. It does not, howver take away the principle problem that you must set the precision from the beginning of your calculation. Christian

1 0

Re: *args and **kwargs
by Gregory Ewing Sept. 3, 2016

Sept. 3, 2016

Ben Finney wrote: > in the definition of the function, parameters are neither positional nor > keyword. In Python 3 that's not quite true -- it's possible to define "keyword only" parameters that can't be passed positionally. However, it's true that any of the "ordinary" parameters (ones that aren't keyword-only) can be specified either positionally or by keyword in a call. -- Greg

1 0

Re: Strange behaviour with numbers in exponential notation
by Christian Gollwitzer Sept. 3, 2016

Sept. 3, 2016

Am 02.09.16 um 19:24 schrieb Marco Sulla: > Excuse me, I forgot to include the python list mail addess. I repost the mail. > > > > On Fri, Sep 2, 2016 at 6:18 PM, Christian Gollwitzer <auriocus(a)gmx.de> wrote: >> 1e26 denotes a *floating point number* Floating point has finite precision, >> in CPython it is a 64bit IEEE number. The largest exact integer there is >> 2**53 (~10^16), everything beyond cannot be accurately represented. > > I see. So python float type is the IEEE 754 double. Why is it not > automatically converted to long double? Because Python has no long double type? And "long double" (assuming 80bit or 128bit) does not solve the problem, it just shifts the bound to some higher arbitrary limit. In fact, it is impossible to make a bigfloat class which acts like a float, but always carries enough bits. A simple counterexample is 0.3, a periodic fraction in binary. Or, if you include arbitrary precision fractions, then e.g. sqrt(2). > And I want to add to my original question: indeed I read from the docs: > https://docs.python.org/3/reference/lexical_analysis.html#floating-point-li… > that float has an 'exponentfloat' syntax. Why integers does not have > an equivalent syntax? Tradition? All languages I know of treat a number with an exponent as floating point. Christian

5 5

Re: Strange behaviour with numbers in exponential notation
by Random832 Sept. 2, 2016

Sept. 2, 2016

On Fri, Sep 2, 2016, at 13:02, Marco Sulla wrote: > On Fri, Sep 2, 2016 at 6:17 PM, Random832 <random832(a)fastmail.com> wrote: > > Trying to add 1 gets it rounded off again, and the value is simply > > printed as 1e+26 by default because this is the shortest representation > > that gives the same number, even if "100000000000000004764729344.0" > > would be more accurate. > > It seems it's not simply a representation. It seems the numbers are just > equal: Yes. The number 1e+16 is represented as a floating point number equal to 100000000000000004764729344. Then when you add 1 to it, the 1 gets rounded off and you get a floating point number that is still equal to 100000000000000004764729344: >>> a = float(10**16) >>> int(a) 100000000000000004764729344 >>> b = a + 1 >>> int(b) 100000000000000004764729344 You switched examples. 10**22 is the largest power of 10 that can be represented exactly, but 10**15+1 is te largest "power of 10 plus one" that can be represented exactly. Float can only represent numbers whose binary representation contains 53 bits or less from the first to the last "1". >>> '{:b}'.format(10**15) '11100011010111111010100100110001101000000000000000' # 35 bits before the run of zeros >>> '{:b}'.format(10**16) '100011100001101111001001101111110000010000000000000000' # 38 bits, works fine in float >>> '{:b}'.format(10**15+1) '11100011010111111010100100110001101000000000000001' # 50 bits, barely fits in a float >>> '{:b}'.format(10**16+1) '100011100001101111001001101111110000010000000000000001' # 54 bits, too many. Anything else gets rounded off. It's the same as if you tried to add a tiny number to 1. >>> 1+1e-15 == 1 False >>> 1+1e-16 == 1 True > This is really weird. I tried to do it just in C: You should be using double instead of long double; double is the C equivalent of the Python float type.

1 0

Re: Strange behaviour with numbers in exponential notation
by Marco Sulla Sept. 2, 2016

Sept. 2, 2016

Excuse me, I forgot to include the python list mail addess. I repost the mail. On Fri, Sep 2, 2016 at 6:18 PM, Christian Gollwitzer <auriocus(a)gmx.de> wrote: > 1e26 denotes a *floating point number* Floating point has finite precision, > in CPython it is a 64bit IEEE number. The largest exact integer there is > 2**53 (~10^16), everything beyond cannot be accurately represented. I see. So python float type is the IEEE 754 double. Why is it not automatically converted to long double? And I want to add to my original question: indeed I read from the docs: https://docs.python.org/3/reference/lexical_analysis.html#floating-point-li… that float has an 'exponentfloat' syntax. Why integers does not have an equivalent syntax? It's not too difficult to lexically distinguish between a integer and a float written in exponential notation, I think.

1 0

Re: urllib2 timeout not working - stalls for an hour or so
by Sumeet Sandhu Sept. 2, 2016

Sept. 2, 2016

On Friday, September 2, 2016 at 6:05:05 AM UTC-7, Peter Otten wrote: > Sumeet Sandhu wrote: > > > Hi, > > > > I use urllib2 to grab google.com webpages on my Mac over my Comcast home > > network. > > > > I see about 1 error for every 50 pages grabbed. Most exceptions are > > ssl.SSLError, very few are socket.error and urllib2.URLError. > > > > The problem is - after a first exception, urllib2 occasionally stalls for > > upto an hour (!), at either the urllib2.urlopen or response.read stages. > > > > Apparently the urllib2 and socket timeouts are not effective here - how do > > I fix this? > > > > ---------------- > > import urllib2 > > import socket > > from sys import exc_info as sysExc_info > > timeout = 2 > > socket.setdefaulttimeout(timeout) > > > > try : > > req = urllib2.Request(url,None,headers) > > response = urllib2.urlopen(req,timeout=timeout) > > html = response.read() > > except : > > e = sysExc_info()[0] > > open(logfile,'a').write('Exception: %s \n' % e) > > < code that follows this : after the first exception, I try again for a > > few tries > > > I'd use separate try...except-s for response = urlopen() and > response.read(). If the problem originates with read() you could try to > replace it with select.select([response.fileno()], [], [], timeout) calls in > a loop. thanks Peter. I will try this. However, I suspect Comcast is rate limiting my home use. Is there a workaround for that? What I really need is to somehow monitor the full url-read loop and break it if Comcast is stalling me too long...

1 0

Strange behaviour with numbers in exponential notation
by Marco Sulla Sept. 2, 2016

Sept. 2, 2016

>>> 10**26 - 1 99999999999999999999999999 >>> 1e26 - 1 1e+26 Why?

2 1