🌐 Mailman 3 June 2002 - Python-list - python.org

Partial Parsing of Python
by Peter Eckersley July 1, 2002

July 1, 2002

Hi, I'm currently working on a python-based alternative to a standard *nix shell. I imagine lots of other people have built such animals, but this one is supposed to be graphical and include nifty features like smart auto-completion and running commentary on one's commands (which should be handy for newbies). Anyway, having implemented some of the completion features the simple way-- using tokenize.tokenize(), string analysis, and the odd call to parser.expr() to guess what the user might want to type next, I've reached the conclusion that doing too much more will lead to Madness and Spaghetti Code. So the alternative is of course to use a proper python parser, which can include hooks suggesting how bits of the grammar can be completed. Basically, what this parser needs to do is attempt to accept its input, noting along the way when it meets the user's cursor. If it fails, it needs to preserve an incomplete parse tree, record which production rule it was trying to use, and which symbol in that rule caused the problem. It's not immediately obvious how to achieve this with a minimum of fuss. A few different strategies might be: * hack the cPython parser to provide this extra stuff (not having emersed myself in the parser, I imagine this could be a lot of work, but would love to hear information to the contrary, or receive few pointers as to how it might be done) * use bison to implement a new parser from scratch (ugly, perilous, and I'd have to learn yacc/bison along the way :( ) * write a minimalist, brute-force recursive descent parser in python, which would be a more-or-less direct translation of the grammar, but would almost certainly be damnably slow Suggestions about which strategy to pursue would be gratefully appreciated! (CCs off-list, IINTMT) -- Peter Eckersley Department of Computer Science & mailto:pde@cs.mu.oz.au IP Research Institute of Australia http://www.cs.mu.oz.au/~pde The University of Melbourne

2 6

dynamic buttons with tkinter
by Mirko Koenig July 1, 2002

July 1, 2002

Hi In my i want to make a button for every file in a special directory, so that i can load it. I did: for file in os.listdir( dir ): Button( load_dlg_frame, text=file, relief=GROOVE, command=load).pack() But how can i find out which button is pressed? Or another question: How can i pass a event funktion more than 1 parameter (the event and other things). I can't give any button a new event function, because it's dynamic, because of the number of files in the directory. thanx Mirko

3 2

pprinting generators
by vvainio＠tp.spt.fi July 1, 2002

July 1, 2002

On 2.2.1, pprint.pprint doesn't seem to be able to handle generators: >>> from __future__ import generators >>> def f(): yield 0 ... >>> import pprint >>> pprint.pprint(f()) <generator object at 0x007F91D0> Surely pprint could be more helpful with generators, iterating through them fully or just n first elements, or whatever. Or is this a design decision with pprint? -- Ville

2 2

telnetlib option negotiation callback should be a method
by Steve Reeves July 1, 2002

July 1, 2002

I have an application using telnet that requires option subnegotiation (specifically, it needs to set a terminal type in the manner of RFC 1091). The standard telnetlib doesn't currently support this. I would like to extend it to do so, but the current design of the option negotiation callback makes this difficult. Currently, the callback is a function taking a socket as an argument. This is adequate for sending data to the server, but not for receiving. It bypasses the Telnet object's internal queue, which may already contain the data you're waiting for. It also requires you to reimplement the special processing the protocol requires (undoubling IAC characters), and the default negotiation behavior. The callback really needs to be a method of the Telnet object, with access to the internal queue. It can implement the default DONT/WONT negotiation. Subclasses can override it to handle the specific options they want and forward the rest to the base class. The set_option_negotiation_callback() method would no longer be needed. (This question of function vs. method was briefly raised when the telnetlib patch was first submitted, <http://www.python.org/sf/416079>, but it was left as a function.) Opinions? -- Steve Reeves stever(a)gate.net

1 1

newbie Q: delete list few lines in file
by xah＠xahlee.org July 1, 2002

July 1, 2002

i have a folder full of individual email text file, whose ending may be the following: _______________________________________________ some mailing list some(a)list.com http://some.com/ttt I want to delete them, but only if they are the last few lines in the file. if you can show me a script that'll be great otherwise my question can be broken down as: * how to write to file inplace? (or, do i write to a new file, then delete original, and rename the new file?) * as far as a newbie, i use "xreadlines" module. I wasn't able to get the total number of lines in a file. li=xreadlines.xreadlines(f) num=len(li) fails. I'm much more interested in learning python idiom and correctness here. I'm not interested in just "get it to work". For the latter, i can do well with perl. (also, is there a site or mailing list that collect very small trivial python programs for learning purposes?) Thanks. Xah xah(a)xahlee.org http://xahlee.org/PageTwo_dir/more.html

7 9

Is this a __slot__ bug?
by Steve Menard July 1, 2002

July 1, 2002

OK, this is probably me not understanding something, but then again, you never know. First things first, I love __slots__. For the paranoid programmer I am, knowing I will never again assing to a variable with a typo and then wonder ehat the hell is going wring is a godsend. However, I also love(need) private variables (those __X). It seems they dont work with slots. try the following simple example under ptyon 2.2 on linux : class O(object) : __slots__ = [ '__a' ] def __init__(self) : object.__init__(self); self.__a = 3 o = O() I get thoe following error : AttributeError: 'O' object has no attribute '_O__a' Now, I know I can easily work around this by manually obfuscating the private variables names in the __slots__, but I would rather not. Feels wrong somehow to rely on what should be a hidden way of generating private variables. So if this is by design, I will not agree but I will do the workaround. If not, well then seems I founda bug :) Steve

4 4

Re: can't import generators
by John Hunter July 1, 2002

July 1, 2002

>>>>> "Tim" == Tim Peters <tim.one(a)comcast.net> writes: Tim> Start Python with the -v switch to see how imports are Tim> getting resolved. __future__.py is an actual module that Tim> actually gets imported. If you're picking up a wrong version Tim> of the Python libraries, that would explain it (maybe due to Tim> a bad PYTHONPATH setting, or some local module named Tim> __future__.{py,pyc,pyo}). Bingo -- thanks. Old musty, crusty PYTHONPATH setting was the culprit. JDH

2 1

Python needs to explain parsing? was Re: Python needs better error reporting
by Antaeus Feldspar July 1, 2002

July 1, 2002

After reading the thread spun out of all of this, this brings up an idea that I've had for quite some time: can the Python interpreter be made to report more information about *how* it is parsing the syntax it finds? For instance, given the valid line "if s == ' ':" it might return something like the following (or at least this might be what is displayed to the user based on information the parser displays): if <condition>: ---- [next 8 lines of code] | == / \ / \ s ' ' thus confirming that what was meant to be a condition is being parsed as a condition. Clearly if the user sees that the parser is putting the block of code to be conditionally executed in where the condition should be, that's a very visual sign of trouble... It's not quite the solution David seems to be looking for, but I think it would address the same problem; is it possible? David LeBlanc wrote: > "Syntax Error: invalid syntax" isn't very informative, yet it's tossed out > frequently. HOW is the syntax invalid? For example: > if s == ' ' > > David LeBlanc > Seattle, WA USA > > > >

2 1

yield equivalent in C/JavaScript?
by Robin Becker June 30, 2002

June 30, 2002

Is there any way to do yielding easily in C? I have a bunch of programs which are normally connected together using unix pipes. I would like to merge them and the problem is then how to get them to communicate nicely using a single process. the typical process currently looks like initialise(); while((c=getc(stdin))>=0){ process(c); putc(c,stdout); } so if the inputs are reasonable I could just run each to completion before proceeding with the next by replacing the file streams with memory ones. That could get messy if the inputs are large. Is there a neat way to simulate the yield idea? In C I can just about consider doing some longjump madness, but that wont wash in the other possible environment which has JavaScript for implementation. -- Robin Becker

5 13

Re: yield equivalent in C/JavaScript?
by bokr＠oz.net June 30, 2002

June 30, 2002

On Sun, 30 Jun 2002 14:01:39 -0400, Oren Tirosh <oren-py-l(a)hishome.net> wrote: >On Sat, Jun 29, 2002 at 07:12:46PM +0100, Robin Becker wrote: >> Is there any way to do yielding easily in C? I have a bunch of programs >> which are normally connected together using unix pipes. I would like to >> merge them and the problem is then how to get them to communicate nicely >> using a single process. > >It's possible to implement this in C using Duff's Device: > >#define BEGIN_GENERATOR static int _GenState=0; \ > switch(_GenState) { \ > case 0: > >#define END_GENERATOR } \ > _GenState = 0; > >#define YIELD(x) _GenState=__line__; return (x); case __line__: > > >int generator(arguments) { > > static int foo, bar, etc; /* Make all variables static */ > > BEGIN_GENERATOR > for(...) { > if(...) { > ... > YIELD(x) > ... > } else { > ... > } > ... > YIELD(y) > ... > } > END_GENERATOR >} > >If you want this to be reentrant use a struct passed as an argument >to the function instead of static variables. > That's cool, but I'm wondering how generally BEGIN_GENERATOR's switch/case hopping into the inside of the for loop like the above will work. Mind-bogglingly, it seems the switch/case can slice just about orthogonally through loop code. Normally jumping into loops is frowned upon, I thought, but it seems like case is practically like a label for a goto with the minor restriction of being inside a switch(){...}, which your BEGIN_/END_GENERATOR guarantees. I never explored this particular arena of possible abuses much ;-) Anyway, you can't generate case numbers from line numbers without some risk of collision: E.g., looking at the preprocessor output (which doesn't mind your ellipses ;-) and putting a contrived switch around the second YIELD like -- switch(y){ case 28: YIELD(y); printf("YIELD will make this case 29 here"); break; case 29: printf("So this won't compile"); break; } -- in place of the plain YIELD(y) line, we get: -- int generator(arguments) { static int foo, bar, etc; /* Make all variables static */ static int _GenState=0; switch(_GenState) { case 0: for(...) { if(...) { ... _GenState=21; return (x); case 21: ... } else { ... } ... switch(y){ case 28: _GenState=29; return (y); case 29:; printf("YIELD will make this case 29 here"); break; case 29: printf("So this won't compile"); break; } ... } } _GenState = 0; } -- BTW, MSVCC6.0 wants #define __line__ __LINE__ at the top to do the above. And for compiling, it doesn't like case __LINE__: because __LINE__ appears not to be a constant expression to it. At the moment I can't find the directive to get around that. It's happy to compile the pre-processor output though ;-) Regards, Bengt Richter

1 0