🌐 Mailman 3 April 1999 - Tutor - python.org

New to this idea (!)
by Good2not＠aol.com May 9, 1999

May 9, 1999

Sir/Madam, I'm fascinated with computer programing, but I have nothing for it to do with it right now! Any ideas/suggestions on what to write while learning? I'm using Win 98 (control that grimace, now...it's alright, I know CPR!), and, as stated before, have nothing to write! Any suggestions would be nice, and remember that I'm a novice! Thanx! Emerson B. Lancaster

3 2

AD:Family Reunion T Shirts & More
by kuppler＠vnet.net April 26, 1999

April 26, 1999

Message sent by: Kuppler Graphics, 32 West Main Street, Maple Shade, New Jersey, 08052, 1-800-810-4330. This list will NOT be sold. All addresses are automatically added to our remove list. Hello. My name is Bill from Kuppler Graphics. We do screenprinting on T Shirts, Sweatshirts, Jackets, Hats, Tote Bags and more! Do you or someone you know have a Family Reunion coming up? Kuppler Graphics would like to provide you with some great looking T Shirts for your Reunion. Kuppler Graphics can also provide you with custom T's and promotional items such as imprinted magnets, keychains, pens, mugs, hats, etc. for your business or any fundraising activity (church, school, business etc.) We also can provide you with quality embroidery. We are a family owned company with over 15 years of experience. All work is done at this location. No middle man. Our prices are great! Click reply to email us or call 1-800-810-4330 for more info Bill Kuppler Graphics

1 0

logging stderr
by Arne Mueller April 19, 1999

April 19, 1999

Hi All, is there a way to write messages normally going to 'sys.stderr' (e.g. pythons error messages) to a file AND to 'sys.stderr' (on unix) without changing all my 'sys.stderr.write('...')' statements of the source files? thanks for help, Arne

2 3

RE: [Tutor] memory management
by Doug Stanfield April 19, 1999

April 19, 1999

Might I respectfully submit that this discussion has exceeded the intent of the 'Tutor List'. I might be wrong but it seems more appropriate to c.l.py or maybe an offline discussion. -Doug- > -----Original Message----- > From: Christian Tismer [mailto:tismer@appliedbiometrics.com] > Sent: Monday, April 19, 1999 6:25 AM > To: David Ascher > Cc: Arne Mueller; tutor > Subject: Re: [Tutor] memory management > > > > > David Ascher wrote: > > > > On Sun, 18 Apr 1999, Christian Tismer wrote: > > > ... > > > and watched my swapfile frow from small to 233 MB. > > > Then I did a del on big. This took even more time than > > > creating it, btw. Touching all the refcounts again seems > > > to be quite a challenge for my harddisk. I have the strong > > > impression that releasing objects should try to do this in > > > reverse order. > > > > Do what in reverse order? > > I believe that trying to release objects in reverse order > might be faster than in-order. At least I'm quite sure that > it will not behave worse. Releasing objects by age is for > sure a good guess. Python cannot know the object's age, > but releasing list elements from back to front might > more often be related to the object's age. > > > > > > Without explicit measure, my guess is it takes > > > 5 times as long to release this chunk. > > > > My guess is that you're seeing OS-specific behavior. > > Well, but there are many who suffer from this OS. > > [most of nice didactic excourse skipped] > > > - Python's memory management is somewhat > object-type-specific -- some > > objects are allocated from buffers which are indeed not > completely > > released when the objects aren't needed -- for example, > frame objects > > (used internally) and integers are taken from > preallocated buffers > > when possible, to speed up the common case of allocation and > > deallocation. You might be seeing an effect of that > with your range() > > test. Details are in the code (see e.g. intobject.c and > > frameobject.c). Interning of strings is another > special-case behavior > > which can lead to apparent memory leask which aren't real memory > > leaks. > > Integers are allocated in a table of 1k blocks. They are chained > to each other, forming a freelist. When a new integer is requested, > it is taken from the freelist. Dropped integers are linked into the > freelist, but never freed but at the program end. > This explains the size of my swap file very well. > An integer takes 12 bytes, this makes about 120 MB in my example, > so this is ok. > > Now, assuming you have a machine with 64 MB of memory, > try this in a fresh PyWin session: > > >>> from ptools import timing > >>> def test(amount): > ... global x > ... x=range(amount) > ... > >>> timing(test, 1000000) > (0.49, None) > >>> timing(test, 0) > (0.27, None) > >>> timing(test, 1000000) > (0.28, None) > >>> timing(test, 0) > (0.22, None) > >>> timing(test, 2000000) > (0.72, None) > >>> timing(test, 0) > (0.5, None) > >>> timing(test, 3000000) > (1.32, None) > >>> timing(test, 0) > (0.77, None) > >>> timing(test, 4000000) > (10.0, None) > >>> timing(test, 0) > (141.65, None) > >>> timing(test, 3000000) > (4.17, None) > >>> timing(test, 0) > (0.71, None) > >>> timing(test, 4000000) > (43.34, None) > >>> timing(test, 0) > (107.6, None) > >>> > > As we can see, everything works fine upto 3 million numbers. > 4 million is really bad, especially for deallocation. > A few calculations: > In the 3 million case, the array created by range is 4 * 3 million > = 12 MB, the data for the integers is 12 * 3 million = 36 MB, > together this is 48 MB which fits into memory after swapping > most of Win98 to the disk, ok. > > With 4 million elements, we reach the 64 MB limit, and the system > goes nuts. Everything is at least 10 times slower, releasing > seems to be at least 20 times slower. So what happens? > > First, the list array is created in memory and filled with NULL. > Then, all the integer objects are created and inserted into > the array. Since the system cannot keep everything in memory, > and since every place in the array and the integer freelist > blocks will be touched once now, the whole memory will be written > out in (small?) chunks in order to make room for the next chunk. > So far ok, this seems to be impossible to avoid. > > On deallocation, the situation seems to be similar. > The list array is read in order, and the numbers refcounts > are decremented. This results in freeing most of the numbers, > but instead of returning them into the global memory pool, > they are inserted into the freelist. > This means, the freed integer object is written with the current > freelist head pointer. This will again pour the whole memory > through the disk, but doesn't explain the time behavior. > > I just tested it with my 16 MB Linux machine, and no, it is not > OS specific. Instead on timing with a function, I did > > >>> x=range(1000000) > >>> x=0 > >>> > > and counted cursor blinks of my telnet shell. > Creating the range took 12 blinks, destroying took 37 blinks. > > But I think I should move this topic to the main list. > > ciao - chris > > -- > Christian Tismer :^) > <mailto:tismer@appliedbiometrics.com> > Applied Biometrics GmbH > : Have a break! Take a ride on Python's > Kaiserin-Augusta-Allee 101 : *Starship* > http://starship.python.net > 10553 Berlin : > PGP key -> http://wwwkeys.pgp.net > PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 > 93E2 1FAE F6DF > we're tired of banana software - shipped green, ripens at home > > _______________________________________________ > Tutor maillist - Tutor(a)python.org > http://www.python.org/mailman/listinfo/tutor >

1 0

memory management
by Arne Mueller April 19, 1999

April 19, 1999

Hi All, I've a very large data structure in memory (100 MB) which I've to delete sequential. THe structure is a tree. it looks like this: (root(list1(dictionary(list2)))) I can traverse the tree using the following loops for i in root.list1: for k in keys(): for l in i.hits[l].list2: action(l) l.remove del i.dictionary.[k] i.remove After action(l) is executed l can be deleted, after all keys in i.dictionary are processed the dictionary can be deleted ... . It's important to delete the structure level by level because 'action' creates another very big structure, so building the new one and deleting 'root' at one is not a good idea. I've monitore memory usage on my computer and after executing the three above loops the memory usage should go down - but it doesn't! Whats' up? Is the memory collected for internal usage by python, would it be reused when creating new objects, or is the above way to free lists and dictionaries of objects wrong? good old malloc -> free thanks very much, Arne

3 7

Teaching Objects
by John Kleinjans April 19, 1999

April 19, 1999

Re: teaching programming (to people who don't know programming) (at all) using Python. So let's say, just to set the scene, that someone who has *never programmed before at all* has now messed around with Python a bit in interactive mode; and then has written a couple short scripts to, maybe, solve quadratic equations or make a calculator, something like that. We told this person about variables and assignments, about sequence and branching and loops... so this person is just *beginning* to get a feeling for (I can't think of the word now... imperative? Like a language like C, or QBasic does.) programming. I'd think that what this person needs is a sequence of some `packages' that introduce a tool (command, function) with an example short, simple application program, and a couple simple projects that use that tool; then another tool, another sample program, and a few more related projects; and so on. And along the way, when appropriate, include objects in the sample code. Explain *why* to make objects and *how* to do it, too; what they are and when to use them; and begin some discussion of programming style (start good habits now). Anybody out there have some cool easy Python tricks? Anybody want to explain Who What When Where Why How of objects in Python? (This is a change: instead of asking others to to my homework for me, it ooks like I'm asking them to give me homework! What would they think in comp.lang.c?) If I remember correctly... john k

2 1

Information
by Dave Brener April 19, 1999

April 19, 1999

I am new to the Python programming language, and have no experience in programming at all. Where should I begin? Are there any books out there to help me? Thanks, davebrener(a)yahoo.com _________________________________________________________ Do You Yahoo!? Get your free @yahoo.com address at http://mail.yahoo.com

3 2

Re: [Tutor] memory management
by Arne Mueller April 18, 1999

April 18, 1999

Dear All, David Ascher wrote: > > On Fri, 16 Apr 1999, Arne Mueller wrote: > > > When posting code, please post working code, not pseudocode -- Python is > so simple and clear, there's no need to make up a new syntax. Sorry for that, yes it was quick & dirty posting and it's not correct, I appoligize for that! > > Christian's comment about rethinking the code is a valid one -- it might > save you time to think a bit about whether you really need to have all 100 > Mb in memory at the same time, or whether you can pipeline the process > somewhat. Without knowing more about the specifics, it's hard to know how > to help. Hm, 100 MB memory usage is not a big problem, 200 MB is (in my case)! However, in I summary: I've to read in data. Whether a dataset is read into memory depends on what is already in the datastructure, it may replace an existing dataset or not, so I've to compare each new dataset to all existing datasets. After reading in the data, a sorting is applied to the structure, the sorted datastructure is written to a file. greetings, Arne

2 1

A special message for tutor
by psychic777＠bigfoot.com April 17, 1999

April 17, 1999

2 1

Classes + inheritance
by Arne Mueller April 17, 1999

April 17, 1999

Hi All (again ...), first of all: thanks very much with help on 'long statements' and now to something completely different ...: I've written a modul 'blast.py' which defines some classes (e.g. Blast, Iteration). A program test.py uses this modul import sys from blast import * and defines it's own version of class 'iteration' (in file test.py) class Iteration(Iteration): def __init__(self, f): sys.stderr.write('My redefined class Iteration ...') Iteration.__init__(self, f) def parseBlock(self, token): sys.stderr.write('My redefined parseBlock ...') return I create an obejct of class Blast f = IO('myfile', 'r', 100) # create an io object b = Blast(f) # create an empty blast object b.createBlastTree() # do something on it ... and call a method of that object (createBlastTree). This method (internally) creates several objects of class 'Iteration' (but class Blast does NOT inherit from class Iteration). I want to change the behaviour of several functions of the 'Iteration' class, so in my program I define a new version of class Iteration (see above) - but it turned out that b.createBlastTree() still creates objects of the old Iteration class! Confusing ??? Hm, I'm confused. How can I make b.createBlastTree() using the redefined/extended version of 'Iteration' without modifying the source in modul 'blast.py'. THese extension are program specific, and I intended to use inheritance to cope with the problem - alas, it doesn't seem to work :-( . So, now tell me that I'm completely wrong, and that all my work of the last 3 days cannot work because I misunderstood python's object orientation :-((((( sorry for frequently asking question ;-), Arne

4 5