🌐 Mailman 3 April 2024 - Python-list - python.org

First two bytes of 'stdout' are lost
by Olivier B. April 11, 2024

April 11, 2024

I am trying to use StringIO to capture stdout, in code that looks like this: import sys from io import StringIO old_stdout = sys.stdout sys.stdout = mystdout = StringIO() print( "patate") mystdout.seek(0) sys.stdout = old_stdout print(mystdout.read()) Well, it is not exactly like this, since this works properly This code is actually run from C++ using the C Python API. This worked quite well, so the code was right at some point. But now, two things changed: - Now using python 3.11.7 instead of 3.7.12 - Now using only the python limited C API And it seems that now, mystdout.read() always misses the first two characters that have been written to stdout. My first ideas was something related to the BOM improperly truncated at some point, but i am manipulating UTF-8, so the bom would be 3 bytes, not 2. I ruled out wrong C++ code to extract the string from the python variable, since running a python print of the content of mystdout in the real stdout also misses the two first characters. Hopefully someone has a clue on what would have changed in Python for this to stop working compared to python 3.7?

3 3

Data Ethics (Virtual) Meeting
by dn April 11, 2024

April 11, 2024

Virtual meeting, Wed 17 April, 1800 for 1830 (NZST, ie 0630 UTC) Data Ethics Emma McDonald is the Director of the Interim Centre for Data Ethics and Innovation at Stats NZ (New Zealand Government Department of Statistics) Emma will talk about why Stats NZ is establishing a Centre for Data Ethics and Innovation, and why it needs to be set up as a network that draws on and leverages knowledge and expertise across relevant work programmes and people across agencies. As an initiative, the Centre is there to help agencies develop and maintain secure and trusted data environments. A large part of this is drawing on a diverse network of people who can support the with sharing the importance of data ethics being a critical component of data driven technologies. Will be of-interest to Quants, Data Science, and Machine Learning folk; as well as those of us with wider interest in what should/not happen with personal, public, and corporate data... She will be wanting to hear what folk have to say, and is interested to recruit competent individuals for hui*, consultations, and the like. WebRef: https://data.govt.nz/leadership/the-interim-centre-for-data-ethics-and-inno… from which you can access their Work Programme and Guidance developed to-date. Please RSVP at https://www.meetup.com/nzpug-auckland/events/299764076/ * hui is the Te Reo Maori word for meeting or conference (Te Reo is one of New Zealand's official languages) -- Regards =dn

1 0

[RELEASE] Python 3.12.3 and 3.13.0a6 released
by Thomas Wouters April 10, 2024

April 10, 2024

*It’s time to eclipse the Python 3.11.9 release with two releases*, one of which is the *very last alpha release of Python 3.13*: <https://discuss.python.org/t/python-3-12-3-and-3-13-0a6-released/50601#pyth…>Python 3.12.3 300+ of the finest commits went into this latest maintenance release of the latest Python version, the most stablest, securest, bugfreeest we could make it. https://www.python.org/downloads/release/python-3123/ <https://discuss.python.org/t/python-3-12-3-and-3-13-0a6-released/50601#pyth…>Python 3.13.0a6 What’s that? The last alpha release? Just one more month until feature freeze! Get your features done, get your bugs fixed, let’s get 3.13.0 ready for people to actually use! Until then, let’s test with alpha 6. The highlights of 3.13 you ask? Well: - In the interactive interpreter, exception tracebacks are now colorized by default <https://docs.python.org/dev/whatsnew/3.13.html#improved-error-messages>. - A preliminary, *experimental* JIT was added <https://docs.python.org/dev/whatsnew/3.13.html#experimental-jit-compiler>, providing the ground work for significant performance improvements. - The (cyclic) garbage collector is now incremental <https://docs.python.org/dev/whatsnew/3.13.html#incremental-garbage-collecti…>, which should mean shorter pauses for collection in programs with a lot of objects. - Docstrings now have their leading indentation stripped <https://docs.python.org/dev/whatsnew/3.13.html#other-language-changes>, reducing memory use and the size of .pyc files. (Most tools handling docstrings already strip leading indentation.) - The dbm module <https://docs.python.org/dev/library/dbm.html> has a new dbm.sqlite3 backend <https://docs.python.org/dev/whatsnew/3.13.html#id1> that is used by default when creating new files. - PEP 594 (Removing dead batteries from the standard library) <https://peps.python.org/pep-0594/> scheduled removals of many deprecated modules: aifc, audioop, chunk, cgi, cgitb, crypt, imghdr, mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau, telnetlib, uu, xdrlib, lib2to3. - Many other removals <https://docs.python.org/dev/whatsnew/3.13.html#removed> of deprecated classes, functions and methods in various standard library modules. - New deprecations <https://docs.python.org/dev/whatsnew/3.13.html#deprecated>, most of which are scheduled for removal from Python 3.15 or 3.16. - C API removals <https://docs.python.org/dev/whatsnew/3.13.html#id10> and deprecations <https://docs.python.org/dev/whatsnew/3.13.html#id9>. (Some removals present in alpha 1 were reverted in alpha 2, as the removals were deemed too disruptive at this time.) (Hey, *fellow core developer,* if a feature you find important is missing from this list, let Thomas know <thomas(a)python.org>. It’s getting to be really important now!) https://www.python.org/downloads/release/python-3130a6/ <https://discuss.python.org/t/python-3-12-3-and-3-13-0a6-released/50601#we-h…>We hope you enjoy the new releases! Thanks to all of the many volunteers who help make Python Development and these releases possible! Please consider supporting our efforts by volunteering yourself, or through contributions to the Python Software Foundation <https://www.python.org/psf-landing/> or CPython itself <https://github.com/sponsors/python>. Thomas “can you tell I haven’t had coffee today” Wouters on behalf of your release team, Ned Deily Steve Dower Pablo Galindo Salgado Łukasz Langa -- Thomas Wouters <thomas(a)python.org>

2 1

ModuleNotFoundError: No module named 'Paramiko'
by Wenyong Wei April 9, 2024

April 9, 2024

Dear Sir/Madam, Recently I encounter a problem that I can't import paramiko in my computer. My PC running on window 10 64 bits. I have investigate this issue via internet, there are a lot of solutions for this issue, after trying most of the steps, I still can't run this module, the major steps I have try are: 1. Install python ver 3.7.1 or 3.11.8 by itself or customer installation (changing the installation folder) and check add python to the path. 2. pip install paramiko, if ver 3.7.1 installed, need to upgrade the pip version. 3. Checking the environment path, there are two path related to the python, one for python.exe, the other for \Lib\site-packages\paramiko can you please provide advice on this issue? BR Ken

5 5

Re: ModuleNotFoundError: No module named 'Paramiko'
by Keith Thompson April 8, 2024

April 8, 2024

Thomas Passin <list1(a)tompassin.net> writes: > On 4/8/2024 2:01 PM, Dietmar Schwertberger via Python-list wrote: >> To be sure, you can always go the the directory of the Python >> interpreter and open a cmd window there. >> (By entering 'cmd' into the explorer address bar.) >> Then enter 'python.exe -mpip install paramiko'. >> This way you can be sure that you're not running a pip.exe that >> belongs to another Python interpreter. > > This is not quite right. The best name of the Python executable may or > may not be "python.exe". The command line needs a space after the > "-m": No, the option and its argument can be bundled. "-mpip" is equivalent to "-m pip". (The space might make it clearer for human readers.) [...] -- Keith Thompson (The_Other_Keith) Keith.S.Thompson+u(a)gmail.com Working, but not speaking, for Medtronic void Void(void) { Void(); } /* The recursive call of the void */

2 1

Running issues
by shannon makasale April 6, 2024

April 6, 2024

Hi there, My name is Shannon. I installed Python 3.12 on my laptop a couple months ago, but realised my school requires me to use 3.11.1. I uninstalled 3.12 and installed 3.11.1. Unfortunately, I am unable to run python now. It keeps asking to be modified, repaired or uninstalled. Do you have any suggestions on how to fix this? Any help you can offer is greatly appreciated. Thank you for your time. Hope to hear from you soon. Shannon Makasale

5 4

Re: A technique from a chatbot
by Mark Bourne April 5, 2024

April 5, 2024

Stefan Ram wrote: > Mark Bourne <nntp.mbourne(a)spamgourmet.com> wrote or quoted: >> I don't think there's a tuple being created. If you mean: >> ( word for word in list_ if word[ 0 ]== 'e' ) >> ...that's not creating a tuple. It's a generator expression, which >> generates the next value each time it's called for. If you only ever >> ask for the first item, it only generates that one. > > Yes, that's also how I understand it! > > In the meantime, I wrote code for a microbenchmark, shown below. > > This code, when executed on my computer, shows that the > next+generator approach is a bit faster when compared with > the procedural break approach. But when the order of the two > approaches is being swapped in the loop, then it is shown to > be a bit slower. So let's say, it takes about the same time. There could be some caching going on, meaning whichever is done second comes out a bit faster. > However, I also tested code with an early return (not shown below), > and this was shown to be faster than both code using break and > code using next+generator by a factor of about 1.6, even though > the code with return has the "function call overhead"! To be honest, that's how I'd probably write it - not because of any thought that it might be faster, but just that's it's clearer. And if there's a `do_something_else()` that needs to be called regardless of the whether a word was found, split it into two functions: ``` def first_word_beginning_with_e(target, wordlist): for w in wordlist: if w.startswith(target): return w return '' def find_word_and_do_something_else(target, wordlist): result = first_word_beginning_with_e(target, wordlist) do_something_else() return result ``` > But please be aware that such results depend on the implementation > and version of the Python implementation being used for the benchmark > and also of the details of how exactly the benchmark is written. > > import random > import string > import timeit > > print( 'The following loop may need a few seconds or minutes, ' > 'so please bear with me.' ) > > time_using_break = 0 > time_using_next = 0 > > for repetition in range( 100 ): > for i in range( 100 ): # Yes, this nesting is redundant! > > list_ = \ > [ ''.join \ > ( random.choices \ > ( string.ascii_lowercase, k=random.randint( 1, 30 ))) > for i in range( random.randint( 0, 50 ))] > > start_time = timeit.default_timer() > for word in list_: > if word[ 0 ]== 'e': > word_using_break = word > break > else: > word_using_break = '' > time_using_break += timeit.default_timer() - start_time > > start_time = timeit.default_timer() > word_using_next = \ > next( ( word for word in list_ if word[ 0 ]== 'e' ), '' ) > time_using_next += timeit.default_timer() - start_time > > if word_using_next != word_using_break: > raise Exception( 'word_using_next != word_using_break' ) > > print( f'{time_using_break = }' ) > print( f'{time_using_next = }' ) > print( f'{time_using_next / time_using_break = }' ) >

1 0

Re: A technique from a chatbot
by Mark Bourne April 5, 2024

April 5, 2024

avi.e.gross(a)gmail.com wrote: > That is an excellent point, Mark. Some of the proposed variants to the requested problem, including mine, do indeed find all instances only to return the first. This can use additional time and space but when done, some of the overhead is also gone. What I mean is that a generator you create and invoke once, generally sits around indefinitely in your session unless it leaves your current range or something. It does only a part of the work and must remain suspended and ready to be called again to do more. It goes out of scope at the end of the function. Unless you return it or store a reference to it elsewhere, it will then be deleted. Or in this case, since the `first_word_beginning_with_e` function doesn't even have a local reference to the generator (it is just created and immediately passed as an argument to `next`), it goes out of scope once the `next` function returns. > If you create a generator inside a function and the function returns, presumably it can be garbage-collected. Exactly. It probably doesn't even need to wait for garbage collection - once the reference count is zero, it can be destroyed. > But if it is in the main body, I have to wonder what happen. If you mean in the top-level module scope outside of any function/method, then it would remain in memory until the process exits. > There seem to be several related scenarios to consider. > > - You may want to find, in our example, a first instance. Right afterwards, you want the generator to disassemble anything in use. > - You may want the generator to stick around and later be able to return the next instance. The generator can only really go away when another call has been made after the last available instance and it cannot look for more beyond some end. > - Finally, you can call a generator with the goal of getting all instances such as by asking it to populate a list. In such a case, you may not necessarily want or need to use a generator expression and can use something straightforward and possible cheaper. Yes, so you create and assign it at an appropriate scope. In the example here, it's just passed to `next` and then destroyed. Passing a generator to the `list` constructor (or the `tuple` constructor in my "FWIW") would behave similarly - you'd get the final list/tuple back, but the generator would be destroyed once that call is done. If you assigned it to a function-local variable, it would exist until the end of that function. > What confuses the issue, for me, is that you can make fairly complex calculations in python using various forms of generators that implement a sort of just-in-time approach as generators call other generators which call yet others and so on. Yes, you can. It can be quite useful when used appropriately. > Imagine having folders full of files that each contain a data structure such as a dictionary or set and writing functionality that searches for the first match for a key in any of the dictionaries (or sets or whatever) along the way? Now imagine that dictionary items can be a key value pair that can include the value being a deeper dictionary, perhaps down multiple levels. > > You could get one generator that generates folder names or opens them and another that generates file names and reads in the data structure such as a dictionary and yet another that searches each dictionary and also any internally embedded dictionaries by calling another instance of the same generator as much as needed. You probably could do that. Personally, I probably wouldn't use generators for that, or at least not custom ones - if you're talking about iterating over directories and files on disk, I'd probably just use `os.walk` (which probably is a generator) and iterate over that, opening each file and doing whatever you want with the contents. > You can see how this creates and often consumes generators along the way as needed and in a sense does the minimum amount of work needed to find a first instance. But what might it leave open and taking up resources if not finished in a way that dismantles it? You'd need to make sure any files are closed (`with open(...)` helps with that). If you're opening files within a generator, I'm pretty sure you can do something like: ``` def iter_files(directory): for filename in directory: with open(filename) as f: yield f ``` Then the file will be closed when the iterator leaves the `with` block and moved on to the next item (presumably there's some mechanism for the context manager's `__exit__` to be called if the generator is destroyed without having iterated over the items - the whole point of using `with` is that `__exit__` is guaranteed to be called whatever happens). Other than that, the generators themselves would be destroyed once they go out of scope. If there are no references to a generator left, nothing is going to be able to call `next` (nor anything else) on it, so no need for it to be kept hanging around in memory. > Perhaps worse, imagine doing the search in parallel and as sone as it is found anywhere, ... > > > > -----Original Message----- > From: Python-list <python-list-bounces+avi.e.gross=gmail.com(a)python.org> On Behalf Of Mark Bourne via Python-list > Sent: Thursday, April 4, 2024 3:04 PM > To: python-list(a)python.org > Subject: Re: A technique from a chatbot > > Thomas Passin wrote: >> On 4/2/2024 1:47 PM, Piergiorgio Sartor via Python-list wrote: >>> On 02/04/2024 19.18, Stefan Ram wrote: >>>> Some people can't believe it when I say that chatbots improve >>>> my programming productivity. So, here's a technique I learned >>>> from a chatbot! >>>> It is a structured "break". "Break" still is a kind of jump, >>>> you know? >>>> So, what's a function to return the first word beginning with >>>> an "e" in a given list, like for example >>>> [ 'delta', 'epsilon', 'zeta', 'eta', 'theta' ] >>>> >>>> ? Well it's >>>> def first_word_beginning_with_e( list_ ): >>>> for word in list_: >>>> if word[ 0 ]== 'e': return word >>>> >>>> . "return" still can be considered a kind of "goto" statement. >>>> It can lead to errors: >>>> >>>> def first_word_beginning_with_e( list_ ): >>>> for word in list_: >>>> if word[ 0 ]== 'e': return word >>>> something_to_be_done_at_the_end_of_this_function() >>>> The call sometimes will not be executed here! >>>> So, "return" is similar to "break" in that regard. >>>> But in Python we can write: >>>> def first_word_beginning_with_e( list_ ): >>>> return next( ( word for word in list_ if word[ 0 ]== 'e' ), None ) >>> >>> Doesn't look a smart advice. >>> >>>> . No jumps anymore, yet the loop is aborted on the first hit >> >> It's worse than "not a smart advice". This code constructs an >> unnecessary tuple, then picks out its first element and returns that. > > I don't think there's a tuple being created. If you mean: > ( word for word in list_ if word[ 0 ]== 'e' ) > > ...that's not creating a tuple. It's a generator expression, which > generates the next value each time it's called for. If you only ever > ask for the first item, it only generates that one. > > When I first came across them, I did find it a bit odd that generator > expressions look like the tuple equivalent of list/dictionary > comprehensions. > > FWIW, if you actually wanted a tuple from that expression, you'd need to > pass the generator to tuple's constructor: > tuple(word for word in list_ if word[0] == 'e') > (You don't need to include an extra set of brackets when passing a > generator a the only argument to a function). >

1 0

Re: A technique from a chatbot
by Mark Bourne April 4, 2024

April 4, 2024

Thomas Passin wrote: > On 4/2/2024 1:47 PM, Piergiorgio Sartor via Python-list wrote: >> On 02/04/2024 19.18, Stefan Ram wrote: >>> Some people can't believe it when I say that chatbots improve >>> my programming productivity. So, here's a technique I learned >>> from a chatbot! >>> It is a structured "break". "Break" still is a kind of jump, >>> you know? >>> So, what's a function to return the first word beginning with >>> an "e" in a given list, like for example >>> [ 'delta', 'epsilon', 'zeta', 'eta', 'theta' ] >>> >>> ? Well it's >>> def first_word_beginning_with_e( list_ ): >>> for word in list_: >>> if word[ 0 ]== 'e': return word >>> >>> . "return" still can be considered a kind of "goto" statement. >>> It can lead to errors: >>> >>> def first_word_beginning_with_e( list_ ): >>> for word in list_: >>> if word[ 0 ]== 'e': return word >>> something_to_be_done_at_the_end_of_this_function() >>> The call sometimes will not be executed here! >>> So, "return" is similar to "break" in that regard. >>> But in Python we can write: >>> def first_word_beginning_with_e( list_ ): >>> return next( ( word for word in list_ if word[ 0 ]== 'e' ), None ) >> >> Doesn't look a smart advice. >> >>> . No jumps anymore, yet the loop is aborted on the first hit > > It's worse than "not a smart advice". This code constructs an > unnecessary tuple, then picks out its first element and returns that. I don't think there's a tuple being created. If you mean: ( word for word in list_ if word[ 0 ]== 'e' ) ...that's not creating a tuple. It's a generator expression, which generates the next value each time it's called for. If you only ever ask for the first item, it only generates that one. When I first came across them, I did find it a bit odd that generator expressions look like the tuple equivalent of list/dictionary comprehensions. FWIW, if you actually wanted a tuple from that expression, you'd need to pass the generator to tuple's constructor: tuple(word for word in list_ if word[0] == 'e') (You don't need to include an extra set of brackets when passing a generator a the only argument to a function). -- Mark.

3 2

Trying to use pyinstaller under python 3.11, and, recently started receiving error message about specific module/distribution
by Jacob Kruger April 4, 2024

April 4, 2024

This started happening this past week, and, while it's worked fine in the past, the moment I try to launch the pyinstaller process at all, to generate compiled output, or even if just launch it with no command line options, I receive the following error message: pkg_resources.DistributionNotFound: The 'altgraph' distribution was not found and is required by the application The full contents of the output string when I even try to just launch pyinstaller with no commands/arguments is the following: Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "C:\pythonScripts\monitoring_nssm\venv\Scripts\pyinstaller.exe\__main__.py", line 7, in <module> File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\__main__.py", line 228, in _console_script_run run() File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\__main__.py", line 170, in run parser = generate_parser() ^^^^^^^^^^^^^^^^^ File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\__main__.py", line 136, in generate_parser import PyInstaller.building.build_main File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\building\build_main.py", line 28, in <module> from PyInstaller.building.api import COLLECT, EXE, MERGE, PYZ File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\building\api.py", line 32, in <module> from PyInstaller.building.splash import Splash # argument type validation in EXE ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\building\splash.py", line 23, in <module> from PyInstaller.depend import bindepend File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\depend\bindepend.py", line 25, in <module> from PyInstaller.depend import dylib, utils File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\depend\utils.py", line 31, in <module> from PyInstaller.lib.modulegraph import modulegraph File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\PyInstaller\lib\modulegraph\modulegraph.py", line 34, in <module> from altgraph.ObjectGraph import ObjectGraph File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\altgraph\__init__.py", line 144, in <module> __version__ = pkg_resources.require("altgraph")[0].version ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\pkg_resources\__init__.py", line 952, in require needed = self.resolve(parse_requirements(requirements)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\pkg_resources\__init__.py", line 813, in resolve dist = self._resolve_dist( ^^^^^^^^^^^^^^^^^^^ File "C:\pythonScripts\monitoring_nssm\venv\Lib\site-packages\pkg_resources\__init__.py", line 854, in _resolve_dist raise DistributionNotFound(req, requirers) pkg_resources.DistributionNotFound: The 'altgraph' distribution was not found and is required by the application # ---end of output--- I have tried completely removing python's installation, and, reinstalling it, but, same issue more or less immediately. If I freeze pip's installed list within this specific virtual environment, it lists the following: altgraph==0.17.4 packaging==24.0 pefile==2023.2.7 pyinstaller==6.5.0 pyinstaller-hooks-contrib==2024.3 pywin32-ctypes==0.2.2 # ---end of requirements.txt--- And, if, just for testing, I launch python interpreter, and, ask it to import altgraph, it provides the same last line of error output? If relevant, running with python 3.11.8, under windows 11 64-bit, and, can't think of anything that specifically occurred/changed this past week, besides normal things like windows updates, etc., but, don't really think that's likely to be relevant, unless something to do with pywin32 has caused an issue? Should I try installing python 3.12 version instead and see if it changes? Also, if relevant, while running under latest up-to-date version of windows 11 64 bit, have in any case enabled case-sensitivity on the folder I store all my python code in, just in case. TIA -- Jacob Kruger +2782 413 4791 "Resistance is futile!...Acceptance is versatile..."

2 5