🌐 Mailman 3 May 2023 - Python-list - python.org

What to use instead of nntplib?
by Grant Edwards May 16, 2023

May 16, 2023

I got a nice warning today from the inews utility I use daily: DeprecationWarning: 'nntplib' is deprecated and slated for removal in Python 3.13 What should I use in place of nntplib? -- Grant

6 8

PythonPath / sys.path
by Grizzy Adams May 15, 2023

May 15, 2023

Hi All My first post (repeated) I am having a problem with PythonPath / sys.path I have a dir where I keep all my current work, but I can't seem to add it to PythonPath / sys.path When I try to import one of my modules I see >>>import My_Working_File Traceback (most recent call last): File "<pyshell#0>", line 1, in <module> import My_Working_File ImportError: No module named 'My_Working_File' I have tried adding my dir in registry to the existing PythonPath [HKEY_CURRENT_USER\Software\Python\PythonCore\3.4\PythonPath] @="D:\\Shades\\Tools\\Python\\Lib;D:\\Shades\\Tools\\Python\\DLLs" that did not help, I tried adding PYTHONPATH as an enviramental variable, that also had no effect (I did re-boot after these two tests) If I go to the editor window and run (F5) any of my modules then I can >>> ================================ RESTART ================================ >>> >>> import My_Working_File >>> without error I can also (from a fresh start) >>> import sys >>> sys.path.append(r'D:\Shades\Tools\Python\MyWork') >>> print(sys.path) ['', 'D:\\Shades\\Tools\\Python\\Lib\\idlelib', 'D:\\Shades\\Tools\\Python\\python34.zip', 'D:\\Shades\\Tools\\Python\\DLLs', 'D:\\Shades\\Tools\\Python\\lib', 'D:\\Shades\\Tools\\Python', 'D:\\Shades\\Tools\\Python\\lib\\site-packages', 'D:\\Shades\\Tools\\Python\\MyWork'] >>> import My_Working_File >>> I don't really want to go this route every time I start work, and more to the point I will need (to learn how) to add dir's to the path at times I did search the net for an answer but only got what I had already tried any pointers what I'm doing wrong (other than using windows '->) Grizz

6 13

Multithreading? How?
by pozz May 13, 2023

May 13, 2023

I need to develop a Python application that is a sort of gateway between to networks: a "serial bus" network (think of a serial port or a USB connection) and a MQTT connection (in my case, it's AWS IoT service). On the bus, a proprietary protocol is implemented. From the bus, the app knows the status of the system (think of a simple ON/OFF status). The user can retrieve the status of the system through MQTT: it sends a message to read/status MQTT topic and receive back a message with the current status on read/status/reply MQTT topic. Of course, they are just examples. On the contrary, when the app detects a status change reported from the serial bus (think of a transition from ON to OFF), it sends a MQTT message. I'm thinking to split the application in three classes: Bus, App and IoT. Bus and IoT are also threads. The thread of Bus manages the serial protocol, while the thread of IoT manages MQTT connection with the broker (the server). However I don't know if it could be a good architecture. Suppone Bus thread receives a new status from the system. In the context of ThreadBus, the object Bus could call a method of App object: app.set_status(new_status_from_the_bus) In the App I have: class App(): .. def set_status(new_status): # Could be called from ThreadBus if new_status != self.new_status: self.new_status = new_status # Do some actions on status change def get_status(): # Could be called from ThreadIoT return self.status Of course, IoT object needs to know the current status of the system when a message is received from MQTT. So ThreadIoT could call app.get_status(). I think this architecture has some problems with race conditions or threads synchronization. What happens if IoT calls get_status() exactly when set_status() called by ThreadBus is executing? If status is a big data structure, set_status() could be interrupted by get_status() that could get a completely corrupted status, because it was only partly updated by set_status(). I know I can use locks or semaphores in get_status() and set_status(), but I don't know if this is a good approach. Consider that the system is complex, it isn't composed by a simple single status. It has many many parameters that are collected from the serial bus. Should I use a lock for every [get|set]_status(), [get|set]_dimensions(), [get|set]_battery_level(), [get|set]_mains_present(), and so on? Another possibility is to use a Queue for Bus and a Queue for IoT. So the set_status(new_status) called from Bus object will be transformed in a put in the queue: app_queue.put({"type": "new_status", "data": ...}) However how could be transformed the get_status() from IoT? How the return value (the current status) is real retrieved? class IoT(): .. def status_request_from_MQTT(): app_queue.put({"type": "get_status"}) # How to get the status to return? return current_status Should the app put the status on the same queue and should IoT waits for a new message in the Queue? def status_request_from_MQTT(): app_queue.put({"type": "get_status"}) try: current_status = app_queue.get(timeout=10) except Empty: # What to do? return current_status Again another approach is to avoid multi-threading at all and create a single "main loop" function that waits at the same time for incoming events on the serial bus and MQTT (how?). I don't know if this could be done in my case, because I'm using awscrt Python module and it works through callbacks that I think is called from another thread. Any suggestions on this architecture?

2 1

Re: Do subprocess.PIPE and subprocess.STDOUT sametime
by Horst Koiner May 12, 2023

May 12, 2023

Using asyncio for this is a good possibility I was not aware of. My best try with asyncio was: import asyncio async def run_command(): # Create subprocess process = await asyncio.create_subprocess_exec( './test.sh', stdout=asyncio.subprocess.PIPE, # Redirect stdout to a pipe stderr=asyncio.subprocess.PIPE # Redirect stderr to a pipe ) # Read stdout and stderr asynchronously captured_output = b'' async for line in process.stdout: print(line.decode().strip()) captured_output += line async for line in process.stderr: print(line.decode().strip()) captured_output += line await process.wait() print(captured_output) # Run the asyncio event loop asyncio.run(run_command()) ######################################## This fulfills all my requirements. A nice to have would that the captured_output has not to be constructed with += 's but with a final seek(0) and read() of process.stdout. But I didn't find anything how to rewind the stream, that i can read the whole output again. Another question is, if this solution is deadlock proof. Thank you all for the already very valuable input! Greetings, Horst

2 1

pip module not found
by David John May 12, 2023

May 12, 2023

Hi, I recently have been experiencing issues with the pip installation module. I have python version 3.11 installed. I've checked the directory installed in the systems variables window and nothing is amiss. Kindly assist. Regards, David

4 6

Help on ctypes.POINTER for Python array
by Jason Qian May 11, 2023

May 11, 2023

Hi, Need some help, in the Python, I have a array of string var_array=["Opt1=DG","Opt1=DG2"] I need to call c library and pass var_array as parameter In the argtypes, how do I set up ctypes.POINTER(???) for var_array? func.argtypes=[ctypes.c_void_p,ctypes.c_int, ctypes.POINTER(????)] In the c code: int func (void* obj, int index, char** opt) Thanks Jason

3 3

Do subprocess.PIPE and subprocess.STDOUT sametime
by Horst Koiner May 10, 2023

May 10, 2023

Hi @all, i'm running a program which is still in development with subprocess.run (Python version 3.10), further i need to capture the output of the program in a python variable. The program itself runs about 2 minutes, but it can also freeze in case of new bugs. For production i run the program with stdout=subprocess.PIPE and i can fetch than the output later. For just testing if the program works, i run with stdout=subprocess.STDOUT and I see all program output on the console, but my program afterwards crashes since there is nothing captured in the python variable. So I think I need to have the functionality of subprocess.PIPE and subprcess.STDOUT sametime. What I tried until now: 1. Poll the the output and use Popen instead: # Start the subprocess process = subprocess.Popen(['./test.sh'], stdout=subprocess.PIPE, stderr=subprocess.PIPE) captured_output = b'' process_running = True while process_running: process_running = (process.poll() is not None) for pipe in [ process.stdout, process.stderr ]: while line := pipe.readline(): print(line) captured_output += line print(captured_output) return_code = process.returncode => But this is discouraged by the python doc, since it says that polling this way is prone to deadlocks. Instead it proposes the use of the communicate() function. 2. Use communicate() with timeout. => This works not at all since when the timeout occurs an exception is thrown and communicate returns at all. 3. Use threading instead => For being that simple and universal like subprocess you will more or less reimplement subprocess with threading, like its done in subprocess.py. Just for a debug output the effort is much to high. ####################################################### Do you have further ideas for implementing such a behavior? Do you think that a feature request should be done of I'm omitting something obvious? Thanks you in advance for your suggestions, Horst.

6 6

Black boxes
by Ondřej Jůn May 10, 2023

May 10, 2023

Good day. After installing the Python program from your site and restarting the computer as requested by the program, problems occurred. Programs like Word are all in black. In addition, a black color appears in the file explorer which sometimes completely covers everything (the black color). This problem also appears everywhere in the "boxes where text is entered" throughout the computer. I tried uninstalling and reinstalling the program but the problem remained. I also tried to repair the program via the button in the program. I didn't get to the modifications part of the program because when I clicked on black color appeared. Python version: Python 3.11.3 (64-bit) Setup The computer I use: ASUS Vivobook Operating System: Windows 11 Home Edition Version 22H2 Operating system build 22621.1555 Thank you in advance for your reply Ondřej

3 2

Re: Problem with accented characters in mailbox.Maildir()
by jak May 9, 2023

May 9, 2023

Peter J. Holzer ha scritto: > On 2023-05-06 16:27:04 +0200, jak wrote: >> Chris Green ha scritto: >>> Chris Green <cl(a)isbd.net> wrote: >>>> A bit more information, msg.get("subject", "unknown") does return a >>>> string, as follows:- >>>> >>>> Subject: =?utf-8?Q?aka_Marne_=C3=A0_la_Sa=C3=B4ne_(Waterways_Continental_Europe)?= > [...] >>> ... and of course I now see the issue! The Subject: with utf-8 >>> characters in it gets spaces changed to underscores. So searching for >>> '(Waterways Continental Europe)' fails. >>> >>> I'll either need to test for both versions of the string or I'll need >>> to change underscores to spaces in the Subject: returned by msg.get(). > > You need to decode the Subject properly. Unfortunately the Python email > module doesn't do that for you automatically. But it does provide the > necessary tools. Don't roll your own unless you've read and understood > the relevant RFCs. > >> >> This is probably what you need: >> >> import email.header >> >> raw_subj = >> '=?utf-8?Q?aka_Marne_=C3=A0_la_Sa=C3=B4ne_(Waterways_Continental_Europe)?=' >> >> subj = email.header.decode_header(raw_subj)[0] >> >> subj[0].decode(subj[1]) >> >> 'aka Marne à la Saône (Waterways Continental Europe)' > > You are an the right track, but that works only because the example > exists only of a single encoded word. This is not always the case (and > indeed not what the RFC recommends). > > email.header.decode_header returns a *list* of chunks and you have to > process and concatenate all of them. > > Here is a snippet from a mail to html converter I wrote a few years ago: > > def decode_rfc2047(s): > if s is None: > return None > r = "" > for chunk in email.header.decode_header(s): > if chunk[1]: > try: > r += chunk[0].decode(chunk[1]) > except LookupError: > r += chunk[0].decode("windows-1252") > except UnicodeDecodeError: > r += chunk[0].decode("windows-1252") > elif type(chunk[0]) == bytes: > r += chunk[0].decode('us-ascii') > else: > r += chunk[0] > return r > > (this is maybe a bit more forgiving than the OP needs, but I had to deal > with malformed mails) > > I do have to say that Python is extraordinarily clumsy in this regard. > > hp > Thanks for the reply. In fact, I gave that answer because I did not understand what the OP wanted to achieve. In addition, the OP opened a second thread on the similar topic in which I gave a more correct answer (subject: "What do these '=?utf-8?' sequences mean in python?", date: "Sat, 6 May 2023 14:50:40 UTC"). I was interested in this thread because a few years ago I wrote a program in C that sent, via email, the log file of an application in the event that it crashed and I had created the attachment based64, however at the time I did not know of the RFC2047 relating to the subject. In addition, investigating the needs of the OP, I discovered that the MAME is not the only format used to compose the subject. I found an example in a thread of same days ago where the subject contained Arabic text (sender: "Uhrda education <Fatmaelhlwany9(a)gmail.com>", date: "Wed, 03 May 2023 00:18:14 UTC"). This is the raw version of the subject: =?UTF-8?B?2LTZh9in2K/YqSDYo9iu2LXYp9im2Yog2K7Yr9mF2Kkg2LnZhdmE2KfYoSDZhdi52KrZhQ==?= =?UTF-8?B?2K8gI9in2YjZhtmE2KfZitmGINio2LHYs9mI2YUg2YXYrtmB2LbYqSDYrtmE2KfZhCDYtNmH2LEg2YU=?= =?UTF-8?B?2KfZitmIMjAyMyDZhNmE2KfYs9iq2YHYs9in2LEg2YjYp9iq2LMgLyAwMDIwMTAwOTMwNjExMQ==?= As you can see, the penultimate letter of the header is not a 'q' as in the OP message but it is a 'b' and the body of the message is covered according to the base64. This made me think that a library could not delegate to the programmer the burden of managing all these exceptions, then I have further investigated to discover that the library also provides the conversion function beyond that of coding and this makes our labors vain: ---------- from email.header import decode_header, make_header subject = make_header(decode_header( raw_subject ))) ---------- This line of code correctly converts the message of the OP and also the one with the text in Arabic. I greet you with cordiality.

2 1

Python-pickle error
by charles wiewiora May 9, 2023

May 9, 2023

Hello, I am experincing problems with the pickle moducle the folowing code was working before, import pickle number=2 my_pickeld_object=pickle.dumps(number) print("this is my pickled object",{my_pickeld_object},) with open('file.pkl', 'rb') as file: number=pickle.load(file) my_unpickeled_object=pickle.loads(my_pickeld_object) print("this is my unpickeled object",{my_unpickeled_object},) but now i get error Traceback (most recent call last): File "C:\Users\lukwi\Desktop\python\tester2.py", line 5, in <module> with open('file.pkl', 'rb') as file: FileNotFoundError: [Errno 2] No such file or directory: 'file.pkl' im get this problem after this, a .pkl came into my Python script files i though this could be a spare file made from python becauce i was doing this first, import pickle number=2 my_pickeld_object=pickle.dumps(number) print("this is my pickled object",{my_pickeld_object},) with open('file.pkl', 'rb') as file: number=pickle.load(file) so i stupidly deleted the file do you know how to fix this? i reinstalled it but it didn't work this is on widnows and on version 3.11.3 on python thank you

3 2