🌐 Mailman 3 February 2020 - Python-list - python.org

Re: Help building python application from source
by Peter Pearson Feb. 29, 2020

Feb. 29, 2020

On Fri, 28 Feb 2020 18:49:58 -0800, Mr. Lee Chiffre wrote: [snip] > I am a python noob. This is why I ask the python masters. There is a > python software I want to install on the server it is called Electrumx. > https://github.com/kyuupichan/electrumx is the link. I am having troubles > with installing this. > The short version is I am wanting to build this python application and > needed dependencies from source code all from a local directory without > relying on the python pip package servers. I only run software I can > compile from source code because this is the only way to trust open source > software. I also want to archive the software I use and be able to install > it on systems in a grid down situation without relying on other servers > such as python package servers. (To make OP's requirements plainly visible, note that this appears to be a cryptocurrency application.) I'd suggest that building everything from source code might not be a realistic solution to your security concerns. I don't know what your threat model is, but if it's something like, "Hackers and gangsters who scatter password-harvesting trojans across the globe and then shlurp up what they can," you might find that you get better security by generating your keys on a computer that never communicates with the outside world. Your concerns are (1) that the random numbers from which your keys have been corrupted to make them predictable, or (2) that malicious software will send your keys to the bad guys. Isolating the key-generation machine takes care of #2. If you have Python code for generating keys, something as simple as XORing a fixed value of your choice with its random numbers will take care of #1. I admit that using an isolated machine introduces a lot of inconveniences, but I bet it compares favorably with building everything from source. -- To email me, substitute nowhere->runbox, invalid->com.

2 1

What is the meaning of Building Wheel for <package name>?
by Souvik Dutta Feb. 29, 2020

Feb. 29, 2020

What is the meaning of the subject I mean what does python internally do when it says this?

2 1

Groovy to Python Converter ?
by Frank Rentmeister Feb. 29, 2020

Feb. 29, 2020

In my new project, I am supposed to bring the current test cases, all written in Groovy, to a Python base. We are talking about several thousand test cases that have accumulated over the last years. Since the test cases are also to be extended towards API gateway testing, and since we work with Python requests here, we want and need to convert everything. But here is the question, which I could not answer even after an intensive search: Is there a converter that can convert Groovy based test cases to Python? An example of how the tests are structured. They are pure web tests that ask for links and other questions with just one click. Actually, it's absolutely simple, but not suitable for us for broad-based tests, even in the direction of expansion.

1 0

Re: Asyncio question (rmlibre)
by rmlibre＠riseup.net Feb. 29, 2020

Feb. 29, 2020

What resources are you trying to conserve? If you want to try conserving time, you shouldn't have to worry about starting too many background tasks. That's because asyncio code was designed to be extremely time efficient at handling large numbers of concurrent async tasks. For your application, it seems starting background tasks that appropriately await execution based on their designated queue is a good idea. This is more time efficient since it takes full advantage of async concurrency, while also allowing you to control the order of execution. Although, there may be other efficiency boosts to be had, for instance, if all but the precise changes that need to be atomic are run concurrently. However, if you want to conserve cpu cycles per unit time, then staggering the processing of requests sequentially is the best option, although, there's little need for async code if this is the case. Or, if you'd like to conserve memory, making the code more generator-based is a good option. Lazy computation is quite efficient on memory and time. Although, rewriting your codebase to run on generators can be a lot of work, and their efficiency won't really be felt unless your code is handling "big data" or very large requests. In any case, you'd probably want to run some benchmark and profiling tools against a mock-up runtime of your code and optimize/experiment only after you've noticed there's an efficiency problem and have deduced its causes. Barring that, it's just guess-work & may just be a waste of time. On 2020-02-21 17:00, python-list-request(a)python.org wrote: > Hi all > I use asyncio in my project, and it works very well without my having to understand what goes on under the hood. It is a multi-user client/server system, and I want it to scale to many concurrent users. I have a situation where I have to decide between two approaches, and I want to choose the least resource-intensive, but I find it hard to reason about which, if either, is better. > > I use HTTP. On the initial connection from a client, I set up a session object, and the session id is passed to the client. All subsequent requests from that client include the session id, and the request is passed to the session object for handling. > > It is possible for a new request to be received from a client before the previous one has been completed, and I want each request to be handled atomically, so each session maintains its own asyncio.Queue(). The main routine gets the session id from the request and 'puts' the request in the appropriate queue. The session object 'gets' from the queue and handles the request. It works well. > > The question is, how to arrange for each session to 'await' its queue. My first attempt was to create a background task for each session which runs for the life-time of the session, and 'awaits' its queue. It works, but I was concerned about having a lot a background tasks active at the same time. > > Then I came up with what I thought was a better idea. On the initial connection, I create the session object, send the response to the client, and then 'await' the method that sets up the session's queue. This also works, and there is no background task involved. However, I then realised that the initial response handler never completes, and will 'await' until the session is closed. > > Is this better, worse, or does it make no difference? If it makes no difference, I will lean towards the first approach, as it is easier to reason about what is going on. > > Thanks for any advice. > > Frank Millman

2 1

Help building python application from source
by Mr. Lee Chiffre Feb. 29, 2020

Feb. 29, 2020

sorry re posting because I forgot subject line in last email. I am a python noob. This is why I ask the python masters. There is a python software I want to install on the server it is called Electrumx. https://github.com/kyuupichan/electrumx is the link. I am having troubles with installing this. The short version is I am wanting to build this python application and needed dependencies from source code all from a local directory without relying on the python pip package servers. I only run software I can compile from source code because this is the only way to trust open source software. I also want to archive the software I use and be able to install it on systems in a grid down situation without relying on other servers such as python package servers. Here is a snippet from https://github.com/lee-chiffre/Announcements/blob/master/02.26.2020 that describes what I am trying to do > 1. I need a way I can download the source of ElectrumX and the dependency tree. 2. In a way that also verifies the integrity of those downloads. I dont think many of these python packages are even signed. 3. Then to be able to build ElectrumX and the needed dependencies from source on a computer that does not have connection to internet. Node2.0 does not connect to internet except for the Tor process. Torsocks is not an option here. I will be running ElectrumX and the python dependencies in a venv virtual environment. If I cannot build ElectrumX and the dependencies for it from source in a way that also verifies the integrity of the downloads then for security reasons I will not run it. If someone has a solution to this I will then run ElectrumX, and give you credit for the help. If you need me to pay bounty for this please reach out to me to negotiate on a price. < Python might be easy to code but simplicity of coding comes at the cost of complexity of the software. With C++ I usually only have only a few dependencies. With python it seems like it is almost 20 dependencies. With Electrumx I counted at least 15 dependencies in the dependency tree. Is it possible to download then build this from source with all needed dependencies? -- lee.chiffre(a)secmail.pro PGP 97F0C3AE985A191DA0556BCAA82529E2025BDE35

1 0

Friday Finking: Poly more thick
by DL Neil Feb. 29, 2020

Feb. 29, 2020

How does one code a function/method signature so that it will accept either a set of key-value pairs, or the same data enclosed as a dict, as part of a general-case and polymorphic solution? Wikipedia: polymorphism is the provision of a single interface to entities of different types. ( https://en.wikipedia.org/wiki/Polymorphism_(computer_science) ) <tldr;> At the end of a code-sprint, one of 'my' teams was presenting. Everything went well. Fatefully, the client-manager then remembered that the latest work would extend a previous sprint's work. Despite it not being a deliverable (?), he asked to be shown the two working together. The team, flushed with pride at their 'success' (foolishly) agreed, and the attempt immediately crashed. (as such live-demos are wont to do!) Wisely, a 'coffee/tea break' was suggested, to give the team time to 'fix' things. Hence a panic-call to me, working from home. It turned-out that a new-ish and less-experienced coder had been given this job, and she had been shown the interface as key-value pairs (her interpretation) - and even on the white-board there were no braces to show it as a dict! Whereas the more experienced programmer who originally assembled the class/signature/interface, had used a dict. Unhappiness was evident, argument threatened... I quickly mocked-up the situation. (code below). The fastest solution, which didn't require altering any calling-code, seemed to be to change the class's signature to use *args and **kwargs, which thereafter required only a single 'normalisation' step to turn any/the kwargs into a dict - which in-turn enabled the existing code, unchanged. Quick and dirty to be sure! However, it allowed the demo to go ahead and recovered everyone's feelings of satisfaction/success! Aside from 'repairing' team spirit, (as regular readers will recognise) I wasn't going to let this pass without some thought and discussion (and perhaps I might learn something about Python interfacing)! </tldr;> Python's powerful polymorphic capabilities [insert expression of thanks here!] allow us to substitute collections - with care. So, a simple function, such as:) def f( colln ): for element in colln: print( element ) will work quite happily when (separately) fed either a tuple or a list. It will also work with a string (a collection of characters) - as long as we are happy with single characters being output. What about dicts as collections? It will work with dicts, as long as we consider the dict's keys to be 'the dict' (cf its collection of values). Well that's very flexible! However, whilst a set of key-value arguments 'quack' somewhat like a dict, or even a tuple (of k-v pairs), they will not be accepted as parameters. Crash! Yes it can be solved, with a 'normalisation' function, as described above, perhaps as a working-theory: def f( *colln_as_tuple, **key_value_pairs_as_dict ): my_dict = key_value_pairs_as_dict if key_value_pairs_as_dict \ else colln_as_tuple[ 0 ] # no change to existing code using my_dict ... However, this seems like a situation that "quacks like a duck", yet the built-in polymorphism doesn't auto-magically extend to cover it. So: - is that the way 'polymorphism' really works/should work? - is expecting the dict to work as a "collection", already 'taking things too far'? - is expecting the key-values to work, really 'taking things too far'? Remember that I am not an OOP-native! The discussion around-the-office diverged into two topics: (1) theoretical: the limits and capability of polymorphism, and (2) practical: the understanding and limits of a Python "collection". How do you see it? How would you solve the Python coding problem - without re-writing both sprints-worth of code? Did we leave polymorphism 'behind' and expect Python to perform magic? ### investigative and prototyping code ### def print_dict( dictionary ): # helper function only for key, value in dictionary.items(): print( "\t", key, value ) def f( **kwargs ): print( kwargs, type( kwargs ) ) print_dict( kwargs ) f( a=1, b=2 ) # arguments as key-value pairs ### {'a': 1, 'b': 2} <class 'dict'> ### a 1 ### b 2 d={ 'a':1, 'b':2 } # arguments as a dictionary try: f( d ) except TypeError: print( "N/A" ) ### N/A ### Traceback (most recent call last): ### File "<stdin>", line 1, in <module> ### TypeError: f() takes 0 positional arguments but 1 was given f( **d ) # yes, easy to edit arguments, but... ### {'a': 1, 'b': 2} <class 'dict'> ### a 1 ### b 2 try: f( { 'a':1, 'b':2 } ) # or modify API to accept single dict except TypeError: print( "N/A" ) ### N/A print( "\nwith g()\n" ) ### with g() def g( *args, **kwargs ): print( args, type( args ), kwargs, type( kwargs ) ) for arg in args: # handle single-objects, eg dict print( arg, type( arg ) ) print_dict( arg ) for k, v in kwargs.items(): # handle key-value pairs print( "\t\t", k, v ) g( a=1, b=2 ) # arguments as key-value pairs ### () <class 'tuple'> {'a': 1, 'b': 2} <class 'dict'> ### a 1 ### b 2 g( d ) # argument as a dictionary ### {'a': 1, 'b': 2},) <class 'tuple'> {} <class 'dict'> ### {'a': 1, 'b': 2} <class 'dict'> ### a 1 ### b 2 -- Regards, =dn

1 0

wxFormBuilder
by Edward Montague Feb. 28, 2020

Feb. 28, 2020

The videos on YouTube by fandangleproductions, have been of some use. However as I also use the Geany editor, I find that I need to append this extra code to the Python script generated from wxFormBuilder. # --------------------- extra code ------------------- class MyApp(wx.App): def OnInit(self): self.frame = frameMain(None) self.SetTopWindow(self.frame) self.frame.Show() return True # end of class MyApp if __name__ == "__main__": app = MyApp(0) app.MainLoop() Where frameMain is the name of this particular frame. This may help you with your Python coding.

1 0

Re: Mental model of lookahead assertions
by Jon Ribbens Feb. 28, 2020

Feb. 28, 2020

On 2020-02-27, Stefan Ram <ram(a)zedat.fu-berlin.de> wrote: > One can count overlapping occurences as follows. > >|>>> print(len(findall('(?=aa)','caaaab'))) >|3 > > Every web page says that lookahead assertions do > not consume nor move the "current position". > > But what mental model can I make of the regex > engine that explains why it is not caught in an > endless loop matching "aa" at the same position > again and again and never advancing to the other > occurences? Simply that when it's returning multiple matches it knows once it's found a match at position <x> to start looking for the next potential match at position <x + 1> ?

1 0

Re: Mental model of lookahead assertions
by Python Feb. 27, 2020

Feb. 27, 2020

Stefan Ram wrote: > One can count overlapping occurences as follows. > > |>>> print(len(findall('(?=aa)','caaaab'))) > |3 > > Every web page says that lookahead assertions do > not consume nor move the "current position". > > But what mental model can I make of the regex > engine that explains why it is not caught in an > endless loop matching "aa" at the same position > again and again and never advancing to the other > occurences? > > (Something with nondeterminism?) Old but EXCELLENT insight into how regular expressions works: https://perl.plover.com/yak/regex/ (pdf and html slides)

1 0

Logging all the requests into a specific file
by valon.januzaj98＠gmail.com Feb. 27, 2020

Feb. 27, 2020

Hello guys, I am new to python and all of this, I am using this FastAPI, to build API, I want when users hit any endpoint for ex /products, that to be written into a file , how do I do it?

2 1