🌐 Mailman 3 October 2020 - Python-list - python.org

Re: Embedding version in command-line program
by Loris Bennett Oct. 20, 2020

Oct. 20, 2020

Tony Flury <tony.flury(a)btinternet.com> writes: > On 07/10/2020 12:06, Loris Bennett wrote: >> Hi, >> >> I have written a program, which I can run on the command-line thus >> >> mypyprog --version >> >> and the get the version, which is currently derived from a variable in >> the main module file. >> >> However, I also have the version in an __init__.py file and in a >> pyproject.toml (as I'm using poetry, otherwise this would be in >> setup.py). >> >> What's the best way of reducing these three places where the version is >> defined to a single one? >> >> Cheers, >> >> Loris > My top level package always has a version.py file which defines __version__, > __author__ etc. I am not sure if that helps in your .toml file though - is it > executed or does it have the ability to read files when it creates the > distributable ? The problem is that the .toml file is not by default part of the distribution, but AFAIU is intended to store the project metadata used to create the METADATA file in dist-info when the package is built. Thus, the version defined in the .toml file is not directly available to the module itself. It seems to me, as a python novice, that poetry's function 'version' needs to be extended. One possibility is that it should not only bump the version in the .toml file, but also bump the version in a version file like 'version.py' as well. Alternatively, the value of the key 'version' in the .toml file, if it is not the version itself, could be path to a file, such as a 'version.py', which holds the version. If that sounds reasonable, I could suggest it to the developers of poetry. Cheers, Loris -- This signature is currently under construction.

1 1

Help with the best practice to learn python
by Azhar Ansari Oct. 20, 2020

Oct. 20, 2020

Hello Python Community, Kindly help me with the best practice to learn python. Lots of material over net but its very confusing. Thanks and Regards, Azhar Ansari

4 3

Simple question - end a raw string with a single backslash ?
by Tony Flury Oct. 19, 2020

Oct. 19, 2020

I am trying to write a simple expression to build a raw string that ends in a single backslash. My understanding is that a raw string should ignore attempts at escaping characters but I get this : >>> a = r'end\' File "<stdin>", line 1 a = r'end\' ^ SyntaxError: EOL while scanning string literal I interpret this as meaning that the \' is actually being interpreted as a literal quote - is that a bug ? If I try to escaped the backslash I get a different problem: >>> a = r'end\\' >>> a 'end\\\\' >>> print(a) end\\ >>> len(a) 5 >>> list(a) ['e', 'n', 'd', '\\', '\\'] So you can see that our string (with the escaped backslash) is now 5 characters with two literal backslash characters The only solution I have found is to do this : >>> a = r'end' + chr(92) >>> a 'end\\' >>> list(a) ['e', 'n', 'd', '\\'] or >>> a = r'end\\'[:-1] >>> list(a) ['e', 'n', 'd', '\\'] Neither of which are nice.

7 8

Are there Python ways to execute queries on PostgreSQL without getting data over?
by Shaozhong SHI Oct. 19, 2020

Oct. 19, 2020

Are there Python ways to execute queries on PostgreSQL without getting data over? Are there ways just to fire off PostgreSQL queries and not get data into Python? Regards, David

3 2

Re: Covariance matrix syntax
by Christian Gollwitzer Oct. 19, 2020

Oct. 19, 2020

Am 19.10.20 um 07:23 schrieb Meghna Karkera: > I am unable to find the *formula for covariance* used in np.cov syntax in > PYTHON given in link > https://numpy.org/doc/stable/reference/generated/numpy.cov.html. Could you > please help me out. As said, you should click on the link [source] just at the right hand side of the page, at the top. There is numpy.cov(m, y=None, rowvar=True, bias=False, ddof=None, fweights=None, aweights=None) and just next to it there is the link. It takes you there: https://github.com/numpy/numpy/blob/v1.19.0/numpy/lib/function_base.py#L227… Which is the source code of the function. Best regards, Christian > Thanks > Meghna > > On Tue, Oct 13, 2020 at 11:46 AM Christian Gollwitzer <auriocus(a)gmx.de> > wrote: > >> Am 13.10.20 um 06:52 schrieb Meghna Karkera: >>> Could you let me know what is the back end calculation of this covariance >>> matrix syntax np.cov >>> >> >> You can look it up yourself: Go to the docs >> https://numpy.org/doc/stable/reference/generated/numpy.cov.html >> >> At the right hand side, just right of the function signature, there is a >> link [source]. Click there and it takes you to the implementation. >> >> Apparently it is done in straightforward Python. >> >> Christian >> >> PS: Snipped a lot of unrelated citation at the bottom >> >>> On Tue, Oct 13, 2020, 10:14 Bruno P. Kinoshita < >> brunodepaulak(a)yahoo.com.br> >>> wrote: >>> [...] >>>> I think the np.cov is from the numpy module (imported/aliased as np?). >> -- >> https://mail.python.org/mailman/listinfo/python-list >>

1 0

Re: Simple question - end a raw string with a single backslash ?
by Mladen Gogala Oct. 19, 2020

Oct. 19, 2020

On Sun, 18 Oct 2020 12:19:18 -0600, Michael Torrie wrote: > Python certainly is procedural. A script starts at the top and executes > through to the bottom and ends, barring any flow control in the middle. > Like Perl you can use it in many different ways and paradigms including > OO if you desire. That's not the point. In Python, regular expressions are a class. So are strings and lists. In Perl, there are no classes. Procedural paradigm means that you are modelling the real world with the objects provided by the computer environment: strings, numbers, lists or hashes (known as "dictionaries" in Python). OO paradigm allows you to create objects and make them behave like in the real world. In other words, in OO paradighm, data and functions are related. The basic idea of the OO was to make code sharing much easier. However, things do not always work as they should: https://medium.com/better-programming/object-oriented-programming-the-trill… https://towardsdatascience.com/object-oriented-programming-is-dead-wait-rea… Of course, not everybody agrees. People never do: https://techbeacon.com/app-dev-testing/object-oriented-programming-dead-not… Sometimes, OO has its funny side: https://www.wearethemighty.com/articles/that-time-the-australian-air-force-… -- Mladen Gogala Database Consultant http://mgogala.byethost5.com

2 1

Re: How to expand and flatten a nested of list of dictionaries of varied lengths?
by dn Oct. 18, 2020

Oct. 18, 2020

If I may, a couple of items of list-etiquette (polite behavior), as I understand them: 1 please reply to the list (cf only myself) because @Mats (who responded earlier) and others on this list are much smarter than me, and might be able to help you more quickly 2 top-posting seems to take the form 'answer, then question' which is illogical to everyone, except apparently Microsoft. It is better to have the conversation 'develop' as it proceeds - all the early information at the beginning, and the more detailed towards the 'end'. That is not to say that we can't "snip" or 'do some gardening', to remove unnecessary or erroneous material, as the conversation progresses. You will notice (as below) that this also enables a posting with multiple questions, to be discussed point-by-point. Now to work... > On Sun, 18 Oct 2020 at 21:48, dn via Python-list <python-list(a)python.org > <mailto:python-list@python.org>> wrote: > > On 19/10/2020 09:09, Shaozhong SHI wrote: > > Even worse is that, in some cases, an addition called > serviceRatings as a > > key occur with new data unexpectedly. > > "Even worse" than what? > > Do you need to keep a list of acceptable/applicable/available keys? > (and reject or deal with others in some alternate fashion) > > > > How to produce a robust Python/Panda script to coping with all these? ... [I often use ellipsis to indicate that I have snipped 'stuff in the middle', others are more overt and will write "<snip>" or similar] > You may find it helpful to use the pprint ("pretty printing" library to > print data-structures in a more readable/structured format). > > To "flatten" a dictionary, you must first be sure that there will be no > keys that will clash (else the second entry will completely replace the > first, without trace). > > Thus, we will need to understand more about this particular definition > of "flatten" in relation to the range of incoming data. Perhaps explain > them in English first... On 19/10/2020 12:14, Shaozhong SHI wrote: > Hi, DN, > > This is the result of pprint. [{u'overall': {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Requires improvement'}, {u'name': u'Well-led', u'rating': u'Requires improvement'}], u'rating': u'Requires improvement'}, u'reportDate': u'2019-10-04', u'reportLinkId': u'63ff05ec-4d31-406e-83de-49a271cfdc43'}, {u'overall': {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Good'}, {u'name': u'Well-led', u'rating': u'Good'}, {u'name': u'Caring', u'rating': u'Good'}, {u'name': u'Responsive', u'rating': u'Good'}, {u'name': u'Effective', u'rating': u'Requires improvement'}], u'rating': u'Good'}, u'reportDate': u'2017-09-08', u'reportLinkId': u'4f20da40-89a4-4c45-a7f9-bfd52b48f286'}, {u'overall': {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Requires improvement'}, {u'name': u'Well-led', u'rating': u'Requires improvement'}, {u'name': u'Caring', u'rating': u'Requires improvement'}, {u'name': u'Responsive', u'rating': u'Requires improvement'}, {u'name': u'Effective', u'rating': u'Good'}], u'rating': u'Requires improvement'}, u'reportDate': u'2016-06-11', u'reportLinkId': u'0cc4226b-401e-4f0f-ba35-062cbadffa8f'}, {u'overall': {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Good'}, {u'name': u'Well-led', u'rating': u'Good'}, {u'name': u'Caring', u'rating': u'Good'}, {u'name': u'Responsive', u'rating': u'Requires improvement'}, {u'name': u'Effective', u'rating': u'Good'}], u'rating': u'Good'}, u'reportDate': u'2015-01-12', u'reportLinkId': u'a11c1e52-ddfd-4cd8-8b56-1b96ac287c96'}] Well done! This looks so much better, and more to the point, it is easier for 'us' to see the structure - but oh dear, doesn't email wrapping make our lives difficult! > Normally, it is like this. > But sometimes, serviceRatings is added to the key list - [u'overall', > u'reportDate', u'reportLinkId'] > > That is what I meant about dynamically growing tree. OK, (and only you/your user can answer this question) why do all the examples (above) not have a service-rating? I am wondering if the use of the word "unexpectedly" has translated accurately between languages - if a data-item is part of the data-input, then our code must be able to handle it or "clean" it, as specified (by the user). - are you able to add a service-rating to each "overall" entry? - where service-ratings are not currently-available, would it be acceptable to add the field with a value of None? (or some other "sentinel-value" - if the analysis-phase does not consider service-ratings, can we write code to read the field from the data-source, but discard it whilst loading everything else into a Pandas matrix? > How best to handle this? This requires understanding how the service-rating value will be used in the analysis, and thus how relevant records may be selected/ignored. Just because it features in the data, doesn't mean it needs to be included in the analysis! Have I understood the question? -- Regards =dn

1 0

How to expand and flatten a nested of list of dictionaries of varied lengths?
by Shaozhong SHI Oct. 18, 2020

Oct. 18, 2020

Even worse is that, in some cases, an addition called serviceRatings as a key occur with new data unexpectedly. How to produce a robust Python/Panda script to coping with all these? Regards, David u'historicRatings': [{u'overall': {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Requires improvement'}, {u'name': u'Well-led', u'rating': u'Requires improvement'}], u'rating': u'Requires improvement'}, u'reportDate': u'2019-10-04', u'reportLinkId': u'63ff05ec-4d31-406e-83de-49a271cfdc43'}, {u'overall': {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Good'}, {u'name': u'Well-led', u'rating': u'Good'}, {u'name': u'Caring', u'rating': u'Good'}, {u'name': u'Responsive', u'rating': u'Good'}, {u'name': u'Effective', u'rating': u'Requires improvement'}], u'rating': u'Good'}, u'reportDate': u'2017-09-08', u'reportLinkId': u'4f20da40-89a4-4c45-a7f9-bfd52b48f286'}, {u'overall': {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Requires improvement'}, {u'name': u'Well-led', u'rating': u'Requires improvement'}, {u'name': u'Caring', u'rating': u'Requires improvement'}, {u'name': u'Responsive', u'rating': u'Requires improvement'}, {u'name': u'Effective', u'rating': u'Good'}], u'rating': u'Requires improvement'}, u'reportDate': u'2016-06-11', u'reportLinkId': u'0cc4226b-401e-4f0f-ba35-062cbadffa8f'}, {u'overall': {u'keyQuestionRatings': [{u'name': u'Safe', u'rating': u'Good'}, {u'name': u'Well-led', u'rating': u'Good'}, {u'name': u'Caring', u'rating': u'Good'}, {u'name': u'Responsive', u'rating': u'Requires improvement'}, {u'name': u'Effective', u'rating': u'Good'}], u'rating': u'Good'}, u'reportDate': u'2015-01-12', u'reportLinkId': u'a11c1e52-ddfd-4cd8-8b56-1b96ac287c96'}]

3 2

Re: File Name issue
by Mladen Gogala Oct. 18, 2020

Oct. 18, 2020

On Sun, 18 Oct 2020 21:00:18 +1300, dn wrote: > On 18/10/2020 12:58, Mladen Gogala via Python-list wrote: >> On Sat, 17 Oct 2020 22:51:11 +0000, Mladen Gogala wrote: >>> On Sat, 17 Oct 2020 18:12:16 -0400, Steve wrote: >>> >>>> with open("HOURLYLOG.txt", 'r') as infile: >>>> works but, when I rename the file, the line: >>>> with open("HOURLY-LOG.txt", 'r') as infile: >>>> does not. The complaint is: Cannot Assign to operator > > >>> with open( "HOURLY-LOG.txt", "r" ) as f: > ... print( f.read() ) > ... > yes > ^^^ the entire contents of a magnificent test-file > > NB: > Python 3.8.5 (default, Aug 12 2020, 00:00:00) > [GCC 10.2.1 20200723 (Red Hat 10.2.1-1)] on linux > > >> BTW, I used this >> cp /var/log/syslog ./in-file.log >> #!/usr/bin/env python3 >> import io >> with open("in-file.log","r") as infile: >> for line in infile: >> print(line) >> I got a different error: >> Traceback (most recent call last): >> File "./test.py", line 4, in <module> >> for line in infile: >> File "/usr/lib/python3.8/codecs.py", line 322, in decode >> (result, consumed) = self._buffer_decode(data, self.errors, final) >> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd8 in position 897: >> invalid continuation byte > > > @Mladen: is syslog a text file or binary format? Hi! Syslog is the system log. It's a text file. This only happens if I use infile as iterable. If I use readline, all is well: #!/usr/bin/env python3 import io with open("in-file.log","r") as infile: while True: line=infile.readline() if not line: break print(line) I don't particularly like this idiom, but it works. That is probably a bug in the utf-8 decoder on Ubuntu. It doesn't happen on my Fedora 32 VM. I haven't tried with infile.reconfigure(encoding=None) -- Mladen Gogala Database Consultant http://mgogala.byethost5.com

2 1

Re: Simple question - end a raw string with a single backslash ?
by Mladen Gogala Oct. 18, 2020

Oct. 18, 2020

On Sun, 18 Oct 2020 16:13:16 +0200, Peter J. Holzer wrote: > > Ah, I see, that the sillyness of Perl's grammar-altering modules (which > let you write Perl in Latin (with proper declensions and conjugations, > of course) or Chinese) has found its way to Python > To tell the truth, I only installed it to test whether it works. I am an old Perl hack forced to learn Python by the market forces. I learned Perl in 1994 which means that it is my first love of a sort. The fundamental difference between the two languages is that Perl is procedural while Python is a fully OO language. Discussion of Perl vs Python necessarily devolves into the discussion of procedural vs OO paradigms. And that has been decided long time ago, with the advent of Java over COBOL. Market has spoken and I had to learn Python. So, I stopped worrying and learned to love Python. Any resemblance to the subtitle of "Dr. Strangelove" is purely accidental. BTW, Visual Studio Code uses pylint to evaluate my adherence to PEP8 and there is a "kite" extension with autocompletion. Nothing like that was ever available for Perl. It certainly is easier to write Python scripts than to write Perl scripts. I do sort of miss $_, @_ and $!. Python's argparse is supreme and much better than Getopt::Long. -- Mladen Gogala Database Consultant http://mgogala.byethost5.com

2 1