🌐 Mailman 3 May 2020 - Python-list - python.org

How to convert csv to netcdf please help me python users
by kotichowdary28＠gmail.com May 29, 2020

May 29, 2020

Hi all I hope all are doing well please help me how to convert CSV to NetCDF. Im trying but its not working #!/usr/bin/env ipython import pandas as pd import numpy as np import netCDF4 import pandas as pd import xarray as xr stn_precip='ts_sept.csv' orig_precip='ts_sept.csv' stations = pd.read_csv(stn_precip) stncoords = stations.iloc[:] orig = pd.read_csv(orig_precip) lats = stncoords['latitude'] lons = stncoords['longitude'] nstations = np.size(lons) ncout = netCDF4.Dataset('Precip_1910-2018_homomod.nc', 'w') ncout.createDimension('station',nstations) ncout.createDimension('time',orig.shape[0]) lons_out = lons.tolist() lats_out = lats.tolist() time_out = orig.index.tolist() lats = ncout.createVariable('latitude',np.dtype('float32').char,('station',)) lons = ncout.createVariable('longitude',np.dtype('float32').char,('station',)) time = ncout.createVariable('time',np.dtype('float32').char,('time',)) precip = ncout.createVariable('precip',np.dtype('float32').char,('time', 'station')) lats[:] = lats_out lons[:] = lons_out time[:] = time_out precip[:] = orig ncout.close()

1 0

Re: Custom logging function
by zljubisic＠gmail.com May 29, 2020

May 29, 2020

Hi Peter. Finally I got it. :) That's it. It works. Thanks. So, in each class, I will in init method execute: self.logger = logging.getLogger() and than everywhere use self.logger.debug('...') in order to produce the message. Best regards.

1 0

EuroPython 2020: Schedule published
by M.-A. Lemburg May 29, 2020

May 29, 2020

We are very excited to announce the first version of our EuroPython 2020 schedule: * EuroPython 2020 Schedule * https://ep2020.europython.eu/schedule/ More sessions than we ever dreamed of ------------------------------------- After the 2nd CFP, we found that we had so many good talk submissions that we were able to open a fourth track. Together with the newly added slots for the Indian / Asian / Pacific and Americas time zones, we now have a fully packed program, with: - more than 110 sessions - more than 110 speakers from around the world - 4 brilliant keynotes, two of which are already published - 2 exciting lightning talk blocks - 4 all-day tracks, with a whole track dedicated to data science topics - a poster track, which we’ll announce next week - a virtual social event - an after party - and lots of socializing on our conference platform We are proud to have reached almost the size of our in-person event with the online version of EuroPython 2020. Never miss a talk ----------------- All talks will be made available to the attendees as live Webinars, with easy switching between tracks, as well as online streams, which will allow rewinding to watch talks you may have missed during the day. Conference Tickets ------------------ Conference tickets are available on our registration page. We have simplified and greatly reduced the prices for the EuroPython 2020 online edition: https://ep2020.europython.eu/registration/buy-tickets/ As always, all proceeds from the conference will go into our grants budget, which we use to fund financial aid for the next EuroPython edition, special workshops and other European conferences and projects: * EuroPython Society Grants Program * https://www.europython-society.org/grants We hope to see lots of you at the conference in July. Rest assured that we’ll make this a great event again — even within the limitations of running the conference online. Sprints ------- On Saturday and Sunday, we will have sprints/hackathons on a variety of topics. Registration of sprint topics has already started. If you would like to run a sprint, please add your sprint topic to the wiki page we have available for this: * EuroPython 2020 Sprints Listing * https://wiki.python.org/moin/EuroPython2020/Sprints If registrations continue as they currently do, we will have a few hundred people waiting to participate in your sprint projects, so this is the perfect chance for you to promote your project and find new contributors. Participation in the sprints is free, but does require registration. We will provide the necessary collaboration tools in form of dedicated Jitsi or Zoom virtual rooms and text channels on our Discord server. EuroPython is your conference ----------------------------- EuroPython has always been a completely volunteer based effort. The organizers work hundreds of hours to make the event happen and will try very hard to create an inspiring and exciting event. However, we can only provide the setting. You, as our attendees, are the ones who fill it with life and creativity. We are very much looking forward to having you at the conference ! Help spread the word -------------------- Please help us spread this message by sharing it on your social networks as widely as possible. Thank you ! Link to the blog post: https://blog.europython.eu/post/619453140468629504/europython-2020-schedule… Tweet: https://twitter.com/europython/status/1266352353961299971 Thanks, -- EuroPython 2020 Team https://ep2020.europython.eu/ https://www.europython-society.org/

1 0

Re: Is there some reason that recent Windows 3.6 releases don't included executable nor msi installers?
by Adam Preble May 29, 2020

May 29, 2020

On Thursday, May 28, 2020 at 7:57:04 PM UTC-5, Terry Reedy wrote: > The OP is so far choosing to not use an installer with those fixes. By > not doing so, he is missing out on the maybe 2000 non-security fixes and > some enhancements that likely would benefit him more than maybe 50 > mostly obscure fixes added between 3.6.8 and 3.6.10*. If a rare user > such as Adam also chooses to not compile the latter, that is his choice. I was going to just stay mute about why I was even looking at 3.6.10, but I felt I should weigh in after some of the other responses. I think somebody would find the issues interesting. We had found what looked like a bug in the Python Launcher where it would eat command line arguments meant for the script. I would find some stuff missing from sys.argv in a script that just imports sys and prints out sys.argv if I ran it directly in cmd.exe as "script.py." If I ran it as "python script.py" then everything was good as usual. So I figured while sorting out what was wrong that I should try the latest 3.6 interpreter since it would be a safe bet. Our organization finally lifted Sisyphus' rock over the 2.7 hump earlier in the year by moving to 3.6. So imagine my surprise when I found the latest 3.6 releases were just source tarballs. This left me with a dilemma and I'm still working through it. I haven't filed an issue about this because I haven't completed my own due diligence on the problem by trying it on a "latest." For the sake of this particular problem, I think I can just use 3.8.3 for exploration, but I'm worrying about my wider organization. I can't count on 3.8 because of some module dependencies our organization's software. 3.7 has a similar issue. So I figured I'd actually just build the thing and see what I can do. I did manage to build it, but there was surprisingly a few quirks. I caused some of it. For example, I didn't care about most of the externals before, but I made sure to include them if I was create a release for others. A few thousand people would be using this and I'm the one that would be accountable if it went bust. So I made sure all the major externals were incorporated, and a lot of those were messing up. Generally, the externals would download, but some would not get moved/renamed to their final name, and then the build would fail when trying to find them. So I wound up with an installation that seemed to run my own code just fine in trials, but I would be terrified to post into it our organization's software stack. I'm now concerned about how long we have with 3.6 because people clearly want us to move on even beyond that. I look online and the official support window for it ends at the end of next year, but it looks like the real support window for that on Windows has already ended. So our organization may have miscalculated this. What does that mean if we managed to make it to 3.8 in a few months? We can't do it right now due to a few missing modules, but now we have to question if we'll only get a year out of 3.8 before we're doing this all over again.

3 2

Create custom types using typing module?
by Chirag Dhyani May 29, 2020

May 29, 2020

I am trying to create a custom type on Python 3.7 typing module. The new type (say Struct) should be same as type tuple. In Python3.6, I was able to do the same by taking cue from Typing.GenericMeta and typing.TupleMeta. With typing module updated in Python3.7, GenericMeta and TupleMeta do not exist, and the special class that I would like to subclass is not possible. e.g. _VariadicGenericAlias cannot be subclassed. What I really want is something similar to: Struct = _VariadicGenericAlias(tuple, (), , inst=False, special=True) and assert _origin(Struct[int, str]) == <class 'Struct'> Note: def _origin(typ: Any) -> Any: """Get the original (the bare) typing class. Get the unsubscripted version of a type. Supports generic types, Union, Callable, and Tuple. Returns None for unsupported types. Examples:: get_origin(int) == None get_origin(ClassVar[int]) == None get_origin(Generic) == Generic get_origin(Generic[T]) == Generic get_origin(Union[T, int]) == Union get_origin(List[Tuple[T, T]][int]) == list """ if isinstance(typ, _GenericAlias): return typ.__origin__ if typ.__origin__ is not ClassVar else None if typ is Generic: return Generic return None Deeply appreciate your help around this !! Thanks, Chirag

1 0

Re: Write tables from Word (.docx) to Excel (.xlsx) using xlsxwriter
by BBT May 29, 2020

May 29, 2020

On Thursday, 28 May 2020 03:07:48 UTC+8, Peter Otten wrote: > BBT wrote: > > > I tried your code by replacing the Document portion: > > > But I received an error: > > TypeError: __init__() takes 1 positional argument but 2 were given > > We seem to have different ideas of what replacing means. > Here is the suggested script spelt out: > > import xlsxwriter > > from docx.api import Document > document = Document('/Users/xxx/Documents/xxx/Clauses Sample - Copy v1 - for merge.docx') > > wb = xlsxwriter.Workbook('C:/Users/xxx/Documents/xx/test clause retrieval.xlsx') > sheet = wb.add_worksheet("Compliance") > > offset = 0 > for table in document.tables: > for y, row in enumerate(table.rows): > for x, cell in enumerate(row.cells): > sheet.write(y + offset, x, cell.text) > offset += len(table.rows) + 1 # one empty row between tables > > wb.close() Yes! That works well, thank you Peter! And I'm impressed by your object-oriented simulation too. Would look into it again :)

1 0

Re: Behaviour of os.path.join
by BlindAnagram May 28, 2020

May 28, 2020

On 27/05/2020 18:37, Roel Schroeven wrote: > BlindAnagram schreef op 27/05/2020 om 18:39: >> On 27/05/2020 16:49, Rhodri James wrote: >>> On 27/05/2020 16:12, BlindAnagram wrote: >>>> I'm sorry that you don't believe me but all I know is how I intend the >>>> path to be used. And the os.path functions aren't helpful here when >>>> they actually_change_ the meanings of paths on Windows: >>>> >>>>>> fp= "C:\\Documents\finance\\" >>>>>> abspath(fp) >>>> 'C:\\Documents\\finance' >>>> >>>> If you believe these 'before' and 'after' paths are the same I can only >>>> assume that you don't work on Windows (where one refers to a directory >>>> and the other a file without an extension). >>> >>> More accurately, one is not a legal filename but both are legal >>> directory names. >> >> If they are to be created, which is my situation, the result will be a >> diretory and a file. > > os.mkdir('C:\\Documents\\finance') creates a directory. > open('C:\\Documents\\finance', 'w') creates a file. > > The difference is in the operation, not in the name. > > 'C:\\Documents\\finance' is a pathname, which can refer to either a > directory or a file. > 'C:\\Documents\\finance\\' could refer to a directory, but to me looks > more like a partial pathname, not a complete one. > > I can't think of any reason for ending pathnames with (back)slashes. > Just use os.path.join(directory, filename) when you need to refer to a > file in the directory. > >> I would be surprised if issues such as these were not, at least in >> significant part, the reason why we now have pathlib. > > That should be easy to verify: the reasons are listed in the PEP: > https://www.python.org/dev/peps/pep-0428 > I don't see your issue there. I don't think anyone has ever considered > it an issue at all really. > >>> How are these unexpected extensionless files getting created? >> >> I believe by attempting to make the directory I send absolute with >> abspath() and then copying a file to this path. They expected this to >> copy the file into the directory with its original name but instead it >> copies it to the file that abspath 'kindly' converts my directory into. > > We're getting closer to the real issue here. What functions were used > copy these files? With which parameters? > Were the destination directories created before copying the files to them? > >> I did complain about their lack of knowledge but I also have a right to >> complain about a function that converts an explicitly specified >> directory into a file :-) > > Again, a pathname is never inherently a directory or a file. You can define a path however you want but it won't change the fact that on Windows a path that ends in '\\' is inherently a path to a directory.

5 5

Behaviour of os.path.join
by BlindAnagram May 28, 2020

May 28, 2020

I came across an issue that I am wondering whether I should report as an issue. If I have a directory, say: base='C:\\Documents' and I use os.path.join() as follows: join(base, '..\\..\\', 'build', '') I obtain as expected from the documentation: 'C:\\Documents\\..\\..\\build\\' But if I try to make the directory myself (as I tried first): join(base, '..\\..\\', 'build', '\\') I obtain: 'C:\\' The documentation says that an absolute path in the parameter list for join will discard all previous parameters but '\\' is not an absoute path! Moreover, if I use join(base, '..\\..\\', 'build', os.sep) I get the same result. This seems to me to be a bug that I should report but to avoid wasting developer time I wanted to hear what others feel about this.

8 9

Constructing mime image attachment
by Joseph L. Casale May 28, 2020

May 28, 2020

I have some json encoded input for nodemailer (https://nodemailer.com/message/embedded-images) where the path key is a string value which contains the base64 encoded data such as: { html: 'Embedded image: <img src="cid:unique@nodemailer.com"/>', attachments: [{ filename: 'image.png', path: 'data:image/png;base64,iVBORw....', cid: 'unique(a)nodemailer.com' }] } Does an approach exist to construct a MIMEImage object with the existing base64 string without parsing it and decoding it? Thanks, jlc

2 1

Ram memory not freed after executing python script on ubuntu system
by Rahul Gupta May 28, 2020

May 28, 2020

I am having a Ubuntu system which has 125 Gb of RAM. I executed few python scripts on that system. Those scripts uses numpy arrays and pandas. Now execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap is occupied. At this moment nothing is running on the system. I have googled it. Most of th result shows that python garbage collector is poor in performance. I want this memory to be cleaned and re claim. One of the easiest way is to restart the system but i dont want to restart i want a way to do this when the system is up and running. Kindly tell me how to do this. Thanks

4 5