🌐 Mailman 3 November 2018 - Python-list - python.org

Re: Reading 'scientific' csv using Pandas?
by Martin Schöön Nov. 19, 2018

Nov. 19, 2018

Den 2018-11-18 skrev Shakti Kumar <shakti.shrivastava13(a)gmail.com>: > On Sun, 18 Nov 2018 at 18:18, Martin Schöön <martin.schoon(a)gmail.com> wrote: >> >> Now I hit a bump in the road when some of the data is not in plain >> decimal notation (xxx,xx) but in 'scientific' (xx,xxxe-xx) notation. >> > > Martin, I believe this should be done by pandas itself while reading > the csv file, > I took an example in scientific notation and checked this out, > > my sample.csv file is, > col1,col2 > 1.1,0 > 10.24e-05,1 > 9.492e-10,2 > That was a quick answer! My pandas is up to date. In your example you use the US convention of using "." for decimals and "," to separate data. This works perfect for me too. However, my data files use European conventions: decimal "," and TAB to separate data: col1 col2 1,1 0 10,24e-05 1 9,492e-10 2 I use EUData = pd.read_csv('file.csv', skiprows=1, sep='\t', decimal=',', engine='python') to read from such files. This works so so. 'Common floats' (3,1415 etc) works just fine but 'scientific' stuff (1,6023e23) does not work. /Martin

4 3

Re: What Python related git pre-commit hooks are you using?
by Jon Ribbens Nov. 19, 2018

Nov. 19, 2018

On 2018-11-18, Malcolm Greene <python(a)bdurham.com> wrote: > Curious to learn what Python related git pre-commit hooks people are > using? What hooks have you found useful and which hooks have you tried > and abandoned? Appreciate any suggestions for those new to this process. > Background: Window, macOS, and Linux dev environments, PyCharm professional edition IDE, 64-bit Python 3.6, private Github repos. Considering black (standardize formatting), pylamas (multiple static code tests) and possibly a hook into our pytest test runner. I wrote the following pre-commit hook, which is fairly specific to my requirements but also fairly easy to customize. It (configurably) checks text-type files for tabs, trailing whitespace and non-Unix line endings, python files for syntax and with flake8 and pylint, and JavaScript files with 'standard'. It works with Python 2 or 3. https://github.com/jribbens/voting/blob/master/.githooks/pre-commit

1 0

bottledaemon stop/start doesn't work if killed elsewhere
by Adam Funk Nov. 18, 2018

Nov. 18, 2018

Hi, I'm using bottledaemon to run a little REST service on a Pi that takes input from other machines on the LAN and stores stuff in a database. I have a cron job to call 'stop' and 'start' on it daily, just in case of problems. Occasionally the oom-killer runs overnight and kills the process using bottledaemon; when this happens (unlike properly stopping the daemon), the pidfile and its lockfile are left on the filesystem, so the 'stop' does nothing and the 'start' gets refusedq because the old pidfile and lockfile are present. At the moment, I eventually notice something wrong with the output data, ssh into the Pi, and rm the two files then call 'start' on the daemon again. Is there a recommended or good way to handle this situation automatically? Thanks, Adam

3 2

What Python related git pre-commit hooks are you using?
by Malcolm Greene Nov. 18, 2018

Nov. 18, 2018

Curious to learn what Python related git pre-commit hooks people are using? What hooks have you found useful and which hooks have you tried and abandoned? Appreciate any suggestions for those new to this process. Background: Window, macOS, and Linux dev environments, PyCharm professional edition IDE, 64-bit Python 3.6, private Github repos. Considering black (standardize formatting), pylamas (multiple static code tests) and possibly a hook into our pytest test runner. Thanks!

3 2

Generators, generator expressions, and loops
by Steve Keller Nov. 18, 2018

Nov. 18, 2018

I have looked at generators, generator expressions, and iterators and I try to get more familiar with these. 1. How would I loop over all (with no upper bound) integers or all powers of two, for example? In C it would be for (int i = 0; ; i++) { ... } or for (int i = 1; ; i *= 2) { ... } In Python, I could define a generator def powers(): i = 1 while True: yield(i) i *= 2 for i in powers(): ... More elegant are generator expressions but I cannot think of a way without giving an upper limit: for i in (2 ** i for i in range(1000000)): ... which looks ugly. Also, the double for-loop (and also the two loops in the above exmaple, for + while in the generator) look unnatural, somehow, i.e. loop over all elements which are created by a loop. Is there a more beautyful way? Steve

4 3

Issue in parsing the string output from the command using "subprocess"
by srinivasan Nov. 18, 2018

Nov. 18, 2018

Dear Python Experts Team, As am newbie to python and learning python, working on embedded linux platform, my intention is to delete all the SSID's before connecting my Wi-Fi module to specific SSID., I am trying to parse command output using the "subprocess" with wrapper "execute_cmd_output_string" written on it described as below, using the nmcli commands "*nmcli -t -f TYPE,UUID con" and "**"nmcli connection delete uuid ".* Could you please help me, what could be the bug in the below method "*def wifi_disconnect_reconnect(self, ssid, pw):" *using the method "execute_cmd_output_string" (in-turn uses "subprocess") which is failing to give correct output of., UUID's for *"nmcli connection delete uuid "* ? But which works fine with "commands.getstatusoutput" (res = commands.getstatusoutput("nmcli -t -f TYPE,UUID con")), but I dont want to include "commands.getstatusoutput" which costs me for including one more python module in my rootfs Typical output for "nmcli -t -f TYPE,UUID con" ~$ nmcli -t -f TYPE,UUID con 802-11-wireless:3f5011d4-5681-4fed-8dea-95dee790e9e2 802-11-wireless:bfb93be0-740d-426e-b215-0fdc2f652877 802-11-wireless:69b60cf4-65aa-442b-a54c-fb82905adb0d 802-11-wireless:dc2a15ec-d3da-491e-9c8f-cb054f375837 802-11-wireless:5164e7f2-4489-462e-b093-76bc51bf1303 802-11-wireless:d462e8c4-fac7-42f2-8f9a-6846f52d4e8c 802-11-wireless:e5020744-5c9c-453c-92ec-7a854fc893e6 802-11-wireless:eed3358e-8635-471d-b7e9-5c2973a05128 802-11-wireless:ceea6707-a929-4941-9047-a75e061914b6 802-11-wireless:dd6b9c83-db7b-42b9-99c0-14a04f6f35f5 802-11-wireless:9f764fff-6288-49c4-9412-902e89230136 802-11-wireless:72c627cd-77a3-4d16-bb2c-058040d8e4fc ~$ *Python Code Snipeet:* *------------------------------* * def wifi_disconnect_reconnect(self, ssid, pw):* * """* * Connect to Access point using SSID and PW.* * :param ssid: SSID of the ACCESS POINT.* * :param pw: password for connecting to the access point.* * :return: command output as True or False.* * """* * cmd = "nmcli -t -f TYPE,UUID con"* * res = self._helper.execute_cmd_output_string(cmd)* * print(res)* * lines = res[1].split('\n') ------------------------> I suspect the issue might be here* * print(lines)* * for line in lines:* * parts = line.split(":")* * print(parts)* * print(parts[1])* * if (parts[0] == "802-11-wireless"):* * print("nmcli connection delete uuid "+ parts[1])* * os.system("nmcli connection delete uuid "+ parts[1])* * cmd = "nmcli device wifi connect '%s' password %s" % (ssid, pw)* * exit_code = self._helper.execute_cmd_return_code(cmd)* * return True if exit_code == 0 else False* def execute_cmd_output_string(self, cmd, enable_shell=False): """ Execute a command and return its output as a string. :param cmd: abs path of the command with arguments :param enable_shell : force the cmd to be run as shell script :return: a string. """ try: result = subprocess.check_output(split(cmd), stderr=subprocess.STDOUT, shell=enable_shell) except subprocess.CalledProcessError as e: s = """While executing '{}' something went wrong. Return code == '{}' Return output:\n'{}' """.format(cmd, e.returncode, e.output, shell=enable_shell) raise AssertionError(s) return result.strip().decode("utf-8") if __name__ == "__main__": m = wifi() print("disconnect and reconnect") print(m.wifi_disconnect_reconnect("NaWiFi", "abcds")) *Errors:* ----------- Traceback (most recent call last): 802-11-wireless:3f5011d4-5681-4fed-8dea-95dee790e9e2 802-11-wireless:dc2a15ec-d3da-491e-9c8f-cb054f375837 802-11-wireless:5164e7f2-4489-462e-b093-76bc51bf1303 File "/home/srinivasan/Downloads/qa_wifi_nov15_after_incorporating_thilo_comments_zip/qa/test_library/wifi.py", line 153, in <module> 802-11-wireless:d462e8c4-fac7-42f2-8f9a-6846f52d4e8c print(m.wifi_connect("NI WiFi", "T.f.o.s.1996!")) 802-11-wireless:e5020744-5c9c-453c-92ec-7a854fc893e6 802-11-wireless:eed3358e-8635-471d-b7e9-5c2973a05128 File "/home/srinivasan/Downloads/qa_wifi_nov15_after_incorporating_thilo_comments_zip/qa/test_library/wifi.py", line 77, in wifi_connect 802-11-wireless:ceea6707-a929-4941-9047-a75e061914b6 print(parts[1]) 802-11-wireless:dd6b9c83-db7b-42b9-99c0-14a04f6f35f5 *IndexError: list index out of range* 802-11-wireless:9f764fff-6288-49c4-9412-902e89230136 802-11-wireless:72c627cd-77a3-4d16-bb2c-058040d8e4fc [u'0'] [u'0'] Process finished with exit code 1 But when I execute the below the command using the python module "commands". I am able to successfully parse the multi line string output and extarct the UUID's from the command "nmcli -t -f TYPE,UUID con" and pass it to *"nmcli connection delete uuid " for* deleting all the SSID's before connecting Wi-Fi to specific SSID. * def wifi_disconnect_reconnect(self, ssid, pw):* * """* * Connect to Access point using SSID and PW.* * :param ssid: SSID of the ACCESS POINT.* * :param pw: password for connecting to the access point.* * :return: command output as True or False.* * """* * res = commands.getstatusoutput("nmcli -t -f TYPE,UUID con")* * lines = res[1].split('\n')* * for line in lines:* * parts = line.split(":")* * if (parts[0] == "802-11-wireless"):* * os.system("nmcli connection delete uuid "+ parts[1])* Kindly do the needful, as I am wandering for any clues what's going wrong with error and the highlighted code snippet... Many Thanks in advance

2 2

Reading 'scientific' csv using Pandas?
by Martin Schöön Nov. 18, 2018

Nov. 18, 2018

I am in this project where I try to get an overview of a bunch of computer generated (finite element program) data. I have it stored in a number of csv files. Reading the data into spreadsheet programs works fine but is very labour intensive so I am working with Pandas in Jupyter notebooks which I find much more efficient. Now I hit a bump in the road when some of the data is not in plain decimal notation (xxx,xx) but in 'scientific' (xx,xxxe-xx) notation. I use read.csv and I read its documentation and poke around for information on this but so far I have failed. Either I have found it already but I don't understand or I ask the wrong question to the search engines. My experience of Pandas is limited and I would appreciate some guidance. /Martin

2 1

Conversion between basic regular expression and extended regular expression
by Peng Yu Nov. 17, 2018

Nov. 17, 2018

Hi, I'd like to use a program to convert between basic regular expression (BRE) and extended regular expression (ERE). (see man grep for the definition of BRE and ERE). Does python has a module for this purpose? Thanks. -- Regards, Peng

1 0

Re: Enums: making a single enum
by Marko Rauhamaa Nov. 17, 2018

Nov. 17, 2018

Ian Kelly <ian.g.kelly(a)gmail.com>: > On Fri, May 25, 2018 at 11:00 PM, Chris Angelico <rosuav(a)gmail.com> wrote: >> On Sat, May 26, 2018 at 2:46 PM, Steven D'Aprano >>> class State(Enum): >>> Maybe = 2 >> >> # Tri-state logic >> Maybe = object() > > The enum has a nice __str__ though. That's why I usually use string sentinels: Maybe = "Maybe" Marko

1 0

Python Interpreters In Eclipse
by Jaxson Baerg Nov. 16, 2018

Nov. 16, 2018

Hey, I'm attempting to use python in eclipse and am having trouble with the python interpreter. I am using Eclipse 2018-09 and the latest PyDev and Python 3.7.1. When declaring python-3.7.1.exe as an interpreter it gives an error saying it was unable to get info on the interpreter. It also says a possibility to why could be using an unsupported version. I don't know where to go from here, another piece of information is that I am doing this on a school computer with teacher permission but I don't know if that could be causing an issue or not? Any help would be appreciated, thanks!

2 1