🌐 Mailman 3 November 2019 - scikit-learn - python.org

logistic regression results are not stable between solvers
by Benoît Presles Jan. 9, 2020

Jan. 9, 2020

Dear scikit-learn users, I am using logistic regression to make some predictions. On my own data, I do not get the same results between solvers. I managed to reproduce this issue on synthetic data (see the code below). All solvers seem to converge (n_iter_ < max_iter), so why do I get different results? If results between solvers are not stable, which one to choose? Best regards, Ben ------------------------------------------ Here is the code I used to generate synthetic data: from sklearn.datasets import make_classification from sklearn.model_selection import StratifiedShuffleSplit from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression # RANDOM_SEED = 2 # X_sim, y_sim = make_classification(n_samples=200, n_features=45, n_informative=10, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=1, random_state=RANDOM_SEED, shuffle=False) # sss = StratifiedShuffleSplit(n_splits=10, test_size=0.2, random_state=RANDOM_SEED) for train_index_split, test_index_split in sss.split(X_sim, y_sim): X_split_train, X_split_test = X_sim[train_index_split], X_sim[test_index_split] y_split_train, y_split_test = y_sim[train_index_split], y_sim[test_index_split] ss = StandardScaler() X_split_train = ss.fit_transform(X_split_train) X_split_test = ss.transform(X_split_test) # classifier_lbfgs = LogisticRegression(fit_intercept=True, max_iter=20000000, verbose=1, random_state=RANDOM_SEED, C=1e9, solver='lbfgs') classifier_lbfgs.fit(X_split_train, y_split_train) print('classifier lbfgs iter:', classifier_lbfgs.n_iter_) classifier_saga = LogisticRegression(fit_intercept=True, max_iter=20000000, verbose=1, random_state=RANDOM_SEED, C=1e9, solver='saga') classifier_saga.fit(X_split_train, y_split_train) print('classifier saga iter:', classifier_saga.n_iter_) # y_pred_lbfgs = classifier_lbfgs.predict(X_split_test) y_pred_saga = classifier_saga.predict(X_split_test) # if (y_pred_lbfgs==y_pred_saga).all() == False: print('lbfgs does not give the same results as saga :-( !') exit()

5 17

Version 0.21! and plot_tree!
by Andrew Howe Dec. 5, 2019

Dec. 5, 2019

I want to say thank you to all the sklearn developers. The breadth and quality of this software is truly breathtaking. Specifically, I want to say thank you very very much for the plot_tree function! I have wasted a lot of effort in the past, on multiple OSes, getting everything to work so I could view the tree.export_graphviz results. Having this new function to plot the trees natively in matplotlib is extremely useful. Thanks again! Andrew <~~~~~~~~~~~~~~~~~~~~~~~~~~~> J. Andrew Howe, PhD LinkedIn Profile <http://www.linkedin.com/in/ahowe42> ResearchGate Profile <http://www.researchgate.net/profile/John_Howe12/> Open Researcher and Contributor ID (ORCID) <http://orcid.org/0000-0002-3553-1990> Github Profile <http://github.com/ahowe42> Personal Website <http://www.andrewhowe.com> I live to learn, so I can learn to live. - me <~~~~~~~~~~~~~~~~~~~~~~~~~~~>

2 4

SVM-RFE
by Malik Yousef Dec. 5, 2019

Dec. 5, 2019

Hello, Let's assume that I have data with 1000 features. I want to apply SVM-RFE on this data where each time 10% for the features are removed. How one can get the accuracy overall the levels of the elimination stages. For example, I want to get performance over 1000 features, 900 features, 800 features,....,2 features, 1 feature. Also, I want to keep track of the features in each level. [image: image.png] Best --------------------------------------------------------------------------------------- *Prof. Malik Yousef (Associate Professor) * *The Head of the** Galilee Digital Health Research Center (GDH)* *Zefat Academic College , Department of Information System * Home Page: https://malikyousef.com/ Google Scholar Profile : https://scholar.google.com/citations?user=9UCZ_q4AAAAJ&hl=en&oi=ao ----------------------------------------------------------------------------------------------------

3 8

scikit-learn twitter account
by Chiara Marmo Dec. 3, 2019

Dec. 3, 2019

Hello everybody, I've taken a look to the last meeting minutes: talking about releases and sprint announcements, it seems that the need for a centralized communication channel is rising, both from user and dev sides. What about starting to use the scikit-learn twitter account for that? This will also help to animate the community, sckit-learn benefits of a lot of mentions which are never answered. I can help with managing the account if needed. WDYT? Chiara

14 35

ANN: scikit-learn 0.22rc3, supporting Python=3.8
by Adrin Nov. 28, 2019

Nov. 28, 2019

Hi everybody, We just released 0.22rc3 which includes python=3.8 wheels which builds on top of 0.22rc2 and has a few issues fixed. We will be releasing 0.22.0 final early next week, and more than happy to have you all test the release candidate. Happy testing, Adrin, on behalf of the scikit-learn maintainer team.

1 0

Maximum Mutual Information value for continuous variables
by Thomas Evangelidis Nov. 27, 2019

Nov. 27, 2019

Greetings, I am thinking of alternative ways of removing the invariant scalar features from my feature vectors before training MLPs. So far I tried removing columns with 0-variance and columns with Pearson's R=1.0 or R=-1.0. If I remove columns with |R|<1.0 the performance drops. However, R measures the linear correlation. Now I am thinking to try removing columns with high Mutual Information, but first I need to normalize it. I found in the documentation under "Univariate Feature Selection" the function "mutual_info_regression". https://scikit-learn.org/stable/modules/feature_selection.html#univariate-f… I used this function to measure the correlation between columns (features) but sometimes returns values >1.0. On the other hand, there is also this function https://scikit-learn.org/stable/modules/generated/sklearn.metrics.adjusted_… which is upper limited to 1.0 but it is for categorical data (clusters). So my question is, is there a way to computer normalized Mutual Information for continuous variables, too? Thanks in advance for any advice. Thomas -- ====================================================================== Dr. Thomas Evangelidis Research Scientist IOCB - Institute of Organic Chemistry and Biochemistry of the Czech Academy of Sciences <https://www.uochb.cz/web/structure/31.html?lang=en>, Prague, Czech Republic & CEITEC - Central European Institute of Technology <https://www.ceitec.eu/>, Brno, Czech Republic email: tevang3(a)gmail.com, Twitter: tevangelidis <https://twitter.com/tevangelidis>, LinkedIn: Thomas Evangelidis <https://www.linkedin.com/in/thomas-evangelidis-495b45125/> website: https://sites.google.com/site/thomasevangelidishomepage/

1 0

PolynomialFeatures
by Sole Galli Nov. 25, 2019

Nov. 25, 2019

Hello team, Can I double check with you that I understand correctly what the PolynomialFeatures() is doing under the hood? If I set it like this: poly = PolynomialFeatures(degree=3, interaction_only=False, include_bias=False) and I fit it on a dataset with 3 variables, a,b and c. Am I correct to say that the fit() method creates all possible combinations like this: a; b; c; (a+b)^2 (a+b)^3 (a+c)^2 (a+c)^3 (c+b)^2 (c+b)^3 (a+b+c)^2 (a+b+c)^3 And the transform() generates the expansion, without the constant that multiplies the interactions and avoiding duplicated terms after the expansion? Thanks for the help. Kind regards Sole

2 2

ANN: Release Candidate for Scikit-learn 0.22
by Adrin Nov. 21, 2019

Nov. 21, 2019

Hello everybody! A new release candidate is out, and PyPI now has source and binary releases for Scikit-learn 0.22rc2.post1. * Documentation at https://scikit-learn.org/0.22 (with a new web design) * Release Notes at https://scikit-learn.org/0.22/whats_new * Download source or wheels at https://pypi.org/project/scikit-learn/0.22rc2.post1/ or install via `pip install --upgrade --pre scikit-learn` We will have another release candidate soon with Python 3.8 wheels included. You can see the release highlights here: https://scikit-learn.org/0.22/auto_examples/release_highlights/plot_release… and it includes: * A new plotting API: https://scikit-learn.org/0.22/visualizations.html#visualizations * Stacking clsasifier and regressor: https://scikit-learn.org/0.22/modules/ensemble.html#stacking * Permutation based feature importance: https://scikit-learn.org/0.22/modules/permutation_importance.html#permutati… * Native support for missing values for gradient boosting: https://scikit-learn.org/0.22/modules/ensemble.html#missing-values-support * ... There are many other enhancements and fixes listed in the release notes. Please note that if you're building from source, now you have Cython and scipy as new build dependencies. Happy Learning! >From the Scikit-learn core dev team.

1 0

Ten years of scikit-learn
by Chiara Marmo Nov. 14, 2019

Nov. 14, 2019

Hello, do you know that, according to wikipedia ( https://en.wikipedia.org/wiki/Scikit-learn) and scikit-learn website ( https://scikit-learn.org/dev/about.html#history) the first release of scikit-learn will date back to ten years on February 1st 2020? What about organizing a "Decade Sprint" around this date? Just let me know... :) Chiara

1 0

Monthly meetings
by Nicolas Hug Nov. 14, 2019

Nov. 14, 2019

Hey everyone, The next monthly meeting is on Monday! As usual, please be nice to the NYC people and *update your project notes before Friday* it'll be 7am for us :) Cheers, Nicolas https://github.com/scikit-learn/scikit-learn/projects/15 <https://www.google.com/url?q=https://github.com/scikit-learn/scikit-learn/p…> https://appear.in/amueller <https://www.google.com/url?q=https://appear.in/amueller&sa=D&ust=1569248921…> https://www.timeanddate.com/worldclock/meetingdetails.html?year=2019&month=…

3 3