🌐 Mailman 3 September 2018 - scikit-learn - python.org

[ANN] Scikit-learn 0.20.0
by Andreas Mueller Oct. 4, 2018

Oct. 4, 2018

Hey everbody! I'm happy to (finally) announce scikit-learn 0.20.0. This release is dedicated to the memory of Raghav Rajagopalan. You can upgrade now with pip or conda! There is many important additions and updates, and you can find the full release notes here: http://scikit-learn.org/stable/whats_new.html#version-0-20 My personal highlights are the ColumnTransformer and the changes to OneHotEncoder, but there's so much more! An important note is that this is the last version to support Python2.7, and the next release will require Python 3.5. A big thank you to everybody who contributed and special thanks to Joel! All the best, Andy

13 34

Explaining pull request #12197 - Fast PolynomialFeatures on CSR matrices
by Andrew Nystrom Sept. 29, 2018

Sept. 29, 2018

Hi scikit-learn community. I've making a pull request <https://github.com/scikit-learn/scikit-learn/pull/12197> to add a method that allows for polynomial features to be computed on compressed sparse row (CSR) matrices directly. It takes advantage of data sparsity, only taking products of nonzero features, resulting in a speedup proportional to the density to the power of the degree of the expansion. The method is laid out in this work <https://arxiv.org/pdf/1803.06418.pdf>. This yields an impressive speedups, as seen in this plot: [image: blippity.png] This is for degree = 2, and the effects become even more pronounced for degree = 3. All the lines for the dense method are overlapping each other. This is because the dense method doesn't leverage sparsity. Also, notice that the CSR algorithm approaches the time of the dense algorithm as the density approaches 1.0. There's even a slight speedup for the fully dense case, which I attribute to the fact that the code's in Cython. One might wonder why I didn't include the times for compressed sparse column (CSC) matrices, which PolynomialFeatures does support. The reason is that this is very, very slow. In fact, it's slower than just passing a dense matrix. I ran a single trial with a 100 x 500 matrix with a density of 0.5. For degree=2, the CSC algorithm took 56.88 seconds while the CSR algorithm took 0.1363. I tried checking for degree=3, but gave up after waiting for what seemed like 20 minutes for the CSC method, but the CSR method finished in 35.38 seconds. The only caveat is that the algorithm requires a function to be derived for each degree. I've done this, as laid out in the paper, for degrees 2 and 3. I don't think people will typically even want higher degrees due to the explosion of the size of the feature space, but I did lay out the pattern for deriving the functions, just in case. Test results: [image: Screen Shot 2018-09-28 at 10.34.19 PM.png]

1 0

Full time jobs to work on scikit-learn in Paris
by Gael Varoquaux Sept. 28, 2018

Sept. 28, 2018

Dear list, I am very happy to announce that the Inria foundation is looking to hire two persons to work with the scikit-learn in France: * One Community and Operation Officer: https://scikit-learn.fondation-inria.fr/job_coo/ We need a good mix of communication, organizational, and technical skills to help the team and the community work best together * One Performance and Quality Engineer: https://scikit-learn.fondation-inria.fr/en/job_performance/ We need someone who care about tests, continuous integration and performance, to help making scikit-learn faster will guaranteeing that it stays as solid as it is. Please forward this announcement to anyone who might be interested. Best, Gaël

1 0

Re: [scikit-learn] Issues with clone for ensemble of, classifiers
by Luiz Gustavo Hafemann Sept. 26, 2018

Sept. 26, 2018

Guillaume - thank you for the comments. Indeed, an approach to "freeze" a fitted classifier would solve our problem. The Github issue seems to be inactive for a while, but I will check if anyone else is working on it. Luiz Gustavo On Wed, Sep 19, 2018 at 12:02 PM <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: Issues with clone for ensemble of classifiers > (Guillaume Lema?tre) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 19 Sep 2018 17:38:46 +0200 > From: Guillaume Lema?tre <g.lemaitre58(a)gmail.com> > To: Scikit-learn user and developer mailing list > <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Issues with clone for ensemble of > classifiers > Message-ID: > <CACDxx9gyszjJP-5ZB_bvH4nCkdn-sb6CCb= > k2j_kOOnFPBQt0g(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > However, there is some issue to frozen a fitted classifier. You can refer > to: > > https://github.com/scikit-learn/scikit-learn/issues/8370 > > with the associated discussion. > On Wed, 19 Sep 2018 at 17:34, Guillaume Lema?tre <g.lemaitre58(a)gmail.com> > wrote: > > > > Ups I misread your comment. I don't think that we have currently a > > mechanism to avoid cloning classifier internally. > > On Wed, 19 Sep 2018 at 17:31, Guillaume Lema?tre <g.lemaitre58(a)gmail.com> > wrote: > > > > > > You don't have anywhere in your class MyClassifier where you are > > > calling base_classifier.fit(...) therefore when calling > > > base_classifier.predict(...) it will let you know that you did not fit > > > it. > > > > > > On Wed, 19 Sep 2018 at 16:43, Luiz Gustavo Hafemann <luiz.gh(a)gmail.com> > wrote: > > > > > > > > Hello, > > > > > > > > I am one of the developers of a library for Dynamic Ensemble > Selection (DES) methods (the library is called DESlib), and we are > currently working to get the library fully compatible with scikit-learn (to > submit it to scikit-learn-contrib). We have "check_estimator" working for > most of the classes, but now I am having problems to make the classes > compatible with GridSearch / other CV functions. > > > > > > > > One of the main use cases of this library is to facilitate research > on this field, and this led to a design decision that the base classifiers > are fit by the user, and the DES methods receive a pool of base classifiers > that were already fit (this allow users to compare many DES techniques with > the same base classifiers). This is creating an issue with GridSearch, > since the clone method (defined in sklearn.base) is not cloning the classes > as we would like. It does a shallow (non-deep) copy of the parameters, but > we would like the pool of base classifiers to be deep-copied. > > > > > > > > I analyzed this issue and I could not find a solution that does not > require changes on the scikit-learn code. Here is the sequence of steps > that cause the problem: > > > > > > > > GridSearchCV calls "clone" on the DES estimator (link) > > > > The clone function calls the "get_params" function of the DES > estimator (link, line 60). We don't re-implement this function, so it gets > all the parameters, including the pool of classifiers (at this point, they > are still "fitted") > > > > The clone function then clones each parameter with safe=False (line > 62). When cloning the pool of classifiers, the result is a pool that is not > "fitted" anymore. > > > > > > > > The problem is that, to my knowledge, there is no way for my > classifier to inform "clone" that a parameter should be always deep copied. > I see that other ensemble methods in sklearn always fit the base > classifiers within the "fit" method of the ensemble, so this problem does > not happen there. I would like to know if there is a solution for this > problem while having the base classifiers fitted elsewhere. > > > > > > > > Here is a short code that reproduces the issue: > > > > > > > > --------------------------- > > > > > > > > from sklearn.model_selection import GridSearchCV, train_test_split > > > > from sklearn.base import BaseEstimator, ClassifierMixin > > > > from sklearn.ensemble import BaggingClassifier > > > > from sklearn.datasets import load_iris > > > > > > > > > > > > class MyClassifier(BaseEstimator, ClassifierMixin): > > > > def __init__(self, base_classifiers, k): > > > > self.base_classifiers = base_classifiers # Base classifiers > that are already trained > > > > self.k = k # Simulate a parameter that we want to do a grid > search on > > > > > > > > def fit(self, X_dsel, y_dsel): > > > > pass # Here we would fit any parameters for the Dynamic > selection method, not the base classifiers > > > > > > > > def predict(self, X): > > > > return self.base_classifiers.predict(X) # In practice the > methods would do something with the predictions of each classifier > > > > > > > > > > > > X, y = load_iris(return_X_y=True) > > > > X_train, X_dsel, y_train, y_dsel = train_test_split(X, y, > test_size=0.5) > > > > > > > > base_classifiers = BaggingClassifier() > > > > base_classifiers.fit(X_train, y_train) > > > > > > > > clf = MyClassifier(base_classifiers, k=1) > > > > > > > > params = {'k': [1, 3, 5, 7]} > > > > grid = GridSearchCV(clf, params) > > > > > > > > grid.fit(X_dsel, y_dsel) # Raises error that the bagging > classifiers are not fitted > > > > > > > > --------------------------- > > > > > > > > Btw, here is the branch that we are using to make the library > compatible with sklearn: > https://github.com/Menelau/DESlib/tree/sklearn-estimators. The failing > test related to this issue is in > https://github.com/Menelau/DESlib/blob/sklearn-estimators/deslib/tests/test… > > > > > > > > Thanks in advance for any help on this case, > > > > > > > > Luiz Gustavo Hafemann > > > > > > > > _______________________________________________ > > > > scikit-learn mailing list > > > > scikit-learn(a)python.org > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > > > > > > > -- > > > Guillaume Lemaitre > > > INRIA Saclay - Parietal team > > > Center for Data Science Paris-Saclay > > > https://glemaitre.github.io/ > > > > > > > > -- > > Guillaume Lemaitre > > INRIA Saclay - Parietal team > > Center for Data Science Paris-Saclay > > https://glemaitre.github.io/ > > > > -- > Guillaume Lemaitre > INRIA Saclay - Parietal team > Center for Data Science Paris-Saclay > https://glemaitre.github.io/ > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > > ------------------------------ > > End of scikit-learn Digest, Vol 30, Issue 14 > ******************************************** >

3 2

Bootstrapping in sklearn
by Daniel Saxton Sept. 23, 2018

Sept. 23, 2018

Hi all, As everyone knows sklearn is excellent for building predictive models, but an area where I believe there is still work to be done is in coming up with measurements for the inherent uncertainties in those models. (That there is an appetite for this is I believe evidenced by the rise in popularity of probabilistic programming.) We can, for example, easily find point estimates for coefficients of linear models in sklearn, but making inferences from those point estimates is not possible without measurements of probable error. To address this and other problems I authored a package called resample which implements the bootstrap and other randomization-based procedures with the goal of performing largely nonparametric statistical inference on a wide class of problems. The package is built entirely in numpy and scipy and so already integrates fairly well with sklearn (there is a tutorial here which among other things shows applications using the Boston housing data: https://github.com/dsaxton/resample/blob/master/doc/resample.ipynb) Might there be interest in including something like this as an sklearn-contrib package? Essentially we are taking what is already in sklearn.utils.resample and extending it to include other forms of the bootstrap (e.g., balanced, parametric, stratified and / or smoothed), algorithms for computing automatic confidence intervals, and procedures for doing nonparametric, randomization-based hypothesis testing. Here is the Github page: https://github.com/dsaxton/resample Of course, I also would greatly appreciate any input that others might have on ways that this package could be made more useful. Thanks, Daniel

3 7

Issues with clone for ensemble of classifiers
by Luiz Gustavo Hafemann Sept. 19, 2018

Sept. 19, 2018

Hello, I am one of the developers of a library for Dynamic Ensemble Selection (DES) methods (the library is called DESlib), and we are currently working to get the library fully compatible with scikit-learn (to submit it to scikit-learn-contrib). We have "check_estimator" working for most of the classes, but now I am having problems to make the classes compatible with GridSearch / other CV functions. One of the main use cases of this library is to facilitate research on this field, and this led to a design decision that the base classifiers are fit by the user, and the DES methods receive a pool of base classifiers that were already fit (this allow users to compare many DES techniques with the same base classifiers). This is creating an issue with GridSearch, since the clone method (defined in sklearn.base) is not cloning the classes as we would like. It does a shallow (non-deep) copy of the parameters, but we would like the pool of base classifiers to be deep-copied. I analyzed this issue and I could not find a solution that does not require changes on the scikit-learn code. Here is the sequence of steps that cause the problem: 1. GridSearchCV calls "clone" on the DES estimator (link <https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/model_sele…> ) 2. The clone function calls the "get_params" function of the DES estimator (link <https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/base.py#L6…>, line 60). We don't re-implement this function, so it gets all the parameters, including the pool of classifiers (at this point, they are still "fitted") 3. The clone function then clones each parameter with safe=False (line 62). When cloning the pool of classifiers, the result is a pool that is not "fitted" anymore. The problem is that, to my knowledge, there is no way for my classifier to inform "clone" that a parameter should be always deep copied. I see that other ensemble methods in sklearn always fit the base classifiers within the "fit" method of the ensemble, so this problem does not happen there. I would like to know if there is a solution for this problem while having the base classifiers fitted elsewhere. Here is a short code that reproduces the issue: --------------------------- from sklearn.model_selection import GridSearchCV, train_test_split from sklearn.base import BaseEstimator, ClassifierMixin from sklearn.ensemble import BaggingClassifier from sklearn.datasets import load_iris class MyClassifier(BaseEstimator, ClassifierMixin): def __init__(self, base_classifiers, k): self.base_classifiers = base_classifiers # Base classifiers that are already trained self.k = k # Simulate a parameter that we want to do a grid search on def fit(self, X_dsel, y_dsel): pass # Here we would fit any parameters for the Dynamic selection method, not the base classifiers def predict(self, X): return self.base_classifiers.predict(X) # In practice the methods would do something with the predictions of each classifier X, y = load_iris(return_X_y=True) X_train, X_dsel, y_train, y_dsel = train_test_split(X, y, test_size=0.5) base_classifiers = BaggingClassifier() base_classifiers.fit(X_train, y_train) clf = MyClassifier(base_classifiers, k=1) params = {'k': [1, 3, 5, 7]} grid = GridSearchCV(clf, params) grid.fit(X_dsel, y_dsel) # Raises error that the bagging classifiers are not fitted --------------------------- Btw, here is the branch that we are using to make the library compatible with sklearn: https://github.com/Menelau/DESlib/tree/sklearn-estimators. The failing test related to this issue is in https://github.com/Menelau/DESlib/blob/sklearn-estimators/deslib/tests/test… Thanks in advance for any help on this case, Luiz Gustavo Hafemann

2 3

Re: [scikit-learn] Bootstrapping in sklearn
by eamanu15 Sept. 18, 2018

Sept. 18, 2018

Hello! > Any help would certainly be welcome, no matter how slow. I appreciate the > interest. > That sound interest! If you need help, let me know! I would be happy to help Regards! Emmanuel

1 0

Announcing scikit-learn at Inria foundation
by Gael Varoquaux Sept. 17, 2018

Sept. 17, 2018

Hi scikit-learn community, I am very happy to announce a foundation to support scikit-learn at Inria: https://scikit-learn.fondation-inria.fr In practice, this gives us a legal vessel to receive money from private entities, and not only research money. This money will provide a stable job to some people working on scikit-learn here, at Inria, and allow us to grow the team and target more ambitious features as well as better quality and hopefully more frequent releases. I have written a blog post about the motivations and the vision behind this new development: http://gael-varoquaux.info/programming/a-foundation-for-scikit-learn-at-inr… Thank you all for being part of the scikit-learn adventure. I am very excited about the new prospects that this is bringing us, Gaël

1 0

sklearn Pipeline decorators
by Uri Goren Sept. 13, 2018

Sept. 13, 2018

Hi, sklearn Pipelines are awesome, I use them all the time for everything. I've been writing a lot custom transformers lately, and since most of my transformers require no fitting (e.g. replacing all number with "{NUM}" token), I started using transformers as decorators. See snippet (example usage at the end): https://github.com/urigoren/decorators4DS/blob/master/decorators4DS/sklearn… Do you think this kind of addition should be a part of sklearn ? -- *Uri Goren,* *Phone: +972-507-649-650* *EMail: uri(a)goren4u.com <uri(a)goren4u.com>* *Linkedin: il.linkedin.com/in/ugoren/ <http://il.linkedin.com/in/ugoren/>*

2 1

Sharing Data Projects knowledge
by Pete Soderling Sept. 12, 2018

Sept. 12, 2018

Allow me to introduce you to DataEngConf <http://bit.ly/DataEngConf>, a new event that helps technical leaders understand data tools, platforms and algorithms like never before. I recently viewed through past SciKit Learn submissions and was intrigued by the similarities between your group and our event. I’m the founder of the annual no bullshit data event for deeply technical professionals hosted annually in SF, NYC, & Barcelona and with many excellent speakers <https://www.dataengconf.com/speakers-bcn18> each year. DataEngConf is not the standard watered-down “big data” conference you might have attended previously - instead we are built and run by software engineers, developers and data geeks. We have teams from across Europe, the US and Asia participating this year from companies like Google, BBVA, Apache Foundation, UBER, Schibsted, LINE, Badi, LetGo, Eurecat, Satellogic, Fishtown, Datadog, Intermix, GO-JEK, Databricks, Datadog, Cabify, Eurecat and Yara and many more. I am so excited about SciKit Learn and what this community can bring to our event that: I would like to offer all of you a 30% discount code Community30 and you can purchase tickets here. <https://www.eventbrite.com/e/dataengconf-bcn-18-tickets-47847605476?discoun…> Hope to hear from you. Cheers, -pete

1 0