🌐 Mailman 3 October 2018 - scikit-learn - python.org

Error with Kfold cross vailidation
by bright silas Aboh Oct. 25, 2018

Oct. 25, 2018

Hi Everyone, I am Bright and am trying to build a machine learning model with sklearn I get the following error however, can someone please help me? kf = KFold(data.shape[0], n_splits=5) TypeError: __init__() got multiple values for argument 'n_splits' Thank you

4 8

Re: Error with Kfold cross vailidation
by eamanu15 Oct. 24, 2018

Oct. 24, 2018

Hello Bright! > I am Bright and am trying to build a machine learning model with sklearn > I get the following error however, can someone please help me? > > kf = KFold(data.shape[0], n_splits=5) > TypeError: __init__() got multiple values for argument 'n_splits' > > Reading the doc [1] I think that the problem is that you are setting the parameters from a wrong way. The first parameter is n_split, but you set data.shape[0] and then you set again "n_splits=5" That is the error. [1] http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.KF… -- Arias Emmanuel http://eamanu.com Github/Gitlab; @eamanu Debian: @eamanu-guest

1 0

Using Scikit-learn graphics for AI workshop, NOKIOS conference
by fabian dietrichson Oct. 20, 2018

Oct. 20, 2018

Hi Scikit-learn! I am employed in Accenture and have been using your machine learning library extensively! It is a well designed library, and my wepon of choice whenever hosting AI workshops. Next week we are will arrange an AI workshop at a conference in Norway, Trondheim, called NOKIOS http://www.nokios.no/english/ <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.nokios.no_english_&…> . For this workshop we would like to use Sciki-learn in the case study and have the participants use your model-chart to select estimator http://scikit-learn.org/stable/tutorial/machine_learning_map/index.html <https://urldefense.proofpoint.com/v2/url?u=http-3A__scikit-2Dlearn.org_stab…> . This will promote your library to several executives, as this is a “how to use AI in your business” course. However, I noticed that your images are protected with copy rights, and I’m asking if I’m allowed to use your illustration for this purpose? Kind regards Fabian Sødal Dietrichson Accenture Developer - IES Norway

3 3

Release of imbalanced-learn 0.4
by Guillaume Lemaître Oct. 12, 2018

Oct. 12, 2018

Hi folks, The new release of imbalanced-learn is available in PyPi and conda-forge. You can find all the new features and changes at: http://imbalanced-learn.org/en/stable/whats_new.html#version-0-4 The documentation is available at: http://imbalanced-learn.org/en/stable/ Hope this helps. Cheers, -- Guillaume Lemaitre INRIA Saclay - Parietal team Center for Data Science Paris-Saclay https://glemaitre.github.io/

1 0

Dimension Reduction - MDS
by Guillaume Favelier Oct. 11, 2018

Oct. 11, 2018

Hi everyone, I'm trying to use some dimension reduction algorithm [1] on my dataset [2] in a python script [3] but for some reason, Python seems to consume a lot of my main memory and even swap on my configuration [4] so I don't have the expected result but a memory error instead. I have the impression that this behaviour is not intended so can you help me know what I did wrong or miss somewhere please? [1]: MDS - http://scikit-learn.org/stable/modules/generated/sklearn.manifold.MDS.html [2]: dragon.csv - 69827 rows, 3 columns (x,y,z) [3]: dragon.py - 10 lines [4]: dragon_swap.png - htop on my workstation TAR archive: https://drive.google.com/open?id=1d1S99XeI7wNEq131wkBUCBrctPQRgpxn Best regards, Guillaume Favelier

3 4

Micro average in classification report
by Kevin Markham Oct. 9, 2018

Oct. 9, 2018

Hello all, Congratulations on the release of 0.20! My questions are about the updated classification_report: http://scikit-learn.org/stable/modules/generated/sklearn.metrics.classifica… Here is the simple example shown in the documentation (apologies for the formatting): >>> from sklearn.metrics import classification_report >>> y_true = [0, 1, 2, 2, 2] >>> y_pred = [0, 0, 2, 2, 1] >>> target_names = ['class 0', 'class 1', 'class 2'] >>> print(classification_report(y_true, y_pred, target_names=target_names)) precision recall f1-score support class 0 0.50 1.00 0.67 1 class 1 0.00 0.00 0.00 1 class 2 1.00 0.67 0.80 3 micro avg 0.60 0.60 0.60 5 macro avg 0.50 0.56 0.49 5 weighted avg 0.70 0.60 0.61 5 I understand how macro average and weighted average are calculated. My questions are in regard to micro average: 1. From this and other examples, it appears to me that "micro average" is identical to classification accuracy. Is that correct? 2. Is there a reason that micro average is listed three times (under the precision, recall, and f1-score columns)? From my understanding, that 0.60 number is being calculated once but is being displayed three times. The display implies (at least in my mind) that 0.60 is being calculated from the three precision numbers, and separately calculated from the three recall numbers, and separately calculated from the three f1-score numbers, which seems misleading. 3. The documentation explains micro average as "averaging the total true positives, false negatives and false positives". If my understanding is correct that micro average is the same as accuracy, then why are true negatives any less relevant to the calculation? (Also, I don't mean to be picky, but "true positives" etc. are whole number counts rather than rates, and so it seems odd to say that you are arriving at a rate by averaging counts.) These may be dumb questions arising from my ignorance... my apologies if so! As well, I don't mean for my questions to criticize the excellent work that has been done by all of the scikit-learn contributors - I deeply appreciate your work! Rather, I'm planning to create a video series explaining some of the new features in 0.20, and I want to make sure that I'm accurately explaining these new features. Thanks very much! Kevin -- Kevin Markham Founder, Data School https://www.dataschool.io https://www.youtube.com/dataschool https://www.patreon.com/dataschool

3 2

Understanding sklearn.tree._tree.value object
by Pranav Ashok Oct. 9, 2018

Oct. 9, 2018

I have a multi-class multi-label decision tree learnt using DecisionTreeClassifier class. The input looks like follows: X = [[2, 51], [3, 20], [5, 30], [7, 1], [20, 46], [25, 25], [45, 70]] Y = [[1,2,3],[1,2,3],[1,2,3],[1,2],[1,2],[1],[1]] I have used MultiLabelBinarizer to convert Y into [[1 1 1] [1 1 1] [1 1 1] [1 1 0] [1 1 0] [1 0 0] [1 0 0]] After training, the _tree.values looks like follows: array([[[7., 0.], [2., 5.], [4., 3.]], [[3., 0.], [0., 3.], [0., 3.]], [[4., 0.], [2., 2.], [4., 0.]], [[2., 0.], [0., 2.], [2., 0.]], [[2., 0.], [2., 0.], [2., 0.]]]) I had the impression that the value array contains for each node, a list of lists [[n_1, y_1], [n_2, y_2], [n_3, y_3]] such that n_i are the number of samples disagreeing with class i and y_i are the number of samples agreeing with class i. But after seeing this output, it does not make sense. For example, the root node has the value [[7,0],[2,5],[4,3]]. According to my interpretation, this would mean 7 samples disagree with class 1; 2 disagree with class 2 and 5 agree with class 2; 4 disagree with class 3 and 3 agree with class 3. which, according to the input dataset is wrong. Could someone please help me understand the semantics of _tree.value for multi-label DTs?

2 3

Re: [scikit-learn] scikit-learn Digest, Vol 30, Issue 25
by Jason Sanchez Oct. 8, 2018

Oct. 8, 2018

The current roadmap is amazing. One feature that would be exciting is better support for multilayer stacking with caching and the ability to add models to already trained layers. I saw this history: https://github.com/scikit-learn/scikit-learn/pull/8960 This library is very close: * API is somewhat awkward, but otherwise good. Does not cache intermediate steps. https://wolpert.readthedocs.io/en/latest/index.html These solutions seem to allow only two layers: * https://github.com/scikit-learn/scikit-learn/issues/4816#issuecomment-21781… * https://rasbt.github.io/mlxtend/user_guide/classifier/StackingClassifier/ * https://github.com/scikit-learn/scikit-learn/pull/6674 The people who put these other libraries together have made an incredibly welcome effort to solve a real need and it would be amazing to see a payoff for their effort in the form of an addition of stacking to scikit-learn's core library. As another data point, I attached a simple implementation I put together to illustrate what I think are core needs of this feature. Feel free to browse the code. Here is the short list: * Infinite layers (or at least 3 ;) ) * Choice of CV or OOB for each model * Ability to add a new model to a layer after the stacked ensemble has been trained and refit the pipeline such that only models that must be retrained are retrained (i.e. train the added model and retrain all models in higher layers) * All standard scikit-learn pipeline goodness (introspection, grid search, serializability, etc) Thanks all! This library is making a real difference for good in the lives of many people. Jason On Fri, Sep 28, 2018 at 11:35 AM <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: [ANN] Scikit-learn 0.20.0 (Sebastian Raschka) > 2. Re: [ANN] Scikit-learn 0.20.0 (Andreas Mueller) > 3. Re: [ANN] Scikit-learn 0.20.0 (Andreas Mueller) > 4. Re: [ANN] Scikit-learn 0.20.0 (Manuel CASTEJ?N LIMAS) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 28 Sep 2018 11:10:50 -0500 > From: Sebastian Raschka <mail(a)sebastianraschka.com> > To: Scikit-learn mailing list <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] [ANN] Scikit-learn 0.20.0 > Message-ID: > <EFC21EB5-CCC4-48CB-AEB5-2C938689EC5E(a)sebastianraschka.com> > Content-Type: text/plain; charset=us-ascii > > > > > > I think model serialization should be a priority. > > > > There is also the ONNX specification that is gaining industrial adoption > and that already includes open source exporters for several families of > scikit-learn models: > > > > https://github.com/onnx/onnxmltools > > > Didn't know about that. This is really nice! What do you think about > referring to it under > http://scikit-learn.org/stable/modules/model_persistence.html to make > people aware that this option exists? > Would be happy to add a PR. > > Best, > Sebastian > > > > > On Sep 28, 2018, at 9:30 AM, Olivier Grisel <olivier.grisel(a)ensta.org> > wrote: > > > > > > > I think model serialization should be a priority. > > > > There is also the ONNX specification that is gaining industrial adoption > and that already includes open source exporters for several families of > scikit-learn models: > > > > https://github.com/onnx/onnxmltools > > > > -- > > Olivier > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > ------------------------------ > > Message: 2 > Date: Fri, 28 Sep 2018 13:38:39 -0400 > From: Andreas Mueller <t3kcit(a)gmail.com> > To: scikit-learn(a)python.org > Subject: Re: [scikit-learn] [ANN] Scikit-learn 0.20.0 > Message-ID: <96edd381-2352-f183-486a-b86e395a78f6(a)gmail.com> > Content-Type: text/plain; charset=utf-8; format=flowed > > > > On 09/28/2018 12:10 PM, Sebastian Raschka wrote: > >>> I think model serialization should be a priority. > >> There is also the ONNX specification that is gaining industrial > adoption and that already includes open source exporters for several > families of scikit-learn models: > >> > >> https://github.com/onnx/onnxmltools > > > > Didn't know about that. This is really nice! What do you think about > referring to it under > http://scikit-learn.org/stable/modules/model_persistence.html to make > people aware that this option exists? > > Would be happy to add a PR. > > > > > I don't think an open source runtime has been announced yet (or they > didn't email me like they promised lol). > I'm quite excited about this as well. > > Javier: > The problem is not so much storing the "model" but storing how to make > predictions. Different versions could act differently > on the same data structure - and the data structure could change. Both > happen in scikit-learn. > So if you want to make sure the right thing happens across versions, you > either need to provide serialization and deserialization for > every version and conversion between those or you need to provide a way > to store the prediction function, > which basically means you need a turing-complete language (that's what > ONNX does). > > We basically said doing the first is not feasible within scikit-learn > given our current amount of resources, and no-one > has even tried doing it outside of scikit-learn (which would be possible). > Implementing a complete prediction serialization language (the second > option) is definitely outside the scope of sklearn. > > > > > ------------------------------ > > Message: 3 > Date: Fri, 28 Sep 2018 13:41:13 -0400 > From: Andreas Mueller <t3kcit(a)gmail.com> > To: scikit-learn(a)python.org > Subject: Re: [scikit-learn] [ANN] Scikit-learn 0.20.0 > Message-ID: <4cfbb327-7489-70ff-8fa3-a21079ec0068(a)gmail.com> > Content-Type: text/plain; charset=utf-8; format=flowed > > > > On 09/28/2018 01:38 PM, Andreas Mueller wrote: > > > > > > On 09/28/2018 12:10 PM, Sebastian Raschka wrote: > >>>> I think model serialization should be a priority. > >>> There is also the ONNX specification that is gaining industrial > >>> adoption and that already includes open source exporters for several > >>> families of scikit-learn models: > >>> > >>> https://github.com/onnx/onnxmltools > >> > >> Didn't know about that. This is really nice! What do you think about > >> referring to it under > >> http://scikit-learn.org/stable/modules/model_persistence.html to make > >> people aware that this option exists? > >> Would be happy to add a PR. > >> > >> > > I don't think an open source runtime has been announced yet (or they > > didn't email me like they promised lol). > > I'm quite excited about this as well. > > > > Javier: > > The problem is not so much storing the "model" but storing how to make > > predictions. Different versions could act differently > > on the same data structure - and the data structure could change. Both > > happen in scikit-learn. > > So if you want to make sure the right thing happens across versions, > > you either need to provide serialization and deserialization for > > every version and conversion between those or you need to provide a > > way to store the prediction function, > > which basically means you need a turing-complete language (that's what > > ONNX does). > > > > We basically said doing the first is not feasible within scikit-learn > > given our current amount of resources, and no-one > > has even tried doing it outside of scikit-learn (which would be > > possible). > > Implementing a complete prediction serialization language (the second > > option) is definitely outside the scope of sklearn. > > > > > Maybe we should add to the FAQ why serialization is hard? > > > ------------------------------ > > Message: 4 > Date: Fri, 28 Sep 2018 20:34:43 +0200 > From: Manuel CASTEJ?N LIMAS <mcasl(a)unileon.es> > To: Scikit-learn user and developer mailing list > <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] [ANN] Scikit-learn 0.20.0 > Message-ID: > <CAAQ3= > UFntYo02YkR9YwrCjicb8A3cutpN47L4MYZWxeNNYP+1A(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > How about a docker based approach? Just thinking out loud > Best > Manuel > > El vie., 28 sept. 2018 19:43, Andreas Mueller <t3kcit(a)gmail.com> escribi?: > > > > > > > On 09/28/2018 01:38 PM, Andreas Mueller wrote: > > > > > > > > > On 09/28/2018 12:10 PM, Sebastian Raschka wrote: > > >>>> I think model serialization should be a priority. > > >>> There is also the ONNX specification that is gaining industrial > > >>> adoption and that already includes open source exporters for several > > >>> families of scikit-learn models: > > >>> > > >>> https://github.com/onnx/onnxmltools > > >> > > >> Didn't know about that. This is really nice! What do you think about > > >> referring to it under > > >> http://scikit-learn.org/stable/modules/model_persistence.html to make > > >> people aware that this option exists? > > >> Would be happy to add a PR. > > >> > > >> > > > I don't think an open source runtime has been announced yet (or they > > > didn't email me like they promised lol). > > > I'm quite excited about this as well. > > > > > > Javier: > > > The problem is not so much storing the "model" but storing how to make > > > predictions. Different versions could act differently > > > on the same data structure - and the data structure could change. Both > > > happen in scikit-learn. > > > So if you want to make sure the right thing happens across versions, > > > you either need to provide serialization and deserialization for > > > every version and conversion between those or you need to provide a > > > way to store the prediction function, > > > which basically means you need a turing-complete language (that's what > > > ONNX does). > > > > > > We basically said doing the first is not feasible within scikit-learn > > > given our current amount of resources, and no-one > > > has even tried doing it outside of scikit-learn (which would be > > > possible). > > > Implementing a complete prediction serialization language (the second > > > option) is definitely outside the scope of sklearn. > > > > > > > > Maybe we should add to the FAQ why serialization is hard? > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > >

4 4

Adding Quickshift clustering algorithm
by Heinrich Jiang Oct. 8, 2018

Oct. 8, 2018

Hello, I'm a researcher at Google Research and I am writing to initiate discussion about adding Quickshift as well as a variant of it as part of scikit-learn's set of clustering algorithms. This somewhat recent algorithm was designed as a faster alternative to Mean Shift and has been used extensively in computer vision (and already part of scikit-image). The method was published independently in these papers [1,2]. [1] has 600 citations and [2] has 1300 citations. [1] Vedaldi, Andrea, and Stefano Soatto. "Quick shift and kernel methods for mode seeking." *European Conference on Computer Vision*. Springer, Berlin, Heidelberg, 2008. [2] Rodriguez, Alex, and Alessandro Laio. "Clustering by fast search and find of density peaks." *Science* 344.6191 (2014): 1492-1496. In addition to Quickshift, I also propose a variant called Quickshift++, which is Quickshift with an additional hyperparameter. We showed in [3] that this substantially improved performance over Quickshift as well as other clustering algorithms implemented in sklearn on benchmark datasets. (i.e. Figure 9 in https://arxiv.org/abs/1805.07909) and was published at ICML 2018. [3] Jiang, Heinrich, Jennifer Jang, and Samory Kpotufe. "Quickshift++: Provably Good Initializations for Sample-Based Mean Shift." ICML 2018 We have an implementation here (https://github.com/google/quickshift). Best, Heinrich

1 0

[ANN] Scikit-learn 0.20.0
by Andreas Mueller Oct. 4, 2018

Oct. 4, 2018

Hey everbody! I'm happy to (finally) announce scikit-learn 0.20.0. This release is dedicated to the memory of Raghav Rajagopalan. You can upgrade now with pip or conda! There is many important additions and updates, and you can find the full release notes here: http://scikit-learn.org/stable/whats_new.html#version-0-20 My personal highlights are the ColumnTransformer and the changes to OneHotEncoder, but there's so much more! An important note is that this is the last version to support Python2.7, and the next release will require Python 3.5. A big thank you to everybody who contributed and special thanks to Joel! All the best, Andy

13 34