🌐 Mailman 3 December 2016 - scikit-learn - python.org

Re: [scikit-learn] Github project management tools
by Raghav R V Dec. 8, 2016

Dec. 8, 2016

> > Okay so in the project, instead of sorting them by Issues / PR why don't >>> we make one column per priority. Let's have 3 levels and one column for >>> Done. We have a label for "Stalled" / "Need Contributor" which shows up in >>> the cards of the project anyway... >>> >>> As I didn't want to disturb the existing project setup, I created one >>> for a demo - https://github.com/scikit-learn/scikit-learn/projects/7 >>> (I'm resending this e-mail as the last one was rejected because the >>> attached image was huge for the mailing list) >>> >> Thanks Raghav

4 5

NuSVC and ValueError: specified nu is infeasible
by Thomas Evangelidis Dec. 8, 2016

Dec. 8, 2016

Greetings, I want to use the Nu-Support Vector Classifier with the following input data: X= [ array([ 3.90387012, 1.60732281, -0.33315799, 4.02770896, 1.82337731, -0.74007214, 6.75989219, 3.68538903, .................. 0. , 11.64276776, 0. , 0. ]), array([ 3.36856769e+00, 1.48705816e+00, 4.28566992e-01, 3.35622071e+00, 1.64046508e+00, 5.66879661e-01, ..................... 4.25335335e+00, 1.96508829e+00, 8.63453394e-06]), array([ 3.74986249e+00, 1.69060713e+00, -5.09921270e-01, 3.76320781e+00, 1.67664455e+00, -6.21126735e-01, .......................... 4.16700259e+00, 1.88688784e+00, 7.34729942e-06]), ....... ] and Y= [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, ............................ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] > Each array of X contains 60 numbers and the dataset consists of 48 > positive and 1230 negative observations. When I train an svm.SVC() > classifier I get quite good predictions, but wit the svm.NuSVC() I keep > getting the following error no matter which value of nu in [0.1, ..., 0.9, > 0.99, 0.999, 0.9999] I try: > /usr/local/lib/python2.7/dist-packages/sklearn/svm/base.pyc in fit(self, > X, y, sample_weight) > 187 > 188 seed = rnd.randint(np.iinfo('i').max) > --> 189 fit(X, y, sample_weight, solver_type, kernel, > random_seed=seed) > 190 # see comment on the other call to np.iinfo in this file > 191 > /usr/local/lib/python2.7/dist-packages/sklearn/svm/base.pyc in > _dense_fit(self, X, y, sample_weight, solver_type, kernel, random_seed) > 254 cache_size=self.cache_size, coef0=self.coef0, > 255 gamma=self._gamma, epsilon=self.epsilon, > --> 256 max_iter=self.max_iter, random_seed=random_seed) > 257 > 258 self._warn_from_fit_status() > /usr/local/lib/python2.7/dist-packages/sklearn/svm/libsvm.so in > sklearn.svm.libsvm.fit (sklearn/svm/libsvm.c:2501)() > ValueError: specified nu is infeasible Does anyone know what might be wrong? Could it be the input data? thanks in advance for any advice Thomas -- ====================================================================== Thomas Evangelidis Research Specialist CEITEC - Central European Institute of Technology Masaryk University Kamenice 5/A35/1S081, 62500 Brno, Czech Republic email: tevang(a)pharm.uoa.gr tevang3(a)gmail.com website: https://sites.google.com/site/thomasevangelidishomepage/

3 8

no positive predictions by neural_network.MLPClassifier
by Thomas Evangelidis Dec. 8, 2016

Dec. 8, 2016

I tried the sklearn.neural_network.MLPClassifier with the default parameters using the input data I quoted in my previous post about Nu-Support Vector Classifier. The predictions are great but the problem is that sometimes when I rerun the MLPClassifier it predicts no positive observations (class 1). I have noticed that this can be controlled by the random_state parameter, e.g. MLPClassifier(random_state=0) gives always no positive predictions. My question is how can I chose the right random_state value in a real blind test case? thanks in advance Thomas -- ====================================================================== Thomas Evangelidis Research Specialist CEITEC - Central European Institute of Technology Masaryk University Kamenice 5/A35/1S081, 62500 Brno, Czech Republic email: tevang(a)pharm.uoa.gr tevang3(a)gmail.com website: https://sites.google.com/site/thomasevangelidishomepage/

3 4

Re: [scikit-learn] Nested Leave One Subject Out (LOSO) cross validation with scikit
by Ludovico Coletta Dec. 7, 2016

Dec. 7, 2016

Thank you for the good news! [😊] Best Ludovico ________________________________ Da: scikit-learn <scikit-learn-bounces+ludo25_90=hotmail.com(a)python.org> per conto di scikit-learn-request(a)python.org <scikit-learn-request(a)python.org> Inviato: mercoledì 7 dicembre 2016 22.06 A: scikit-learn(a)python.org Oggetto: scikit-learn Digest, Vol 9, Issue 24 Send scikit-learn mailing list submissions to scikit-learn(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... or, via email, send a message with subject or body 'help' to scikit-learn-request(a)python.org You can reach the person managing the list at scikit-learn-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of scikit-learn digest..." Today's Topics: 1. Re: Nested Leave One Subject Out (LOSO) cross validation with scikit (Andreas Mueller) ---------------------------------------------------------------------- Message: 1 Date: Wed, 7 Dec 2016 16:06:29 -0500 From: Andreas Mueller <t3kcit(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Nested Leave One Subject Out (LOSO) cross validation with scikit Message-ID: <e3971a8a-a4eb-2536-4e78-95edd2fece2e(a)gmail.com> Content-Type: text/plain; charset="windows-1252"; Format="flowed" PredefinedSplit allows to define your split but I think it's better if you look at this pr: https://github.com/scikit-learn/scikit-learn/pull/7990 [https://avatars2.githubusercontent.com/u/7454015?v=3&s=400]<https://github.com/scikit-learn/scikit-learn/pull/7990> [WIP] Cache pipeline by glemaitre · Pull Request #7990 · scikit-learn/scikit-learn<https://github.com/scikit-learn/scikit-learn/pull/7990> github.com Reference Issue Address the discussions in #3951 Other related issues and PR: #2086 #5082 #5080 What does this implement/fix? Explain your changes. It implements a version of Pipeline which allows ... That allows you to cache steps in a pipeline so you don't have to recompute the feature selection for each parameter. On 12/07/2016 03:13 PM, Ludovico Coletta wrote: > > Thank you for the answer. > > > I also thought about ShuffleSplit (n_splits=1), but I need to control > which indices are used for training and which for testing in the > nested folds. The problem is that I did feature selection before > hyperparameters optimization (with a nested Leave One Out schema) and > now I need the same partitioning for hyperparameters optimization. The > reason why I did this is that the feature selection step is incredibly > slow, I hope I can get rid of that step in the permutation test. Is > not clear to me if I have to include feature selection in the > permutation test as well. > > > Maybe LeavePOut is what I need. > > > Best > > Ludovico > > > > ------------------------------------------------------------------------ > *Da:* scikit-learn > <scikit-learn-bounces+ludo25_90=hotmail.com(a)python.org> per conto di > scikit-learn-request(a)python.org <scikit-learn-request(a)python.org> > *Inviato:* mercoled? 7 dicembre 2016 17.48 > *A:* scikit-learn(a)python.org > *Oggetto:* scikit-learn Digest, Vol 9, Issue 22 > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > <https://mail.python.org/mailman/listinfo/scikit-learn> scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > scikit-learn Info Page - Python > <https://mail.python.org/mailman/listinfo/scikit-learn> scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > mail.python.org > To see the collection of prior postings to the list, visit the > scikit-learn Archives. Using scikit-learn: To post a message to all > the list members ... > > > > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: New to scikit (Andreas Mueller) > 2. Re: Nested Leave One Subject Out (LOSO) cross validation with > scikit (Andreas Mueller) > 3. return type of StandardScaler (Nilay Shrivastava) > 4. Re: return type of StandardScaler (Bharat Didwania .) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 7 Dec 2016 11:33:38 -0500 > From: Andreas Mueller <t3kcit(a)gmail.com> > To: Scikit-learn user and developer mailing list > <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] New to scikit > Message-ID: <f1d5d579-0646-d05d-4bde-9d44c34ec2a3(a)gmail.com> > Content-Type: text/plain; charset="windows-1252"; Format="flowed" > > http://scikit-learn.org/dev/developers/contributing.html#deprecation > > On 12/07/2016 09:42 AM, Chinmay Talegaonkar wrote: > > Yeah, I found an easy bug. Looking for some help in writing > > deprecation cycles for a bug. > > > > On Wed, Dec 7, 2016 at 8:05 PM, Siddharth Gupta > > <siddharthgupta234(a)gmail.com <mailto:siddharthgupta234@gmail.com>> > wrote: > > > > Great! Welcome to the community. I would suggest you to check out > > the issues page on the github repo, raise hand to the issues you > > feel like you can give a go to, check out the issues that are > > tagged as require contributor. Issues are a good way to start, > > they will direct you about the areas of the code base to explore. > > > > On Dec 7, 2016 6:02 PM, "Chinmay Talegaonkar" > > <chinmay0301(a)gmail.com <mailto:chinmay0301@gmail.com>> wrote: > > > > Hi everyone, > > I have a prior experience in python, and > > have started learning machine learning recently. I wanted to > > contribute to scikit, can anyone suggest a relatively easy > > codebase to explore. > > Thanks in advance! > > > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org <mailto:scikit-learn@python.org> > > https://mail.python.org/mailman/listinfo/scikit-learn > > <https://mail.python.org/mailman/listinfo/scikit-learn> > > > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org <mailto:scikit-learn@python.org> > > https://mail.python.org/mailman/listinfo/scikit-learn > > <https://mail.python.org/mailman/listinfo/scikit-learn> > > > > > > > > > > -- > > -- > > *Chinmay Talegaonkar* > > Cultural and Events Coordinator, Mood Indigo > > .............................................. > > > > > > +91-8879178724 > > chinmay0301(a)gmail.com <mailto:bajajkshitij19@gmail.com> > > www.moodi.org<http://www.moodi.org> <http://www.moodi.org> <http://www.moodi.org/> > > > > > > > > > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://mail.python.org/pipermail/scikit-learn/attachments/20161207/4f86464e…> > > ------------------------------ > > Message: 2 > Date: Wed, 7 Dec 2016 11:33:00 -0500 > From: Andreas Mueller <t3kcit(a)gmail.com> > To: Scikit-learn user and developer mailing list > <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Nested Leave One Subject Out (LOSO) cross > validation with scikit > Message-ID: <e9e342f5-d09b-92b7-e304-a0bfa37aecb6(a)gmail.com> > Content-Type: text/plain; charset="windows-1252"; Format="flowed" > > > > On 12/07/2016 07:41 AM, Ludovico Coletta wrote: > > > > Dear scikit experts, > > > > > > I did as you suggested, but it is not exactly what I would like to do > > ( I also read this: > > > http://stackoverflow.com/questions/40400351/nested-cross-validation-with-st…) > > > > Perhaps I should ask my question in another way: it is possible to > > split the nested cv folds just once? It seems to me that this is not > > possible, do you have any hints? > > > > > Not sure I understand your question. > You can do a single split by using ShuffleSplit(n_splits=1) for example. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://mail.python.org/pipermail/scikit-learn/attachments/20161207/063e6a44…> > > ------------------------------ > > Message: 3 > Date: Wed, 7 Dec 2016 22:14:18 +0530 > From: Nilay Shrivastava <nilay.euler16(a)gmail.com> > To: scikit-learn(a)python.org > Subject: [scikit-learn] return type of StandardScaler > Message-ID: > <CAKLfaQRA_qGdYPrDdtG58nhyBQurWBCT2_2NZKzRiOmww5rgJw(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > StandardScaler returns numpy array even if the object passed is a pandas > dataframe, shouldn't it return a dataframe? > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://mail.python.org/pipermail/scikit-learn/attachments/20161207/feecd6be…> > > ------------------------------ > > Message: 4 > Date: Wed, 7 Dec 2016 08:48:47 -0800 > From: "Bharat Didwania ." <bharat.didwania.eee14(a)itbhu.ac.in> > To: Scikit-learn user and developer mailing list > <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] return type of StandardScaler > Message-ID: > <CAA3g_m_jqWaLgkqA530vP5pshdry+mCWqiUtS5QhDV-EjtcThQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > you can use pandas.get_dummies() > <http://pandas.pydata.org/pandas-docs/stable/generated/pandas.get_dummies.ht…>. > It will perform one hot encoding on categorical columns, and produce a > dataframe as the result. From there you can use > pandas.concat([existing_df, > new_df],axis=0) to add the new columns to your existing dataframe. This > will avoid the use of a numpy array. > > > On Wed, Dec 7, 2016 at 8:44 AM, Nilay Shrivastava > <nilay.euler16(a)gmail.com> > wrote: > > > StandardScaler returns numpy array even if the object passed is a pandas > > dataframe, shouldn't it return a dataframe? > > > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://mail.python.org/pipermail/scikit-learn/attachments/20161207/d6afb4d5…> > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > > ------------------------------ > > End of scikit-learn Digest, Vol 9, Issue 22 > ******************************************* > > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161207/8e3450c9…> ------------------------------ Subject: Digest Footer _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://mail.python.org/mailman/listinfo/scikit-learn ------------------------------ End of scikit-learn Digest, Vol 9, Issue 24 *******************************************

1 0

Nested Leave One Subject Out (LOSO) cross validation with scikit
by Ludovico Coletta Dec. 7, 2016

Dec. 7, 2016

Thank you for the answer. I also thought about ShuffleSplit (n_splits=1), but I need to control which indices are used for training and which for testing in the nested folds. The problem is that I did feature selection before hyperparameters optimization (with a nested Leave One Out schema) and now I need the same partitioning for hyperparameters optimization. The reason why I did this is that the feature selection step is incredibly slow, I hope I can get rid of that step in the permutation test. Is not clear to me if I have to include feature selection in the permutation test as well. Maybe LeavePOut is what I need. Best Ludovico ________________________________ Da: scikit-learn <scikit-learn-bounces+ludo25_90=hotmail.com(a)python.org> per conto di scikit-learn-request(a)python.org <scikit-learn-request(a)python.org> Inviato: mercoledì 7 dicembre 2016 17.48 A: scikit-learn(a)python.org Oggetto: scikit-learn Digest, Vol 9, Issue 22 Send scikit-learn mailing list submissions to scikit-learn(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... or, via email, send a message with subject or body 'help' to scikit-learn-request(a)python.org You can reach the person managing the list at scikit-learn-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of scikit-learn digest..." Today's Topics: 1. Re: New to scikit (Andreas Mueller) 2. Re: Nested Leave One Subject Out (LOSO) cross validation with scikit (Andreas Mueller) 3. return type of StandardScaler (Nilay Shrivastava) 4. Re: return type of StandardScaler (Bharat Didwania .) ---------------------------------------------------------------------- Message: 1 Date: Wed, 7 Dec 2016 11:33:38 -0500 From: Andreas Mueller <t3kcit(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] New to scikit Message-ID: <f1d5d579-0646-d05d-4bde-9d44c34ec2a3(a)gmail.com> Content-Type: text/plain; charset="windows-1252"; Format="flowed" http://scikit-learn.org/dev/developers/contributing.html#deprecation On 12/07/2016 09:42 AM, Chinmay Talegaonkar wrote: > Yeah, I found an easy bug. Looking for some help in writing > deprecation cycles for a bug. > > On Wed, Dec 7, 2016 at 8:05 PM, Siddharth Gupta > <siddharthgupta234(a)gmail.com <mailto:siddharthgupta234@gmail.com>> wrote: > > Great! Welcome to the community. I would suggest you to check out > the issues page on the github repo, raise hand to the issues you > feel like you can give a go to, check out the issues that are > tagged as require contributor. Issues are a good way to start, > they will direct you about the areas of the code base to explore. > > On Dec 7, 2016 6:02 PM, "Chinmay Talegaonkar" > <chinmay0301(a)gmail.com <mailto:chinmay0301@gmail.com>> wrote: > > Hi everyone, > I have a prior experience in python, and > have started learning machine learning recently. I wanted to > contribute to scikit, can anyone suggest a relatively easy > codebase to explore. > Thanks in advance! > > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org <mailto:scikit-learn@python.org> > https://mail.python.org/mailman/listinfo/scikit-learn > <https://mail.python.org/mailman/listinfo/scikit-learn> > > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org <mailto:scikit-learn@python.org> > https://mail.python.org/mailman/listinfo/scikit-learn > <https://mail.python.org/mailman/listinfo/scikit-learn> > > > > > -- > -- > *Chinmay Talegaonkar* > Cultural and Events Coordinator, Mood Indigo > .............................................. > > > +91-8879178724 > chinmay0301(a)gmail.com <mailto:bajajkshitij19@gmail.com> > www.moodi.org<http://www.moodi.org> <http://www.moodi.org/> > > > > > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn

2 1

return type of StandardScaler
by Nilay Shrivastava Dec. 7, 2016

Dec. 7, 2016

StandardScaler returns numpy array even if the object passed is a pandas dataframe, shouldn't it return a dataframe?

3 2

New to scikit
by Chinmay Talegaonkar Dec. 7, 2016

Dec. 7, 2016

Hi everyone, I have a prior experience in python, and have started learning machine learning recently. I wanted to contribute to scikit, can anyone suggest a relatively easy codebase to explore. Thanks in advance!

3 3

Re: [scikit-learn] Nested Leave One Subject Out (LOSO) cross validation with scikit
by Ludovico Coletta Dec. 7, 2016

Dec. 7, 2016

thank you for the quick answer! The problem is that I have a different number of features for each cv folds, therefore I thought that I had to handle each cv fold separately. I did like you suggested, but training set of the outer cv is then further splitted 3 times (stratified kfold), which I think is suboptimal (for feature selection I indeed implemented a nested loso). One question: would it be so bad if I had nested loso for feature selection but the default stratified kfold for hyperparameter optimization? It would be some kind of double dipping in the nested cv, but the final set left out for test is not concerned. The other point is that maybe I got something wrong in the whole process. I have 26 subjects. CV 1: subject 26 is left out for the final test, subjects 1:24 are used for hyperparameter optimization, subject 25 is used to select the best hyperpameters CV 2: subject 1 is left out for the final test, subjects 2:25 are used for hyperparameter optimization, subject 26 is used to select the best hyperpameters. So until the end. Is that correct? Sorry for the trivial questions, but I am quite a beginner with both Python and ML Best Ludovico ________________________________ Da: scikit-learn <scikit-learn-bounces+ludo25_90=hotmail.com(a)python.org> per conto di scikit-learn-request(a)python.org <scikit-learn-request(a)python.org> Inviato: lunedì 5 dicembre 2016 14.54 A: scikit-learn(a)python.org Oggetto: scikit-learn Digest, Vol 9, Issue 15 Send scikit-learn mailing list submissions to scikit-learn(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... or, via email, send a message with subject or body 'help' to scikit-learn-request(a)python.org You can reach the person managing the list at scikit-learn-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of scikit-learn digest..." Today's Topics: 1. Re: Markov Clustering? (Gael Varoquaux) 2. Re: Markov Clustering? (Raphael C) 3. Re: Nested Leave One Subject Out (LOSO) cross validation with scikit (Andy) 4. Re: Nested Leave One Subject Out (LOSO) cross validation with scikit (Andy) ---------------------------------------------------------------------- Message: 1 Date: Mon, 5 Dec 2016 14:45:52 +0100 From: Gael Varoquaux <gael.varoquaux(a)normalesup.org> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Markov Clustering? Message-ID: <20161205134552.GK2327874(a)phare.normalesup.org> Content-Type: text/plain; charset=iso-8859-1 Interestingly, a couple of days before this thread was started a researcher in a top lab of a huge private-sector company had mentionned to me that they found this algorithm very useful in practice (sorry for taking time to point this out, I just needed to check with him that indeed it was this specific algorithm). G On Sun, Dec 04, 2016 at 08:18:54AM +0000, Raphael C wrote: > I think you get a better view of the importance of Markov Clustering in > academia from https://scholar.google.co.uk/scholar?hl=en&as_sdt=0,5&q= > Markov+clustering . > Raphael > On Sat, 3 Dec 2016 at 22:43 Allan Visochek <avisochek3(a)gmail.com> wrote: > Thanks for pointing that out, I sort of picked it up by word of mouth so > I'd assumed it had a bit more precedence in the academic world. ? > I'll look into it a little more, but I'd definitely be interested in > contributing something else if that doesn't work out. > -Allan > On Sat, Dec 3, 2016 at 4:45 PM, Andy <t3kcit(a)gmail.com> wrote: > Hey Allan. > None of the references apart from the last one seems to be published in > a peer-reviewed place, is that right? > And "A stochastic uncoupling process for graphs" has 13 citations since > 2000. Unless there is a more prominent > publication or evidence of heavy use, I think it's disqualified. > Academia is certainly not the only metric for evaluation, so if you > have others, that's good, too ;) > Best, > Andy > On 12/03/2016 04:33 PM, Allan Visochek wrote: > Hey Andy, > This algorithm does operate on sparse graphs so it may be beyond > the scope of sci-kit learn, let me know what you think.? > The website is here, it includes a brief description of how the > algorithm operates under Documentation -> Overview1 and Overview2.? > The references listed on the website are included below. > Best, > -Allan > [1]?Stijn van Dongen.?Graph Clustering by Flow Simulation. PhD > thesis, University of Utrecht, May 2000. > http://www.library.uu.nl/digiarchief/dip/diss/1895620/inhoud.htm > [2]?Stijn van Dongen.?A cluster algorithm for graphs. Technical > Report INS-R0010, National Research Institute for Mathematics and > Computer Science in the Netherlands, Amsterdam, May 2000. > http://www.cwi.nl/ftp/CWIreports/INS/INS-R0010.ps.Z > [3]?Stijn van Dongen.?A stochastic uncoupling process for graphs. > Technical Report INS-R0011, National Research Institute for > Mathematics and Computer Science in the Netherlands, Amsterdam, May > 2000. > http://www.cwi.nl/ftp/CWIreports/INS/INS-R0011.ps.Z > [4]?Stijn van Dongen.?Performance criteria for graph clustering and > Markov cluster experiments. Technical Report INS-R0012, National > Research Institute for Mathematics and Computer Science in the > Netherlands, Amsterdam, May 2000. > http://www.cwi.nl/ftp/CWIreports/INS/INS-R0012.ps.Z > [5]?Enright A.J., Van Dongen S., Ouzounis C.A.?An efficient > algorithm for large-scale detection of protein families, Nucleic > Acids Research 30(7):1575-1584 (2002). > On Sat, Dec 3, 2016 at 3:34 PM, Andy <t3kcit(a)gmail.com> wrote: > Hi Allan. > Can you provide the original paper? > It this something usually used on sparse graphs? We do have > algorithms that operate on data-induced > graphs, like SpectralClustering, but we don't really implement > general graph algorithms (there's no PageRank or community > detection). > Andy > On 12/03/2016 12:19 PM, Allan Visochek wrote: > Hi there, > My name is Allan Visochek, I'm a data scientist and web > developer and I love scikit-learn so first of all, thanks > so much for the work that you do.? > I'm reaching out because I've found the markov clustering > algorithm to be quite useful for me in some of my work and > noticed that there is no implementation in scikit-learn, is > anybody working on this? If not, id be happy to take this > on. I'm new to open source, but I've been working with > python for a few years now.? > Best, > -Allan > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > _______________________________________________ scikit-learn > mailing list scikit-learn(a)python.org https://mail.python.org/ mail.python.org Mailing Lists<https://mail.python.org/> mail.python.org mail.python.org Mailing Lists: Welcome! Below is a listing of all the public mailing lists on mail.python.org. Click on a list name to get more information ... > mailman/listinfo/scikit-learn > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... -- Gael Varoquaux Researcher, INRIA Parietal NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France Phone: ++ 33-1-69-08-79-68 http://gael-varoquaux.info http://twitter.com/GaelVaroquaux Gael Varoquaux (@GaelVaroquaux) | Twitter<http://twitter.com/GaelVaroquaux> twitter.com The latest Tweets from Gael Varoquaux (@GaelVaroquaux). Researcher and geek: ►Brain, Data, & Computational science ►#python #pydata #sklearn ►Machine learning for fMRI ►Photography on @artgael. Paris, France Gaël Varoquaux: computer / data / brain science<http://gael-varoquaux.info/> gael-varoquaux.info Gaël Varoquaux, computer / data / brain science ... Latest posts . misc personnal programming science Data science instrumenting social media for advertising is ... ------------------------------ Message: 2 Date: Mon, 5 Dec 2016 13:51:26 +0000 From: Raphael C <drraph(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Markov Clustering? Message-ID: <CAFHc1QZEEwBWWP7BNRCGKEVvNAgUDEQ=3TKBWjkjjtTgc+SmAA(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 And... [1] Stijn van Dongen. Graph Clustering by Flow Simulation. PhD thesis, University of Utrecht, May 2000. http://www.library.uu.nl/digiarchief/dip/diss/1895620/inhoud.htm has 1201 citations. I think it's fair to say the method is very widely known and used. Raphael On 5 December 2016 at 13:45, Gael Varoquaux <gael.varoquaux(a)normalesup.org> wrote: > Interestingly, a couple of days before this thread was started a > researcher in a top lab of a huge private-sector company had mentionned > to me that they found this algorithm very useful in practice (sorry for > taking time to point this out, I just needed to check with him that > indeed it was this specific algorithm). > > G > > On Sun, Dec 04, 2016 at 08:18:54AM +0000, Raphael C wrote: >> I think you get a better view of the importance of Markov Clustering in >> academia from https://scholar.google.co.uk/scholar?hl=en&as_sdt=0,5&q= >> Markov+clustering . > >> Raphael > >> On Sat, 3 Dec 2016 at 22:43 Allan Visochek <avisochek3(a)gmail.com> wrote: > >> Thanks for pointing that out, I sort of picked it up by word of mouth so >> I'd assumed it had a bit more precedence in the academic world. > >> I'll look into it a little more, but I'd definitely be interested in >> contributing something else if that doesn't work out. > >> -Allan > >> On Sat, Dec 3, 2016 at 4:45 PM, Andy <t3kcit(a)gmail.com> wrote: > >> Hey Allan. > >> None of the references apart from the last one seems to be published in >> a peer-reviewed place, is that right? >> And "A stochastic uncoupling process for graphs" has 13 citations since >> 2000. Unless there is a more prominent >> publication or evidence of heavy use, I think it's disqualified. >> Academia is certainly not the only metric for evaluation, so if you >> have others, that's good, too ;) > >> Best, >> Andy > >> On 12/03/2016 04:33 PM, Allan Visochek wrote: > >> Hey Andy, > >> This algorithm does operate on sparse graphs so it may be beyond >> the scope of sci-kit learn, let me know what you think. >> The website is here, it includes a brief description of how the >> algorithm operates under Documentation -> Overview1 and Overview2. >> The references listed on the website are included below. > >> Best, >> -Allan > > >> [1] Stijn van Dongen. Graph Clustering by Flow Simulation. PhD >> thesis, University of Utrecht, May 2000. >> http://www.library.uu.nl/digiarchief/dip/diss/1895620/inhoud.htm > >> [2] Stijn van Dongen. A cluster algorithm for graphs. Technical >> Report INS-R0010, National Research Institute for Mathematics and >> Computer Science in the Netherlands, Amsterdam, May 2000. >> http://www.cwi.nl/ftp/CWIreports/INS/INS-R0010.ps.Z > >> [3] Stijn van Dongen. A stochastic uncoupling process for graphs. >> Technical Report INS-R0011, National Research Institute for >> Mathematics and Computer Science in the Netherlands, Amsterdam, May >> 2000. >> http://www.cwi.nl/ftp/CWIreports/INS/INS-R0011.ps.Z > >> [4] Stijn van Dongen. Performance criteria for graph clustering and >> Markov cluster experiments. Technical Report INS-R0012, National >> Research Institute for Mathematics and Computer Science in the >> Netherlands, Amsterdam, May 2000. >> http://www.cwi.nl/ftp/CWIreports/INS/INS-R0012.ps.Z > >> [5] Enright A.J., Van Dongen S., Ouzounis C.A. An efficient >> algorithm for large-scale detection of protein families, Nucleic >> Acids Research 30(7):1575-1584 (2002). > > >> On Sat, Dec 3, 2016 at 3:34 PM, Andy <t3kcit(a)gmail.com> wrote: > >> Hi Allan. >> Can you provide the original paper? >> It this something usually used on sparse graphs? We do have >> algorithms that operate on data-induced >> graphs, like SpectralClustering, but we don't really implement >> general graph algorithms (there's no PageRank or community >> detection). > >> Andy > > >> On 12/03/2016 12:19 PM, Allan Visochek wrote: > >> Hi there, > >> My name is Allan Visochek, I'm a data scientist and web >> developer and I love scikit-learn so first of all, thanks >> so much for the work that you do. > >> I'm reaching out because I've found the markov clustering >> algorithm to be quite useful for me in some of my work and >> noticed that there is no implementation in scikit-learn, is >> anybody working on this? If not, id be happy to take this >> on. I'm new to open source, but I've been working with >> python for a few years now. > >> Best, >> -Allan > > > >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > >> _______________________________________________ scikit-learn >> mailing list scikit-learn(a)python.org https://mail.python.org/ mail.python.org Mailing Lists<https://mail.python.org/> mail.python.org mail.python.org Mailing Lists: Welcome! Below is a listing of all the public mailing lists on mail.python.org. Click on a list name to get more information ... >> mailman/listinfo/scikit-learn > >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > > >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > > > >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > > >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > > > -- > Gael Varoquaux > Researcher, INRIA Parietal > NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France > Phone: ++ 33-1-69-08-79-68 > http://gael-varoquaux.info http://twitter.com/GaelVaroquaux Gael Varoquaux (@GaelVaroquaux) | Twitter<http://twitter.com/GaelVaroquaux> twitter.com The latest Tweets from Gael Varoquaux (@GaelVaroquaux). Researcher and geek: ►Brain, Data, & Computational science ►#python #pydata #sklearn ►Machine learning for fMRI ►Photography on @artgael. Paris, France Gaël Varoquaux: computer / data / brain science<http://gael-varoquaux.info/> gael-varoquaux.info Gaël Varoquaux, computer / data / brain science ... Latest posts . misc personnal programming science Data science instrumenting social media for advertising is ... > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... ------------------------------ Message: 3 Date: Mon, 5 Dec 2016 08:51:47 -0500 From: Andy <t3kcit(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Nested Leave One Subject Out (LOSO) cross validation with scikit Message-ID: <2864ea23-e6ca-cf83-599f-f8ec149e8d67(a)gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed On 12/04/2016 04:27 PM, Raghav R V wrote: > Hi! > > It looks like you are using the old `sklearn.cross_validation`'s > LeaveOneLabelOut cross-validator. It has been deprecated since v0.18. > > Use the `LeaveOneLabelOut` from `sklearn.model_selection`, that should > fix your issue I think (thought I have not looked into your code in > detail). > You mean LeaveOneGroupOut, right? ------------------------------ Message: 4 Date: Mon, 5 Dec 2016 08:54:01 -0500 From: Andy <t3kcit(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Nested Leave One Subject Out (LOSO) cross validation with scikit Message-ID: <6e970af0-faeb-bf81-3e9f-28dcc5df9168(a)gmail.com> Content-Type: text/plain; charset="windows-1252"; Format="flowed" I'm not sure what the issue with your custom CV is but this seems like a complicated way to implement this. Try model_selection.LeaveOneGroupOut, which directly implements LOSO On 12/04/2016 03:12 PM, Ludovico Coletta wrote: > Dear scikit experts, > > I'm struggling with the implementation of a nested cross validation. > > My data: I have 26 subjects (13 per class) x 6670 features. I used a > feature reduction algorithm (you may have heard about Boruta) to > reduce the dimensionality of my data. Problems start now: I defined > LOSO as outer partitioning schema. Therefore, for each of the 26 cv > folds I used 24 subjects for feature reduction. This lead to a > different number of features in each cv fold. Now, for each cv fold I > would like to use the same 24 subjects for hyperparameter optimization > (SVM with rbf kernel). > > This is what I did: > > /cv = list(LeaveOneout(len(y))) # in y I stored the labels/ > / > / > /inner_train = [None] * len(y)/ > / > / > /inner_test = [None] * len(y)/ > / > / > /ii = 0/ > / > / > /while ii < len(y):/ > / cv = list(LeaveOneOut(len(y))) / > / a = cv[ii][0]/ > / a = a[:-1]/ > / inner_train[ii] = a/ > / > / > / b = cv[ii][0]/ > / b = np.array(b[((len(cv[0][0]))-1)])/ > / inner_test[ii]=b/ > / > / > / ii = ii + 1/ > / > / > /custom_cv = zip(inner_train,inner_test) # inner cv/ > / > / > / > / > /pipe_logistic = Pipeline([('scl', StandardScaler()),('clf', > SVC(kernel="rbf"))])/ > / > / > /parameters = [{'clf__C': np.logspace(-2, 10, 13), > 'clf__gamma':np.logspace(-9, 3, 13)}]/ > / > / > / > / > / > / > /scores = [None] * (len(y)) / > / > / > /ii = 0/ > / > / > /while ii < len(scores):/ > / > / > / a = data[ii][0] # data for train/ > / b = data[ii][1] # data for test/ > / c = np.concatenate((a,b)) # shape: number of subjects * number of > features/ > / d = cv[ii][0] # labels for train/ > / e = cv[ii][1] # label for test/ > / f = np.concatenate((d,e))/ > / > / > / grid_search = GridSearchCV(estimator=pipe_logistic, > param_grid=parameters, verbose=1, scoring='accuracy', cv= > zip(([custom_cv[ii][0]]), ([custom_cv[ii][1]])))/ > / > / > / scores[ii] = cross_validation.cross_val_score(grid_search, c, > y[f], scoring='accuracy', cv = zip(([cv[ii][0]]), ([cv[ii][1]])))/ > / > / > / ii = ii + 1/ > However, I got the following error message: index 25 is out of bounds > for size 25 > > Would it be so bad if I do not perform a nested LOSO but I use the > default setting for hyperparameter optimization? > > Any help would be really appreciated > > > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161205/6584ecee…> ------------------------------ Subject: Digest Footer _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... ------------------------------ End of scikit-learn Digest, Vol 9, Issue 15 *******************************************

2 2

Latent Dirichlet Allocation transformation of data with pre-determined topic_word distribution
by Dylan Kotliar Dec. 7, 2016

Dec. 7, 2016

Hello, I am running Latent Dirichlet Allocation 100 times on bootstrapped versions of a dataset, gathering up the topic_word matrix from each run (components_), and merging it into a final cleaner topic_word matrix. Because I am bootstrapping documents, not every document is in every run and so it isn't clear how to get a final merged doc_topic distribution. I was wondering if there is any way to run the LatentDirichletAllocation transform method with a pre-determined components_ matrix. I tried this out in a few ways none of which worked. from sklearn.decomposition import LatentDirichletAllocation as skLDA mod = skLDA(n_topics=7, learning_method='batch', doc_topic_prior=.1, topic_word_prior=.1, evaluate_every=1) mod.components_ = median_beta # my collapsed estimates of this matrix topic_usage = mod.transform(word_matrix) crashes with: AttributeError: 'LatentDirichletAllocation' object has no attribute 'exp_dirichlet_component_' I try to correct this with: mod.components_ = median_beta mod.exp_dirichlet_component_ = np.exp( _dirichlet_expectation_2d(mod.components_)) mod._init_latent_vars(components_.shape[1]) and now transform will complete will run but the results don't match in the least what I would expect after looking at multiple LDA runs. Note that this kind of functionality is available for NMF where you can run: (W, H, niter) = non_negative_factorization(wordmatrix, H=median_beta, n_components=median_beta.shape[0], update_H=False) Thanks for any insight or help you can provide. Best, Dylan

1 0

Fwd: Scikit-learn MLPRegressor Help
by Alekh Karkada Ashok Dec. 6, 2016

Dec. 6, 2016

Hi all, I want use the Scikit-learn's MLPRegressor to map image to image. That is I have a numpy array of size [1000,2030400] (1000 samples, 76800x3 (RGB) pixels). Corresponding labelled images I have. Therefore Y is also [1000,230400]. But according to documentation: *fit(X, y)* Fit the model to data matrix X and target y. *Parameters:* *X : *{array-like, sparse matrix}, shape (n_samples, n_features) The input data. *y : *array-like, shape (n_samples,) The target values. *Returns:* self : returns a trained MLP model. We can see that Y should be a column matrix. Does this mean Scikit-learn doesn't support multiple outputs? I am getting MemoryError when I try to fit now. More: http://stackoverflow.com/questions/40945791/ memoryerror-in-scikit-learn Please help. Thanks!

6 10