🌐 Mailman 3 August 2017 - scikit-learn - python.org

transform categorical data to numerical representation
by Georg Heiler Aug. 7, 2017

Aug. 7, 2017

Hi, the LabelEncooder is only meant for a single column i.e. target variable. Is the DictVectorizeer or a manual chaining of multiple LabelEncoders (one per categorical column) the desired way to get values which can be fed into a subsequent classifier? Is there some way I have overlooked which works better and possibly also can handle unseen values by applying most frequent imputation? regards, Georg

3 6

Re: [scikit-learn] Problems with running GridSearchCV on a pipeline with a custom transformer
by Sam Barnett Aug. 4, 2017

Aug. 4, 2017

Hi Andy, I have since been able to resolve the pickling issue, though I am now getting an error message saying that an error message does not include the expected string 'fit'. In general, I am trying to use the fit() method of my classifier to instantiate a separate SVC() classifier with a custom kernel, fit THAT to the data, then return this instance as the fitted version of the new classifier. Is this possible in theory? If so, what is the best way to implement it? As before, the requisite code and a .ipynb file is attached. Best, Sam On Thu, Aug 3, 2017 at 6:35 PM, Andreas Mueller <t3kcit(a)gmail.com> wrote: > Hi Sam. > You need to put these into a reachable namespace (possibly as private > functions) so that they can be pickled. > Please stay on the sklearn mailing list, I might not have time to reply. > > Andy > > > On 08/03/2017 01:24 PM, Sam Barnett wrote: > > Hi Andy, > > I've since tried a different solution: instead of a pipeline, I've simply > created a classifier that is for the most part like svm.SVC, though it > takes a few extra inputs for the sequentialisation step. I've used a Python > function that can compute the Gram matrix between two datasets of any shape > to pass into SVC(), though I'm now having trouble with pickling on the > check_estimator test. It appears that SeqSVC.fit() doesn't like to have > methods defined within it. Can you see how to pass this test? (the .ipynb > file shows the error). > > Best, > Sam > > On Wed, Aug 2, 2017 at 9:44 PM, Sam Barnett <sambarnett95(a)gmail.com> > wrote: > >> You're right: it does fail without GridSearchCV when I change the size of >> seq_test. I will look at the transform tomorrow to see if I can work this >> out. Thank you for your help so far! >> >> On Wed, Aug 2, 2017 at 9:20 PM, Andreas Mueller <t3kcit(a)gmail.com> wrote: >> >>> Change the size of seq_test in your notebook and you'll see the failure >>> without GridSearchCV. >>> I haven't looked at your code in detail, but transform is supposed to >>> work on arbitrary new data with the same number of features. >>> Your code requires the test data to have the same shape as the training >>> data. >>> Cross-validation will lead to training data and test data having >>> different sizes. But I feel like something is already wrong if your >>> test data size depends on your training data size. >>> >>> >>> >>> On 08/02/2017 03:08 PM, Sam Barnett wrote: >>> >>> Hi Andy, >>> >>> The purpose of the transformer is to take an ordinary kernel (in this >>> case I have taken 'rbf' as a default) and return a 'sequentialised' kernel >>> using a few extra parameters. Hence, the transformer takes an ordinary >>> data-target pair X, y as its input, and the fit_transform(X, y) method will >>> output the Gram matrix for X that is associated with this sequentialised >>> kernel. In the pipeline, this Gram matrix is passed into an SVC classifier >>> with the kernel parameter set to 'precomputed'. >>> >>> Therefore, I do not think your hacky solution would be possible. >>> However, I am still unsure how to implement your first solution: won't the >>> Gram matrix from the transformer contain all the necessary kernel values? >>> Could you elaborate further? >>> >>> >>> Best, >>> Sam >>> >>> On Wed, Aug 2, 2017 at 5:05 PM, Andreas Mueller <t3kcit(a)gmail.com> >>> wrote: >>> >>>> Hi Sam. >>>> GridSearchCV will do cross-validation, which requires to "transform" >>>> the test data. >>>> The shape of the test-data will be different from the shape of the >>>> training data. >>>> You need to have the ability to compute the kernel between the training >>>> data and new test data. >>>> >>>> A more hacky solution would be to compute the full kernel matrix in >>>> advance and pass that to GridSearchCV. >>>> >>>> You probably don't need it here, but you should also checkout what the >>>> _pairwise attribute does in cross-validation, >>>> because that it likely to come up when playing with kernels. >>>> >>>> Hth, >>>> Andy >>>> >>>> >>>> On 08/02/2017 08:38 AM, Sam Barnett wrote: >>>> >>>> Dear all, >>>> >>>> I have created a 2-step pipeline with a custom transformer followed by >>>> a simple SVC classifier, and I wish to run a grid-search over it. I am able >>>> to successfully create the transformer and the pipeline, and each of these >>>> elements work fine. However, when I try to use the fit() method on my >>>> GridSearchCV object, I get the following error: >>>> >>>> 57 # during fit. >>>> 58 if X.shape != self.input_shape_: >>>> ---> 59 raise ValueError('Shape of input is different from >>>> what was seen ' >>>> 60 'in `fit`') >>>> 61 >>>> >>>> ValueError: Shape of input is different from what was seen in `fit` >>>> >>>> For a full breakdown of the problem, I have written a Jupyter notebook >>>> showing exactly how the error occurs (this also contains all .py files >>>> necessary to run the notebook). Can anybody see how to work through this? >>>> >>>> Many thanks, >>>> Sam Barnett >>>> >>>> >>>> >>>> _______________________________________________ >>>> scikit-learn mailing listscikit-learn@python.orghttps://mail.python.org/mailman/listinfo/scikit-learn >>>> >>>> >>>> >>>> _______________________________________________ >>>> scikit-learn mailing list >>>> scikit-learn(a)python.org >>>> https://mail.python.org/mailman/listinfo/scikit-learn >>>> >>>> >>> >>> >>> _______________________________________________ >>> scikit-learn mailing listscikit-learn@python.orghttps://mail.python.org/mailman/listinfo/scikit-learn >>> >>> >>> >> > >

2 1

Extra trees tuning parameters
by muhammad waseem Aug. 4, 2017

Aug. 4, 2017

Hi All, I was wondering if you could please tell me what is the "nmin , the minimum sample size for splitting a node" (referred by Geurts et al., 2006) in scikit-learn API for Extra trees? Is it min_samples_split in skearn? Regards Waseem

2 1

OneClassSvm | Different results on different runs
by Abhishek Raj Aug. 4, 2017

Aug. 4, 2017

Hi, I am using one class svm for developing an anomaly detection model. I observed that different runs of training on the same data set outputs different accuracy. One run takes the accuracy as high as 98% and another run on the same data brings it down to 93%. Googling a little bit I found out that this is happening because of the random_state <http://scikit-learn.org/stable/modules/generated/sklearn.utils.check_random…> parameter but I am not clear of the details. Can anyone expand on how is the parameter exactly affecting my training and how I can figure out the best value to get the model with best accuracy? Thanks, Abhishek

5 6

Help With Text Classification
by pybokeh Aug. 3, 2017

Aug. 3, 2017

Hello, I am studying this example from scikit-learn's site: http://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_ data.html The problem that I need to solve is very similar to this example, except I have one additional feature column (part #) that is categorical of type string. My label or target values consist of just 2 values: 0 or 1. With that additional feature column, I am transforming it with a LabelEncoder and then I am encoding it with the OneHotEncoder. Then I am concatenating that one-hot encoded column (part #) to the text/document feature column (complaint), which I had applied the CountVectorizer and TfidfTransformer transformations. Then I chose the MultinomialNB model to fit my concatenated training data with. The problem I run into is when I invoke the prediction, I get a dimension mis-match error. Here's my jupyter notebook gist: http://nbviewer.jupyter.org/gist/anonymous/59ba930a783571c85ef86ba41424b311 I would gladly appreciate it if someone can guide me where I went wrong. Thanks! - Daniel

2 5

Re: [scikit-learn] Problems with running GridSearchCV on a pipeline with a custom transformer
by Andreas Mueller Aug. 3, 2017

Aug. 3, 2017

Hi Sam. You need to put these into a reachable namespace (possibly as private functions) so that they can be pickled. Please stay on the sklearn mailing list, I might not have time to reply. Andy On 08/03/2017 01:24 PM, Sam Barnett wrote: > Hi Andy, > > I've since tried a different solution: instead of a pipeline, I've > simply created a classifier that is for the most part like svm.SVC, > though it takes a few extra inputs for the sequentialisation step. > I've used a Python function that can compute the Gram matrix between > two datasets of any shape to pass into SVC(), though I'm now having > trouble with pickling on the check_estimator test. It appears that > SeqSVC.fit() doesn't like to have methods defined within it. Can you > see how to pass this test? (the .ipynb file shows the error). > > Best, > Sam > > On Wed, Aug 2, 2017 at 9:44 PM, Sam Barnett <sambarnett95(a)gmail.com > <mailto:sambarnett95@gmail.com>> wrote: > > You're right: it does fail without GridSearchCV when I change the > size of seq_test. I will look at the transform tomorrow to see if > I can work this out. Thank you for your help so far! > > On Wed, Aug 2, 2017 at 9:20 PM, Andreas Mueller <t3kcit(a)gmail.com > <mailto:t3kcit@gmail.com>> wrote: > > Change the size of seq_test in your notebook and you'll see > the failure without GridSearchCV. > I haven't looked at your code in detail, but transform is > supposed to work on arbitrary new data with the same number of > features. > Your code requires the test data to have the same shape as the > training data. > Cross-validation will lead to training data and test data > having different sizes. But I feel like something is already > wrong if your > test data size depends on your training data size. > > > > On 08/02/2017 03:08 PM, Sam Barnett wrote: >> Hi Andy, >> >> The purpose of the transformer is to take an ordinary kernel >> (in this case I have taken 'rbf' as a default) and return a >> 'sequentialised' kernel using a few extra parameters. Hence, >> the transformer takes an ordinary data-target pair X, y as >> its input, and the fit_transform(X, y) method will output the >> Gram matrix for X that is associated with this sequentialised >> kernel. In the pipeline, this Gram matrix is passed into an >> SVC classifier with the kernel parameter set to 'precomputed'. >> >> Therefore, I do not think your hacky solution would be >> possible. However, I am still unsure how to implement your >> first solution: won't the Gram matrix from the transformer >> contain all the necessary kernel values? Could you elaborate >> further? >> >> >> Best, >> Sam >> >> On Wed, Aug 2, 2017 at 5:05 PM, Andreas Mueller >> <t3kcit(a)gmail.com <mailto:t3kcit@gmail.com>> wrote: >> >> Hi Sam. >> GridSearchCV will do cross-validation, which requires to >> "transform" the test data. >> The shape of the test-data will be different from the >> shape of the training data. >> You need to have the ability to compute the kernel >> between the training data and new test data. >> >> A more hacky solution would be to compute the full kernel >> matrix in advance and pass that to GridSearchCV. >> >> You probably don't need it here, but you should also >> checkout what the _pairwise attribute does in >> cross-validation, >> because that it likely to come up when playing with kernels. >> >> Hth, >> Andy >> >> >> On 08/02/2017 08:38 AM, Sam Barnett wrote: >>> Dear all, >>> >>> I have created a 2-step pipeline with a custom >>> transformer followed by a simple SVC classifier, and I >>> wish to run a grid-search over it. I am able to >>> successfully create the transformer and the pipeline, >>> and each of these elements work fine. However, when I >>> try to use the fit() method on my GridSearchCV object, I >>> get the following error: >>> >>> 57 # during fit. >>> 58 if X.shape != self.input_shape_: >>> ---> 59 raise ValueError('Shape of input is >>> different from what was seen ' >>> 60 'in `fit`') >>> 61 >>> >>> ValueError: Shape of input is different from what was >>> seen in `fit` >>> >>> For a full breakdown of the problem, I have written a >>> Jupyter notebook showing exactly how the error occurs >>> (this also contains all .py files necessary to run the >>> notebook). Can anybody see how to work through this? >>> >>> Many thanks, >>> Sam Barnett >>> >>> >>> >>> _______________________________________________ >>> scikit-learn mailing list >>> scikit-learn(a)python.org <mailto:scikit-learn@python.org> >>> https://mail.python.org/mailman/listinfo/scikit-learn >>> <https://mail.python.org/mailman/listinfo/scikit-learn> >> >> >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org <mailto:scikit-learn@python.org> >> https://mail.python.org/mailman/listinfo/scikit-learn >> <https://mail.python.org/mailman/listinfo/scikit-learn> >> >> >> >> >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org <mailto:scikit-learn@python.org> >> https://mail.python.org/mailman/listinfo/scikit-learn >> <https://mail.python.org/mailman/listinfo/scikit-learn> > > >

1 0

question about class_weights in LogisticRegression
by Johnson, Jeremiah Aug. 3, 2017

Aug. 3, 2017

Hello all, I'm looking for confirmation on an implementation detail that is somewhere in liblinear, but I haven't found documentation for yet. When the class_weights='balanced' parameter is set in LogisticRegression, then the regularisation parameter for an observation from class I is class_weight[I] * C, where C is the usual regularization parameter - is this correct? Thanks, Jeremiah

3 3

Re: [scikit-learn] Nearest neighbor search with 2 distance measures
by "Умнов Алексей (Alexey Umnov)" Aug. 3, 2017

Aug. 3, 2017

Здравствуйте! Я в данный момент в отпуске, и вернусь 15 августа. -- Умнов Алексей

3 8

Problems with running GridSearchCV on a pipeline with a custom transformer
by Sam Barnett Aug. 2, 2017

Aug. 2, 2017

Dear all, I have created a 2-step pipeline with a custom transformer followed by a simple SVC classifier, and I wish to run a grid-search over it. I am able to successfully create the transformer and the pipeline, and each of these elements work fine. However, when I try to use the fit() method on my GridSearchCV object, I get the following error: 57 # during fit. 58 if X.shape != self.input_shape_: ---> 59 raise ValueError('Shape of input is different from what was seen ' 60 'in `fit`') 61 ValueError: Shape of input is different from what was seen in `fit` For a full breakdown of the problem, I have written a Jupyter notebook showing exactly how the error occurs (this also contains all .py files necessary to run the notebook). Can anybody see how to work through this? Many thanks, Sam Barnett

2 2

minibatchkmeans deprecation warning?
by Chris Carrion Aug. 2, 2017

Aug. 2, 2017

Hi, I’m working in an environment provided by Quantopian, an algorithmic-traders hub for research. I imported the minibatch kmeans from sklearn.clusters in the environment they provided, but I’m getting a deprecation warning. After reaching out to Quantopian support, they claim it’s something with the way sklearn is coded, and nothing can be done on their end. I was wondering whether this was true or not. Curious, Chris

2 6