🌐 Mailman 3 February 2019 - scikit-learn - python.org

Re: [scikit-learn] Sprint discussion points?
by Hanmin Qin Feb. 18, 2019

Feb. 18, 2019

Maybe it's worthwhile to discuss (and release) 0.20.3 during the sprint. We're almost ready except for a few test failures on specific platforms. I've labeled all the related PRs (i.e., PRs with a what's new entry in 0.20.3) as 0.20.3. We need to decide whether we want to backport more bug fixes (maybe more doc/example corrections) to 0.20.3. Joel mentions this several times but seems that he hasn't made the decision. I tend to do so, though technically we should only include bug fixes related to features introduced in 0.20.X (but I won't argue if someone make the decision). Some bugs seems not trivial (e.g., #13142 related to BaseMixture and #13124 related to StratifiedKFold). Hanmin Qin ----- Original Message ----- From: Olivier Grisel <olivier.grisel(a)ensta.org> To: Scikit-learn mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Sprint discussion points? Date: 2019-02-15 23:06 I would also add generalizing early stopping options to most estimators. This is a bit related to Joel's point on max_iter consistency in LogisticRegression. -- Olivier _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://mail.python.org/mailman/listinfo/scikit-learn

3 2

Re: [scikit-learn] Inclusion of an LSTM Classifier
by David Burns Feb. 17, 2019

Feb. 17, 2019

There is an sklearn wrapper for Keras models in the Keras library. That's an easy way to use LSTM in sklearn. Also the sklearn estimator API is pretty easy to figure out if you want to roll your own wrapper for any model really.

2 1

Fw: Inclusion of an LSTM Classifier
by Yash Raj Rai Feb. 17, 2019

Feb. 17, 2019

________________________________ From: Yash Raj Rai Sent: Sunday, February 17, 2019 2:34 PM To: scikit-learn(a)python.org Subject: Inclusion of an LSTM Classifier Hello I wanted to know if there are any on-going projects on LSTM Classifier model in sklearn. If no, is there any possibility of its inclusion in the library? Is there anything beyond the contributor's guidelines that I need to know for the introduction of a new model? Than you.

2 1

inconsistency across version
by Huan Tran Feb. 16, 2019

Feb. 16, 2019

Dear community, I did a very small pca analysis on a 3D data to print out the explained_variance. I found that with scikit-learn 0.18.1 AND 0.20.2, the results are significantly different. In particular, for 0.18.1 I got +3.875925353581E+00 +3.270175297443E+00 +2.207814537475E+00 and with 0.20.2, I got +4.651110424297E+00 +3.924210356932E+00 +2.649377444970E+00 Could anyone has a hint on what is going on? FYI, my data and code are enclosed. Many thanks. Huan My data is -3.117642E+00, 1.453819E+00, -7.952874E-02 3.081224E+00, 1.453819E+00, -7.952874E-02 1.376932E-01, -2.491454E+00, -1.908521E-01 9.578602E-02, 3.632759E+00, -1.908521E-01 -1.238644E-01, 5.396424E-02, -3.147031E+00 6.335262E-01, 1.393937E+00, 2.500474E+00 and my code is import pandas as pd import numpy as np from sklearn import decomposition df = pd.read_csv('data', delimiter=',', header=None) data = np.array(df) X = data[:,:] data_size = X.shape[0] feature_dim = X.shape[1] print X pca = decomposition.PCA(n_components=feature_dim) X_transformed = pca.fit_transform(X) print "%+4.12E %+4.12E %+4.12E" %(pca.explained_variance_[0], pca.explained_variance_[1], pca.explained_variance_[2])

2 2

[Question & Help]The criterion of data size for choosing a right algorithm.
by skim22 Feb. 14, 2019

Feb. 14, 2019

Dear Sir or Madam, Good morning, My name is Steven Kim from Memphis, and I am a graduate student at the University of Memphis. Recently, I found the page of choosing the right estimator on the official website (https://scikit-learn.org/stable/tutorial/machine_learning_map/index.html). It was greatly helpful to distinguish what algorithms I should use. But, I would like to know something in detail on the page. My question is that there are several criteria of sample size such as ">50" and "100k>" before the decision. Could you let me know that the grounds( ex) academic papers ) for the sample sizes? It would be helpful to understand them more deeply. Regards, Steven Kim

1 0

cross_validate() with HMM
by Anni Bauer Feb. 13, 2019

Feb. 13, 2019

Hi! I want to be able to run each fold of a k-fold cross validation fold in parallel, using all of my 6 CPUs at once. My model is a hidden markov model and I want to train it using the training portion of the data and then extract the anomaly score (negative log-likelihood) of each test sequence of the test portion with every fold and use ROC as an evaluation technique with every fold. I have found the function cross_validate() which seems to provide the option of running things in parralel with n_jobs = -1. I assume the estimator is then my HMM model. As of now I'm using pomegranate to train the model and extract the anomaly score of the test sequences. I don't understand how to call the cross_validate function with the right arguments for my HMM model. All examples I've seen havn't used HMM. I'm confused on where to specify the hidden states number if Im not callign my usual pomegranate function from_samples(), which I've used before. Also how can I extract the anomay scores within each fold using this function? I'm unsure what exactly is happening with in the cross_validate function and how to control it the way I need. If anyone has an example or explanation or another idea on how to run the folds in parallel, I would really appreciate it! This is my attempt of using cross_validate, which gets stuck or seems to not be running through (although I'm quite sure I'm not using it properly): import pomegranate import sklearn model = pomegranate.HiddenMarkovModel() results = cross_validate(model, listToUse, y=None, groups=None, scoring=None, cv=3, n_jobs=-1, verbose=10) print(results) Below is how I've manually set my cross-validation up as of now: listExample = [] kfold = KFold(10, True) for train, test in kfold.split(listToUse): listExample.append([listToUse[train], listToUse[test]]) scoreList = [] for ex in listExample: hmmModel = hmm.hmm(ex[0]) scoreListFold = [] mid = time.time() for li in ex[1]: prob = hmmModel.log_probability(li) scoreListFold.append(prob) scoreList.append(numpy.mean(scoreListFold)) avg = numpy.mean(scoreList) Thanks again! Anni

1 0

How to deal with hierarchical and real-time analysis in machine learning?
by lampahome Feb. 13, 2019

Feb. 13, 2019

For example, I may have huge different regions and every regions have many or less points. And I also want to real-time to analyze the newest data and older data, but I don't want to put data into memory cuz I don't have enough memory. What I thought I can use is partial_fit to accept streaming data when new data comes in. But the incoming data has hierarchical, it's hard to cluster them cuz I don't have older and newer data together to cluster. How to design the system better? thx

2 1

How to design system if I have huge items to real time analysis?
by lampahome Feb. 11, 2019

Feb. 11, 2019

Hello, I'm figuring out some way to deal with real time regression on disk block access times. But I have multiple patterns of each block. Ex: Some block were accessed once a month, some blocks were accessed everyday. They all have different access patterns. The question is that how to predict access pattern of each block well in real time? I tried regression.ensemble but they don't have partial_fit to fit real time. I found leanr_model.SGDRegressor and neural_network.MLPRegressor, they have partial_fit. But they only predict one result.(But result of each block shouldn't be the same cuz they have different access times) I want to predict access times of each block in real time but I don't know how to reach the same effect. Should I change algo? thx

1 0

VOTE: scikit-learn governance document
by Andreas Mueller Feb. 10, 2019

Feb. 10, 2019

Hey all. I want to call a vote on the final version on the scikit-learn governance document, which can be found in this PR: https://github.com/scikit-learn/scikit-learn/pull/12878 It underwent some significant changes in the last couple of weeks. The two-sentence summary is: conflicts are resolved by vote among core devs, with a technical committee resolving anything that can not be decided by at least a 2/3 majority. The initial technical committee is Alexander Gramfort, Olivier Grisel, Joel Nothman, Hanmin Qin, Gaël Varoquaux and myself (Andreas Müller). I would ask all of the *core developers* to either vote +1 for the governance doc, -1 against it, or to explicitly abstain here on the public mailing list (which is the way any vote will be conducted according to the new governance document). I suggest we leave the vote open for two weeks, so that the decision is made before the sprint and we can take actions. Anyone can still comment on the PR or here, though I would rather not make more changes as this has already been discussed to some length. Thank you for participating, Andy

4 4

Probabilities for LogisticRegression and LDA
by Guillaume Lemaître Feb. 8, 2019

Feb. 8, 2019

I was earlier looking at the code of predict_proba of LDA and LogisticRegression. While we certainly some bugs I was a bit confused and I thought an email would be better than opening an issue since that might not be one. In the case of multiclass classification, the probabilities could be computed with two different assumptions - either as a set of independent binary regression or as a log-linear model ( https://en.wikipedia.org/wiki/Multinomial_logistic_regression). Then, we can compute the probabilities either by using a class as a pivot and computing exp(beta_c X) / 1 + sum(exp(beta_k X)) or using all classes and computing a softmax. My question is related to the LogisticRegression in the OvR scheme. Naively, I thought that it was corresponding to the former case (case of a set of independent regression). However, we are using another normalization there which was first implemented in liblinear. I search on liblinear's issue tracker and found: https://github.com/cjlin1/liblinear/pull/20 It is related to the following paper: https://www.csie.ntu.edu.tw/~cjlin/papers/generalBT.pdf My skill in math is limited and I am not sure to grasp what is going on? Anybody could bring some lights on this OvR normalization and why is it different from the case of a set of independent regression describe in Wikipedia? Cheers, -- Guillaume Lemaitre INRIA Saclay - Parietal team Center for Data Science Paris-Saclay https://glemaitre.github.io/

2 2