🌐 Mailman 3 December 2016 - scikit-learn - python.org

Markov Clustering?
by Allan Visochek Dec. 6, 2016

Dec. 6, 2016

Hi there, My name is Allan Visochek, I'm a data scientist and web developer and I love scikit-learn so first of all, thanks so much for the work that you do. I'm reaching out because I've found the markov clustering algorithm to be quite useful for me in some of my work and noticed that there is no implementation in scikit-learn, is anybody working on this? If not, id be happy to take this on. I'm new to open source, but I've been working with python for a few years now. Best, -Allan

5 11

[semi-supervised learning] Using a pre-existing graph with LabelSpreading API
by Delip Rao Dec. 5, 2016

Dec. 5, 2016

Hello, I have an existing graph dataset in the edge format: node_i node_j weight The number of nodes are around 3.6M, and the number of edges are around 72M. I also have some labeled data (around a dozen per class with 16 classes in total), so overall, a perfect setting for label propagation or its variants. In particular, I want to try the LabelSpreading implementation for the regularization. I looked at the documentation and can't find a way to plug in a pre-computed graph (or adjacency matrix). So two questions: 1. What are any scaling issues I should be aware of for a dataset of this size? I can try sparsifying the graph, but would love to learn any knobs I should be aware of. 2. How do I plugin an existing weighted graph with the current API? Happy to use any undocumented features. Thanks in advance! Delip

2 1

Nested Leave One Subject Out (LOSO) cross validation with scikit
by Ludovico Coletta Dec. 5, 2016

Dec. 5, 2016

Dear scikit experts, I'm struggling with the implementation of a nested cross validation. My data: I have 26 subjects (13 per class) x 6670 features. I used a feature reduction algorithm (you may have heard about Boruta) to reduce the dimensionality of my data. Problems start now: I defined LOSO as outer partitioning schema. Therefore, for each of the 26 cv folds I used 24 subjects for feature reduction. This lead to a different number of features in each cv fold. Now, for each cv fold I would like to use the same 24 subjects for hyperparameter optimization (SVM with rbf kernel). This is what I did: cv = list(LeaveOneout(len(y))) # in y I stored the labels inner_train = [None] * len(y) inner_test = [None] * len(y) ii = 0 while ii < len(y): cv = list(LeaveOneOut(len(y))) a = cv[ii][0] a = a[:-1] inner_train[ii] = a b = cv[ii][0] b = np.array(b[((len(cv[0][0]))-1)]) inner_test[ii]=b ii = ii + 1 custom_cv = zip(inner_train,inner_test) # inner cv pipe_logistic = Pipeline([('scl', StandardScaler()),('clf', SVC(kernel="rbf"))]) parameters = [{'clf__C': np.logspace(-2, 10, 13), 'clf__gamma':np.logspace(-9, 3, 13)}] scores = [None] * (len(y)) ii = 0 while ii < len(scores): a = data[ii][0] # data for train b = data[ii][1] # data for test c = np.concatenate((a,b)) # shape: number of subjects * number of features d = cv[ii][0] # labels for train e = cv[ii][1] # label for test f = np.concatenate((d,e)) grid_search = GridSearchCV(estimator=pipe_logistic, param_grid=parameters, verbose=1, scoring='accuracy', cv= zip(([custom_cv[ii][0]]), ([custom_cv[ii][1]]))) scores[ii] = cross_validation.cross_val_score(grid_search, c, y[f], scoring='accuracy', cv = zip(([cv[ii][0]]), ([cv[ii][1]]))) ii = ii + 1 However, I got the following error message: index 25 is out of bounds for size 25 Would it be so bad if I do not perform a nested LOSO but I use the default setting for hyperparameter optimization? Any help would be really appreciated

3 4

Re: [scikit-learn] Nested Leave One Subject Out (LOSO) cross validation with scikit
by Ludovico Coletta Dec. 5, 2016

Dec. 5, 2016

Unfortunately, it did not work. I think I am doing something wrong when passing the nested cv, but I do not understand where. If I omit the cv argument in the grid search it runs smoothly. I would like to have LeaveOneOut in both the outer and inner cv, how would you implement such a thing? Best Ludovico ________________________________ Da: scikit-learn <scikit-learn-bounces+ludo25_90=hotmail.com(a)python.org> per conto di scikit-learn-request(a)python.org <scikit-learn-request(a)python.org> Inviato: domenica 4 dicembre 2016 22.27 A: scikit-learn(a)python.org Oggetto: scikit-learn Digest, Vol 9, Issue 13 Send scikit-learn mailing list submissions to scikit-learn(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... or, via email, send a message with subject or body 'help' to scikit-learn-request(a)python.org You can reach the person managing the list at scikit-learn-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of scikit-learn digest..." Today's Topics: 1. Nested Leave One Subject Out (LOSO) cross validation with scikit (Ludovico Coletta) 2. Re: Adding samplers for intersection/Jensen-Shannon kernels (avn(a)mccme.ru) 3. Re: Nested Leave One Subject Out (LOSO) cross validation with scikit (Raghav R V) ---------------------------------------------------------------------- Message: 1 Date: Sun, 4 Dec 2016 20:12:29 +0000 From: Ludovico Coletta <ludo25_90(a)hotmail.com> To: "scikit-learn(a)python.org" <scikit-learn(a)python.org> Subject: [scikit-learn] Nested Leave One Subject Out (LOSO) cross validation with scikit Message-ID: <BLUPR0301MB2017B71792C520F6ACE525478C800(a)BLUPR0301MB2017.namprd03.prod.outlook.com> Content-Type: text/plain; charset="iso-8859-1" Dear scikit experts, I'm struggling with the implementation of a nested cross validation. My data: I have 26 subjects (13 per class) x 6670 features. I used a feature reduction algorithm (you may have heard about Boruta) to reduce the dimensionality of my data. Problems start now: I defined LOSO as outer partitioning schema. Therefore, for each of the 26 cv folds I used 24 subjects for feature reduction. This lead to a different number of features in each cv fold. Now, for each cv fold I would like to use the same 24 subjects for hyperparameter optimization (SVM with rbf kernel). This is what I did: cv = list(LeaveOneout(len(y))) # in y I stored the labels inner_train = [None] * len(y) inner_test = [None] * len(y) ii = 0 while ii < len(y): cv = list(LeaveOneOut(len(y))) a = cv[ii][0] a = a[:-1] inner_train[ii] = a b = cv[ii][0] b = np.array(b[((len(cv[0][0]))-1)]) inner_test[ii]=b ii = ii + 1 custom_cv = zip(inner_train,inner_test) # inner cv pipe_logistic = Pipeline([('scl', StandardScaler()),('clf', SVC(kernel="rbf"))]) parameters = [{'clf__C': np.logspace(-2, 10, 13), 'clf__gamma':np.logspace(-9, 3, 13)}] scores = [None] * (len(y)) ii = 0 while ii < len(scores): a = data[ii][0] # data for train b = data[ii][1] # data for test c = np.concatenate((a,b)) # shape: number of subjects * number of features d = cv[ii][0] # labels for train e = cv[ii][1] # label for test f = np.concatenate((d,e)) grid_search = GridSearchCV(estimator=pipe_logistic, param_grid=parameters, verbose=1, scoring='accuracy', cv= zip(([custom_cv[ii][0]]), ([custom_cv[ii][1]]))) scores[ii] = cross_validation.cross_val_score(grid_search, c, y[f], scoring='accuracy', cv = zip(([cv[ii][0]]), ([cv[ii][1]]))) ii = ii + 1 However, I got the following error message: index 25 is out of bounds for size 25 Would it be so bad if I do not perform a nested LOSO but I use the default setting for hyperparameter optimization? Any help would be really appreciated

1 0

Adding samplers for intersection/Jensen-Shannon kernels
by avn＠mccme.ru Dec. 4, 2016

Dec. 4, 2016

Hello, In the course of my work, I've made samplers for intersection/Jensen-Shannon kernels, just by small modifications to sklearn.kernel_approximation.AdditiveChi2Sampler code. Intersection kernel proved to be the best one for my task (clustering Docstrum feature vectors), so perhaps it'd be good to add those samplers alongside AdditiveChi2Sampler? Should I proceed with creating a pull request? Or, perhaps, those kernels were not already included for some good reason? With best regards, -- Valery

2 2

Github project management tools
by Joel Nothman Dec. 3, 2016

Dec. 3, 2016

One of the biggest issues with scikit-learn as a project is managing its backlog of issues; another is release scheduling. Some of this cannot be fixed as long as our model of voluntary contribution (with a couple of important exceptions) does not change. However, it may be worth considering the new project management features in Github. At the moment we have the following management: * labels corresponding to type (bug, enhancement, new feat, question), scope (API, Build/CI, ?Large Scale, Documentation), difficulty (easy, moderate), status/scheduling (needs contributor, needs review, sprint). * PR status management with title prefixes [WIP], [MRG], [MRG+1], [MRG+2] Firstly, we might benefit from prefixing labels by category, i.e. difficulty:easy so that complementary labels appear together. In truth, PRs have roughly these statuses: * WIP (not ready for review) * waiting for review * waiting for changes (with or without one of the following) * in dispute (i.e. fundamental doubts about the PR) * the above together with 1 or 2 "official" approvals * ready for merge (pending minor changes such as what's new documentation) New github features: * reviews with "approved" or "request changes". A list of approvers can be found in the merge/CI panel. We could replace the MRG+1 annotation with this and use it to track disputation too. I'm not sure how it works with changes that are added after approval. I think it would have avoided one improper merge by me... One downside is that there does not yet seem to be a way to search for PRs with a specified level of approval (while searching for "MRG+1" sort-of works). * Milestone prioritising: issues in a milestone, such as https://github.com/scikit-learn/scikit-learn/milestone/21, can be ranked with drag-and-drop. I think this could help with release scheduling as it would allow us to identify the top priorities for a release and see when enough of them are completed. * The Kanban-style workflow management of the new Projects tool https://github.com/scikit-learn/scikit-learn/projects is another way of managing status and, I think, priority, for a small set of related issues. This might be an alternative way of managing milestone scope, or of working towards big changes like the one just completed for model selection; like proposed expansions to get_feature_names expansion; like estimator tags; making utilities public/private... So with the goal of making it easier to track where attention is most needed, and when to move to release: What's worth trying?

13 37

Trying to get learning curves with custom scorer and leave one group out
by Matteo Niccoli Dec. 3, 2016

Dec. 3, 2016

HI all, I want to plot learning curves on a trained SVM classifier, using a custom scorer, and using Leave One Group Out as the method of crossvalidation. I thought I had it figured out, but two different scorers - 'f1_micro' and 'accuracy' - will yield identical values. I am confused, is that supposed to be the case? Here's my code (unfortunately I cannot share the data as it is not open): from sklearn import svm SVC_classifier_LOWO_VC0 = svm.SVC(cache_size=800, class_weight=None, coef0=0.0, decision_function_shape=None, degree=3, gamma=0.01, kernel='rbf', max_iter=-1, probability=False, random_state=1, shrinking=True, tol=0.001, verbose=False) training_data = pd.read_csv('training_data.csv') scaler = preprocessing.StandardScaler().fit(X) X = scaler.transform(X) y = training_data['Targets'].values groups = training_data["Groups"].values Fscorer = make_scorer(f1_score, average = 'micro') logo = LeaveOneGroupOut() parm_range0 = np.logspace(-2, 6, 9) train_scores0, test_scores0 = validation_curve(SVC_classifier_LOWO_VC0, X, y, "C", parm_range0, cv =logo.split(X, y, groups=groups), scoring = Fscorer) Now, from: train_scores_mean0 = np.mean(train_scores0, axis=1) train_scores_std0 = np.std(train_scores0, axis=1) test_scores_mean0 = np.mean(test_scores0, axis=1) test_scores_std0 = np.std(test_scores0, axis=1) print test_scores_mean0 print np.amax(test_scores_mean0) print np.logspace(-2, 6, 9)[test_scores_mean0.argmax(axis=0)] I get: [ 0.20257407 0.35551122 0.40791047 0.49887676 0.5021742 0.50030438 0.49426622 0.48066419 0.4868987 ] 0.502174200206 100.0 If I create a new classifier, but with the same parameters, and run everything exactly as before, except for the scoring, e.g.: parm_range1 = np.logspace(-2, 6, 9) train_scores1, test_scores1 = validation_curve(SVC_classifier_LOWO_VC1, X, y, "C", parm_range1, cv =logo.split(X, y, groups=wells), scoring = 'accuracy') train_scores_mean1 = np.mean(train_scores1, axis=1) train_scores_std1= np.std(train_scores1, axis=1) test_scores_mean1 = np.mean(test_scores1, axis=1) test_scores_std1 = np.std(test_scores1, axis=1) print test_scores_mean1 print np.amax(test_scores_mean1) print np.logspace(-2, 6, 9)[test_scores_mean1.argmax(axis=0)] I get exactly the same answer: [ 0.20257407 0.35551122 0.40791047 0.49887676 0.5021742 0.50030438 0.49426622 0.48066419 0.4868987 ] 0.502174200206 100.0 How is that possible, am I doing something wrong, or missing something? Thanks

2 2

random forests using grouped data
by Thomas Evangelidis Dec. 1, 2016

Dec. 1, 2016

Greetings I have grouped data which are divided into actives and inactives. The features are two different types of normalized scores (0-1), where the higher the score the most probable is an observation to be an "active". My data look like this: Group1: score1 = [0.56, 0.34, 0.42, 0.12, 0.08, 0.21, ...] score2 = [ y=[1,1,1,0,0,0, ...] Group2: score1 = [0 score2 = [ y=[1,1,1,1,1] ...... Group24: score1 = [0 score2 = [ y=[1,1,1,1,1] I searched in the documentation about treatment of grouped data, but the only thing I found was how do do cross-validation. My question is whether there is any special algorithm that creates random forests from these type of grouped data. thanks in advance Thomas -- ====================================================================== Thomas Evangelidis Research Specialist CEITEC - Central European Institute of Technology Masaryk University Kamenice 5/A35/1S081, 62500 Brno, Czech Republic email: tevang(a)pharm.uoa.gr tevang3(a)gmail.com website: https://sites.google.com/site/thomasevangelidishomepage/

3 3