🌐 Mailman 3 September 2017 - scikit-learn - python.org

Accessing Clustering Feature Tree in Birch
by Sema Atasever Oct. 3, 2017

Oct. 3, 2017

Dear scikit-learn members, I have written about this subject before but I have not completely solved my question. - How can i *access Clustering Feature Tree* in Birch? - For example, how many clusters are there in the hierarchy under the *root node* and what are the data samples in this clusters? - Can I get them separately for 3 trees? Best. *Birch Implementation Code:* from sklearn.cluster import Birch from sklearn.externals import joblib import numpy as np import matplotlib.pyplot as plt from time import time from sklearn.datasets.samples_generator import make_blobs from sklearn.neighbors import NearestCentroid X=np.loadtxt(open("C:\dataset.txt", "rb"), delimiter=";") brc = Birch(branching_factor=50, n_clusters=None, threshold=0.5,compute_labels=True,copy=True) brc.fit(X) birch_predict=brc.predict(X) print ("\nClustering_result:\n") print (birch_predict) np.savetxt('birch_predict_CLASS_0.csv', birch_predict,fmt="%i", delimiter=',') myroot = brc.root_ centroids = brc.subcluster_centers_ plt.plot(X[:,0], X[:,1], '+') plt.plot(centroids[:,0], centroids[:,1], 'o') plt.show() labels = brc.subcluster_labels_ n_clusters = np.unique(labels).size print("n_clusters : %d" % n_clusters + "\n")

4 5

TF-IDF
by Apurva Nandan Oct. 2, 2017

Oct. 2, 2017

Hello, Could anybody tell me the difference between using augmented frequency (which is used for weighting term frequencies to eliminate the bias towards larger documents) and cosine normalization (l2 norm which scikit-learn uses for TfidfTransformer). Augmented frequency is given by the following equation. It tries to divide the natural term frequency by the maximum frequency of any term in the document. [image: Inline image 1] Do they both do the same thing when it comes to eliminating bias towards larger documents? I suppose scikit-learn uses the natural term freq, and using cosine normalization is enabled with using norm=l2 Any help would be appreciated! - Apurva

2 1

anti-correlated predictions by SVR
by Thomas Evangelidis Sept. 26, 2017

Sept. 26, 2017

Greetings, I don't know if anyone encountered this before, but sometimes I get anti-correlated predictions by the SVR I that am training. Namely, the Pearson's R and Kendall's tau are negative when I compare the predictions on the external test set with the true values. However, the SVR predictions on the training set have positive correlations with the experimental values and hence I can't think of a way to know in advance if the trained SVR will produce anti-correlated predictions in order to change their sign and avoid the disaster. Here is an example of what I mean: Training set predictions: R=0.452422, tau=0.333333 External test set predictions: R=-0.537420, tau-0.300000 Obviously, in a real case scenario where I wouldn't have the external test set I would have used the worst observation instead of the best ones. Has anybody any idea about how I could prevent this? thanks in advance Thomas -- ====================================================================== Dr Thomas Evangelidis Post-doctoral Researcher CEITEC - Central European Institute of Technology Masaryk University Kamenice 5/A35/2S049, 62500 Brno, Czech Republic email: tevang(a)pharm.uoa.gr tevang3(a)gmail.com website: https://sites.google.com/site/thomasevangelidishomepage/

3 4

batch_size for small training sets
by Thomas Evangelidis Sept. 24, 2017

Sept. 24, 2017

Greetings, I traing MLPRegressors using small datasets, usually with 10-50 observations. The default batch_size=min(200, n_samples) for the adam optimizer, and because my n_samples is always < 200, it is eventually batch_size=n_samples. According to the theory, stochastic gradient-based optimizers like adam perform better in the small batch regime. Considering the above, what would be a good batch_size value in my case (e.g. 4)? Is there any rule of thump to select the batch_size when the n_samples is small or must the choice be based on trial and error? -- ====================================================================== Dr Thomas Evangelidis Post-doctoral Researcher CEITEC - Central European Institute of Technology Masaryk University Kamenice 5/A35/2S049, 62500 Brno, Czech Republic email: tevang(a)pharm.uoa.gr tevang3(a)gmail.com website: https://sites.google.com/site/thomasevangelidishomepage/

2 1

Decision Tree Regressor - DepthFirstTreeBuilder vs BestFirstTreeBuilder
by hanzi mao Sept. 22, 2017

Sept. 22, 2017

Hi, I am reading the source code of the Decision Tree Regressor in sklearn. To build a tree, there are two fashions: depth first and best first. Best first fashion is adopted only when user set max_leaf_nodes. Otherwise, the tree will be built using the DepthFirstTreeBuilder. My questions are: 1. Are there any practical considerations when to use depth-first or best-first? Dose the depth-first fashion has a overwhelming advantage / popularity compared with the best-first one which makes it a default choice? 2. I am kind of confused why using a optional parameter max_leaf_nodes to decide whether to use BestFirstTreeBuilder or not. I am wondering if there are some considerations when you decide to develop like this. Thanks! Best, Hanna

2 2

Re: [scikit-learn] LatentDirichletAllocation failing to find topics in NLTK Gutenberg corpus?
by Markus Konrad Sept. 20, 2017

Sept. 20, 2017

Hi Chyi-Kwei, thanks for digging into this. I made similar observations with Gensim when using only a small number of (big) documents. Gensim also uses the Online Variational Bayes approach (Hoffman et al.). So could it be that the Hoffman et al. method is problematic in such scenarios? I found that Gibbs sampling based implementations provide much more informative topics in this case. If this was the case, then if I'd slice the documents in some way (say every N paragraphs become a "document") then I should get better results with scikit-learn and Gensim, right? I think I'll try this out tomorrow. Best, Markus > Date: Sun, 17 Sep 2017 23:52:51 +0000 > From: chyi-kwei yau <chyikwei.yau(a)gmail.com> > To: Scikit-learn mailing list <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] LatentDirichletAllocation failing to find > topics in NLTK Gutenberg corpus? > Message-ID: > <CAK-jh0Ygd8fSdJom+gdDOHvAYCPuJVHHX77qcd+d4_xm6vi9yA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi Markus, > > I tried your code and find the issue might be there are only 18 docs > in the Gutenberg > corpus. > if you print out transformed doc topic distribution, you will see a lot of > topics are not used. > And since there is no words assigned to those topics, the weights will be > equal to`topic_word_prior` parameter. > > You can print out the transformed doc topic distributions like this: > ------------- >>>> doc_distr = lda.fit_transform(tf) > >>>> for d in doc_distr: > ... print np.where(d > 0.001)[0] > ... > [17 27] > [17 27] > [17 27 28] > [14] > [ 2 4 28] > [ 2 4 15 21 27 28] > [1] > [ 1 2 17 21 27 28] > [ 2 15 17 22 28] > [ 2 17 21 22 27 28] > [ 2 15 17 28] > [ 2 17 21 27 28] > [ 2 14 15 17 21 22 27 28] > [15 22] > [ 8 11] > [8] > [ 8 24] > [ 2 14 15 22] > > and my full test scripts are here: > https://gist.github.com/chyikwei/1707b59e009d381e1ce1e7a38f9c7826 > > Best, > Chyi-Kwei > > > On Thu, Sep 14, 2017 at 7:26 AM Markus Konrad <markus.konrad(a)wzb.eu> wrote: > >> Hi there, >> >> I'm trying out sklearn's latent Dirichlet allocation implementation for >> topic modeling. The code from the official example [1] works just fine and >> the extracted topics look reasonable. However, when I try other corpora, >> for example the Gutenberg corpus from NLTK, most of the extracted topics >> are garbage. See this example output, when trying to get 30 topics: >> >> Topic #0: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) >> fatiguing (0.01) >> Topic #1: mr (1081.61) emma (866.01) miss (506.94) mrs (445.56) jane >> (301.83) >> Topic #2: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) >> fatiguing (0.01) >> Topic #3: thee (82.64) thou (70.0) thy (66.66) father (56.45) mother >> (55.27) >> Topic #4: anne (498.74) captain (303.01) lady (173.96) mr (172.07) charles >> (166.21) >> Topic #5: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) >> fatiguing (0.01) >> Topic #6: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) >> fatiguing (0.01) >> Topic #7: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) >> fatiguing (0.01) >> ... >> >> Many topics tend to have the same weights, all equal to the >> `topic_word_prior` parameter. >> >> This is my script: >> >> import nltk >> from sklearn.feature_extraction.text import CountVectorizer >> from sklearn.decomposition import LatentDirichletAllocation >> >> def print_top_words(model, feature_names, n_top_words): >> for topic_idx, topic in enumerate(model.components_): >> message = "Topic #%d: " % topic_idx >> message += " ".join([feature_names[i] + " (" + str(round(topic[i], >> 2)) + ")" >> for i in topic.argsort()[:-n_top_words - >> 1:-1]]) >> print(message) >> >> >> data_samples = [nltk.corpus.gutenberg.raw(f_id) >> for f_id in nltk.corpus.gutenberg.fileids()] >> >> tf_vectorizer = CountVectorizer(max_df=0.95, min_df=2, >> stop_words='english') >> tf = tf_vectorizer.fit_transform(data_samples) >> >> lda = LatentDirichletAllocation(n_components=30, >> learning_method='batch', >> n_jobs=-1, # all CPUs >> verbose=1, >> evaluate_every=10, >> max_iter=1000, >> doc_topic_prior=0.1, >> topic_word_prior=0.01, >> random_state=1) >> >> lda.fit(tf) >> tf_feature_names = tf_vectorizer.get_feature_names() >> print_top_words(lda, tf_feature_names, 5) >> >> Is there a problem in how I set up the LatentDirichletAllocation instance >> or pass the data? I tried out different parameter settings, but none of >> them provided good results for that corpus. I also tried out alternative >> implementations (like the lda package [2]) and those were able to find >> reasonable topics. >> >> Best, >> Markus >> >> >> [1] >> http://scikit-learn.org/stable/auto_examples/applications/plot_topics_extra… >> [2] http://pythonhosted.org/lda/

2 5

LatentDirichletAllocation failing to find topics in NLTK Gutenberg corpus?
by Markus Konrad Sept. 17, 2017

Sept. 17, 2017

Hi there, I'm trying out sklearn's latent Dirichlet allocation implementation for topic modeling. The code from the official example [1] works just fine and the extracted topics look reasonable. However, when I try other corpora, for example the Gutenberg corpus from NLTK, most of the extracted topics are garbage. See this example output, when trying to get 30 topics: Topic #0: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) fatiguing (0.01) Topic #1: mr (1081.61) emma (866.01) miss (506.94) mrs (445.56) jane (301.83) Topic #2: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) fatiguing (0.01) Topic #3: thee (82.64) thou (70.0) thy (66.66) father (56.45) mother (55.27) Topic #4: anne (498.74) captain (303.01) lady (173.96) mr (172.07) charles (166.21) Topic #5: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) fatiguing (0.01) Topic #6: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) fatiguing (0.01) Topic #7: zoological (0.01) fathoms (0.01) fatigued (0.01) fatigues (0.01) fatiguing (0.01) ... Many topics tend to have the same weights, all equal to the `topic_word_prior` parameter. This is my script: import nltk from sklearn.feature_extraction.text import CountVectorizer from sklearn.decomposition import LatentDirichletAllocation def print_top_words(model, feature_names, n_top_words): for topic_idx, topic in enumerate(model.components_): message = "Topic #%d: " % topic_idx message += " ".join([feature_names[i] + " (" + str(round(topic[i], 2)) + ")" for i in topic.argsort()[:-n_top_words - 1:-1]]) print(message) data_samples = [nltk.corpus.gutenberg.raw(f_id) for f_id in nltk.corpus.gutenberg.fileids()] tf_vectorizer = CountVectorizer(max_df=0.95, min_df=2, stop_words='english') tf = tf_vectorizer.fit_transform(data_samples) lda = LatentDirichletAllocation(n_components=30, learning_method='batch', n_jobs=-1, # all CPUs verbose=1, evaluate_every=10, max_iter=1000, doc_topic_prior=0.1, topic_word_prior=0.01, random_state=1) lda.fit(tf) tf_feature_names = tf_vectorizer.get_feature_names() print_top_words(lda, tf_feature_names, 5) Is there a problem in how I set up the LatentDirichletAllocation instance or pass the data? I tried out different parameter settings, but none of them provided good results for that corpus. I also tried out alternative implementations (like the lda package [2]) and those were able to find reasonable topics. Best, Markus [1] http://scikit-learn.org/stable/auto_examples/applications/plot_topics_extra… [2] http://pythonhosted.org/lda/

2 1

Terminating a Pipeline with a NearestNeighbors search
by Ryan Conway Sept. 16, 2017

Sept. 16, 2017

Hello, I'm wondering if sklearn provides a means of terminating pipelines with a NearestNeighbors search. For example, my workflow is DictVectorizer -> TfidfTransformer -> NearestNeighbors. I'd like to capture this in an sklearn Pipeline. Unfortunately, Pipeline does not expose a kneighbors() method that would run all intermediate transforms and then return the result of NearestNeighbors.kneighbors(). I went through Pipeline's source and noticed its decision_function(), predict() etc. all capture this functionality with different terminating operation names. Maybe there is some way to specify the terminating operation method name rather than relying on these Pipeline methods? Thank you, Ryan

4 5

Help needed
by L Ali Sept. 15, 2017

Sept. 15, 2017

Hi guys, I am totally new to the scikit-learn, I am going to submit a pull request to the repository, but always got following error message, I could not find any usefully information from Google, my last hope is our community. Is there anyone can give me some advise about this error: ModuleNotFoundError: No module named 'matplotlib' Thanks so much! Li Yuan

3 4

custom loss function
by Thomas Evangelidis Sept. 14, 2017

Sept. 14, 2017

Greetings, I know this is a recurrent question, but I would like to use my own loss function either in a MLPRegressor or in an SVR. For the MLPRegressor case so far my conclusion was that it is not possible unless you modify the source code. On the other hand, for the SVR I was looking at setting custom kernel functions. But I am not sure if this is the same thing. Could someone please clarify this to me? Finally, I read about the "scoring" parameter is cross-validation, but this is just to select a Regressor that has been trained already with the default loss function, so it would be harder to find one that minimizes my own loss function. For the record, my loss function is the centered root mean square error. Thanks in advance for any advice. -- ====================================================================== Dr Thomas Evangelidis Post-doctoral Researcher CEITEC - Central European Institute of Technology Masaryk University Kamenice 5/A35/2S049, 62500 Brno, Czech Republic email: tevang(a)pharm.uoa.gr tevang3(a)gmail.com website: https://sites.google.com/site/thomasevangelidishomepage/

4 11