🌐 Mailman 3 February 2018 - scikit-learn - python.org

New Transformer to Support Multiple Column Pipelines & One Hot Encoding
by Dale Jacques Feb. 20, 2018

Feb. 20, 2018

Hello all, Long time lurker, first time emailer. I have two small contributions I would like to propose to the email list. I was working on a project this weekend that was using both categorical and numerical columns to predict a final output. I needed to save my transformations to make future predictions and grid search over multiple models and parameters, so sklearn pipelines were the obvious answer. I setup a pipeline, grid searched, then pickled the best model to use for future predictions. This worked well, but I ran into two issues. *1). I needed a transformer to select individual columns in my pipeline. *I needed to apply unique transformations to each column in my data, then recombine with a FeatureUnion. I realized there is not a supported transformer to extract a specific column within pipelines. See this issue here as an example <https://stackoverflow.com/questions/39001956/sklearn-pipeline-how-to-apply-…>. I created a transformation that explicitly extracts columns of interest for use in a pipeline with FeatureUnion. A FunctionTransformer will solve this issue, but I feel as if sklearn should directly and explicitly support this functionality. I believe this will make pipelines significantly more intuitive and accessible for most users. *2). One hot encoding requires arrays that are already integers.* You can find a similar issue here <https://stackoverflow.com/questions/40456867/labelbinarizer-for-multiple-co…>. This can be accomplished using Pandas.get_dummies() (where the transformation cannot be saved to apply to future predictions) or by using a scikit-learn LabelBinarizer <http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.Labe…> transformation. LabelBinarizer is designed to transform y and does not have a method to pass x and y in a pipeline. This breaks scikit-learn pipelines. I built a LabelBinarizer transformation that can be used with FeatureUnion in pipelines. This issue may be moot with the new CategoricalEncoder <http://scikit-learn.org/dev/modules/generated/sklearn.preprocessing.Categor…> that is about to be released. Does the community believe I should pursue contributing either of these? -- Cheers, DJ

2 1

Announcing modAL: a modular active learning framework
by Tivadar Danka Feb. 19, 2018

Feb. 19, 2018

Dear scikit-learn community! It is my pleasure to announce modAL, a modular active learning framework for Python3, built on top of scikit-learn. Designed with modularity, flexibility and extensibility in mind, it allows the rapid development of active learning workflows with nearly complete freedom. It is aimed for researchers and practitioners, where fast prototyping is essential for testing and developing active learning pipelines. modAL is quite young and under constant improvement. Any feedback, feature request or contribution are very welcome! The package can be installed via pip: pip3 install modAL The repository, tutorials and documentation are available at - GitHub: https://github.com/cosmic-cortex/modAL - Webpage: https://cosmic-cortex.github.io/modAL Cheers, Tivadar -------------------------------------- Tivadar Danka postdoctoral researcher BIOMAG group, MTA-BRC http://www.tivadardanka.com twitter: @TivadarDanka <https://twitter.com/TivadarDanka>

3 3

PieGraph: First examples and documentation
by Manuel Castejón Limas Feb. 16, 2018

Feb. 16, 2018

Dear all, We have produced some documentation for the PipeGraph module. Essentially it consists of the API for the two main interfaces: PipeGraphRegressor and PipeGraphClassifier. I guess that at this point the best experience comes from reading the examples and watching the diagrams. These examples are more suggestive than exhaustive though. Our purpose is to present the project in this initial form in order to hear all your comments for making it as useful for you all as possible. These are the links: - The documentation: https://mcasl.github.io/PipeGraph/auto_examples/index.html - The module sources: https://mcasl.github.io/PipeGraph/ Best Manuel

1 0

Building models for recommendation system
by prince gosavi Feb. 14, 2018

Feb. 14, 2018

Hi, I have a *1000x1000* euclidean* distance matrix.* The distance is calculated pairwise between each item i.e *distance between an ITEM with remaining ITEMS.* I would* like to know* the *next step after calculating the distance matrix.* Further, please* link some resources* so that I can get deep understanding because *as far as I have researched most of the websites provide examples with toy dataset which are pretty straight forward.* https://github.com/maxyodedara5/BE_Project/blob/master/test.ipynb is the link to the code CODE FOR READ ONLY PURPOSE. -- Regards

2 3

Handle uncertainties in NMF
by Samuël Weber Feb. 13, 2018

Feb. 13, 2018

Dears, First of all, thanks for scikit-learn! I was wondering if handling uncertainties in NMF would be possible. Indeed, in NMF we minimize a Frobenius norm ||X - WH||², so we may quite easily minimize ||(X - WH) / U||², with U the matrix of uncertainty. This would be really helpfull in many field (atmospheric chemistry [1], biology [2], etc). Does such feature is planed to be implemented in scikit-learn? Best regards, Samuël Weber [1] : https://www.epa.gov/air-research/positive-matrix-factorization-model-enviro… [2] : https://bmcbioinformatics.biomedcentral.com/track/pdf/10.1186/1471-2105-7-1…

2 1

Pipegraph is on its way!
by Manuel Castejón Limas Feb. 12, 2018

Feb. 12, 2018

Dear all, after some playing with the concept we have developed a module for implementing the functionality of Pipeline in more general contexts as first introduced in a former thread ( https://mail.python.org/ pipermail/scikit-learn/2018-January/002158.html ) In order to expand the possibilities of Pipeline for non linearly sequential workflows a graph like structure has been deployed while keeping as much as possible the already known syntax we all love and honor: X = pd.DataFrame(dict(X=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])) y = 2 * X sc = MinMaxScaler() lm = LinearRegression() steps = [('scaler', sc), ('linear_model', lm)] connections = {'scaler': dict(X='X'), 'linear_model': dict(X=('scaler', 'predict'), y='y')} pgraph = PipeGraph(steps=steps, connections=connections, use_for_fit='all', use_for_predict='all') As you can see the biggest difference for the final user is the dictionary describing the connections. Another major contribution for developers wanting to expand scikit learn is a collection of adapters for scikit learn models in order to provide them a common API irrespectively of whether they originally implemented predict, transform or fit_predict as an atomic operation without predict. These adapters accept as many positional or keyword parameters in their fit predict methods through *pargs and **kwargs. As general as PipeGraph is, it cannot work under the restrictions imposed by GridSearchCV on the input parameters, namely X and y since PipeGraph can accept as many input signals as needed. Thus, an adhoc GridSearchCv version is also needed and we will provide a basic initial version in a later version. We need to write the documentation and we will propose it as a contrib-project in a few days. Best wishes, Manuel Castejón-Limas

4 5

Re: [scikit-learn] Applying clustering to cosine distance matrix
by prince gosavi Feb. 12, 2018

Feb. 12, 2018

Will look into it.Although I have problem generating cluster as my data is 14000x14000 distance_matrix and it says "Memory Error". I have 6GB RAM. Any insight on this error is welcomed. Regards On Tue, Feb 13, 2018 at 3:19 AM, federico vaggi <vaggi.federico(a)gmail.com> wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (vaggi.federico(a)gmail.com) Add cleanup rule > <https://www.boxbe.com/popup?url=https%3A%2F%2Fwww.boxbe.com%2Fcleanup%3Fkey…> > | More info > <http://blog.boxbe.com/general/boxbe-automatic-cleanup?tc_serial=36655412130…> > > As a caveat, a lot of clustering algorithms assume that the distance > matrix is a proper metric. If your distance is not a proper metric then > the results might be meaningless (the narrative docs do a good job of > discussing this). > > On Mon, 12 Feb 2018 at 13:30 prince gosavi <princegosavi12(a)gmail.com> > wrote: > >> Hi, >> Thanks for those tips Sebastian.That just saved my day. >> >> Regards, >> Rajkumar >> >> On Tue, Feb 13, 2018 at 12:44 AM, Sebastian Raschka <se.raschka(a)gmail.com >> > wrote: >> >>> [image: Boxbe] <https://www.boxbe.com/overview> This message is >>> eligible for Automatic Cleanup! (se.raschka(a)gmail.com) Add cleanup rule >>> <https://www.boxbe.com/popup?url=https%3A%2F%2Fwww.boxbe.com%2Fcleanup%3Fkey…> >>> | More info >>> <http://blog.boxbe.com/general/boxbe-automatic-cleanup?tc_serial=36653890807…> >>> >> >>> Hi, >>> >>> by default, the clustering classes from sklearn, (e.g., DBSCAN), take an >>> [num_examples, num_features] array as input, but you can also provide the >>> distance matrix directly, e.g., by instantiating it with >>> metric='precomputed' >>> >>> my_dbscan = DBSCAN(..., metric='precomputed') >>> my_dbscan.fit(my_distance_matrix) >>> >>> Not sure if it helps in that particular case (depending on how many zero >>> elements you have), you can also use a sparse matrix in CSR format ( >>> https://docs.scipy.org/doc/scipy-1.0.0/reference/ >>> generated/scipy.sparse.csr_matrix.html). >>> >>> Also, you don't need to for-loop through the rows if you want to compute >>> the pair-wise distances, you can simply do that on the complete array. E.g., >>> >>> from sklearn.metrics.pairwise import cosine_distances >>> from scipy import sparse >>> >>> distance_matrix = cosine_distances(sparse.csr_matrix(X), >>> dense_output=False) >>> >>> where X is your "[num_examples, num_features]" array. >>> >>> Best, >>> Sebastian >>> >>> >>> > On Feb 12, 2018, at 1:10 PM, prince gosavi <princegosavi12(a)gmail.com> >>> wrote: >>> > >>> >> > I have generated a cosine distance matrix and would like to apply >>> clustering algorithm to the given matrix. >>> > np.shape(distance_matrix)==(14000,14000) >>> > >>> > I would like to know which clustering suits better and is there any >>> need to process the data further to get it in the form so that a model can >>> be applied. >>> > Also any performance tip as the matrix takes around 3-4 hrs of >>> processing. >>> > You can find my code here https://github.com/ >>> maxyodedara5/BE_Project/blob/master/main.ipynb >>> > Code for READ ONLY PURPOSE. >>> > -- >>> > Regards >>> > _______________________________________________ >>> > scikit-learn mailing list >>> > scikit-learn(a)python.org >>> > https://mail.python.org/mailman/listinfo/scikit-learn >>> >>> _______________________________________________ >>> scikit-learn mailing list >>> scikit-learn(a)python.org >>> https://mail.python.org/mailman/listinfo/scikit-learn >>> >>> >> >> >> -- >> Regards >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://mail.python.org/mailman/listinfo/scikit-learn >> > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > -- Regards

1 0

Re: [scikit-learn] Applying clustering to cosine distance matrix
by prince gosavi Feb. 12, 2018

Feb. 12, 2018

Hi, Thanks for those tips Sebastian.That just saved my day. Regards, Rajkumar On Tue, Feb 13, 2018 at 12:44 AM, Sebastian Raschka <se.raschka(a)gmail.com> wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (se.raschka(a)gmail.com) Add cleanup rule > <https://www.boxbe.com/popup?url=https%3A%2F%2Fwww.boxbe.com%2Fcleanup%3Fkey…> > | More info > <http://blog.boxbe.com/general/boxbe-automatic-cleanup?tc_serial=36653890807…> > > Hi, > > by default, the clustering classes from sklearn, (e.g., DBSCAN), take an > [num_examples, num_features] array as input, but you can also provide the > distance matrix directly, e.g., by instantiating it with > metric='precomputed' > > my_dbscan = DBSCAN(..., metric='precomputed') > my_dbscan.fit(my_distance_matrix) > > Not sure if it helps in that particular case (depending on how many zero > elements you have), you can also use a sparse matrix in CSR format ( > https://docs.scipy.org/doc/scipy-1.0.0/reference/ > generated/scipy.sparse.csr_matrix.html). > > Also, you don't need to for-loop through the rows if you want to compute > the pair-wise distances, you can simply do that on the complete array. E.g., > > from sklearn.metrics.pairwise import cosine_distances > from scipy import sparse > > distance_matrix = cosine_distances(sparse.csr_matrix(X), > dense_output=False) > > where X is your "[num_examples, num_features]" array. > > Best, > Sebastian > > > > On Feb 12, 2018, at 1:10 PM, prince gosavi <princegosavi12(a)gmail.com> > wrote: > > > > I have generated a cosine distance matrix and would like to apply > clustering algorithm to the given matrix. > > np.shape(distance_matrix)==(14000,14000) > > > > I would like to know which clustering suits better and is there any need > to process the data further to get it in the form so that a model can be > applied. > > Also any performance tip as the matrix takes around 3-4 hrs of > processing. > > You can find my code here https://github.com/ > maxyodedara5/BE_Project/blob/master/main.ipynb > > Code for READ ONLY PURPOSE. > > -- > > Regards > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > -- Regards

2 1

Re: [scikit-learn] Multi-Output Decision Trees for mixedclassification-regerssion problems
by prince gosavi Feb. 12, 2018

Feb. 12, 2018

Thanks for the reply will definitely try to PR this issue. Regrads On Tue, Feb 13, 2018 at 2:10 AM, Joel Nothman <joel.nothman(a)gmail.com> wrote: > [image: Boxbe] <https://www.boxbe.com/overview> This message is eligible > for Automatic Cleanup! (joel.nothman(a)gmail.com) Add cleanup rule > <https://www.boxbe.com/popup?url=https%3A%2F%2Fwww.boxbe.com%2Fcleanup%3Fkey…> > | More info > <http://blog.boxbe.com/general/boxbe-automatic-cleanup?tc_serial=36654532645…> > > presuming there are clear applications for this, other models should be > able to support mixed targets similarly, like MLP. since we don't really > have an API design for this, it might take some time to find consensus on > what it should look like. but a PR would be a good way to concretely > consider it. > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > -- Regards

1 0

Multi-Output Decision Trees for mixed classification-regerssion problems
by Evgeniya Korneva Feb. 12, 2018

Feb. 12, 2018

Dear all, For my research, I'm working with multi-output decision trees. In the current sklearn implementation, a tree can predict either several numerical or several categorical targets simultaneously, but not a mixture of those. However, predicting various targets jointly is often beneficial both in terms of speed and accuracy. Because of that, I'm willing to add this functionality. It seems that the only thing to be done is to implement a new node splitting criteria that handles a mixture of nominal and numerical attributes, and then define a new class of models (such as DecisionTreeRegressor or DecisionTreeClassifier, but for mixed output). However, since I'm not an experienced sklearn contributor, I am looking for any hints on how to implement this in effective way, re-using as much functionality already available as possible. Your advice is very welcome. Best, Evgeniya

3 2