🌐 Mailman 3 April 2019 - scikit-learn - python.org

Using sklearn-crfsuite on Production Systems
by Astha Agarwal Aug. 5, 2019

Aug. 5, 2019

Hi, I'm wondering if anyone is using sklearn-crfsuite on production systems? Is this library suitable for usage in industry on production systems (and not academia) for non-big data problems? Thanks, Astha

3 2

Any other clustering algo cluster incrementally?
by lampahome April 30, 2019

April 30, 2019

I read this : https://scikit-learn.org/0.15/modules/scaling_strategies.html There's only one clustering algo cluster incrementally, that is minibatch kmeans. Is there any clustering algo can reach this? On github is okay. thanks.

3 2

2019 John Hunter Excellence in Plotting Contest Reminder
by Nelle Varoquaux April 26, 2019

April 26, 2019

Hi everybody, My apologies to those of you getting this on multiple lists. In memory of John Hunter, we are pleased to be announce the SciPy John Hunter Excellence in Plotting Competition for 2019. This open competition aims to highlight the importance of data visualization to scientific progress and showcase the capabilities of open source software. Participants are invited to submit scientific plots to be judged by a panel. The winning entries will be announced and displayed at the conference. John Hunter’s family and NumFocus are graciously sponsoring cash prizes for the winners in the following amounts: - 1st prize: $1000 - 2nd prize: $750 - 3rd prize: $500 - Entries must be submitted by June, 8th to the form at https://goo.gl/forms/cFTB3FUBrMPfQ7Vz1 - Winners will be announced at Scipy 2019 in Austin, TX. - Participants do not need to attend the Scipy conference. - Entries may take the definition of “visualization” rather broadly. Entries may be, for example, a traditional printed plot, an interactive visualization for the web, or an animation. - Source code for the plot must be provided, in the form of Python code and/or a Jupyter notebook, along with a rendering of the plot in a widely used format. This may be, for example, PDF for print, standalone HTML and Javascript for an interactive plot, or MPEG-4 for a video. If the original data can not be shared for reasons of size or licensing, "fake" data may be substituted, along with an image of the plot using real data. - Each entry must include a 300-500 word abstract describing the plot and its importance for a general scientific audience. - Entries will be judged on their clarity, innovation and aesthetics, but most importantly for their effectiveness in communicating a real-world problem. Entrants are encouraged to submit plots that were used during the course of research or work, rather than merely being hypothetical. - SciPy reserves the right to display any and all entries, whether prize-winning or not, at the conference, use in any materials or on its website, with attribution to the original author(s). SciPy John Hunter Excellence in Plotting Competition Co-Chairs Hannah Aizenman Thomas Caswell Madicken Munk Nelle Varoquaux

1 0

Re: [scikit-learn] Feature engineering functionality - new package
by Sole Galli April 24, 2019

April 24, 2019

> > Dear Scikit-Learn team, > > Feature engineering is a big task ahead of building machine learning > models. It involves imputation of missing values, encoding of categorical > variables, discretisation, variable transformation etc. > > Sklearn includes some functionality for feature engineering, which is > useful, but it has a few limitations: > > 1) it does not allow for feature specification - it will do the same > process on all variables, for example SimpleImputer. Typically, we want > to impute different columns with different values. > 2) It does not capture information from the training set, this is it does > not learn, therefore, it is not able to perpetuate the values learnt from > the train set, to unseen data. > > The 2 limitations above apply to all the feature transformers in sklearn, > I believe. > > Therefore, if these transformers are used as part of a pipeline, we could > end up doing different transformations to train and test, depending on the > characteristics of the datasets. For business purposes, this is not a > desired option. > > I think that building transformers that learn from the train set would be > of much use for the community. > > To this end, I built a python package called feature engine > <https://pypi.org/project/feature-engine/> which expands the sklearn-api > with additional feature engineering techniques, and the functionality that > allows the transformer to learn from data and store the parameters learnt. > > The techniques included have been used worldwide, both in business and in > data competitions, and reported in kdd reports and other articles. I also > cover them in an udemy course > <https://www.udemy.com/feature-engineering-for-machine-learning> which > has enrolled several thousand students. > > The package capitalises on the use of pandas to capture the features, but > I am confident that the columns names could be captured and the df > transformed to a numpy array to comply with sklearn requirements. > > I wondered whether it would be of interest to include the functionality of > this package within sklearn? > If you would consider extending the sklearn api to include these > transformers, I would be happy to help. > > Alternatively, would you consider to add the package to your website? > where you mention the libaries that extend sklearn functionality? > > All feedback is welcome. > > Many thanks and I look forward to hearing from you > > Thank you so much fur such an awesome contribution through the sklearn api > > Kind regards > > Sole > >

3 4

Categorical Encoding of high cardinality variables
by federico vaggi April 24, 2019

April 24, 2019

Hi everyone, I wanted to use the scikit-learn transformer API to clean up some messy data as input to a neural network. One of the steps involves converting categorical variables (of very high cardinality) into integers for use in an embedding layer. Unfortunately, I cannot quite use LabelEncoder to do solve this. When dealing with categorical variables with very high cardinality, I found it useful in practice to have a threshold value for the frequency under which a variable ends up with the 'unk' or 'rare' label. This same label would also end up applied at test time to entries that were not observed in the train set. This is relatively straightforward to add to the existing label encoder code, but it breaks the contract slightly: if we encode some variables with a 'rare' label, then the transform operation is no longer a bijection. Is this feature too niche for the main sklearn? I saw there was a package ( https://feature-engine.readthedocs.io/en/latest/RareLabelCategoricalEncoder…) that implemented a similar feature discussed in the mailing list.

2 1

The 2nd International Workshop on Machine Learning in Clinical Neuroimaging (MLCN 2019): ENTERING THE ERA OF BIG DATA VIA TRANSFER LEARNING AND DATA HARMONIZATION
by MLCN Workshop April 23, 2019

April 23, 2019

Dear Colleagues,We are delighted to invite you to join us for the MLCN 2019 <https://mlcnws.com/> workshop as a satellite event of the MICCAI 2019 conference, Shenzhen, China. Call for Papers Recent advances in neuroimaging and machine learning provide an exceptional opportunity for investigators and physicians to discover complex relationships between brain, behaviors, and mental and neurological disorders. The MLCN 2019 workshop (https://mlcnws.com), as a satellite event of MICCAI 2019 (https://www.miccai2019.org), aims to bring together researchers in both theory and application from various fields in domains such as *e.g.* machine learning, neuroimaging, predictive clinical neuroscience, *etc.* Topics of interests include, but are not limited to: - Transfer learning in clinical neuroimaging - Model stability in transfer learning - Data prerequisites for successful transfer learning - Domain adaptation in neuroimaging - Data harmonization across sites - Data pooling – practical issues - Cross-domain learning in neuroimaging - Interpretability for transfer learning - Unsupervised methods for domain adaptation - Multi-site data analysis, from preprocessing to modeling - Big data in clinical neuroimaging - Scalable machine learning methods - Benefits, problems, and solutions of working with very large datasets SUBMISSION PROCESS: The workshop seeks high quality, original, and unpublished work on algorithms, theory, and applications of machine learning in clinical neuroimaging related to big data, transfer learning, and data harmonization. Papers should be submitted electronically in Springer Lecture Notes in Computer Science (LCNS) style ( https://www.springer.com/gp/computer-science/lncs/conference-proceedings-gu…) with up to 8-pages and using the CMT system at https://cmt3.research.microsoft.com/MLCN2019. The MLCN workshop uses a double-blind review process in the evaluation phase, thus authors must ensure anonymous submissions. Accepted papers will be published in a joint proceeding with the MICCAI 2019 conference. IMPORTANT DATES: - Paper submission deadline: July 1, 2019 (23:59 PST) - Notification of Acceptance: August 5, 2019 - Camera-ready Submission: August 12, 2019 - Workshop Date: October 13, 2019 Best regards, MLCN 2019 Organizing Committee <https://mlcnws.com/organization/>, Email: mlcnworkshop(a)gmail.com Website: https://mlcnws.com/ twitter: @MLCNworkshop <https://twitter.com/MLCNworkshop>

1 0

PyDataLondon 2019 (July 12-14) Call for Proposals closing this Friday
by Ian Ozsvald April 17, 2019

April 17, 2019

On July 12-14 we host the sixth PyDataLondon conference in central London. As last year we'll be hosted close to Tower Bridge at the Tower Hotel with 700 attendees over 3 days: https://pydata.org/london2019/ Our Call for Proposals has been open for several weeks, it closes this Friday. If anyone here would like to spread the good word about scikit-learn (and any scikit/scipy/Python data science related topics) we'd love to see a proposal. We also offer first-time speaker mentoring, it is a bit late for this now so I'll offer to answer any questions anyone has personally - just email me directly. The Call for Proposals closes this Friday, please submit your talk here: https://pydata.org/london2019/cfp If you've not been to PyDataLondon before - here's last year's schedule and my write-up of all of the events that we covered. Gael Varoquaux and others spoke for us, we'd love to see scikit-learn well represented again: https://pydata.org/london2018/schedule/ https://ianozsvald.com/2018/04/30/pydatalondon-2018-and-creating-correct-an… Regards, Ian (PyDataLondon co-founder) -- Ian Ozsvald (Data Scientist, PyDataLondon co-chair) ian(a)IanOzsvald.com https://IanOzsvald.com https://MorConsulting.com https://twitter.com/IanOzsvald

1 0

Predict Method of OneVsRestClassifier Integration with Google Cloud ML
by Liam Geron April 11, 2019

April 11, 2019

Hi all, I was hoping to get some guidance re: changing the result of the predict method of the OneVsRestClassifier to return a dense array rather than a sparse array, given that Google Cloud ML only accepts dense numpy arrays as a result of a given models predict method. Right now my model architecture looks like: model = Pipeline([('tfidf', TfidfVectorizer()), ('clf', OneVsRestClassifier(XGBClassifier()))]) Which returns a sparse array with the predict method. I saw the Stack Overflow post here: https://stackoverflow.com/questions/52151548/google-cloud-ml-engine-scikit-… which recommends overwriting the predict method with the predict_proba method, however I found that I can't serialize the model after doing so. I also have a stack overflow post here: https://stackoverflow.com/questions/55366454/how-to-convert-scikit-learn-on… which details the specific pickling error. Is this a known issue? Is there an accepted way to convert this into a dense array? Thanks, Liam Geron

3 6

API Discussion: Where shall we put the plotting functions?
by Hanmin Qin April 7, 2019

April 7, 2019

See https://github.com/scikit-learn/scikit-learn/issues/13448 We've introduced several plotting functions (e.g., plot_tree and plot_partial_dependence) and will introduce more (e.g., plot_decision_boundary) in the future. Consequently, we need to decide where to put these functions. Currently, there're 3 proposals: (1) sklearn.plot.plot_YYY (e.g., sklearn.plot.plot_tree) (2) sklearn.plot.XXX.plot_YYY (e.g., sklearn.plot.tree.plot_tree) (3) sklearn.XXX.plot.plot_YYY (e.g., sklearn.tree.plot.plot_tree, note that we won't support from sklearn.XXX import plot_YYY) Joel Nothman, Gael Varoquaux and I decided to post it on the mailing list to invite opinions. Thanks Hanmin Qin

10 12

Starting to contribute
by Heitor Boschirolli April 7, 2019

April 7, 2019

Hello! First of all, I'm apologize if this email is not for such questions, but I never contributed to open source code before and I'm not sure how to proceed, could someone help me with that? Should I just pick an issue, solve it following the guidelines described in the website and open a PR? If I have any trouble, can I post it on the mailing list? Att, Heitor Boschirolli

2 1