🌐 Mailman 3 October 2019 - scikit-learn - python.org

scikit supervised learning order
by Mike Smith Oct. 11, 2019

Oct. 11, 2019

I see that the list of regressors ordered at https://scikit-learn.org/stable/supervised_learning.html#supervised-learning seems to be from simplest to most complex regressor. For example, 1.6 KNearest Neighbor and 1.10 Decision Tree, are clearly more basic models than 1.11 Ensemble methods, and worst results are expected from 1.6,1.10 vs 1.11. So, does this mean that the ordering 1.1-1-17 signifies 1.1 are weaker models than 1.17, in that order? And that 1.17 Neural Networks is presumed to give the best results? On Fri, Oct 11, 2019 at 9:02 AM <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: logistic regression results are not stable between > solvers (Andreas Mueller) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 11 Oct 2019 15:42:58 +0200 > From: Andreas Mueller <t3kcit(a)gmail.com> > To: scikit-learn(a)python.org > Subject: Re: [scikit-learn] logistic regression results are not stable > between solvers > Message-ID: <d55949d6-3355-f892-f6b3-030edf1c7947(a)gmail.com> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > > > On 10/10/19 1:14 PM, Beno?t Presles wrote: > > > > Thanks for your answers. > > > > On my real data, I do not have so many samples. I have a bit more than > > 200 samples in total and I also would like to get some results with > > unpenalized logisitic regression. > > What do you suggest? Should I switch to the lbfgs solver? > Yes. > > Am I sure that with this solver I will not have any convergence issue > > and always get the good result? Indeed, I did not get any convergence > > warning with saga, so I thought everything was fine. I noticed some > > issues only when I decided to test several solvers. Without comparing > > the results across solvers, how to be sure that the optimisation goes > > well? Shouldn't scikit-learn warn the user somehow if it is not the case? > We should attempt to warn in the SAGA solver if it doesn't converge. > That it doesn't raise a convergence warning should probably be > considered a bug. > It uses the maximum weight change as a stopping criterion right now. > We could probably compute the dual objective once in the end to see if > we converged, right? Or is that not possible with SAGA? If not, we might > want to caution that no convergence warning will be raised. > > > > > At last, I was using saga because I also wanted to do some feature > > selection by using l1 penalty which is not supported by lbfgs... > You can use liblinear then. > > > > > > Best regards, > > Ben > > > > > > Le 09/10/2019 ? 23:39, Guillaume Lema?tre a ?crit?: > >> Ups I did not see the answer of Roman. Sorry about that. It is coming > >> back to the same conclusion :) > >> > >> On Wed, 9 Oct 2019 at 23:37, Guillaume Lema?tre > >> <g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> wrote: > >> > >> Uhm actually increasing to 10000 samples solve the convergence > issue. > >> SAGA is not designed to work with a so small sample size most > >> probably. > >> > >> On Wed, 9 Oct 2019 at 23:36, Guillaume Lema?tre > >> <g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> wrote: > >> > >> I slightly change the bench such that it uses pipeline and > >> plotted the coefficient: > >> > >> > https://gist.github.com/glemaitre/8fcc24bdfc7dc38ca0c09c56e26b9386 > >> > >> I only see one of the 10 splits where SAGA is not converging, > >> otherwise the coefficients > >> look very close (I don't attach the figure here but they can > >> be plotted using the snippet). > >> So apart from this second split, the other differences seems > >> to be numerical instability. > >> > >> Where I have some concern is regarding the convergence rate > >> of SAGA but I have no > >> intuition to know if this is normal or not. > >> > >> On Wed, 9 Oct 2019 at 23:22, Roman Yurchak > >> <rth.yurchak(a)gmail.com <mailto:rth.yurchak@gmail.com>> wrote: > >> > >> Ben, > >> > >> I can confirm your results with penalty='none' and C=1e9. > >> In both cases, > >> you are running a mostly unpenalized logisitic > >> regression. Usually > >> that's less numerically stable than with a small > >> regularization, > >> depending on the data collinearity. > >> > >> Running that same code with > >> ? - larger penalty ( smaller C values) > >> ? - or larger number of samples > >> ? yields for me the same coefficients (up to some > tolerance). > >> > >> You can also see that SAGA convergence is not good by the > >> fact that it > >> needs 196000 epochs/iterations to converge. > >> > >> Actually, I have often seen convergence issues with SAG > >> on small > >> datasets (in unit tests), not fully sure why. > >> > >> -- > >> Roman > >> > >> On 09/10/2019 22:10, serafim loukas wrote: > >> > The predictions across solver are exactly the same when > >> I run the code. > >> > I am using 0.21.3 version. What is yours? > >> > > >> > > >> > In [13]: import sklearn > >> > > >> > In [14]: sklearn.__version__ > >> > Out[14]: '0.21.3' > >> > > >> > > >> > Serafeim > >> > > >> > > >> > > >> >> On 9 Oct 2019, at 21:44, Beno?t Presles > >> <benoit.presles(a)u-bourgogne.fr > >> <mailto:benoit.presles@u-bourgogne.fr> > >> >> <mailto:benoit.presles@u-bourgogne.fr > >> <mailto:benoit.presles@u-bourgogne.fr>>> wrote: > >> >> > >> >> (y_pred_lbfgs==y_pred_saga).all() == False > >> > > >> > > >> > _______________________________________________ > >> > scikit-learn mailing list > >> > scikit-learn(a)python.org <mailto:scikit-learn@python.org> > >> > https://mail.python.org/mailman/listinfo/scikit-learn > >> > > >> > >> _______________________________________________ > >> scikit-learn mailing list > >> scikit-learn(a)python.org <mailto:scikit-learn@python.org> > >> https://mail.python.org/mailman/listinfo/scikit-learn > >> > >> > >> > >> -- > >> Guillaume Lemaitre > >> Scikit-learn @ Inria Foundation > >> https://glemaitre.github.io/ > >> > >> > >> > >> -- > >> Guillaume Lemaitre > >> Scikit-learn @ Inria Foundation > >> https://glemaitre.github.io/ > >> > >> > >> > >> -- > >> Guillaume Lemaitre > >> Scikit-learn @ Inria Foundation > >> https://glemaitre.github.io/ > >> > >> _______________________________________________ > >> scikit-learn mailing list > >> scikit-learn(a)python.org > >> https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > >

1 0

Regarding design decision for putting Data Scaler and Feature Transformers under same module
by Hemanth Kota Oct. 8, 2019

Oct. 8, 2019

Hi Team, I'm beginner in using sklearn library. I have doubt regarding reason for putting Data scalers like StandardScaler, RobustScaler etc and feature transformers like QuantileTransformer, PolynomialFeatures in preprocessing module ? What relationship made to put them together? Thanks Hemanth

2 2

Re: [scikit-learn] scikit-learn Digest, Vol 43, Issue 11
by Mike Smith Oct. 6, 2019

Oct. 6, 2019

Can I call an MSExcel cell range in a function such as model.predict(), instead of typing the data in for each element? On Sat, Oct 5, 2019 at 11:58 AM <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: scikit-learn Digest, Vol 43, Issue 10 (Mike Smith) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 5 Oct 2019 11:55:33 -0700 > From: Mike Smith <javaeurusd(a)gmail.com> > To: scikit-learn(a)python.org > Subject: Re: [scikit-learn] scikit-learn Digest, Vol 43, Issue 10 > Message-ID: > <CAEWZffDWv8mOUVaKSSBzpiEebjcVrRD-t8zxuBSFCKxqTGi3= > A(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > 1. Re: Can Scikit-learn decision tree (CART) have both > continuous and categorical features? (C W) > > What I'd ask in reply to this is if regression and classification module > results can be entered into an input for one resultant output. > > > > On Sat, Oct 5, 2019, 11:50 AM , <scikit-learn-request(a)python.org> wrote: > > > Send scikit-learn mailing list submissions to > > scikit-learn(a)python.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://mail.python.org/mailman/listinfo/scikit-learn > > or, via email, send a message with subject or body 'help' to > > scikit-learn-request(a)python.org > > > > You can reach the person managing the list at > > scikit-learn-owner(a)python.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of scikit-learn digest..." > > > > > > Today's Topics: > > > > 1. Re: Can Scikit-learn decision tree (CART) have both > > continuous and categorical features? (C W) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Sat, 5 Oct 2019 14:50:09 -0400 > > From: C W <tmrsg11(a)gmail.com> > > To: Scikit-learn mailing list <scikit-learn(a)python.org> > > Subject: Re: [scikit-learn] Can Scikit-learn decision tree (CART) have > > both continuous and categorical features? > > Message-ID: > > < > > CAE2FW2nHDJGNky2VWk-U8fU3gqwBqWEgidzTAWnUq+NzAK68VA(a)mail.gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Thanks, great material! I got pydotplus with graphviz to work. > > > > Using the code on sklean website [1], tree.plot_tree(clf.fit(iris.data, > > iris.target)) gives an error: > > AttributeError: module 'sklearn.tree' has no attribute 'plot_tree' > > > > Both my colleague and I got the same error message. Per this post > > https://github.com/Microsoft/LightGBM/issues/1844, a PyPI update is > > needed. > > > > [1] sklearn link: > > https://scikit-learn.org/stable/modules/tree.html#classification > > > > > > On Fri, Oct 4, 2019 at 11:52 PM Sebastian Raschka < > > mail(a)sebastianraschka.com> > > wrote: > > > > > The docs show a way such that you don't need to write it as png file > > using > > > tree.plot_tree: > > > https://scikit-learn.org/stable/modules/tree.html#classification > > > > > > I don't remember why, but I think I had problems with that in the past > (I > > > think it didn't look so nice visually, but don't remember), which is > why > > I > > > still stick to graphviz. For my use cases, it's not much hassle -- it > > used > > > to be a bit of a hassle to get GraphViz working, but now you can do > > > > > > conda install pydotplus > > > conda install graphviz > > > > > > Coincidentally, I just made an example for a lecture I was teaching on > > > Tue: > > > > > > https://github.com/rasbt/stat479-machine-learning-fs19/blob/master/06_trees… > > > > > > Best, > > > Sebastian > > > > > > > > > > On Oct 4, 2019, at 10:09 PM, C W <tmrsg11(a)gmail.com> wrote: > > > > > > > > On a separate note, what do you use for plotting? > > > > > > > > I found graphviz, but you have to first save it as a png on your > > > computer. That's a lot work for just one plot. Is there something like > a > > > matplotlib? > > > > > > > > Thanks! > > > > > > > > On Fri, Oct 4, 2019 at 9:42 PM Sebastian Raschka < > > > mail(a)sebastianraschka.com> wrote: > > > > Yeah, think of it more as a computational workaround for achieving > the > > > same thing more efficiently (although it looks inelegant/weird)-- > > something > > > like that wouldn't be mentioned in textbooks. > > > > > > > > Best, > > > > Sebastian > > > > > > > > > On Oct 4, 2019, at 6:33 PM, C W <tmrsg11(a)gmail.com> wrote: > > > > > > > > > > Thanks Sebastian, I think I get it. > > > > > > > > > > It's just have never seen it this way. Quite different from what > I'm > > > used in Elements of Statistical Learning. > > > > > > > > > > On Fri, Oct 4, 2019 at 7:13 PM Sebastian Raschka < > > > mail(a)sebastianraschka.com> wrote: > > > > > Not sure if there's a website for that. In any case, to explain > this > > > differently, as discussed earlier sklearn assumes continuous features > for > > > decision trees. So, it will use a binary threshold for splitting along > a > > > feature attribute. In other words, it cannot do sth like > > > > > > > > > > if x == 1 then right child node > > > > > else left child node > > > > > > > > > > Instead, what it does is > > > > > > > > > > if x >= 0.5 then right child node > > > > > else left child node > > > > > > > > > > These are basically equivalent as you can see when you just plug in > > > values 0 and 1 for x. > > > > > > > > > > Best, > > > > > Sebastian > > > > > > > > > > > On Oct 4, 2019, at 5:34 PM, C W <tmrsg11(a)gmail.com> wrote: > > > > > > > > > > > > I don't understand your answer. > > > > > > > > > > > > Why after one-hot-encoding it still outputs greater than 0.5 or > > less > > > than? Does sklearn website have a working example on categorical input? > > > > > > > > > > > > Thanks! > > > > > > > > > > > > On Fri, Oct 4, 2019 at 3:48 PM Sebastian Raschka < > > > mail(a)sebastianraschka.com> wrote: > > > > > > Like Nicolas said, the 0.5 is just a workaround but will do the > > > right thing on the one-hot encoded variables, here. You will find that > > the > > > threshold is always at 0.5 for these variables. I.e., what it will do > is > > to > > > use the following conversion: > > > > > > > > > > > > treat as car_Audi=1 if car_Audi >= 0.5 > > > > > > treat as car_Audi=0 if car_Audi < 0.5 > > > > > > > > > > > > or, it may be > > > > > > > > > > > > treat as car_Audi=1 if car_Audi > 0.5 > > > > > > treat as car_Audi=0 if car_Audi <= 0.5 > > > > > > > > > > > > (Forgot which one sklearn is using, but either way. it will be > > fine.) > > > > > > > > > > > > Best, > > > > > > Sebastian > > > > > > > > > > > > > > > > > >> On Oct 4, 2019, at 1:44 PM, Nicolas Hug <niourf(a)gmail.com> > wrote: > > > > > >> > > > > > >> > > > > > >>> But, decision tree is still mistaking one-hot-encoding as > > > numerical input and split at 0.5. This is not right. Perhaps, I'm doing > > > something wrong? > > > > > >> > > > > > >> You're not doing anything wrong, and neither is the tree. Trees > > > don't support categorical variables in sklearn, so everything is > treated > > as > > > numerical. > > > > > >> > > > > > >> This is why we do one-hot-encoding: so that a set of numerical > > (one > > > hot encoded) features can be treated as if they were just one > categorical > > > feature. > > > > > >> > > > > > >> > > > > > >> > > > > > >> Nicolas > > > > > >> > > > > > >> On 10/4/19 2:01 PM, C W wrote: > > > > > >>> Yes, you are right. it was 0.5 and 0.5 for split, not 1.5. So, > > > typo on my part. > > > > > >>> > > > > > >>> Looks like I did one-hot-encoding correctly. My new variable > > names > > > are: car_Audi, car_BMW, etc. > > > > > >>> > > > > > >>> But, decision tree is still mistaking one-hot-encoding as > > > numerical input and split at 0.5. This is not right. Perhaps, I'm doing > > > something wrong? > > > > > >>> > > > > > >>> Is there a good toy example on the sklearn website? I am only > see > > > this: > > > > > > https://scikit-learn.org/stable/auto_examples/tree/plot_tree_regression.html > > > . > > > > > >>> > > > > > >>> Thanks! > > > > > >>> > > > > > >>> > > > > > >>> > > > > > >>> On Fri, Oct 4, 2019 at 1:28 PM Sebastian Raschka < > > > mail(a)sebastianraschka.com> wrote: > > > > > >>> Hi, > > > > > >>> > > > > > >>>> The funny part is: the tree is taking one-hot-encoding (BMW=0, > > > Toyota=1, Audi=2) as numerical values, not category.The tree splits at > > 0.5 > > > and 1.5 > > > > > >>> > > > > > >>> that's not a onehot encoding then. > > > > > >>> > > > > > >>> For an Audi datapoint, it should be > > > > > >>> > > > > > >>> BMW=0 > > > > > >>> Toyota=0 > > > > > >>> Audi=1 > > > > > >>> > > > > > >>> for BMW > > > > > >>> > > > > > >>> BMW=1 > > > > > >>> Toyota=0 > > > > > >>> Audi=0 > > > > > >>> > > > > > >>> and for Toyota > > > > > >>> > > > > > >>> BMW=0 > > > > > >>> Toyota=1 > > > > > >>> Audi=0 > > > > > >>> > > > > > >>> The split threshold should then be at 0.5 for any of these > > > features. > > > > > >>> > > > > > >>> Based on your email, I think you were assuming that the DT does > > > the one-hot encoding internally, which it doesn't. In practice, it is > > hard > > > to guess what is a nominal and what is a ordinal variable, so you have > to > > > do the onehot encoding before you give the data to the decision tree. > > > > > >>> > > > > > >>> Best, > > > > > >>> Sebastian > > > > > >>> > > > > > >>>> On Oct 4, 2019, at 11:48 AM, C W <tmrsg11(a)gmail.com> wrote: > > > > > >>>> > > > > > >>>> I'm getting some funny results. I am doing a regression > decision > > > tree, the response variables are assigned to levels. > > > > > >>>> > > > > > >>>> The funny part is: the tree is taking one-hot-encoding (BMW=0, > > > Toyota=1, Audi=2) as numerical values, not category. > > > > > >>>> > > > > > >>>> The tree splits at 0.5 and 1.5. Am I doing one-hot-encoding > > > wrong? How does the sklearn know internally 0 vs. 1 is categorical, not > > > numerical? > > > > > >>>> > > > > > >>>> In R for instance, you do as.factor(), which explicitly states > > > the data type. > > > > > >>>> > > > > > >>>> Thank you! > > > > > >>>> > > > > > >>>> > > > > > >>>> On Wed, Sep 18, 2019 at 11:13 AM Andreas Mueller < > > > t3kcit(a)gmail.com> wrote: > > > > > >>>> > > > > > >>>> > > > > > >>>> On 9/15/19 8:16 AM, Guillaume Lema?tre wrote: > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> On Sat, 14 Sep 2019 at 20:59, C W <tmrsg11(a)gmail.com> wrote: > > > > > >>>>> Thanks, Guillaume. > > > > > >>>>> Column transformer looks pretty neat. I've also heard though, > > > this pipeline can be tedious to set up? Specifying what you want for > > every > > > feature is a pain. > > > > > >>>>> > > > > > >>>>> It would be interesting for us which part of the pipeline is > > > tedious to set up to know if we can improve something there. > > > > > >>>>> Do you mean, that you would like to automatically detect of > > > which type of feature (categorical/numerical) and apply a > > > > > >>>>> default encoder/scaling such as discuss there: > > > > > > https://github.com/scikit-learn/scikit-learn/issues/10603#issuecomment-4011… > > > > > >>>>> > > > > > >>>>> IMO, one a user perspective, it would be cleaner in some > cases > > > at the cost of applying blindly a black box > > > > > >>>>> which might be dangerous. > > > > > >>>> Also see > > > > > > https://amueller.github.io/dabl/dev/generated/dabl.EasyPreprocessor.html#da… > > > > > >>>> Which basically does that. > > > > > >>>> > > > > > >>>> > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> Jaiver, > > > > > >>>>> Actually, you guessed right. My real data has only one > > numerical > > > variable, looks more like this: > > > > > >>>>> > > > > > >>>>> Gender Date Income Car Attendance > > > > > >>>>> Male 2019/3/01 10000 BMW Yes > > > > > >>>>> Female 2019/5/02 9000 Toyota No > > > > > >>>>> Male 2019/7/15 12000 Audi Yes > > > > > >>>>> > > > > > >>>>> I am predicting income using all other categorical variables. > > > Maybe it is catboost! > > > > > >>>>> > > > > > >>>>> Thanks, > > > > > >>>>> > > > > > >>>>> M > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> On Sat, Sep 14, 2019 at 9:25 AM Javier L?pez <jlopez(a)ende.cc > > > > > wrote: > > > > > >>>>> If you have datasets with many categorical features, and > > perhaps > > > many categories, the tools in sklearn are quite limited, > > > > > >>>>> but there are alternative implementations of boosted trees > that > > > are designed with categorical features in mind. Take a look > > > > > >>>>> at catboost [1], which has an sklearn-compatible API. > > > > > >>>>> > > > > > >>>>> J > > > > > >>>>> > > > > > >>>>> [1] https://catboost.ai/ > > > > > >>>>> > > > > > >>>>> On Sat, Sep 14, 2019 at 3:40 AM C W <tmrsg11(a)gmail.com> > wrote: > > > > > >>>>> Hello all, > > > > > >>>>> I'm very confused. Can the decision tree module handle both > > > continuous and categorical features in the dataset? In this case, it's > > just > > > CART (Classification and Regression Trees). > > > > > >>>>> > > > > > >>>>> For example, > > > > > >>>>> Gender Age Income Car Attendance > > > > > >>>>> Male 30 10000 BMW Yes > > > > > >>>>> Female 35 9000 Toyota No > > > > > >>>>> Male 50 12000 Audi Yes > > > > > >>>>> > > > > > >>>>> According to the documentation > > > > > > https://scikit-learn.org/stable/modules/tree.html#tree-algorithms-id3-c4-5-… > > , > > > it can not! > > > > > >>>>> > > > > > >>>>> It says: "scikit-learn implementation does not support > > > categorical variables for now". > > > > > >>>>> > > > > > >>>>> Is this true? If not, can someone point me to an example? If > > > yes, what do people do? > > > > > >>>>> > > > > > >>>>> Thank you very much! > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> _______________________________________________ > > > > > >>>>> scikit-learn mailing list > > > > > >>>>> scikit-learn(a)python.org > > > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >>>>> _______________________________________________ > > > > > >>>>> scikit-learn mailing list > > > > > >>>>> scikit-learn(a)python.org > > > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >>>>> _______________________________________________ > > > > > >>>>> scikit-learn mailing list > > > > > >>>>> scikit-learn(a)python.org > > > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> -- > > > > > >>>>> Guillaume Lemaitre > > > > > >>>>> INRIA Saclay - Parietal team > > > > > >>>>> Center for Data Science Paris-Saclay > > > > > >>>>> https://glemaitre.github.io/ > > > > > >>>>> > > > > > >>>>> > > > > > >>>>> _______________________________________________ > > > > > >>>>> scikit-learn mailing list > > > > > >>>>> > > > > > >>>>> scikit-learn(a)python.org > > > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >>>> > > > > > >>>> _______________________________________________ > > > > > >>>> scikit-learn mailing list > > > > > >>>> scikit-learn(a)python.org > > > > > >>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >>>> _______________________________________________ > > > > > >>>> scikit-learn mailing list > > > > > >>>> scikit-learn(a)python.org > > > > > >>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >>> > > > > > >>> _______________________________________________ > > > > > >>> scikit-learn mailing list > > > > > >>> scikit-learn(a)python.org > > > > > >>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >>> > > > > > >>> > > > > > >>> _______________________________________________ > > > > > >>> scikit-learn mailing list > > > > > >>> > > > > > >>> scikit-learn(a)python.org > > > > > >>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >> _______________________________________________ > > > > > >> scikit-learn mailing list > > > > > >> scikit-learn(a)python.org > > > > > >> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > > > > > > > _______________________________________________ > > > > > > scikit-learn mailing list > > > > > > scikit-learn(a)python.org > > > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > _______________________________________________ > > > > > > scikit-learn mailing list > > > > > > scikit-learn(a)python.org > > > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > > > > > _______________________________________________ > > > > > scikit-learn mailing list > > > > > scikit-learn(a)python.org > > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > _______________________________________________ > > > > > scikit-learn mailing list > > > > > scikit-learn(a)python.org > > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > > > _______________________________________________ > > > > scikit-learn mailing list > > > > scikit-learn(a)python.org > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > > > > scikit-learn mailing list > > > > scikit-learn(a)python.org > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > _______________________________________________ > > > scikit-learn mailing list > > > scikit-learn(a)python.org > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > >

2 1

Can Scikit-learn decision tree (CART) have both continuous and categorical features?
by C W Oct. 6, 2019

Oct. 6, 2019

Hello all, I'm very confused. Can the decision tree module handle both continuous and categorical features in the dataset? In this case, it's just CART (Classification and Regression Trees). For example, Gender Age Income Car Attendance Male 30 10000 BMW Yes Female 35 9000 Toyota No Male 50 12000 Audi Yes According to the documentation https://scikit-learn.org/stable/modules/tree.html#tree-algorithms-id3-c4-5-…, it can not! It says: "scikit-learn implementation does not support categorical variables for now". Is this true? If not, can someone point me to an example? If yes, what do people do? Thank you very much!

6 25

Re: [scikit-learn] scikit-learn Digest, Vol 43, Issue 10
by Mike Smith Oct. 5, 2019

Oct. 5, 2019

1. Re: Can Scikit-learn decision tree (CART) have both continuous and categorical features? (C W) What I'd ask in reply to this is if regression and classification module results can be entered into an input for one resultant output. On Sat, Oct 5, 2019, 11:50 AM , <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: Can Scikit-learn decision tree (CART) have both > continuous and categorical features? (C W) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 5 Oct 2019 14:50:09 -0400 > From: C W <tmrsg11(a)gmail.com> > To: Scikit-learn mailing list <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Can Scikit-learn decision tree (CART) have > both continuous and categorical features? > Message-ID: > < > CAE2FW2nHDJGNky2VWk-U8fU3gqwBqWEgidzTAWnUq+NzAK68VA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Thanks, great material! I got pydotplus with graphviz to work. > > Using the code on sklean website [1], tree.plot_tree(clf.fit(iris.data, > iris.target)) gives an error: > AttributeError: module 'sklearn.tree' has no attribute 'plot_tree' > > Both my colleague and I got the same error message. Per this post > https://github.com/Microsoft/LightGBM/issues/1844, a PyPI update is > needed. > > [1] sklearn link: > https://scikit-learn.org/stable/modules/tree.html#classification > > > On Fri, Oct 4, 2019 at 11:52 PM Sebastian Raschka < > mail(a)sebastianraschka.com> > wrote: > > > The docs show a way such that you don't need to write it as png file > using > > tree.plot_tree: > > https://scikit-learn.org/stable/modules/tree.html#classification > > > > I don't remember why, but I think I had problems with that in the past (I > > think it didn't look so nice visually, but don't remember), which is why > I > > still stick to graphviz. For my use cases, it's not much hassle -- it > used > > to be a bit of a hassle to get GraphViz working, but now you can do > > > > conda install pydotplus > > conda install graphviz > > > > Coincidentally, I just made an example for a lecture I was teaching on > > Tue: > > > https://github.com/rasbt/stat479-machine-learning-fs19/blob/master/06_trees… > > > > Best, > > Sebastian > > > > > > > On Oct 4, 2019, at 10:09 PM, C W <tmrsg11(a)gmail.com> wrote: > > > > > > On a separate note, what do you use for plotting? > > > > > > I found graphviz, but you have to first save it as a png on your > > computer. That's a lot work for just one plot. Is there something like a > > matplotlib? > > > > > > Thanks! > > > > > > On Fri, Oct 4, 2019 at 9:42 PM Sebastian Raschka < > > mail(a)sebastianraschka.com> wrote: > > > Yeah, think of it more as a computational workaround for achieving the > > same thing more efficiently (although it looks inelegant/weird)-- > something > > like that wouldn't be mentioned in textbooks. > > > > > > Best, > > > Sebastian > > > > > > > On Oct 4, 2019, at 6:33 PM, C W <tmrsg11(a)gmail.com> wrote: > > > > > > > > Thanks Sebastian, I think I get it. > > > > > > > > It's just have never seen it this way. Quite different from what I'm > > used in Elements of Statistical Learning. > > > > > > > > On Fri, Oct 4, 2019 at 7:13 PM Sebastian Raschka < > > mail(a)sebastianraschka.com> wrote: > > > > Not sure if there's a website for that. In any case, to explain this > > differently, as discussed earlier sklearn assumes continuous features for > > decision trees. So, it will use a binary threshold for splitting along a > > feature attribute. In other words, it cannot do sth like > > > > > > > > if x == 1 then right child node > > > > else left child node > > > > > > > > Instead, what it does is > > > > > > > > if x >= 0.5 then right child node > > > > else left child node > > > > > > > > These are basically equivalent as you can see when you just plug in > > values 0 and 1 for x. > > > > > > > > Best, > > > > Sebastian > > > > > > > > > On Oct 4, 2019, at 5:34 PM, C W <tmrsg11(a)gmail.com> wrote: > > > > > > > > > > I don't understand your answer. > > > > > > > > > > Why after one-hot-encoding it still outputs greater than 0.5 or > less > > than? Does sklearn website have a working example on categorical input? > > > > > > > > > > Thanks! > > > > > > > > > > On Fri, Oct 4, 2019 at 3:48 PM Sebastian Raschka < > > mail(a)sebastianraschka.com> wrote: > > > > > Like Nicolas said, the 0.5 is just a workaround but will do the > > right thing on the one-hot encoded variables, here. You will find that > the > > threshold is always at 0.5 for these variables. I.e., what it will do is > to > > use the following conversion: > > > > > > > > > > treat as car_Audi=1 if car_Audi >= 0.5 > > > > > treat as car_Audi=0 if car_Audi < 0.5 > > > > > > > > > > or, it may be > > > > > > > > > > treat as car_Audi=1 if car_Audi > 0.5 > > > > > treat as car_Audi=0 if car_Audi <= 0.5 > > > > > > > > > > (Forgot which one sklearn is using, but either way. it will be > fine.) > > > > > > > > > > Best, > > > > > Sebastian > > > > > > > > > > > > > > >> On Oct 4, 2019, at 1:44 PM, Nicolas Hug <niourf(a)gmail.com> wrote: > > > > >> > > > > >> > > > > >>> But, decision tree is still mistaking one-hot-encoding as > > numerical input and split at 0.5. This is not right. Perhaps, I'm doing > > something wrong? > > > > >> > > > > >> You're not doing anything wrong, and neither is the tree. Trees > > don't support categorical variables in sklearn, so everything is treated > as > > numerical. > > > > >> > > > > >> This is why we do one-hot-encoding: so that a set of numerical > (one > > hot encoded) features can be treated as if they were just one categorical > > feature. > > > > >> > > > > >> > > > > >> > > > > >> Nicolas > > > > >> > > > > >> On 10/4/19 2:01 PM, C W wrote: > > > > >>> Yes, you are right. it was 0.5 and 0.5 for split, not 1.5. So, > > typo on my part. > > > > >>> > > > > >>> Looks like I did one-hot-encoding correctly. My new variable > names > > are: car_Audi, car_BMW, etc. > > > > >>> > > > > >>> But, decision tree is still mistaking one-hot-encoding as > > numerical input and split at 0.5. This is not right. Perhaps, I'm doing > > something wrong? > > > > >>> > > > > >>> Is there a good toy example on the sklearn website? I am only see > > this: > > > https://scikit-learn.org/stable/auto_examples/tree/plot_tree_regression.html > > . > > > > >>> > > > > >>> Thanks! > > > > >>> > > > > >>> > > > > >>> > > > > >>> On Fri, Oct 4, 2019 at 1:28 PM Sebastian Raschka < > > mail(a)sebastianraschka.com> wrote: > > > > >>> Hi, > > > > >>> > > > > >>>> The funny part is: the tree is taking one-hot-encoding (BMW=0, > > Toyota=1, Audi=2) as numerical values, not category.The tree splits at > 0.5 > > and 1.5 > > > > >>> > > > > >>> that's not a onehot encoding then. > > > > >>> > > > > >>> For an Audi datapoint, it should be > > > > >>> > > > > >>> BMW=0 > > > > >>> Toyota=0 > > > > >>> Audi=1 > > > > >>> > > > > >>> for BMW > > > > >>> > > > > >>> BMW=1 > > > > >>> Toyota=0 > > > > >>> Audi=0 > > > > >>> > > > > >>> and for Toyota > > > > >>> > > > > >>> BMW=0 > > > > >>> Toyota=1 > > > > >>> Audi=0 > > > > >>> > > > > >>> The split threshold should then be at 0.5 for any of these > > features. > > > > >>> > > > > >>> Based on your email, I think you were assuming that the DT does > > the one-hot encoding internally, which it doesn't. In practice, it is > hard > > to guess what is a nominal and what is a ordinal variable, so you have to > > do the onehot encoding before you give the data to the decision tree. > > > > >>> > > > > >>> Best, > > > > >>> Sebastian > > > > >>> > > > > >>>> On Oct 4, 2019, at 11:48 AM, C W <tmrsg11(a)gmail.com> wrote: > > > > >>>> > > > > >>>> I'm getting some funny results. I am doing a regression decision > > tree, the response variables are assigned to levels. > > > > >>>> > > > > >>>> The funny part is: the tree is taking one-hot-encoding (BMW=0, > > Toyota=1, Audi=2) as numerical values, not category. > > > > >>>> > > > > >>>> The tree splits at 0.5 and 1.5. Am I doing one-hot-encoding > > wrong? How does the sklearn know internally 0 vs. 1 is categorical, not > > numerical? > > > > >>>> > > > > >>>> In R for instance, you do as.factor(), which explicitly states > > the data type. > > > > >>>> > > > > >>>> Thank you! > > > > >>>> > > > > >>>> > > > > >>>> On Wed, Sep 18, 2019 at 11:13 AM Andreas Mueller < > > t3kcit(a)gmail.com> wrote: > > > > >>>> > > > > >>>> > > > > >>>> On 9/15/19 8:16 AM, Guillaume Lema?tre wrote: > > > > >>>>> > > > > >>>>> > > > > >>>>> On Sat, 14 Sep 2019 at 20:59, C W <tmrsg11(a)gmail.com> wrote: > > > > >>>>> Thanks, Guillaume. > > > > >>>>> Column transformer looks pretty neat. I've also heard though, > > this pipeline can be tedious to set up? Specifying what you want for > every > > feature is a pain. > > > > >>>>> > > > > >>>>> It would be interesting for us which part of the pipeline is > > tedious to set up to know if we can improve something there. > > > > >>>>> Do you mean, that you would like to automatically detect of > > which type of feature (categorical/numerical) and apply a > > > > >>>>> default encoder/scaling such as discuss there: > > > https://github.com/scikit-learn/scikit-learn/issues/10603#issuecomment-4011… > > > > >>>>> > > > > >>>>> IMO, one a user perspective, it would be cleaner in some cases > > at the cost of applying blindly a black box > > > > >>>>> which might be dangerous. > > > > >>>> Also see > > > https://amueller.github.io/dabl/dev/generated/dabl.EasyPreprocessor.html#da… > > > > >>>> Which basically does that. > > > > >>>> > > > > >>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> Jaiver, > > > > >>>>> Actually, you guessed right. My real data has only one > numerical > > variable, looks more like this: > > > > >>>>> > > > > >>>>> Gender Date Income Car Attendance > > > > >>>>> Male 2019/3/01 10000 BMW Yes > > > > >>>>> Female 2019/5/02 9000 Toyota No > > > > >>>>> Male 2019/7/15 12000 Audi Yes > > > > >>>>> > > > > >>>>> I am predicting income using all other categorical variables. > > Maybe it is catboost! > > > > >>>>> > > > > >>>>> Thanks, > > > > >>>>> > > > > >>>>> M > > > > >>>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> On Sat, Sep 14, 2019 at 9:25 AM Javier L?pez <jlopez(a)ende.cc> > > wrote: > > > > >>>>> If you have datasets with many categorical features, and > perhaps > > many categories, the tools in sklearn are quite limited, > > > > >>>>> but there are alternative implementations of boosted trees that > > are designed with categorical features in mind. Take a look > > > > >>>>> at catboost [1], which has an sklearn-compatible API. > > > > >>>>> > > > > >>>>> J > > > > >>>>> > > > > >>>>> [1] https://catboost.ai/ > > > > >>>>> > > > > >>>>> On Sat, Sep 14, 2019 at 3:40 AM C W <tmrsg11(a)gmail.com> wrote: > > > > >>>>> Hello all, > > > > >>>>> I'm very confused. Can the decision tree module handle both > > continuous and categorical features in the dataset? In this case, it's > just > > CART (Classification and Regression Trees). > > > > >>>>> > > > > >>>>> For example, > > > > >>>>> Gender Age Income Car Attendance > > > > >>>>> Male 30 10000 BMW Yes > > > > >>>>> Female 35 9000 Toyota No > > > > >>>>> Male 50 12000 Audi Yes > > > > >>>>> > > > > >>>>> According to the documentation > > > https://scikit-learn.org/stable/modules/tree.html#tree-algorithms-id3-c4-5-… > , > > it can not! > > > > >>>>> > > > > >>>>> It says: "scikit-learn implementation does not support > > categorical variables for now". > > > > >>>>> > > > > >>>>> Is this true? If not, can someone point me to an example? If > > yes, what do people do? > > > > >>>>> > > > > >>>>> Thank you very much! > > > > >>>>> > > > > >>>>> > > > > >>>>> > > > > >>>>> _______________________________________________ > > > > >>>>> scikit-learn mailing list > > > > >>>>> scikit-learn(a)python.org > > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > >>>>> _______________________________________________ > > > > >>>>> scikit-learn mailing list > > > > >>>>> scikit-learn(a)python.org > > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > >>>>> _______________________________________________ > > > > >>>>> scikit-learn mailing list > > > > >>>>> scikit-learn(a)python.org > > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > >>>>> > > > > >>>>> > > > > >>>>> -- > > > > >>>>> Guillaume Lemaitre > > > > >>>>> INRIA Saclay - Parietal team > > > > >>>>> Center for Data Science Paris-Saclay > > > > >>>>> https://glemaitre.github.io/ > > > > >>>>> > > > > >>>>> > > > > >>>>> _______________________________________________ > > > > >>>>> scikit-learn mailing list > > > > >>>>> > > > > >>>>> scikit-learn(a)python.org > > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > >>>> > > > > >>>> _______________________________________________ > > > > >>>> scikit-learn mailing list > > > > >>>> scikit-learn(a)python.org > > > > >>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > >>>> _______________________________________________ > > > > >>>> scikit-learn mailing list > > > > >>>> scikit-learn(a)python.org > > > > >>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > >>> > > > > >>> _______________________________________________ > > > > >>> scikit-learn mailing list > > > > >>> scikit-learn(a)python.org > > > > >>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > >>> > > > > >>> > > > > >>> _______________________________________________ > > > > >>> scikit-learn mailing list > > > > >>> > > > > >>> scikit-learn(a)python.org > > > > >>> https://mail.python.org/mailman/listinfo/scikit-learn > > > > >> _______________________________________________ > > > > >> scikit-learn mailing list > > > > >> scikit-learn(a)python.org > > > > >> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > > > > > _______________________________________________ > > > > > scikit-learn mailing list > > > > > scikit-learn(a)python.org > > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > _______________________________________________ > > > > > scikit-learn mailing list > > > > > scikit-learn(a)python.org > > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > > > _______________________________________________ > > > > scikit-learn mailing list > > > > scikit-learn(a)python.org > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > > > > scikit-learn mailing list > > > > scikit-learn(a)python.org > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > _______________________________________________ > > > scikit-learn mailing list > > > scikit-learn(a)python.org > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > _______________________________________________ > > > scikit-learn mailing list > > > scikit-learn(a)python.org > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > >

1 0

Re: [scikit-learn] scikit-learn Digest, Vol 43, Issue 8
by Mike Smith Oct. 5, 2019

Oct. 5, 2019

Are Nearest Neighbor models better than decision trees for Adaboost? On Sat, Oct 5, 2019 at 9:02 AM <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: Can Scikit-learn decision tree (CART) have both > continuous and categorical features? (Sebastian Raschka) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 4 Oct 2019 22:28:46 -0500 > From: Sebastian Raschka <mail(a)sebastianraschka.com> > To: Scikit-learn mailing list <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Can Scikit-learn decision tree (CART) have > both continuous and categorical features? > Message-ID: > <4FC33890-94D3-4AA8-8FA9-EF1FADFD4C20(a)sebastianraschka.com> > Content-Type: text/plain; charset=utf-8 > > The docs show a way such that you don't need to write it as png file using > tree.plot_tree: > https://scikit-learn.org/stable/modules/tree.html#classification > > I don't remember why, but I think I had problems with that in the past (I > think it didn't look so nice visually, but don't remember), which is why I > still stick to graphviz. For my use cases, it's not much hassle -- it used > to be a bit of a hassle to get GraphViz working, but now you can do > > conda install pydotplus > conda install graphviz > > Coincidentally, I just made an example for a lecture I was teaching on > Tue: > https://github.com/rasbt/stat479-machine-learning-fs19/blob/master/06_trees… > > Best, > Sebastian > > > > On Oct 4, 2019, at 10:09 PM, C W <tmrsg11(a)gmail.com> wrote: > > > > On a separate note, what do you use for plotting? > > > > I found graphviz, but you have to first save it as a png on your > computer. That's a lot work for just one plot. Is there something like a > matplotlib? > > > > Thanks! > > > > On Fri, Oct 4, 2019 at 9:42 PM Sebastian Raschka < > mail(a)sebastianraschka.com> wrote: > > Yeah, think of it more as a computational workaround for achieving the > same thing more efficiently (although it looks inelegant/weird)-- something > like that wouldn't be mentioned in textbooks. > > > > Best, > > Sebastian > > > > > On Oct 4, 2019, at 6:33 PM, C W <tmrsg11(a)gmail.com> wrote: > > > > > > Thanks Sebastian, I think I get it. > > > > > > It's just have never seen it this way. Quite different from what I'm > used in Elements of Statistical Learning. > > > > > > On Fri, Oct 4, 2019 at 7:13 PM Sebastian Raschka < > mail(a)sebastianraschka.com> wrote: > > > Not sure if there's a website for that. In any case, to explain this > differently, as discussed earlier sklearn assumes continuous features for > decision trees. So, it will use a binary threshold for splitting along a > feature attribute. In other words, it cannot do sth like > > > > > > if x == 1 then right child node > > > else left child node > > > > > > Instead, what it does is > > > > > > if x >= 0.5 then right child node > > > else left child node > > > > > > These are basically equivalent as you can see when you just plug in > values 0 and 1 for x. > > > > > > Best, > > > Sebastian > > > > > > > On Oct 4, 2019, at 5:34 PM, C W <tmrsg11(a)gmail.com> wrote: > > > > > > > > I don't understand your answer. > > > > > > > > Why after one-hot-encoding it still outputs greater than 0.5 or less > than? Does sklearn website have a working example on categorical input? > > > > > > > > Thanks! > > > > > > > > On Fri, Oct 4, 2019 at 3:48 PM Sebastian Raschka < > mail(a)sebastianraschka.com> wrote: > > > > Like Nicolas said, the 0.5 is just a workaround but will do the > right thing on the one-hot encoded variables, here. You will find that the > threshold is always at 0.5 for these variables. I.e., what it will do is to > use the following conversion: > > > > > > > > treat as car_Audi=1 if car_Audi >= 0.5 > > > > treat as car_Audi=0 if car_Audi < 0.5 > > > > > > > > or, it may be > > > > > > > > treat as car_Audi=1 if car_Audi > 0.5 > > > > treat as car_Audi=0 if car_Audi <= 0.5 > > > > > > > > (Forgot which one sklearn is using, but either way. it will be fine.) > > > > > > > > Best, > > > > Sebastian > > > > > > > > > > > >> On Oct 4, 2019, at 1:44 PM, Nicolas Hug <niourf(a)gmail.com> wrote: > > > >> > > > >> > > > >>> But, decision tree is still mistaking one-hot-encoding as > numerical input and split at 0.5. This is not right. Perhaps, I'm doing > something wrong? > > > >> > > > >> You're not doing anything wrong, and neither is the tree. Trees > don't support categorical variables in sklearn, so everything is treated as > numerical. > > > >> > > > >> This is why we do one-hot-encoding: so that a set of numerical (one > hot encoded) features can be treated as if they were just one categorical > feature. > > > >> > > > >> > > > >> > > > >> Nicolas > > > >> > > > >> On 10/4/19 2:01 PM, C W wrote: > > > >>> Yes, you are right. it was 0.5 and 0.5 for split, not 1.5. So, > typo on my part. > > > >>> > > > >>> Looks like I did one-hot-encoding correctly. My new variable names > are: car_Audi, car_BMW, etc. > > > >>> > > > >>> But, decision tree is still mistaking one-hot-encoding as > numerical input and split at 0.5. This is not right. Perhaps, I'm doing > something wrong? > > > >>> > > > >>> Is there a good toy example on the sklearn website? I am only see > this: > https://scikit-learn.org/stable/auto_examples/tree/plot_tree_regression.html > . > > > >>> > > > >>> Thanks! > > > >>> > > > >>> > > > >>> > > > >>> On Fri, Oct 4, 2019 at 1:28 PM Sebastian Raschka < > mail(a)sebastianraschka.com> wrote: > > > >>> Hi, > > > >>> > > > >>>> The funny part is: the tree is taking one-hot-encoding (BMW=0, > Toyota=1, Audi=2) as numerical values, not category.The tree splits at 0.5 > and 1.5 > > > >>> > > > >>> that's not a onehot encoding then. > > > >>> > > > >>> For an Audi datapoint, it should be > > > >>> > > > >>> BMW=0 > > > >>> Toyota=0 > > > >>> Audi=1 > > > >>> > > > >>> for BMW > > > >>> > > > >>> BMW=1 > > > >>> Toyota=0 > > > >>> Audi=0 > > > >>> > > > >>> and for Toyota > > > >>> > > > >>> BMW=0 > > > >>> Toyota=1 > > > >>> Audi=0 > > > >>> > > > >>> The split threshold should then be at 0.5 for any of these > features. > > > >>> > > > >>> Based on your email, I think you were assuming that the DT does > the one-hot encoding internally, which it doesn't. In practice, it is hard > to guess what is a nominal and what is a ordinal variable, so you have to > do the onehot encoding before you give the data to the decision tree. > > > >>> > > > >>> Best, > > > >>> Sebastian > > > >>> > > > >>>> On Oct 4, 2019, at 11:48 AM, C W <tmrsg11(a)gmail.com> wrote: > > > >>>> > > > >>>> I'm getting some funny results. I am doing a regression decision > tree, the response variables are assigned to levels. > > > >>>> > > > >>>> The funny part is: the tree is taking one-hot-encoding (BMW=0, > Toyota=1, Audi=2) as numerical values, not category. > > > >>>> > > > >>>> The tree splits at 0.5 and 1.5. Am I doing one-hot-encoding > wrong? How does the sklearn know internally 0 vs. 1 is categorical, not > numerical? > > > >>>> > > > >>>> In R for instance, you do as.factor(), which explicitly states > the data type. > > > >>>> > > > >>>> Thank you! > > > >>>> > > > >>>> > > > >>>> On Wed, Sep 18, 2019 at 11:13 AM Andreas Mueller < > t3kcit(a)gmail.com> wrote: > > > >>>> > > > >>>> > > > >>>> On 9/15/19 8:16 AM, Guillaume Lema?tre wrote: > > > >>>>> > > > >>>>> > > > >>>>> On Sat, 14 Sep 2019 at 20:59, C W <tmrsg11(a)gmail.com> wrote: > > > >>>>> Thanks, Guillaume. > > > >>>>> Column transformer looks pretty neat. I've also heard though, > this pipeline can be tedious to set up? Specifying what you want for every > feature is a pain. > > > >>>>> > > > >>>>> It would be interesting for us which part of the pipeline is > tedious to set up to know if we can improve something there. > > > >>>>> Do you mean, that you would like to automatically detect of > which type of feature (categorical/numerical) and apply a > > > >>>>> default encoder/scaling such as discuss there: > https://github.com/scikit-learn/scikit-learn/issues/10603#issuecomment-4011… > > > >>>>> > > > >>>>> IMO, one a user perspective, it would be cleaner in some cases > at the cost of applying blindly a black box > > > >>>>> which might be dangerous. > > > >>>> Also see > https://amueller.github.io/dabl/dev/generated/dabl.EasyPreprocessor.html#da… > > > >>>> Which basically does that. > > > >>>> > > > >>>> > > > >>>>> > > > >>>>> > > > >>>>> Jaiver, > > > >>>>> Actually, you guessed right. My real data has only one numerical > variable, looks more like this: > > > >>>>> > > > >>>>> Gender Date Income Car Attendance > > > >>>>> Male 2019/3/01 10000 BMW Yes > > > >>>>> Female 2019/5/02 9000 Toyota No > > > >>>>> Male 2019/7/15 12000 Audi Yes > > > >>>>> > > > >>>>> I am predicting income using all other categorical variables. > Maybe it is catboost! > > > >>>>> > > > >>>>> Thanks, > > > >>>>> > > > >>>>> M > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> On Sat, Sep 14, 2019 at 9:25 AM Javier L?pez <jlopez(a)ende.cc> > wrote: > > > >>>>> If you have datasets with many categorical features, and perhaps > many categories, the tools in sklearn are quite limited, > > > >>>>> but there are alternative implementations of boosted trees that > are designed with categorical features in mind. Take a look > > > >>>>> at catboost [1], which has an sklearn-compatible API. > > > >>>>> > > > >>>>> J > > > >>>>> > > > >>>>> [1] https://catboost.ai/ > > > >>>>> > > > >>>>> On Sat, Sep 14, 2019 at 3:40 AM C W <tmrsg11(a)gmail.com> wrote: > > > >>>>> Hello all, > > > >>>>> I'm very confused. Can the decision tree module handle both > continuous and categorical features in the dataset? In this case, it's just > CART (Classification and Regression Trees). > > > >>>>> > > > >>>>> For example, > > > >>>>> Gender Age Income Car Attendance > > > >>>>> Male 30 10000 BMW Yes > > > >>>>> Female 35 9000 Toyota No > > > >>>>> Male 50 12000 Audi Yes > > > >>>>> > > > >>>>> According to the documentation > https://scikit-learn.org/stable/modules/tree.html#tree-algorithms-id3-c4-5-…, > it can not! > > > >>>>> > > > >>>>> It says: "scikit-learn implementation does not support > categorical variables for now". > > > >>>>> > > > >>>>> Is this true? If not, can someone point me to an example? If > yes, what do people do? > > > >>>>> > > > >>>>> Thank you very much! > > > >>>>> > > > >>>>> > > > >>>>> > > > >>>>> _______________________________________________ > > > >>>>> scikit-learn mailing list > > > >>>>> scikit-learn(a)python.org > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > >>>>> _______________________________________________ > > > >>>>> scikit-learn mailing list > > > >>>>> scikit-learn(a)python.org > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > >>>>> _______________________________________________ > > > >>>>> scikit-learn mailing list > > > >>>>> scikit-learn(a)python.org > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > >>>>> > > > >>>>> > > > >>>>> -- > > > >>>>> Guillaume Lemaitre > > > >>>>> INRIA Saclay - Parietal team > > > >>>>> Center for Data Science Paris-Saclay > > > >>>>> https://glemaitre.github.io/ > > > >>>>> > > > >>>>> > > > >>>>> _______________________________________________ > > > >>>>> scikit-learn mailing list > > > >>>>> > > > >>>>> scikit-learn(a)python.org > > > >>>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > >>>> > > > >>>> _______________________________________________ > > > >>>> scikit-learn mailing list > > > >>>> scikit-learn(a)python.org > > > >>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > >>>> _______________________________________________ > > > >>>> scikit-learn mailing list > > > >>>> scikit-learn(a)python.org > > > >>>> https://mail.python.org/mailman/listinfo/scikit-learn > > > >>> > > > >>> _______________________________________________ > > > >>> scikit-learn mailing list > > > >>> scikit-learn(a)python.org > > > >>> https://mail.python.org/mailman/listinfo/scikit-learn > > > >>> > > > >>> > > > >>> _______________________________________________ > > > >>> scikit-learn mailing list > > > >>> > > > >>> scikit-learn(a)python.org > > > >>> https://mail.python.org/mailman/listinfo/scikit-learn > > > >> _______________________________________________ > > > >> scikit-learn mailing list > > > >> scikit-learn(a)python.org > > > >> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > > > _______________________________________________ > > > > scikit-learn mailing list > > > > scikit-learn(a)python.org > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > > > > scikit-learn mailing list > > > > scikit-learn(a)python.org > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > _______________________________________________ > > > scikit-learn mailing list > > > scikit-learn(a)python.org > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > _______________________________________________ > > > scikit-learn mailing list > > > scikit-learn(a)python.org > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > > ------------------------------ > > End of scikit-learn Digest, Vol 43, Issue 8 > ******************************************* >

1 0

AffinityProp to classify 2D points
by Randy Heiland Oct. 1, 2019

Oct. 1, 2019

This is surely a well-studied problem, but I'm enjoying just playing with it for now. I have a bunch of 2D points (they are actually circles with possibly varying radii... later) and I'd like to devise a metric of sorts to quantify their arrangement. At first I was thinking K-means, but I don't know how many clusters there might be. So I began playing with AffinityPropagation (for my first time). The results weren't exactly what I was expecting, and I was wondering what parameters I should tweak to get different results? In the 2 sample datasets/outcomes at https://github.com/rheiland/PhysiCell_tools/tree/master/cell_metrics, I have what I call "uniform" and "clumpy". Can someone offer a general explanation of why they both have ~25 clusters? I'm probably making false assumptions about the AP alg. Thanks for any insights. Next, I'll probably explore some image processing algs and graph algs. But I'd welcome other ideas. Randy

1 0