8

So i have a PandasDataFrame with categorical variables in a column which i want to one hot encode i've used the following code from an ML udemy course

from sklearn.preprocessing import OneHotEncoder
onehotencoder=OneHotEncoder(categorical_features=[10])
Y= onehotencoder.fit_transform(X).toarray()

However i get the following error 

ValueError: could not convert string to float:

A bit of information Y is converted to an object from a df using 

Y=df.iloc[:,:].values

I want to oneHotencode the 10th column of y which contains string values. The type of Y in variable explorer is object and if execute 

type(Y)

i get numpy.ndarray

I'm new to Pandas and sklearn and would really appreciate any help.

Iltl
  • 253
  • 1
  • 3
  • 7
  • 1
    Use [LabelBinarizer](http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelBinarizer.html) instead of OneHotEncoder – MaxU - stand with Ukraine Oct 18 '17 at 19:53
  • I think first step is label encoding your array before actually one_hot encoding. See, for instance, this webpage: http://www.ritchieng.com/machinelearning-one-hot-encoding/ – tagoma Oct 18 '17 at 21:21

5 Answers5

5

There is an easy way to use one hot encoding in pandas and you can read about it in the following link:

https://pandas.pydata.org/pandas-docs/stable/generated/pandas.get_dummies.html

The get_dummies method is easy and optimized for use in pandas Data Frames.

Good luck!

AR_
  • 91
  • 6
  • This has the downside of not being directly pluggable into a sklearn pipeline, so re-running transforms on validation and test data will require separate handling from the pipe, or else creating a custom transformer to wrap `get_dummies`. – holocronweaver Apr 11 '23 at 18:47
5

It may be a late answer, but I got the same problem and below is the solution

# Don't use categorical_features=[10] in encoder init
from sklearn.preprocessing import OneHotEncoder
onehotencoder=OneHotEncoder()
Y= onehotencoder.fit_transform(X[:,[10]]).toarray()
vikas sharma
  • 51
  • 1
  • 1
  • 2
    This works with the newer versions of sklearn. (At the time of the question OneHotEncoder couldn't handle string data.) – Ben Reiniger Dec 17 '19 at 14:55
3

You need two steps:

  1. Use LabelEncoder to encode your string variables to integers
  2. Then use OneHotEncoder on your integer variables
DontDivideByZero
  • 131
  • 3
1

You could just use a LabelBinarizer. Label binarizer will skip the two step process(converting string to integer and then integer to float) as mentioned by DontDivideByZero.

from sklearn.preprocessing import labelBinarizer
encoder = LabelBinarizer()
Y = encoder.fit_transform(X)

This way you will convert the entire X matrix, but later you can quite easily extract Y[10] which is the one hot encoded matrix that you are looking for.

Stephen Rauch
  • 1,783
  • 11
  • 21
  • 34
Mayank Khanna
  • 29
  • 2
1

I was also facing the same issue. I tried every possible way to do and got into why it wasn't happening. Actually, earlier OneHotEncoding needed numerical value first (earlier we couldn't directly encode string type data to numerical using OneHotEncoding, so first we used to apply LabelEncoding first) and then we used to apply OneHotEncoding.

But Now, OneHotEncoding could directly work with String data types also, but here we need data to be of type -either a DATAFRAME or a 2D Array (Not Object type) example [50,1].

Now you can do so, this way:

from sklearn.preprocessing import OneHotEncoder
    onehotencoder = OneHotEncoder()
    X_new_enc= onehotencoder.fit_transform(X[:,[3]]).toarray()  #[String_Column Index]

OR you rather use get_dummies directly (pandas based)

X= pd.get_dummies(X)

Feel free to ask any doubts over this.

HumbleBee
  • 11
  • 1