I am new in python and data science (and not great in math). I am learning machine learning. I got following normalize function. Can you please explain what does this normalize function do?
def normalize(array):
return (array - array.mean()) / array.std()
Also please explain what this array - array.mean() does?
Also please explain what this array - array.mean() do?
Basically, it is doing memberwise subtraction operation after broadcasting. np.mean function finds the mean in your array and its result will be a scalar, a single number. Your array is a numpy array and the result of the latter term is a single value as mentioned. Consequently, the single value gets extended to the shape of the former term. then a memberwise subtraction will be performed for each entry of the array and the result will have the same shape as the former term.
Can you please explain what does this normalize function do?
Normalizing data is done for accelerating optimization. If you have features with different scales, it will take too much time for your optimizer function to find optimal points. Suppose you have age feature which can change between 0 to 150 (!) and salary which can be changed from 0 to whatever, like 500,000,000 $. your optimization algorithm used in your ML model will take too much time, if possible, to find appropriate weights for each feature. Moreover, if you don't scale your data, your ML algorithm may take too much care to features with large scales.
The black box answer is you can’t train models when your features have different ranges (1-5 vs 1-5000).
I really recommend writing a simple gradient descent solver (plenty of samples online) and train a simple linear model (y=mX+b) where the target solution is something like y=5x+500. With a fixed learning rate for both m and b - the training goes super slow or doesn’t converge and goes backwards. The solution is normalizing or scaling.
As I said the best way to learn this is code it up and experiment.
It is a standard approach is to scale the inputs to have zero mean unit variance.
Mathematically:
if you have some observed data (your training examples), called the empirical distribution, which is a proxy for the true data distribution that is generated from the unknown data distribution.
you could assume that your input features, each dimension is drawn from a uni-variate Gaussian distribution, which you could estimate using maximum likelihood estimation in closed form solution see MLE for Gaussian and you could obtain unbiased mean and biased variance could be fixed to be unbiased. following that, it's straightforward to just subtract all your data points from the original mean, you end up with zero mean, and scale the variance to be one by dividing by the fixed unbiased variance.
