7

I'm using the HMMLearn python package for hidden markov models. That implementation is build on multivariate gaussian distributions.

So I have a string of features. How sensitive are gaussians to vastly different feature scales? Will it be really skewed if one feature is scaled between 0 and 1, and another is scaled between 0 and 1e8?

Clayton
  • 73
  • 3
  • Why would you want to work with data that hasn't been standardized? I'm not familiar enough with hidden markov, but there are so many pitfalls to not scaling, it is so easy to do, and it is reversible, so why would you not just scale the data and move on? – AN6U5 Jul 07 '15 at 21:30
  • 1
    Upon further reflection of my question, scaling the data seemed like the right thing to do. Thank you for confirming my thought process. – Clayton Jul 08 '15 at 15:42

1 Answers1

2

There are many pitfalls to not scaling your data and it is generally very advisable to scale it. It is so easy to do, it is reversible, and it useful in other operations like removing outliers.

Upon further analysis of the specifics of Hidden Markov Models using Multivariate Gaussians the theoretical accuracy should not suffer as a result of drastic differences in the scales of your features. But, an important operation involving multivariate Gaussian distribution is matrix inversion.

Though the theoretical invertibility won't change, the practical numerical solution to your problem will likely suffer inaccuracy due to the difference in scales. Iterative methods will have issues with convergence and direct solve methods will suffer from stiffness. This is especially true when common complications like linear dependence are present.

Here is a set of 3 lectures on HMM with some specifics on multivariate Gaussians (1-2-3)

I know we've already discussed this in the comments, but I wanted to close out this question, so have added it as an answer.

I hope this helps!

AN6U5
  • 6,798
  • 1
  • 24
  • 42