5

Given a data set with features, that you want to check for normality, one feature at a time w/o a multivariate normal test, how do you decided which test of normality to use? For example, using the python module scipy I could either use:

http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.shapiro.html

or I could use:

http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.stats.mstats.normaltest.html

For example on the same feature I get these outputs: 

scipy.stats.shapiro(data[:,0])
(0.9985173940658569, 0.77845299243927)


scipy.stats.mstats.normaltest(data[:,0])
NormaltestResult(statistic=1.492603328675163, pvalue=0.47411675723570479)


scipy.stats.anderson(data[:,0], dist='norm')
AndersonResult(statistic=0.17542490527580412, critical_values=array([ 0.573,  0.653,  0.783,  0.913,  1.086]), significance_level=array([ 15. ,  10. ,   5. ,   2.5,   1. ]))
FakeBrain
  • 109
  • 1
  • 6

1 Answers1

2

Mathematical detail of the two different tests can be found in two papers:

[1]. http://sci2s.ugr.es/keel/pdf/algorithm/articulo/shapiro1965.pdf

[2]. http://www.jstor.org/stable/2334522?seq=1#page_scan_tab_contents

To summarize, both methods are based on hypothesis testing. But they are using different test statistics.

Shapiro-Wilk test has many good analytical properties and was designed to apply to data with sample size less than $50$. However, when the sample size becomes greater, Shapiro-Walk test might be unreliable. As I quote from the second paper:

Shapiro and Wilk did not extend their test beyond samples of size 50. A number of reasons indicate that it is best not to make such an extension.

Essentially, that is why people invented the second normality test. As I quote from the second paper:

We present a new test of normality applicable for samples of size 50 or larger which possesses the desirable omnibus property

3x89g2
  • 144
  • 4