4

I read through "ImageNet Classification with Deep Convolutional Neural Networks" again specifically for details on how to implement PCA color augmentation. I am unsure if I have it right. Here is how I did this in numpy:

# original image is 224x224x3 of dtype uint8 

renorm_image = np.reshape(original_image,(original_image.shape[0]*original_image.shape[1],3))

renorm_image = renorm_image.astype('float32')
renorm_image -= np.mean(renorm_image, axis=0)
renorm_image /= np.std(renorm_image, axis=0)

cov = np.cov(renorm_image, rowvar=False)

lambdas, p = np.linalg.eig(cov)
alphas = np.random.normal(0, 0.1, 3)

delta = np.dot(p, alphas*lambdas)

delta = (delta*255.).astype('int8')

pca_color_image = np.maximum(np.minimum(original_image + delta, 255), 0).astype('uint8')

One serious doubt is the line "delta = (delta*255.)". I have to do this to rescale things such that the numbers make sense. I hope someone can give me feedback if this is right.

Alaa M.
  • 140
  • 5
kawingkelvin
  • 211
  • 1
  • 2
  • 7
  • Just an anecdotal update, i find color PCA not contributing too much in improving accuracy in recent work, esp. anything after resnet50. So due to the complexity added to processing, depending on your task, this may not worth your time worrying over implementing. – kawingkelvin Nov 27 '19 at 17:14

1 Answers1

2

You should not apply *255.

delta was supposed to be added to renorm_image, because you calculated this delta using cov, which was based on renorm_image.

Then how would you restore renorm_image to your original image? *std + mean or *255?

Obviously you should apply *std + mean.

Therefore, 

delta = (delta*255.).astype('int8')
pca_color_image = np.maximum(np.minimum(original_image + delta, 255), 0).astype('uint8')

should be changed to:

mean = np.mean(renorm_image, axis=0)
std = np.std(renorm_image, axis=0)
pca_augmentation_version_renorm_image = renorm_image + delta
pca_color_image = pca_augmentation_version_renorm_image * std + mean
pca_color_image = np.maximum(np.minimum(pca_color_image, 255), 0).astype('uint8')
user10253771
  • 153
  • 8