12

What is the best optimizer for Convolutional neural network (CNN)?

Can I use RMSProp for CNN or only for RNN?

Ethan
  • 1,625
  • 8
  • 23
  • 39
Noran
  • 758
  • 3
  • 8
  • 21

1 Answers1

9

Yes, you can use the same optimizers you are familiar with for CNNs.

I don't think that there is a best optimizer for CNNs. The most popular in my opinion is Adam. However some people like to use a plain SGD optimizer with custom parameters.

An excellent article explaining the differences between most popular gradient descent based optimizers can be found here.

JkBk
  • 422
  • 2
  • 7
  • I get better accuracy with default valued SGD than ADAM. I read somewhere that that tends to be the case with CNN's. I'm actually looking for a more definitive answer that does recommend and explain why one is better than the other, as I don't seem to be the only one noticing this when optimizing a CNN. https://towardsdatascience.com/the-3-best-optimization-methods-in-neural-networks-40879c887873 – Elegant Code Feb 28 '20 at 01:18
  • Here is a link to a white paper on why SGD tends to generalize better than ADAM even though the latter tends to converge faster: https://proceedings.neurips.cc/paper/2020/file/f3f27a324736617f20abbf2ffd806f6d-Paper.pdf – Zexelon Dec 18 '22 at 03:57