1

Disclaimer: I am very new to using stackexchange so bear with me.

I am trying to gain experience using neural networks and would like to do a few projects involving them. Right now I am trying to train on the cifar10 dataset and built a quick CNN model in keras. I then realized that this model has 2 million trainable parameters, and as a result the kernel restarts before the model is trained on even 1 epoch. I think this is due to a lack of memory (I have 8gb RAM), but this confuses me somewhat because I've usually heard that GPUs were usually the factor that make machines inadequate for deep learning tasks.

The questions:

  1. Is my kernel restarting because of a lack of memory or processing power?
  2. What are my options? If it's memory, should I just buy another 8GB of RAM and plug it into my laptop? If it's the GPU, should I use AWS (I have before) or just build my own desktop? I would really like to engage in the iterative process of improving a model -- building a baseline model, training, tweak the architecture of the NN, train again, read a paper about how people improved results on cifar10, train again, etc., so I'd imagine using AWS all the time could get expensive quick. I probably would not build a desktop right away since I have to make sure the money would be well spent but would still appreciate being pointed towards some resources by someone experienced.
  3. My third option is just reducing the amount of computation my computer has to do by using smaller/less fully connected layers and using more pooling layers. If I went this route, how limited would I be in terms of what projects I can tackle? I'm particularly interested in image processing projects.

I'm just trying to fully understand all my options from someone that's dealt with this issue. Thank you for taking the time to read my post, I really appreciate any advice given.

valentinocc
  • 11
  • 1
  • I would advise using Google Collab, It's free and comes with a very nice GPU ! – Ran Elgiser May 30 '19 at 16:44
  • Adding more RAM will not increase training speed (at least not much). Training on GPU is the thing to do (my GPU alone has 8 GB RAM which is fully employed during training). Try with AWS or Google. Kaggle offers free GPU use (but it is not very practical). Dont buy a GPU yourself unlike you really want to go for expensiv model training at home. Reducing complexity of the network may help, but if you want to train a good model, you simply might need the extra computation power. – Peter May 30 '19 at 18:42
  • There is not information provided to answer your question. Since you don't get a memory error this might as well be a problem with your environment. For example, there is a known issue that leads to silent crashes in jupyter notebooks when you have multiple copies of the OpenMP linked to your code. – oW_ May 30 '19 at 19:20
  • @oW_ I don’t get an error which is frustrating. I did try running my program in Jupyter notebooks and spyder. Also tried installing an older version of tensorflow but that didn’t work. – valentinocc May 30 '19 at 23:37

0 Answers0