Model accuracy when training on GPU and then inferencing on CPU

Question

When we are concerned about speed, GPU is way better than CPU.

But if I train a model on a GPU and then deploy the same trained model (no quantization techniques used) on a CPU, will this affect the accuracy of my model? Can the accuracy of the same model degrade on a CPU?

My intuition says, GPU vs CPU should not make any difference if accuracy is concerned.

But I have one doubt that, the GPUs and CPUs have their own different ways to process the information internally. Both of them have different architecture. When a model is trained on a GPU, does the exact same way of processing happens when trained on a CPU but in much slower way? I am not concerned about the accuracy while training, but if a model was trained on a GPU, will it perform exactly with same accuracy on a CPU?

Answer 1

The results you get will be identical* for identical inputs. So for any practical purposes, the accuracy you get will not depend on whether you use a cpu or gpu.

*up to floating point precision. From this post on the pytorch forum:

Both are implementing the floating point number computation standard. So they are both correct (even though [they may be] different)

Answer 2

A trained model is just a well-defined set of floats i.e parameters

If we go for prediction,
What it will do are a large number of multiplication and addition of input data and the parameters. This will be done by the ALU (Both CPU and GPU)
A CPU will have a few ALU and a GPU will have thousands of them. So it will complete its task very quickly.

So, unless there is a difference of precision for holding the floats which might accumulate in a Deep Network, I don't see any reason that the prediction should differ.