I am working on a node classification model, My friend implemented a simple 2 layer GCN and got an accuracy of 62%, I implemented the same code and got an accuracy of 50% we are both working on google collab, anyone knows why this is happening.
Asked
Active
Viewed 32 times
0
-
First thing I would check is the proportion of the majority class. Is it binary classification? If yes, it's obvious that your model with 50% accuracy learns nothing, but if the majority class is 62% or more then even the other model learns nothing. Please give more detail about the task/data. – Erwan Oct 11 '22 at 10:42
-
No the classification is not binary there are 40 classes I copied his code and I still got 50% and he got 62% – edak Oct 11 '22 at 11:39
-
Ok. I assume that the classes are imbalanced, right? If so accuracy is not a great measure. What is the distribution of the classes? at least what is the proportion of the largest class? And how mamy instances? Ideally a confusion matrix could reveal things, but with 40 classes it's not convenient (maybe only largest classes). Anyway you can't investigate a ML problem without taking such basic info into account imho. – Erwan Oct 11 '22 at 16:27
2 Answers
0
Are you working on the same dataset?
If yes - Somewhere in the code, there could be some parameter that is initialized randomly, usually called the random seed. This is why you are getting different answers.
-
Yes we are working on the same dataset, and to top that off i copied his entire code and still got 50% test accuracy – edak Oct 11 '22 at 12:19
0
Apart from what @Rina said, another possibility could be that the algorithm runs with parallelization, and thus sometimes it is not guaranteed that all threads will finish at the correct order etc. Although this would not affect the result all that much in my experience, but it can explain some variance.
liakoyras
- 626
- 4
- 15