I am curious to experiment with a project like this below where one can use an open-source LLM and then retune it with their own data where in this repo its a PDF in a folder called SOURCE_DOCUMENTS. The question I have is I don't have a GPU machine, but considering looking to one if I can find something on a less expensive side.
https://github.com/PromtEngineer/localGPT
Can I run a project like this locally some hardware like this I think meant for computer vision on the edge for FP16 Yolo models? These links for NVIDIA edge AI devices where it seems like a rasp pi on steroids' which I think would be nice for a home test bench application:
https://www.seeedstudio.com/reComputer-J1020-v2-w-o-power-adapter-p-5608.html https://www.seeedstudio.com/reComputer-J3011-w-o-power-adapter-p-5630.html
Its just for experimentation and learning purposes. This most expensive model has 16 gig of memory, what is best recommended just for experimentation purposes? The cheap side would be nice especially it just being for experimentation purposes just to see if I can get it to run : )
