So, a couple of you strongly suggested that I not use my subscriptions to Grok or ChatGPT as models for Hermes Agent, since that eats up so many tokens. You suggested I use ollama and the gemma4 models. I gotta tell you, it worked great. The key for me was that ChatGPT was profoundly helpful in teaching me to set it up. Basically, the workflow goes something like this:
I do a first pass that goes recursively through the directory and uses local tools (e.g. pdftotext, tesseract) for text retrieval and OCR. It uses ollama with gemma4:E4b to do the initial classification. Those documents that aren’t classified with high confidence then go through a second pass that uses gemma4:12B to do better OCR and classification.
And it does it all locally, so no token costs! It takes about 5-6 seconds per document on average for classification. I have about 15000 documents per year in my backups (of which there are really only about 4000-5000 unique documents). So, it’s about one night to go through one year of backups. It’s working great.
I have beome a big fan of ollama and local models. It all loads and runs on my NVIDIA GPU.
The key for me, though, is that I would never have figured out how to write the schema and commands for ollama without being tutored by ChatGPT. It turned out that there were some issues with using Hermes agent on ollama, but it was just as easy to write a little python script to do it. One odd thing is that parallelization slowed things down. Doing this with one “worker” was faster than two.
When this is done, I’ll move to images. ChatGPT says the ollama and gemma should be great for classifying images in my backups. We’ll see. I would include the schemata and prompts, but it was vibe coded with ChatGPT and each pass runs around 3000 lines.
In any case, if any of you are like me and have 20 years of backups sitting there. this turns out to be a great way to get a handle on all those old files. Thanks to those of you who told me to use ollama and a local model.