making local models run on a GPU server as easy as ollama

running a model on a GPU server usually means ssh, a pile of flags, guessing a context size, and finding out it didn't fit when it crashes. i wanted the ollama experience, but for real GPU hardware sitting in a closet.

so i built two pieces:

Pantheon Local runs on the server. it knows what cards it has, how much memory is free, and which models fit.
Argus Desktop runs on your laptop. it finds the server on your network by itself.

how it works:

open the app, the server shows up
pair once with a code the server prints
pick a model, install, load, chat

the part i care most about: every model in the catalog is one i built and measured for that exact card. the app only offers what fits your hardware, with the real tok/s shown next to it. the catalog is signed, so the server only installs what i've tested.

right now that's Ornith 9B on a Tenstorrent p150 and Qwen3.6 35B on a pair of ten-year-old K80s. the whole stack is my own C: the engine, the server side, and the desktop app (no electron).

it supports two cards for now: the Tenstorrent p150 and the Tesla K80. those are the cards i own, and i only ship what i can build and measure myself.

both are open source and free to use, launching in the next couple of weeks. anyone else running K80s or a Tenstorrent card? which models would you want on them?

0 comments

making local models run on a GPU server as easy as ollama