running a model on a GPU server usually means ssh, a pile of flags, guessing a context size, and finding out it didn't fit when it crashes. i wanted the ollama experience, but for real GPU hardware sitting in a closet. so i built two pieces: - Pantheon Local runs on the server. it knows what cards it has, how much memory is free, and which models fit. - Argus Desktop runs on your laptop. it finds the server on your network by itself. how it works: - open the app, the server shows up - pair once with a code the server prints - pick a model, install, load, chat the part i care most about: every model in the catalog is one i built and measured for that exact card. the app only offers what fits your hardware, with the real tok/s shown next to it. the catalog is signed, so the server only installs what i've tested. right now that's Ornith 9B on a Tenstorrent p150 and Qwen3.6 35B on a pair of ten-year-old K80s. the whole stack is my own C: the engine, the server side, and the desktop app (no electron). it supports two cards for now: the Tenstorrent p150 and the Tesla K80. those are the cards i own, and i only ship what i can build and measure myself. both are open source and free to use, launching in the next couple of weeks. anyone else running K80s or a Tenstorrent card? which models would you want on them?

my 9B model (Ornith 1.5, 8-bit, one Tenstorrent Blackhole card) uses speculative decoding: a small head built into the model (MTP) guesses the next few tokens, and the big model checks them all in one pass. every token is still the big model's own choice, so the output never changes. the speed depends only on how often the guesses are right. the official head's first guess was right 84% of the time. but the 2nd and 3rd guesses dropped to 50-69%. - the reason: the head was trained to guess from the big model's own hidden state. when drafting, it chains on its own guesses instead, and it never practiced that. - the fix (the FastMTP idea): let the big model write ~100k tokens of its own answers, greedy and sampled. save its hidden state at each position. then train the head to chain on its own outputs, the way it's actually used. - cheap: a LoRA on my M4 Mac, ~9 minutes per run. the data was ~80 minutes of card time. the result, through the real server, single user: - greedy: 40.0 → 41.7 tok/s (+4.2%) - sampled: 31.3 → 32.9 tok/s (+4.9%) - short answers: 45.4 → 48.8 tok/s - all 50 test outputs byte-identical to before. the head only drafts; it can't change what the model says. my weak spot: offline it predicted +7.6%, the card gives about half that. the card runs an 8-bit copy of the head, and the offline test set was small. the gain was still climbing with more data, so it isn't done. small model, small trick, free speed. anyone else tuning their draft heads?

Hi Clief Notes Community! Please note that this Saturday's Vibe Film Session (October 31, 2026) has been canceled. We are excited to share the reason behind the change. Our very own Danney Trieu will be attending the Hello America Concert in Las Vegas, where he has been selected to serve as the AI Visual Director for this major production. This is an incredible professional milestone and a testament to the creativity, innovation, and dedication Danney brings to his work. Opportunities like this don't come around often, and we could not be prouder to see his talents recognized on such a significant stage. The leadership team at Clief Notes is honored to support and sponsor Danney as he represents not only his creative vision but also the spirit of excellence and innovation that our community values. We believe in investing in the growth of our creators, builders / leaders, and we're thrilled to cheer him on during this exciting chapter. Please join us in congratulating Danney and wishing him an amazing experience. We look forward to hearing about the event and sharing updates with the community upon his return. Thank you for your understanding, continued support, and commitment to the Clief Notes community. Congratulations, Danney! We're rooting for you! 🎉

the expensive part of using Claude for real work isn't the thinking. it's the doing: hours of running commands, waiting on builds, polling jobs, re-reading output. every one of those steps burns tokens, and that's what pushes you onto the pricier plans. so i split it. Claude Code writes the spec and the plan, and checks the result at the end. a local model does the long, mechanical work in between, on my own machine, for free. i've been doing this with Ornith 9B (8-bit, on my own engine). on my last run it merged 75 commits across two repos and got every test passing. the planning and the final check were a small slice of what doing the whole job with Claude would have cost. the catch: a small local model only works if the plan is written for a small model. so i wrote down everything i learned into a file you hand to Claude along with your task: https://git.teamide.dev/David/TeamIDE-methodology/src/branch/main/Model-Best-Practices/Ornith-q8-best-practices.md what's in it: - what the local model is good at: git work, running and polling long jobs, following written rules - what to expect: its work is more reliable than its reports. it doesn't go looking for context on its own. it doesn't know when it's done. it slows down at long context - how to brief it: exact files and line ranges to read, absolute paths, small edits, every rule written down, every step ending in a command with the exact output it should print, stop rules ("if it's already done, say so and stop") - how to run it: one task per fresh session, keep the context small, make the plan file the memory so any session can pick up where the last one stopped - before anything lands: check its claims against git and the logs, never its summary. on my runs that caught every wrong note it wrote how to use it: give Claude Code the file plus your task, and ask it to write a spec and plan in that style for your local model. run the plan locally, then bring the result back to Claude to verify before you merge.

That's the baseline, not the edge. What separates people is whether they can do it safely. Safe comes from constraints: rules you force yourself to follow. Clear stages. A defined order for what the model reads and what it does. I looked at eleven recent ways to give AI the right context (check my other post). They differ a lot, but they share one thing: hierarchy. A short summary first, detail only when needed. Design patterns matter because you stop improvising. The good result stops depending on a lucky prompt and starts depending on a structure you can repeat. Reusable templates, in my view, is what lets you vibe code safely.