Do you run an LLM locally? · Automate What Academy

Do you run an LLM locally?

If you do share your set-up!

I use LM Studio to load some open source LLM. It was fun. I have a Mackbook M3 Pro 18 Ram. The biggest issue i found was the context windows. I don't have too much ram so, the model load and process fast but the context windows was reduce to 17325 tokens....

If you are running any open source llm share the set-up, was model are you running, and what was the main challenge.

0 comments

Do you run an LLM locally?