Coding with a local agent

Coding agents are the hot new thing. The most popular ones are Github copilot, Claude, Devin and Cursor. These are all decent, however they require subscriptions to do any extensive work. For a single developer, that might not be a problem, but trying to scale that across a whole organization might require some financial justification.

Considering that limitation, I looked into free open source models and decided to see how effective they are at performing coding tasks. These models are capable of running on a local machine. Since a good number of them are free as well, that can be a cost effective option for teams with limited resources.

Installing ollama and choosing a model

To try this out, I decided to install OpenCode and Ollama. OpenCode is a model-agnostic coding agent, while Ollama is a tool for running Large Language Models locally. With OpenCode as the agent and Ollama as the model provider, it is possible to have a fully local setup without relying on a cloud service. The model I chose to run through Ollama was granite4.2:3b by IBM. At 2.2 GB, it doesn't take up too much space. There are also larger versions that will provide better results, but I chose to start with something small.

Ollama has installation scripts for mac, linux and windows here.

Here are some command line instructions for managing models:

#list installed models
ollama list

#install a model
ollama run <modelname> 
#eg
ollama run granite4.2:3b

#delete a model
ollama rm <modelname>

Installing opencode

Opencode can be installed through a curl script from their site opencode.ai or through homebrew brew install opencode. Once that is installed, you can start a terminal session in a folder using opencode with ollama by running ollama launch opencode.

Screenshot of terminal with opencode using the granite4.2 model

Observations

I was able to produce some code with this setup, however, it was slow and took a while getting any output from the model. I am on an M2 Macbook pro. It's not the latest hardware but a fairly decent device. For the first time in 3 years, I heard the laptop fan switch on. It's a clear tradeoff between purchasing powerful hardware or offloading that to a third party provider.