Cloud-backed coding agents are useful, but their model access, rate limits, and pricing depend on a provider. I wanted a small agent whose model endpoint I could choose independently. LillyCoder runs locally and speaks to a compatible LLM server. With a local server and downloaded weights, it can work offline. Pointing it at a remote endpoint sends prompts and included code to that endpoint. Other agents can also support alternative or local models; the point here is the small, interchangeable setup I wanted.
The shape of the tool is small on purpose. You run lillycoder in a project directory and it drops you into a chat REPL. REPL stands for read-eval-print loop: an interactive prompt that takes an input, runs it, shows the result, and waits for the next one. You type a request, and the model on the other end picks tools to do it: read a file, write a file, edit a file, run a shell command, install a package, grep the project, list a directory. Each tool call comes back as a structured action, the agent executes it inside your current working directory, and the result feeds back into the conversation. From the outside, this is the same loop every coding agent runs. From the inside, the difference is what is doing the picking.
What is doing the picking is whatever you want. lillycoder speaks the OpenAI /v1/chat/completions shape, which is by now the lingua franca of local LLM servers. llama.cpp publishes it. ollama publishes it on its compatibility surface. LM Studio publishes it. hydra-llm publishes it. So the agent does not care which one is running, and on first launch it scans common ports and offers to use whatever it finds. If nothing is running, you point it at a URL with --api. The model is not part of the package; it is whatever you started.
Model location and execution permissions are separate questions. A cloud-backed coding CLI can still write files and run commands on your machine. A local model can do the same through LillyCoder. The risks depend on the tools, filesystem access, network access, and permissions you give the agent. Keeping inference local addresses one data-sharing concern; it does not make shell commands safe.
Every mutating action is gated by a prompt before it runs. When the model decides to write a file, you see the path and the size; when it wants to run a command, you see the command. You answer with one of four options:
🦊 lilly wants to: write_file("src/index.js", 142 chars)
[y]es [n]o [a]lways for this tool [p]ath: always for this exact target
>The four options are not decoration. y is the obvious one. n rejects this single call and lets the model retry or change tack. a grants a session-scoped pass for the tool itself, which is what you want once you trust the model to read files but still want to look at every bash. p grants a pass for this exact path, which is what you want when the model is iterating on a single file and you do not want to keep approving each save. Both passes are session-scoped: they vanish when the REPL exits.
Above the permission prompt sits a hard-deny classifier. There is a list of commands that simply will not run, no matter what you answer at the prompt and no matter what flag you pass. sudo, rm -rf /, mkfs, dd of=/dev/*, fork bombs, recursive chmod or chown against / or ~. These are refused before they reach the executor. The --bypass-permissions flag (which is there for headless or scripted use) skips the per-call prompt, but it does not skip the safety classifier. That is by design; bypass should be for tedium, never for danger.
The classifier is small and dumb on purpose. It is a string-and-pattern checker, not a sandbox. A malicious shell pipeline that does the same damage in a clever way will get past it; sandboxing the entire tool is a bigger project than this one is. What the classifier buys you is protection against the kind of mistake that comes from an LLM hallucinating a path or copy-pasting a destructive command from training data. That is the realistic failure mode, and the thing the classifier exists to catch.
The development container gives the agent a separate environment, but it is not confined to one disposable folder. The Compose file, checked on 15 October 2026 mounts WORKINGDIR/ at /workspace, the repository at /opt/lillycoder, and your LillyCoder configuration directory. All three mounts are writable in that file. Changes there affect the host, and network access is available. Inspect those mounts before using it with valuable work.
docker compose up -d
docker compose exec lillycoder bash
# inside the container, in /workspace:
lillycoder --api http://host.docker.internal:11434/v1The --bypass-permissions flag removes prompts while leaving the pattern-based deny list active. It does not strengthen isolation. Containers share a kernel with their Docker host, and their protection depends on configuration and vulnerabilities; see the Docker security documentation. docker compose down -v removes containers and applicable Docker volumes. It does not undo changes or delete data in host bind mounts. Use a disposable checkout when experimenting.
The model on the other end matters. Tool-calling reliability is not free; it requires a model that was trained or fine-tuned to emit structured tool calls without going off the rails. The community-distilled tier of "good at tools" is not the same as the tier of "good at chat," and a model that is excellent at writing prose can be useless at picking the right tool with the right arguments. lillycoder keeps a small allowlist of model families that have, in my testing, proven reliable enough: Qwen 2.5 and 3, Gemma 3, Llama 3.1, Mistral Small 3, Dolphin 3 R1. If you point the tool at a model outside that list, it warns you. --force silences the warning when you know what you are doing.
For the small coding tasks I use it for, 14B to 32B models at Q4_K_M can be useful starting points. A 32B model needs roughly 19–20 GB just for weights, so it will not fit wholly on a 16 GB GPU; even 24 GB can become tight once the context cache and runtime buffers are included. Tool support depends on the model, chat template, and server. My experience with a few tasks is not a benchmark establishing parity with cloud agents.
There is a sibling project called hydra-llm that I built before this one. It manages local LLM servers: download a model, start it on a port, get an OpenAI-compatible endpoint back. lillycoder talks that exact shape. So the two compose into a fully local coding stack with two commands:
# in hydra-llm:
hydra-llm start qwen2.5-32b
hydra-llm api qwen2.5-32b # prints the URL
# in your project directory:
lillycoder --api http://localhost:18087/v1The split stays simple. hydra-llm handles the model lifecycle, and lillycoder is the agent on top. Either half is replaceable. If you already have ollama running, skip hydra-llm. If you have a different agent you like better, point it at hydra-llm's endpoint instead. Tools that combine the runtime and the agent tend to become worse at one of those jobs.
Installation is the boring part, which is how it should be. On Debian and Ubuntu there is a signed apt repository, so it is a normal package after a one-time keyring setup:
sudo install -d -m 0755 /etc/apt/keyrings
curl -fsSL https://ra-yavuz.github.io/apt/pubkey.gpg \
| sudo tee /etc/apt/keyrings/ra-yavuz.gpg >/dev/null
echo "deb [signed-by=/etc/apt/keyrings/ra-yavuz.gpg] https://ra-yavuz.github.io/apt stable main" \
| sudo tee /etc/apt/sources.list.d/ra-yavuz.list
sudo apt update
sudo apt install lillycoderFrom source on any Linux it is the usual git clone and pip install --user -e .. After that, cd into a project, run lillycoder, and start typing. If a local LLM server is already listening on a common port, the tool finds it. If not, pass --api with the URL.
The bundled persona is a kid-coder voice, deliberately playful, because I find it pleasant to work with and the alternative was yet another terse-junior-engineer voice that the world has plenty of. If that is not your taste, drop a markdown file at ~/.config/lillycoder/personas/<name>.md and run with --persona <name>. The persona is the system prompt; everything else, the tools, the gates, the loop, is unchanged.
Personas can also encode project conventions. A persona that says "always run the test suite after edits, never modify files outside src/, prefer pure functions" turns into a behavior the model leans on across the whole session. It cannot provide a sandbox or a guarantee. It can shape what the model picks before it gets to the permission prompt, which means fewer prompts to refuse and more to accept on autopilot.
The bigger point behind all of this is that local agentic coding is no longer a research project. The pieces are there: capable models, an open endpoint shape, fast enough hardware, and the realisation that "agent" and "model server" are two different concerns that should not be welded together. lillycoder is one specific composition of those pieces. It is opinionated about safety (gates by default, hard-deny on top), uninterested in being the model runtime (point it at one), and small enough that the entire codebase is readable in one sitting if you want to know what runs on your machine.
If you want to try it, the install lines above are real and the source is on GitHub. If something local is already listening, just running lillycoder in a folder will pick it up. The first thing to type, the same thing I type, is "what files are in this folder?" From the answer you can already tell whether the model on the other end is going to be a useful collaborator, and that is the only thing that really matters about which one you picked.