For vibe coding with a local LLM, start with a model that can use your coding agent’s tools and still leave enough memory for a working session. Our shortlist is Qwen3.8-27B for general coding and reasoning, Devstral Small 2 for software-engineering agents, gpt-oss-20b for tighter memory budgets, and Qwen3-Coder-Next for machines with more memory.
The choice gets harder after the first generated page. Your model needs to read files, make changes, run commands through an agent, and recover when the app fails. Each step adds context and waiting time.
That is the problem behind this guide—and the reason we built Edge0-145B and Edge0-120B. Here is how to choose a starting point, where local coding works well, and how our new models approach the capability-versus-memory tradeoff.
Which local LLMs are worth trying for vibe coding?
These recommendations draw on published model and runtime documentation. Treat them as a shortlist to evaluate against your own project and hardware.
| Model | A good starting point for | What to check before downloading |
|---|---|---|
| Qwen3.8-27B | General coding, debugging, and reasoning | The current Ollama package is about 18GB. Budget additional memory for context and the rest of your development environment. |
| Devstral Small 2 24B | Agents that explore a codebase and edit multiple files | Mistral names a single RTX 4090 or a Mac with 32GB RAM as local deployment options. Your quantization and context settings still matter. |
| gpt-oss-20b | Reasoning and function calling with a tighter memory budget | Its official card says it can run within 16GB of memory using MXFP4. Leave headroom for the agent, context, operating system, and other apps; use a runtime that handles its Harmony format correctly. |
| Qwen3-Coder-Next | Coding-agent workflows on a larger-memory machine | It has 80B total parameters and 3B active per token, but the current Ollama Q4 package is still about 52GB. Low active parameter count does not make the download small. |
The table separates two questions that model rankings often blur: can the model do the work, and can your machine serve it at a useful speed?
At the large end, Qwen3-Coder-480B-A35B is a coding-oriented model with 480B total parameters. Its smaller active parameter count reduces computation per token, but the weights still need storage and a way to reach the processor. It belongs in a very different hardware discussion from an ordinary laptop.
At the compact end, Qwen2.5-Coder-7B has a roughly 4.7GB Ollama package. It is a reasonable budget option for scripts, code explanations, and tightly scoped edits. We would start there with a small task and an explicit check before asking it to manage a whole application.
“Too small for my workflow” is more useful than “small models cannot code.” The failure may come from the task, the context, or the tool integration. Equally, choosing a larger model helps only if your hardware can keep the interaction moving.
A realistic scenario: build a private expense dashboard
Imagine you have a CSV export of business expenses and want a small app that runs on your computer. You want monthly totals, category filters, and a way to find unusually large transactions. Keeping the data local is part of the appeal.
A useful first request would be:
Build a local expense dashboard that imports a CSV with Date, Description, Amount, and Category columns. Show monthly totals and category filters. Explain how to run it, and add tests for the calculations. Use local sample data during development.
Once the first version works, ask for one addition: keep imported expenses after restarting the app. Then introduce a concrete bug report: importing the same file twice doubles the totals.
This is an illustrative project, with clear checkpoints. Does the CSV import? Do the totals match a known sample? Does the filter change the displayed rows? Does restarting preserve the data? Does a repeated import leave the totals unchanged?
Those checkpoints give the coding agent feedback it can act on. They also give you a way to decide whether another iteration improved the app.
What works: small changes with fast feedback
Generating a bounded first version
A dashboard gives the model a recognizable task: parse input, transform records, and render a view. You can constrain the stack, define the CSV columns, and supply examples of correct output.
The important part is deciding what the first version must do. A local model has less work to infer when you specify that amounts use one currency, dates follow one format, and invalid rows should produce a visible error.
For the initial calculation, provide a tiny CSV and the expected total yourself. Ask the model to implement that behavior and run the test. You can then expand the example to include a refund or a missing category.
Editing one behavior at a time
After the app runs, requests such as “add a date filter” or “show a helpful error for a missing Amount column” make good next steps. The agent can inspect the relevant files, change the implementation, and check a defined result.
Keep each change reviewable. Ask which files it expects to edit and how it will verify the result. This also makes mistakes easier to locate: if the totals change after a styling update, you have a small diff to inspect.
Tests help most when they capture a requirement independently of the implementation. For duplicate imports, decide what counts as the same transaction before letting the model write the test. Otherwise, the model may encode its own mistaken assumption in both places.
Repeating private, routine work
Once the model and dependencies are installed, local inference can support repeated edits without a hosted model request for each turn. That is useful for proprietary code, private sample data, and development with unreliable connectivity.
For this dashboard, the value is practical: you can keep iterating on a personal tool without turning each experiment into another hosted inference call. The quality of those iterations still depends on your model and feedback loop.
What breaks: memory, tool calls, and long repair loops
The model loads, but the session runs out of room
A downloaded model is only part of the memory budget. The runtime also needs working memory, including the context cache that holds information from the conversation. Your editor, browser, application, and test processes need room too.
As the agent reads files and receives command output, the session grows. A configuration that handles a short chat can struggle with a longer coding session. Ollama’s context documentation explicitly notes that increasing context length increases memory requirements.
Separate GPU memory from system RAM when checking requirements. Offloading work between them can make a configuration possible, but performance depends on the runtime and hardware. A package size or maximum context window alone cannot tell you how a full session will behave.
The model answers, but the agent cannot execute its tools
A coding model generates instructions and code. The agent around it reads files, applies edits, and runs commands. These components have to agree on the tool-call format and how results return to the model.
A successful chat response does not establish that this loop works. Continue’s Ollama guide notes that some models advertising tool support still fail in agent mode.
Before asking for the dashboard, try a small integration check: have the agent read a file, make one defined edit, and run a command that verifies it. Check the file and command output. If that fails, investigate the runtime, template, and agent configuration before judging the model’s coding ability.
A repair loop consumes more time than it saves
The duplicate-import bug is a good example. The agent might suppress identical rows, then discover that two legitimate purchases can share a date and amount. A vague requirement can turn into repeated patches that move the problem around.
Each attempt includes prompt processing, generated tokens, tool execution, and possibly another test run. Decode speed matters, but it measures only part of that cycle.
When the same failure returns, stop adding prompts to the growing conversation. Reduce the problem to a small example, state the expected behavior, and ask for a focused repair. If the model cannot make progress against that check, reconsider the model or the approach.
Where Edge0-120B and Edge0-145B fit
The capability-versus-memory squeeze is why we built Edge0-145B and Edge0-120B. We changed the expert architecture, post-trained the models, and paired them with our dedicated inference stack to make larger-model capability usable on local hardware.
Edge0-145B is the quality-oriented choice. Edge0-120B favors faster decoding and a smaller memory footprint. Our internal model-level evaluations report 86.5 on Terminal-Bench 2.1 for Edge0-145B. In the published RTX 4090 24GB, Q3_K configuration, agent decode rates are 91.7 tokens per second for 145B and 104.9 for 120B. These measurements cover model capability and decoding separately; they do not measure completion time for the dashboard example. See the conditions in our model and deployment documentation.
Both models use host memory alongside the GPU, so plan around the documented GPU, RAM, and model-format requirements together. Use Q3_K on Apple Silicon; NVFP4 targets CUDA setups. Follow the Edge0 deployment instructions and use its runtime for your platform.
For the architecture and release background, read Introducing Edge0-145B: Frontier AI Capabilities on Your Own Computer.
How to make vibe coding with a local LLM work
Start with the smallest useful version of your project, then make the feedback loop dependable:
- Choose a model for the available hardware. Check the actual model format, GPU and host-memory requirements, and supported runtime. Leave room for context and development tools.
- Verify the agent integration. Confirm that reading files, applying edits, running commands, and returning tool results all work.
- Write acceptance checks before requesting the implementation. For the dashboard, use a small CSV with known totals, then define persistence and duplicate-import behavior.
- Add one feature per iteration. Supply relevant files and a concise task. Avoid filling the session with unrelated logs or the entire repository by default.
- Check the result after every change. Run tests, inspect the diff, and use the app. Save a working checkpoint before the next request.
For vibe coding with a local LLM, the useful outcome is a change you can verify and build on. Match the task to the model, and choose a deployment that leaves enough room for the work surrounding inference. Edge0-145B and Edge0-120B give you two new options to evaluate within that workflow.
Frequently asked questions
How much RAM do I need for local vibe coding?
It depends on the model format, runtime, context length, and GPU offloading. Use the model’s deployment requirements and account for both GPU memory and system RAM. A download that fits on disk does not establish that a coding session will fit in memory.
Can a local LLM replace a coding agent?
The model and agent perform different jobs. The model proposes code and tool calls; the agent executes file operations and commands, then returns the results. Choose a supported combination and test the integration before starting a project.
Can the whole workflow run offline?
Local model inference can run offline after downloading the model and completing setup. Your workflow can still depend on package downloads, documentation services, or remote tools. Prepare the dependencies and use local tools if you need to develop without a connection.
Get Started with Edge0
Download the models to run locally, evaluate or fine-tune them, and build them into your own products:
- Model weights: Edge0-145B and Edge0-120B on Hugging Face.
- Inference framework and deployment guides: Edge0 on GitHub.
- Try it without environment setup: Download the Edge0 App. Its core model capabilities work offline, without an account or usage quota.
- Community: Join us on Discord.
For hardware or product integrations, reach the Edge0 team through GitHub or contact@autoark.ai.