[SYSTEM] Technical Overview

Agentic Coding

Local AI Agents with llama-cpp & open-code

2026.06.29

Agentic Coding

A software development approach where an AI agent:

Plans the development steps
Writes the code
Tests the implementation
Modifies based on feedback

user request → LLM answer → execution

open-code

AI coding agent orchestrator (open-source).

Supports multiple LLM backends / AI providers
Like: claude-code, gemini-cli, cursor-agent, openai-codex, openclaw

llama-cpp: Local Inference

Runs LLM inference locally on consumer hardware.

Supports any GGUF pre-trained model
Written in C++ — remarkable optimization
Advanced quantization techniques
Peers: ollama, vllm, lm studio

Two Main Orchestrators

OpenClaw

Grew from clawdbot, moltbot
Named after Anthropic's Claude
"Digital employee that works for you"
General task orchestration

OpenCode

Grew as alternative to claude-code
"Pair programmer in the terminal"
Focused coding workflows
Open-source, multiple LLM backends

Request → Execution

1

open-code

Assembles the request

→

2

llama-cpp

Loads & runs the LLM

→

3

open-code

Translates LLM instructions to actions

→ offline environment · free · experimental

Disk Space Management

Use baobab to understand disk usage.

Clean your computer. Make space for GGUF models.

Install llama-cpp

Easier with admin privileges.

Guide: llama-cpp.com/getting-started/

Download GGUF Model

Example: Gemma-4-31B-it-QAT by Google DeepMind.

Source: HuggingFace — unsloth/gemma-4-31B-it-qat-GGUF

LLM Quantization

Lowers model precision to reduce memory usage.

Size

Model takes space on disk + RAM
4-bit ≈ 7–15 GB, 8-bit ≈ 15–30 GB

Speed · Accuracy

Speed: Weights reduction → faster loading
Accuracy: Rounding → perplexity ↑

Suffixes: 4-bit, 6-bit, 8-bit, etc.

Perplexity (PPL)

Measure of LLM uncertainty.

Average number of equally likely continuations
Lower PPL = more confident predictions
Used to compare quantization levels

Ref: huggingface.co/docs/transformers/perplexity

GGUF Model Card

Quantization

4-bit (Q4)	~8 GB
6-bit (Q6)	~12 GB
8-bit (Q8)	~16 GB

RAM Required

7B model	~4–8 GB
31B model	~17–32 GB
70B model	~40–80 GB

Disk Size

Q4_K_M	~7–15 GB
Q8_0	~15–30 GB

Overhead

Context window ≈ 20% extra RAM
RAM ≈ model file size

What is an LLM?

Probabilistic model: predicts the next token in a sequence.

P(xt+1 | x1, ..., xt; θ)

where xt+1 = target token, x1, ..., xt = input context, θ = learned parameters

llama-cpp Installation

Step 5: Build & verify the binaries.

Install open-code

Easier with admin privileges.

Website: opencode.ai
Docs: opencode.ai/docs/

Open Network Connection: open-code ↔ llama-cpp

$ cat ~/.config/opencode/opencode.json

opencode.json

{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "llama.cpp": {
      "name": "llama-server (local)",
      "npm": "@ai-sdk/openai-compatible",
      "options": {"baseURL": "http://localhost:8080/v1"},
      "models": {
        "gemma-4-31B": {
          "name": "gemma-4-31B",
          "mode": "primary",
          "limit": {"context": 256000, "output": 65536},
          "modalities": {
            "input": ["text", "image"],
            "output": ["text"]
          }
        }
      }
    }
  }
}

without modalities it often won't believe it can read images

Config Location

File path for standard user configuration.

Terminal

~/.config/opencode/opencode.json

Path: /Users/mecatronik/.config/opencode/opencode.json

Verify with: cat .config/opencode/opencode.json

Unix User Isolation

Users cannot access another user's files (except admin/root).

Risk

open-code can erase important files
Admin/root can see everything

Mitigation

Run open-code as standard user
llama-cpp does not execute commands

User Execution Flow

1

Admin

kayane

runs llama-server

→

2

Standard

mecatronik

runs open-code

→

3

Browser

localhost:8080

127.0.0.1:8080

Localhost Connection

Local network connection within the computer.

Endpoints

http://localhost:8080
http://127.0.0.1:8080/v1

CLI vs Server

llama-cli

./build/bin/llama-cli
Interactive Command Line Interface
-m models/model.gguf
Direct terminal I/O

llama-server

./build/bin/llama-server
Starts an HTTP API server
-m models/model.gguf
REST endpoint for agents

Launch llama-server

Admin terminal command.

Terminal (admin)

./build/bin/llama-server -m models/model.gguf

Launch open-code

Standard user terminal command.

Terminal (user)

opencode

Shortcuts: Ctrl+P → switch model

Extending LLM Context

An LLM needs 'context' to work.

MCP — Model Context Protocol servers
Skills — Structured instructions to condition the LLM

Context via: copy-paste docs, detailed prompts, context engineering

Results & Performance

Attempt 1

Time	41m 46s
Status	HTML output

Attempt 2

Time	31m 45s
Status	Error → server crash

Final Result

Time	4h 05m
Status	HTML output ✓

Hardware

MacBook Pro

M1 Max, 32GB, 8 cores

Cumulative Mentions (Google Scholar)

Source: not accurate, but conveying the trend

NLP (2.23M)

LLM (409K)

World Model (18K)

Agentic (1.7K)

MCP Protocol

Universal interface for external service APIs.

MCP Functions

Abstract API from external services
Manage authentication & permissions
Instruct LLM: get, post records

Use Cases

Look up customer information
Use container information
Stand-alone AWS permissions

Skills Architecture

Structured instructions to condition the LLM.

Auto-loading in context when judged necessary
Specific output formats, debugging, checks
Gives LLM expertise it might not have

Cost Comparison

Claude-code

Requires 20x Max plan
$200 / month
Multi-tasking (docs, coding, synthesis)

open-code + llama-cpp

Free (open-source)
One-time hardware cost
Requires a good computer!

[SYSTEM] Session Complete

Thank You

Welcome to the era of Agentic Coding.

opencode.ai llama-cpp.com github.com

Press S speaker notes · F fullscreen