I took the local LLMs too "seriously" - Built my own AI Agent

Hello everyone,

After checking out local LLMs (llama.cpp setup), I decided to take things a bit further: orchestrating that already-built service with an agent framework to get some agentic tasks done and experiment. While searching around, I came across a video from PewDiePie — a semi-retired YouTuber who posts very periodically — where he had built his own AI agent, Odysseus. I was amazed by the project he’d created, and I decided to make my own.

Meet Oceano.

Oceano is a local-first - integrates with cloud and also claude code itself if you want - AI agent that runs entirely on my own machine. The idea is no Cloud, no API keys, no monthly bill, no data leaving the house. It’s built on top of llama.cpp — the thing that already let me run models locally — but wrapped in an agent framework that can actually do things: read and write files, run shell commands, search the web, remember what we talked about last week, and chain all of that together to get real tasks done.

The name is a bit on the nose. It’s an ocean of small models and tools working together, and I liked the idea of an agent quietly exploring the deep on its own.

Why local?

This was the whole point for me. The big hosted assistants are incredible, but they’re also a black box you rent. I wanted something that:

• Runs on my hardware — my GPU, my rules, offline if I want.
• Is actually private — chats, notes, and documents stay on disk, in plain files and a local database.
• Costs nothing per token — so I can let it run experiments, schedule jobs, and “waste” tokens without watching a meter.
• Is mine to break — every part is open and hackable.

Under the hood

Oceano is a small stack of local services talking to each other:

• llama-swap serves the chat models over an OpenAI-compatible API and hot-swaps between them, keeping one resident on the GPU at a time. So a light, fast model (Qwen3-4B) handles everyday chat, and a heavier one can be swapped in for the hard stuff.
• A local embedding server (nomic-embed) powers all the semantic search — memory, documents, past conversations.
• SearXNG gives it private web search, so “look this up” doesn’t phone home to a search giant.
• A FastAPI engine ties it all together and runs as a background service, with a web UI on localhost.

On top of that sits the actual agent loop: the model thinks, calls tools, sees the results, and keeps going until the job is done — the same pattern the big agents use, just running on my desk.

It remembers, and it learns.

Two things turned Oceano from “a chatbot with tools” into something I hope to rely on.

• Memory. It keeps a persistent, semantic memory of facts, preferences, and ongoing projects — stored locally and wired into a little knowledge graph you can literally look at. It also tidies itself up over time, so the memory doesn’t rot. Start a conversation in the morning and it already knows the context.
• Skills. Oceano can distill a useful conversation into a reusable skill — a small playbook it can pull up later for similar tasks. There’s even a self-improving loop: it drafts a skill, has it evaluated, and promotes it once it’s good enough. The agent slowly gets better at the things I keep asking it to do.

It can also index my documents and answer questions over them (local RAG), so it’s part assistant, part second brain.

Visual workflows

For anything multi-step, there’s a workflow canvas — boxes, arrows, and decision branches you drag around like a flowchart. Each node can run a tool, follow an instruction, branch on a decision, or hand work off, and you can run a workflow on demand or on a schedule. It’s the difference between “do this one thing” and “do this whole process for me, every morning”. For the example workflow I designed a workflow for the agent to search for google dorks information and build a skill on how to use it and have a stronger model review it.

Also added a feature that I believe it’s a cool one: Delegation.

Small local models are great until a task genuinely needs more horsepower. So Oceano has an abstracted delegation layer: it can hand a hard sub-task off to a stronger model (or another backend entirely) for things like evaluating its own skills or double-checking work — without that being hardcoded to any one provider. The local model does the bulk of the work; the heavy lifter only gets pulled in when it’s worth it.

Three ways to talk to it

Because the frontends are thin and everything runs through the same agent core, I can reach Oceano however I want:

• A web UI with little floating “apps”: an artifact preview (markdown, diagrams, charts, slides), the memory graph, a system-health dashboard, semantic search, notes/kanban, and a voice console.
• A Telegram bot, so it’s in my pocket.
• A terminal client — an oceano command with streamed answers, rendered markdown, colored diffs when it edits files, sessions that sync with the web UI, and a slash-command palette you can fuzzy-search.

And the conversations are shared terminal and web, so a chat I start in the terminal shows up on the web and vice-versa.

Keeping it sane

There’s a job queue so two heavy tasks don’t fight over the GPU (if you enable it), a scheduler for recurring work (it re-indexes my documents and memory overnight), and a few safety rails — untrusted web content gets fenced off, and risky tools can ask before they run.

Asking him to do something:

Prompt:

Hello. Create a simple chess platform for 2 players please. This chess must follow chess rules, have all the logic that is to be expected, with properly designed elements please.

Since we had delegation enabled, our local model decided to delegate the game creation to a stronger model. Since my hardware is super limited, with limited context, our LLM knew it might fail so it just delegated the heavy lifting to our stronger model. To be mentioned if you have a strong setup, you can just throw this job to a stronger, local model (through the cloud model in delegation — it says cloud, but you can point it to another local LLM).

After a while it got done with the game.

Sadly it made a small mistake regarding hint squares, however we can simply ask it to only consider hints on piece selection based on turns and after a small edit, voilà:

Where it’s going

Oceano is very much a personal project and a moving target — I keep finding new things to bolt on. But it already does something I genuinely wanted to exist: a capable, private agent that lives on my own machine, remembers me, learns over time, and doesn’t cost a cent to let loose.

Huge thanks to PewDiePie and Odysseus for the spark. If you’ve been running local models and wondering how far you can push them — this far, at least.

Surely this is lacking a bunch of stuff, including tests.. but I don’t know, someone might find this cool. Have a look at Oceano’s repo

I will surely keep having my fun with this and add things in the future.