vakforge

The decision layer for open voice AI.

vakforge helps you build a voice assistant from the data your company already has, and run it on your own servers. Before you spend anything on GPUs, it tells you what actually needs training. Usually that is less than you think.

• Bring any data: documents, FAQs, database tables, chat logs, CRM records, recorded calls.
• Open models only: the assistant itself runs on your servers and never calls a hosted API. (Generating synthetic training dialogues may use a provider you choose, only after you approve it.)
• Any language: through locale packs. Launching with English (US, UK, India) and Hinglish.

Status: pre-alpha. init, inspect, recommend, validate and the locale packs work today. See docs/ROADMAP.md.

Why vakforge exists

Open voice models are good enough today. Moshi, PersonaPlex, LFM2.5-Audio and Qwen-Omni can listen and talk, and strong open speech-to-text and text-to-speech models exist too. Yet most companies still pay per minute for a closed voice API.

That is not because the open models are worse. It is because nobody helps them answer four questions:

• Should we fine-tune, or is retrieval enough?
• Which model fits our language, our speed needs and our hardware?
• What data do we need, and are we allowed to train on our calls?
• How do we prove the result is better before a customer hears it?

vakforge answers them from your own data.

The hard part was never the models. It was the decision.

"Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty." Jensen Huang, NVIDIA, on X, 24 July 2026

vakforge is about that last word: owning your voice AI instead of renting it.

How it works

1. init creates a project folder for your language and market.
2. inspect reads your data folder and reports what is in it: languages, personal data, tables that could become tools, audio quality.
3. recommend decides what needs changing. Often the answer is retrieval and tools, with no training at all.
4. prepare, train, eval, serve are built for your project by a coding agent using the vakforge skill. It trains only what recommend asked for, and ships only if eval shows the result beats the base model.

Today the CLI covers init, inspect, recommend, validate and the locale packs.

How the decision is made

Every route ends the same way: test it against your own held-out data, and ship only if it beats the base model.

Each goal has its own fix, and most of them are not a full fine-tune. The detailed rules, with how much data each route needs and what hardware it takes, are in docs/DECISION_GUIDE.md.

The problem, for engineers

Open speech-to-speech models exist. Fine-tuning scripts exist for some of them. Evaluation tools and serving frameworks exist. What is missing is one path from "here is what my company knows" to "here is a voice assistant that handles my workflow, I can prove it is better than the base model, and my existing voice client can talk to it without a rewrite." Every team rebuilds that path, and most of them fine-tune when they should have used retrieval.

What's in the repo

1. A small library and command-line tool (pip install vakforge). No machine-learning dependencies, so it runs on any laptop. It holds the dataset format, the data checks, the data inspector, the decision rules and the locale packs.
2. An agent skill (skill/vakforge/). Add it to Claude Code or another coding agent. The agent reads your data, follows the decision rules, and writes the training and serving code for your project. It checks every library it uses against the installed source first, so it does not guess at APIs.
3. Recipes (docs/RECIPES.md). Researched paths from an open base model to a running assistant: which model suits which language and GPU, what it can and cannot learn, and the licence to read first. None have been run end to end yet, and the file says so on every page — a recipe is only marked tested once its report is committed.

Use it with a coding agent

The skill is where the knowledge lives. Copy it into your project and the agent follows the vakforge workflow: inspect, recommend, then write and test the prepare, train, eval and serve code for your case, verifying every upstream API against installed source first.

# Claude Code: project-level skill (or ~/.claude/skills/vakforge for every project)
mkdir -p .claude/skills
cp -r path/to/vakforge/skill/vakforge .claude/skills/vakforge

Then ask: "Build a Hinglish support assistant from the data in data/raw." Other agents that read SKILL.md-style instructions work the same way; point them at the folder.

Bring any data

Locale packs

The core works in any language. Everything that changes by language or country lives in a locale pack: how money, dates and phone numbers are written, which ID numbers count as personal data, the local privacy and call-recording rules, and which open models support the language. Adding a new market means adding a pack, not changing the core. See docs/LOCALE_PACKS.md, or run vakforge locales.

Recipes

"Duplex" means the assistant can listen while it talks, so callers can interrupt it naturally. Details in docs/RECIPES.md.

Connecting your app

Your assistant runs an open model on your own servers. Nothing calls OpenAI or any other hosted API.

To make switching easy, the server accepts the same WebSocket messages as OpenAI's Realtime API. If your app already talks to GPT Realtime, you change one URL. Tools like Pipecat, LiveKit and Twilio keep working as they are. More connection types can be added without touching the model:

Why English and Hinglish first

English has the strongest open voice models, so every recipe works for US, UK and Indian English. Hinglish is the hard test: people switch between Hindi and English mid-sentence, write Hindi in Roman or Devanagari script, and say amounts like "2 lakh rupees" on noisy phone lines. If vakforge handles that, adding another language is mostly filling in a new pack.

Quick start

pip install vakforge
vakforge init my-assistant --locale hi-Latn-IN
# put your documents, tables, chat exports and call recordings in my-assistant/data/raw/
vakforge inspect my-assistant/data/raw

inspect prints a summary and writes inspect.json. vakforge recommend inspect.json reads it and writes recommend.json.

No data of your own yet? examples/hinglish-shop is a small synthetic project (Hinglish FAQ, two table exports, a WhatsApp chat) with both reports committed next to it, so you can see the output before installing anything.

Documentation

• docs/DECISION_GUIDE.md: what needs customizing, and when not to fine-tune
• docs/LOCALE_PACKS.md: what a locale pack contains, and how to add one
• docs/DATA_FORMAT.md: the dataset format
• docs/ARCHITECTURE.md: how the code fits together, with diagrams
• docs/RECIPES.md: per-model training recipes
• docs/EVALUATION.md: metrics, report format, per-locale benchmarks
• docs/DATA_ETHICS.md: consent, PII, licences, privacy law by region
• docs/ROADMAP.md: status
• CONTRIBUTING.md

Related projects (we build on these, not against them)

Unsloth, LLaMA-Factory, ms-swift, kyutai-labs/moshi-finetune, NVIDIA PersonaPlex, liquid-audio, Qwen-Omni, Pipecat, LiveKit Agents, vLLM-omni, UltraEval-Audio, AI4Bharat, Common Voice

Licence

Apache-2.0 for this code. Each open model keeps its own licence; see docs/RECIPES.md. Datasets you create with vakforge are yours. The consent records vakforge asks for are there to keep it that way.