pre-alpha building in public →

Your data. Your voice assistant. Your hardware.

vakforge reads what your company already has, documents, tables, chat logs, CRM exports, recorded calls, and tells you what a self-hosted voice assistant actually needs: retrieval, tools, a locale pack or fine-tuning. Often the answer is not to train yet, and it says why. inspect and recommend run today; the rest of the path is written into your project by an agent skill.

Read the docs How it decides

pip install vakforge

examples/hinglish-shop · actual output recommend complete

recommend.json

baseline first

inspectrecommendbaseline

Recommended routing4 files

FAQ · 3 languagesretrieval

Orders, products · 3 toolstools

Chat · 16 messagesbehaviour fine-tune

Hinglish, Devanagarilocale pack

Fine-tune?blocked

turnsfloor

That run is the Hinglish shop example. Try it yourself.A small synthetic project (FAQ, two tables, a WhatsApp chat) with both reports committed next to it.

For developers and ML engineers

Apache-2.0Open models onlyNo telemetryCPU-only corePython 3.11+

One opinionated path

Seven steps, two decision gates, no menu of four hundred models.

Open speech-to-speech models exist. Fine-tuning scripts exist. Eval tools and serving frameworks exist. What was missing is one path from “here is what my company knows” to a served assistant you can prove beats the base model.

In the CLI — ships today in the vakforge package, installable now.Skill generates — the agent skill writes this step into your project, against your data and your models. vakforge does not ship a training or serving stack.

Inspect

vakforge inspect ./datain the CLI

Inventories every document, table, chat export and recording: formats, languages, personal data it can detect, tables that could become tools. Says what it skipped and why.

02decision gate

Recommend

vakforge recommendin the CLI

Knowledge to retrieval, behaviour to tuning, language to a locale pack. Says “don’t fine-tune” when that is the answer, and cites what the answer rests on.

Prepare

prepare/skill generates

Ingest, transcribe, diarize, redact PII, record consent, write one canonical dataset.

Synth

synth/skill generates

Grounded dialogues in your locale that exercise every tool and scenario you are missing. Coverage, not a substitute for real data.

Train

train/skill generates

One recipe per base model, each in an isolated environment because model libraries pin conflicting versions.

06decision gate

Evaluate

eval/skill generates

Base and tuned on the same held-out split, and the split stays real. No metric is reported without its baseline.

Serve

serve/skill generates

Your open model on your hardware, behind protocols your clients already speak. Nothing calls a hosted API.

OpenAI Realtime WebSocketfirstWebRTC · LiveKit, PipecatnextSIP · telephonynextHTTP · one turn per requestplanned

Bring any data

Start with what you have. Even nothing.

Fine-tuning is not the default answer. Every kind of input gets the least expensive fix that can actually solve it. Pick a source to see its route.

Every threshold behind these routes is written down with its source, and the ones we chose ourselves are labelled as guesses rather than requirements: the evidence, and what it changed →

vakforge recommend

RetrievalFacts stay out of weights; answers come from your source.no training

Tools over your tablesA schema and a prompt first. Published tool-use corpora run 8k–60k examples, so fine-tune only what the tools still miss.usually no training

Behaviour fine-tuneYour call flow, tone and hand-offs, rendered to audio.fine-tune

Contextual biasing, then recognitionBiasing towards your product and customer names cuts entity errors with no training. Collect accent hours only if it still misses.free first

Synthetic dataScenario templates in your locale for a v0.no training

What ships

Three parts. One repository.

Stable primitives in the core, fast-changing model knowledge in the skill, and only verified paths in recipes.

vakforge / vakforge-ai

open sourceApache-2.0main

Foundationvakforge/

Core library and CLI

Canonical dataset schema, validator, data inspector, decision engine, locale packs. Zero ML dependencies. Tested on CPU with no downloads.

working today · CPU test suite, no downloads

Knowledge layerskill/

Agent skill

Drop it into Claude Code or any coding agent. It reads your data, runs the decision guide, writes the recipe glue for your project, and checks every upstream API against installed source first.

written · not yet run end to end

Verified pathsrecipes/

Recipes

Researched paths from an open base model to a served assistant: LFM2.5-Audio, Moshi and PersonaPlex, Qwen-Omni, and a cascade for any language. A recipe is marked tested only once its report is committed.

none built yet · GPU help welcome

Locale packs

Language‑agnostic core. Everything local lives in a pack.

Currency, dates, phone and ID formats, personal-data patterns, language detection, consent and privacy-law notes, and which recipes can speak the language. Core never branches on a language string. Adding a market is a contribution, not a rewrite.

How to add a locale pack

vakforge/locales

one folder per pack
locales/<id>/

Evaluation

“Fine‑tuned” is not a result.

Show that the assistant got better on your own held-out data, broken down by locale, language, audio condition and source. Show what got worse, too. Or do not ship the change.

WER / CERentity accuracytool-call accuracytask completionhallucination rateTTFT / TTFAbarge-in stop timevoice similarity

report.md

layout only · numbers illustrative SHIP · 1 regression ok

Task completion+23pts · 61→84%

Tool-call accuracy+23pts · 68→91%

Hallucination ↓−9pts · 14→5%

entity accuracynames, ₹ amounts, PIN+16 pts

hallucination ↓lower is better−9 pts

TTFA p50 ↓time to first audio+40 ms

same split for base and tunedno number without its baseline

These figures show the shape of the report, not a benchmark. vakforge does not publish numbers it has not measured; yours come from your own held-out split.

Data safety

Redact before you write.

Every record carries its consent basis and redaction status, and vakforge validate re-scans the text to check that a redaction claim is true rather than trusting the flag. Cutting the personal data — from text and from audio, by locale-aware patterns and NER — is the prepare step the skill generates, shown below as the contract it has to meet.

prepare · generated for your projectconv_0142 · hi-Latn-IN3 spans redacted

user0:03.8

Haan, main Rahul Sharma<PERSON_1> bol raha hoon. Mera number 74281 96530<PHONE_1> hai, Aadhaar 9999 4105 7058<AADHAAR_1>. Order ORD-2291 ka status batao.

audio24 kHz

logredactions/conv_0142.json · PERSON, PHONE, AADHAAR · audio → tone

keepORD-2291 · configs/keep_list.yaml

metaconsent=recorded_verbal · pii_redacted=true · source=real

For coding agents

The knowledge is the product. The glue is generated.

Model libraries churn monthly. Instead of shipping adapters that rot, vakforge ships the decision guide, the data format, the locale rules and the upstream pitfalls as an agent skill. Your agent writes current glue for your project and verifies every API against source.

skill/vakforge/SKILL.mdClaude Code or any agent

vakforge skill · project analysisanalysis complete

Commands run

vakforge init shop -l hi-Latn-IN

vakforge inspect shop/data/raw

vakforge recommend inspect.json

Decision report · Hinglish shop example

Primary problem: tools
Fine-tune: blocked · 16 of 200 turns
Knowledge: FAQ · retrieval, not weights
Next: measure the baseline · then synth

Personal data found (phone, email): redact before any training row

Build in the open

Your data. Your voice assistant. Your hardware.

Install it, point it at your data, and let recommend tell you what is worth training, and what is not.

Star vakforge on GitHub Contribute

Apache-2.0Self-hostedNo telemetry

01Installpip install vakforge
02Start a projectvakforge init shop -l en-US
03Add files, get the decisionvakforge recommend shop/data/raw