pre-alpha building in public →

Your data. Your voice assistant. Your hardware.

vakforge turns what your company already has, documents, tables, chat logs, CRM records, recorded calls, into a real-time voice assistant you run yourself. It works out what actually needs changing, generates the conversations you lack, trains only what needs training, and proves the result on your own data before you ship.

Star on GitHub Read the decision guide

git clone https://github.com/vakforge-ai/vakforge && cd vakforge && uv sync

my-assistant · illustrative run, not a benchmark eval complete

recommend.json

held-out split

inspectrecommendpreparesynthtrainevalserve

Recommended routing4 sources

Product docs · 3 PDFsretrieval

CRM tables · 3 toolstools

Hinglish callerslocale pack

Call audio · 12 hredact + consent

Completionillustrative

basetuned

For developers and ML engineers

Apache-2.0Open models onlyNo telemetryCPU-only corePython 3.11+

One opinionated path

Seven steps, two decision gates, no menu of four hundred models.

Open speech-to-speech models exist. Fine-tuning scripts exist. Eval tools and serving frameworks exist. What was missing is one path from “here is what my company knows” to a served assistant you can prove beats the base model.

In the CLI — ships today in the vakforge package, installable now.Skill generates — the agent skill writes this step into your project, against your data and your models. vakforge does not ship a training or serving stack.

Inspect

vakforge inspect ./datain the CLI

Formats, languages, quality and consent status of every document, table, chat export and recording.

02decision gate

Recommend

vakforge recommendin the CLI

Knowledge to retrieval, behaviour to tuning, language to a locale pack. Says “don’t fine-tune” when that is the answer, and cites what the answer rests on.

Prepare

prepare/skill generates

Ingest, transcribe, diarize, redact PII, record consent, write one canonical dataset.

Synth

synth/skill generates

Grounded dialogues in your locale that exercise every tool and scenario you are missing. Coverage, not a substitute for real data.

Train

train/skill generates

One recipe per base model, each in an isolated environment because model libraries pin conflicting versions.

06decision gate

Evaluate

eval/skill generates

Base and tuned on the same held-out split, and the split stays real. No metric is reported without its baseline.

Serve

serve/skill generates

Your open model on your hardware, behind protocols your clients already speak. Nothing calls a hosted API.

OpenAI Realtime WebSocketfirstWebRTC · LiveKit, PipecatnextSIP · telephonynextHTTP · one turn per requestplanned

Bring any data

Start with what you have. Even nothing.

Fine-tuning is not the default answer. Every kind of input gets the least expensive fix that can actually solve it. Pick a source to see its route.

Every threshold behind these routes is written down with its source, and the ones we chose ourselves are labelled as guesses rather than requirements: the evidence, and what it changed →

vakforge recommend

RetrievalFacts stay out of weights; answers come from your source.no training

Tools over your tablesA schema and a prompt first. Published tool-use corpora run 8k–60k examples, so fine-tune only what the tools still miss.usually no training

Behaviour fine-tuneYour call flow, tone and hand-offs, rendered to audio.fine-tune

Contextual biasing, then recognitionBiasing towards your product and customer names cuts entity errors with no training. Collect accent hours only if it still misses.free first

Synthetic dataScenario templates in your locale for a v0.no training

What ships

Three parts. One repository.

Stable primitives in the core, fast-changing model knowledge in the skill, and only verified paths in recipes.

vakforge / vakforge-ai

open sourceApache-2.0main

Foundationvakforge/

Core library and CLI

Canonical dataset schema, validator, data inspector, decision engine, locale packs. Zero ML dependencies. Tested on CPU with no downloads.

working today · 226 CPU tests, no downloads

Knowledge layerskill/

Agent skill

Drop it into Claude Code or any coding agent. It reads your data, runs the decision guide, writes the recipe glue for your project, and checks every upstream API against installed source first.

phase 2 · glue is generated

Verified pathsrecipes/

Recipes

Tested paths from base model to served assistant: LFM2.5-Audio, Moshi and PersonaPlex, Qwen-Omni, and a cascade for any language. Only recipes run end to end get listed.

phase 5+ · GPU help welcome

Locale packs

Language‑agnostic core. Everything local lives in a pack.

Currency, dates, phone and ID formats, PII patterns, privacy-law notes, name generators, preferred models, a benchmark. Core never branches on a language string. Adding a market is a contribution, not a rewrite.

How to add a locale pack

vakforge/locales

one folder per pack
locales/<id>/

Evaluation

“Fine‑tuned” is not a result.

Show that the assistant got better on your own held-out data, broken down by locale, language, audio condition and source. Show what got worse, too. Or do not ship the change.

WER / CERentity accuracytool-call accuracytask completionhallucination rateTTFT / TTFAbarge-in stop timevoice similarity

report.md

layout only · numbers illustrative SHIP · 1 regression ok

Task completion+23pts · 61→84%

Tool-call accuracy+23pts · 68→91%

Hallucination ↓−9pts · 14→5%

entity accuracynames, ₹ amounts, PIN+16 pts

hallucination ↓lower is better−9 pts

TTFA p50 ↓time to first audio+40 ms

same split for base and tunedno number without its baseline

These figures show the shape of the report, not a benchmark. vakforge does not publish numbers it has not measured; yours come from your own held-out split.

Data safety

Redact before you write.

Every record carries its consent basis and redaction status. Personal data is cut from text and audio by locale-aware patterns and NER, and logged, before anything reaches a training row.

prepare · generated for your projectconv_0142 · hi-Latn-IN3 spans redacted

user0:03.8

Haan, main Rahul Sharma<PERSON_1> bol raha hoon. Mera number 98765 43210<PHONE_1> hai, Aadhaar 4821 7730 1956<AADHAAR_1>. Order ORD-2291 ka status batao.

audio24 kHz

logredactions/conv_0142.json · PERSON, PHONE, AADHAAR · audio → tone

keepORD-2291 · configs/keep_list.yaml

metaconsent=recorded_verbal · pii_redacted=true · source=real

For coding agents

The knowledge is the product. The glue is generated.

Model libraries churn monthly. Instead of shipping adapters that rot, vakforge ships the decision guide, the data format, the locale rules and the upstream pitfalls as an agent skill. Your agent writes current glue for your project and verifies every API against source.

skill/vakforge/SKILL.mdClaude Code or any agent

vakforge skill · project analysisanalysis complete

Commands run

vakforge init my-assistant -l en-IN -l hi-Latn-IN

vakforge inspect ./data/raw

vakforge recommend

Decision report

Primary problem: tool-use behaviour
Recipe: lfm25-audio · 1× 24 GB GPU
Knowledge: 3 documents · retrieval, not weights
Data gap: 0 of ~600 turns · run synth

Consent: call recordings need a basis before prepare

Build in the open

Your data. Your voice assistant. Your hardware.

Clone it, inspect your sources, and let the decision guide tell you what is worth training.

Star vakforge on GitHub Contribute

Apache-2.0Self-hostedNo telemetry

Quick startgit clone https://github.com/vakforge-ai/vakforge && cd vakforge && uv sync