pre-alpha building in public →

Your data. Your voice assistant. Your hardware.

vakforge turns what your company already has, documents, tables, chat logs, CRM records, recorded calls, into a real-time voice assistant you run yourself. It works out what actually needs changing, generates the conversations you lack, trains only what needs training, and proves the result on your own data before you ship.

Star on GitHub Read the decision guide

git clone https://github.com/vakforge-ai/vakforge && cd vakforge && uv sync

my-assistant · example run eval complete

recommend.json

held-out split

inspectrecommendpreparesynthtrainevalserve

Recommended routing4 sources

Product docs · 3 PDFsretrieval

CRM tables · 3 toolsfine-tune

Hinglish callerslocale pack

Call audio · 12 hredact + consent

Completionn=180

basetuned

For developers and ML engineers

Apache-2.0Open models onlyNo telemetryCPU-only corePython 3.11+

One opinionated path

Seven commands, two decision gates, no menu of four hundred models.

Open speech-to-speech models exist. Fine-tuning scripts exist. Eval tools and serving frameworks exist. What was missing is one path from “here is what my company knows” to a served assistant you can prove beats the base model.

Inspect

vakforge inspect ./data

Formats, languages, quality and consent status of every document, table, chat export and recording.

02decision gate

Recommend

vakforge recommend

Knowledge to retrieval, behaviour to tuning, language to a locale pack. Says “don’t fine-tune” when that is the answer.

Prepare

vakforge prepare

Ingest, transcribe, diarize, redact PII, record consent, write one canonical dataset.

Synth

vakforge synth

Grounded dialogues in your locale that exercise every tool and scenario you are missing.

Train

vakforge train --recipe

One tested recipe per base model, each in an isolated environment because model libraries conflict.

06decision gate

Evaluate

vakforge eval

Base and tuned on the same held-out split. No metric is reported without its baseline.

Serve

vakforge serve

Your open model on your hardware, behind protocols your clients already speak. Nothing calls a hosted API.

OpenAI Realtime WebSocketfirstWebRTC · LiveKit, PipecatnextSIP · telephonynextHTTP · one turn per requestplanned

Bring any data

Start with what you have. Even nothing.

Fine-tuning is not the default answer. Every kind of input gets the least expensive fix that can actually solve it. Pick a source to see its route.

vakforge recommend

RetrievalFacts stay out of weights; answers come from your source.no training

Tool-use fine-tuneTools over your tables, dialogues that call every one.fine-tune

Behaviour fine-tuneYour call flow, tone and hand-offs, rendered to audio.fine-tune

Voice and full-duplexAccents, timing, interruptions, backchannels.fine-tune

Synthetic dataScenario templates in your locale for a v0.no training

What ships

Three parts. One repository.

Stable primitives in the core, fast-changing model knowledge in the skill, and only verified paths in recipes.

vakforge / vakforge-ai

open sourceApache-2.0main

Foundationvakforge/

Core library and CLI

Canonical dataset schema, validator, data inspector, decision engine, locale packs. Zero ML dependencies. Tested on CPU with no downloads.

phase 0 done · 32 CPU tests

Knowledge layerskill/

Agent skill

Drop it into Claude Code or any coding agent. It reads your data, runs the decision guide, writes the recipe glue for your project, and checks every upstream API against installed source first.

phase 2 · glue is generated

Verified pathsrecipes/

Recipes

Tested paths from base model to served assistant: LFM2.5-Audio, Moshi and PersonaPlex, Qwen-Omni, and a cascade for any language. Only recipes run end to end get listed.

phase 5+ · GPU help welcome

Locale packs

Language‑agnostic core. Everything local lives in a pack.

Currency, dates, phone and ID formats, PII patterns, privacy-law notes, name generators, preferred models, a benchmark. Core never branches on a language string. Adding a market is a contribution, not a rewrite.

How to add a locale pack

vakforge/locales

one folder per pack
locales/<id>/

Evaluation

“Fine‑tuned” is not a result.

Show that the assistant got better on your own held-out data, broken down by locale, language, audio condition and source. Show what got worse, too. Or do not ship the change.

WER / CERentity accuracytool-call accuracytask completionhallucination rateTTFT / TTFAbarge-in stop timevoice similarity

report.md

en-IN · phone · n=180 SHIP · 1 regression ok

Task completion+23pts · 61→84%

Tool-call accuracy+23pts · 68→91%

Hallucination ↓−9pts · 14→5%

entity accuracynames, ₹ amounts, PIN+16 pts

hallucination ↓lower is better−9 pts

TTFA p50 ↓time to first audio+40 ms

same split for base and tunedno number without its baseline

Data safety

Redact before you write.

Every record carries its consent basis and redaction status. Personal data is cut from text and audio by locale-aware patterns and NER, and logged, before anything reaches a training row.

vakforge prepareconv_0142 · hi-Latn-IN3 spans redacted

user0:03.8

Haan, main Rahul Sharma<PERSON_1> bol raha hoon. Mera number 98765 43210<PHONE_1> hai, Aadhaar 4821 7730 1956<AADHAAR_1>. Order ORD-2291 ka status batao.

audio24 kHz

logredactions/conv_0142.json · PERSON, PHONE, AADHAAR · audio → tone

keepORD-2291 · configs/keep_list.yaml

metaconsent=recorded_verbal · pii_redacted=true · source=real

For coding agents

The knowledge is the product. The glue is generated.

Model libraries churn monthly. Instead of shipping adapters that rot, vakforge ships the decision guide, the data format, the locale rules and the upstream pitfalls as an agent skill. Your agent writes current glue for your project and verifies every API against source.

skill/vakforge/SKILL.mdClaude Code or any agent

vakforge skill · project analysisanalysis complete

Commands run

vakforge init my-assistant -l en-IN -l hi-Latn-IN

vakforge inspect ./data/raw

vakforge recommend

Decision report

Primary problem: tool-use behaviour
Recipe: lfm25-audio · 1× 24 GB GPU
Knowledge: 3 documents · retrieval, not weights
Data gap: 0 of ~600 turns · run synth

Consent: call recordings need a basis before prepare

Build in the open

Your data. Your voice assistant. Your hardware.

Clone it, inspect your sources, and let the decision guide tell you what is worth training.

Star vakforge on GitHub Contribute

Apache-2.0Self-hostedNo telemetry

Quick startgit clone https://github.com/vakforge-ai/vakforge && cd vakforge && uv sync