Every task gets the right model. The promotion gate has teeth, empirically.
SoloFrame's AI layer is a thin, typed boundary that lets each vertical compose its own AI stack from shared primitives - centralized model routing, a classifier adapter interface, a git-backed prompt registry, and a typed manifest that wires up compliance posture before any model is called. The vertical decides what AI looks like; the engine guarantees it gets wired in correctly.
CPU FastAPI sidecar. Four classifiers registered. One production-loaded.
FastAPI single-process ASGI service, CPU-only, deployed to Dokploy as the platform-level alias maia-mono:8001. The live service code is app.py (376 LOC), db.py (250 LOC), classifiers/base.py (233 LOC), and four classifier modules.
- POST /v1/classify/{classifier} · 50-150 ms/req
- GET /v1/health · per-classifier load status
- GET /v1/metrics[/{classifier}]
- GET /v1/admin/version-stats · Bearer auth
- POST /v1/admin/reload · hot-swap
- POST /v1/rerank · cross-encoder re-ranking (ms-marco-MiniLM-L-6-v2)
sentinet/suicidality
- →ELECTRA-base, ~110M params, CC0 license
- →Trained on C-SSRS clinical labels + Reddit + Twitter + suicide notes
- →Pinned at
services/maia/classifiers/distress.py - →22.3 ms/sample on CPU FastAPI; ~$0.0001/inference
| Classifier | Status | Purpose |
|---|---|---|
| distress | production-loaded | Production-loaded, sentinet baseline, validated F1 0.93 / crisis_recall 1.000 / trap_FP 0 |
| harm | scaffolded · awaiting weights | Registered, empty fallback_model. Skips at boot until trained weights ship via the feedback loop. |
| boundary | scaffolded · awaiting weights | Registered, empty fallback_model. Same scaffold; awaiting training data. |
| escalation | scaffolded · awaiting weights | Registered, empty fallback_model. Same scaffold; awaiting training data. |
Three independent validation surfaces. Model card: services/maia/MODEL_CARD.md
SVTech's first fine-tune was REJECTED on adversarial recall.
First SVTech-built fine-tune (DistilBERT-base, 5,000 stratified thePixel42 samples, 4.7 minutes on M2 Pro MPS) passed in-distribution F1 (0.929) but failed adversarial recall (0.647 vs sentinet 0.765). Decision: REJECT. Sentinet remains the production model. A weaker promotion gate would have promoted v1 on its in-distribution F1 alone. The single strongest technical-credibility moment on the platform.
SVTech DistilBERT v1
- →DistilBERT-base, 5,000 stratified samples
- →4.7 min training on M2 Pro MPS
- ✓In-distribution F1: 0.929
- ✗Adversarial recall: 0.647 (vs sentinet 0.765)
- ⚠REJECTED — sentinet remains production
sentinet/suicidality
- →ELECTRA-base, ~110M params, CC0
- →C-SSRS + Reddit + Twitter + suicide notes
- ✓F1 0.93, crisis_recall 1.000, trap_FP 0
- ✓Adversarial recall 0.765
- ✓PRODUCTION — measurably wins
A retrain candidate must (a) beat sentinet on the 24-sample synthetic regression suite, (b) show measurable delta on F1 + sensitivity on real labeled data, (c) get human sign-off for the first N retrains. Hot-swap via models/distress/v{N}/ + active.json + POST /v1/admin/reload; no container rebuild. Each inference tags the model version it ran on.
Signal capture today. Spot-check labels today. Validation harness, retrain, promotion script next.
Documented at docs/maia-feedback-loop/spec.md. The data flywheel claim made empirical, not aspirational.
Signal capture
shippedMAIA writes one row to classification_event in maia-signals-db (pgvector for Day-4 active-learning sample selection) per inference. Text hash only - never raw text. Connection-pooled via psycopg_pool. Capture is fire-and-forget; inference returns regardless of DB health.
Clinical spot-check surface
shippedDWA mirrors with ai_classification_event + clinical_spot_check (provider labels: agree/disagree/unsure). Surfaces in DWA's provider portal at /provider/spot-check. Per-version inference rollups via /provider/spot-check/version-stats-panel.tsx.
Versioned model layout
nextmodels/distress/v{N}/ + active.json pointer. R2 sync. Hot-swap via POST /v1/admin/reload.
Retrain job
nextManual trigger first; eventual Nebius. Pulls labeled samples from clinical_spot_check + active-learning samples. Held-out validation harness runs nightly on Apple Silicon MPS.
Promotion script + dashboard
nextMeasured-delta dashboard surfaces F1, sensitivity, adversarial recall vs the current production model. Promotion requires beating baseline on synthetic regression suite + measurable delta on real corpus + human sign-off for the first N retrains.
6.2x measured on GTM. 4.4x measured on DWA. 13.6x ambitious with Layer 6.
Each layer is implemented in repo-traceable code. The CSV reproduces the numbers from vendor-published pricing. Documented at docs/positioning/unit-economics-one-session.md.
| Layer | What it is | Where in the engine |
|---|---|---|
| 1 | Tier routing Manifest-declared per-task model selection. Heavy turns to Sonnet/Haiku-class; light turns to Flash/Llama-class. Per-call model arg on the OpenRouter client. | manifest.coaching.models → @svtech/llm ModelRegistry |
| 2 | OpenRouter as model bus Single OpenAI-SDK client pointed at OpenRouter. One bill, many models. Voice (Whisper STT + tts-1) bypasses to OpenAI direct - the only place OpenAI is hit. | @svtech/llm + @svtech/voice |
| 3 | Prompt + context caching Anthropic-style cache_control: ephemeral breakpoints on stable prefixes. 5-min TTL, ~90% read discount on cached portion. | vertical lib/ai/openai-coaching.ts |
| 4 | RAG (bounded conversation) Last-turn summary + retrieved chunks per heavy turn replaces cumulative-history scroll. GTM uses pgvector; DWA uses JSONB float arrays + pure-JS cosine. | vertical lib/services/ragService.ts |
| 5 | CPU sidecar classifiers (MAIA) DWA only. ~$0.0001/inference, 22.3 ms/sample. Negligible cost; load-bearing safety. Production distress + scaffolded harm/boundary/escalation. | services/maia/ FastAPI sidecar |
| 6 | In-house fine-tunes Coach v0.1 trained on Apple Silicon: perplexity 11.32 → 8.98 (20.7% reduction) in 5.9 min on the 217 .md therapeutic-school slice. Production-class v1+ requires Nebius. | services/maia/training/ + Nebius (pending) |
Shared across verticals. Each vertical's manifest.ai picks adapters from the same shared primitives.
Multi-model router
Centralized routing through OpenRouter with direct-API fallbacks for provider features that aren't proxied (voice, embeddings). Task-to-model assignment in @svtech/llm's ModelRegistry, configured by env vars per environment.
Classifier adapter interface
A typed Classifier interface accepts text and returns a structured label. DWA's @adapter/classifier-maia plugs in by manifest flag. Safety-critical classifiers stay on CPU; training is GPU-out-of-band.
Prompt registry (git-backed)
YAML prompt files in the repo are the source of truth. Parsed, validated, and cached in memory. Every prompt is version-controlled, diff-reviewable, pinned to a vertical. No SaaS prompt store; no environment drift.
RAG adapter (pgvector + JSONB)
Semantic retrieval over vertical-owned content corpora. GTM uses pgvector with HNSW; DWA uses JSONB + pure-JS cosine. Embeddings via configurable provider through OpenRouter. Source attribution on every response.
Compliance wire-up from manifest
The orchestrator reads manifest.compliance before any model is called. phi: true flips the stack to HIPAA-aware behavior: classifier sidecar required, crisis responses bypass the LLM, retention enforced, guardrails refused at orchestration layer.
Structural improvements shipped across both flagships.
Each improvement has a specific measurable effect. None are config tweaks — each changes code paths, prompt structure, or retrieval logic.
RAG similarity threshold gate
GTM: minSimilarity raised from 0.3 → 0.72. text-embedding-3-small space puts <0.5 as noise, 0.72+ as strong relevance. DWA: added threshold parameter (0.72 default, 0.65 for coach path filtered further on lessonId presence). Previous implementation: no threshold, all cosine scores accepted.
Temperature tuning per task type
Analytical/structured JSON (icp-validation, roleplay-eval, quiz-reflection): 0.2. Persona/creative (roleplay, outreach-drafts): 0.9. Coaching voice: 0.85. RAG synthesis (DWA): 0.3 → 0.2. Provider session prep: 0.4 → 0.2.
Chain-of-thought for eval tasks
reasoning: z.string() added as first field in every structured-output schema for reasoning-heavy tasks (RoleplayEvalOutput, QuizReflectionOutput, PipelineInsightsOutput, OutreachAnalysisOutput, ICPValidationOutput, MiniAssessmentOutput). Forces model to commit reasoning before score.
Localized coaching system prompt
prompts/founder-coach.es.md — Spanish-language prompt localized with locale-specific response rules. loadSystemPrompt() now accepts locale; cache keyed by locale. Chat route threads NEXT_LOCALE cookie through to openaiCoachingReply().
Kimi K2 few-shot examples
VOICE EXAMPLES section added to EN and ES coaching prompts — 3 examples each covering: vague question (narrow it), execution data (diagnose it), positioning feeling (Prescription Frame). Symmetric across both language versions.
Cross-encoder re-ranking (MAIA)
Two-stage RAG retrieval: cosine similarity pulls candidateK = topK×4 candidates above threshold, MAIA /v1/rerank (ms-marco-MiniLM-L-6-v2, ~85 MB) re-ranks and returns topK. Falls back to cosine order when MAIA unreachable. Lazy singleton model load on first call.
Semantic session memory
Previously: always inject last 3 coaching sessions. Now: embed current message in parallel with session DB fetch, compute cosine similarity against stored session embeddings, surface 3 most semantically relevant past sessions. Cap at 20 sessions (~120 KB with embeddings). Fail-safe: never blocks the coaching path.
Intent-based context routing
Classify message intent via regex (O(1), no LLM call): outreach / positioning / learning / sales / general. Gate heavy context blocks on relevance — a subject-line question no longer injects assessment scores and XP progress. Focused context → lower input token cost + more grounded responses.
Outreach draft review-and-refine loop
Two-pass generation: Generate at temp 0.9 (Kimi K2 creative), then Refine critic pass at temp 0.2 checks: opening line leads with buyer pain, subject under 50 chars, body under 120 words, CTA is single low-commitment ask. Failing elements rewritten; passing kept. fix[] field stripped before returning.
MAIA SAFETY SIGNAL feature payload
safetySignalBlock() now includes detected clinical feature categories: 8 categories (suicidality/SI, self-harm, hopelessness, sleep disruption, anhedonia, worthlessness, isolation, substance use). Absent features explicitly noted ('may be indirect expression or affect-only signal'). Only computed when alertedUrgency is set.
The concrete AI story lives in each vertical.
How MAIA is trained, how the GPU-bound roadmap unfolds, what the clinical data flywheel looks like - those are DWA concerns. How DISC-based AI pod matching, four named AI bot personas, durable-delay persona responses, and sales roleplay evaluation work - those are GTM-OS concerns. The platform guarantees they both wire in consistently.