Positioning and Trade-off Reference
Cross-reference of industry products and open-source implementations, used to decide what vanling should and should not do. Priorities and the backlog live only in the Roadmap; nothing here is a commitment.
Open Source Voice Models
Open-source TTS/voice models, sorted by quality/size/license. vanling can integrate directly.
| Model | Params | License | Latency | Voice Cloning | Best for vanling |
|---|
| Piper | ~20M | MIT | 55ms (10s audio) | No | ★★★★★ CPU run, 30+ languages, edge deployment first choice |
| Kokoro v1.0 | 82M | Apache 2.0 | CPU realtime | No (KokoClone extension) | ★★★★★ Best quality/size ratio, 54 voices |
| Step-Audio-TTS-3B | 3B | Apache 2.0 | Needs GPU | Yes | ★★★★ Best Chinese voice quality + emotion + dialects |
| Coqui XTTS v2 | 467M | CPML (non-commercial) | <200ms | Yes (6s sample) | ★★★★ Best cloning, 17 languages, license restricts |
| F5-TTS | 335M | CC-BY-NC | Needs GPU | Yes (zero-shot) | ★★★ Excellent cloning, Chinese-English mixed, non-commercial |
| Orpheus TTS | 3B | — | Needs GPU | No | ★★★ Llama backbone, style control |
| Bark (Suno) | — | MIT | 0.8x (too slow) | No | ★★ Expressive (laughter/sighs/music), poor real-time |
Recommended Architecture
ESP32 (Edge) Server (vanling)
┌──────────┐ ┌─────────────────────┐
│ VAD │ │ ASR: sherpa-onnx │
│ Piper/ │◄── WebSocket ──│ LLM: Qwen3 candle │
│ Kokoro │ │ TTS: Kokoro/ │
│ (Local │ │ Step-Audio-TTS-3B │
│ TTS) │ └─────────────────────┘
└──────────┘
- Piper or Kokoro for ESP32 edge TTS (lightweight, fast, CPU)
- Kokoro for server TTS (best quality/size ratio, Apache 2.0)
- Step-Audio-TTS-3B for server high-quality Chinese (emotion control, dialect support)
Reference Projects
Key reference projects for vanling positioning and trade-off decisions, organized by category.
Backend Services (Alternative Implementations)
Client Implementations
Home Assistant Integration
MQTT Gateway
Embedded Hardware Adaptation
Management & Monitoring
Commercial Products Reference
Key technologies and UX patterns from closed-source commercial products, for vanling trade-off reference.
| Product | Key Technologies | Value for vanling |
|---|
| OpenAI Realtime | E2E S2S, semantic VAD, WebRTC, ~232ms latency | Semantic VAD is gold standard for turn-taking/interruption; WebRTC transport reference |
| Cartesia Sonic 3.5 | SSM architecture TTS, <90ms latency, 3-second voice cloning | SSM is new TTS direction, faster than Transformer |
| ElevenLabs | Voice cloning (1-25 samples), Expressive Mode, 11.ai MCP voice assistant | Voice + MCP integration pattern; Agent coaching/evaluation |
| MiniMax Speech-2.8 | Insertion tags (laughs) (sighs), 7 emotions + 0-100% intensity, per-sentence emotion control | Simple practical emotion expression, no SSML needed |
| Inworld AI | 15-second voice cloning, text-describe-generate voice, <130ms TTS | Lowest barrier voice personalization |
| Hume AI EVI | 600+ emotion tags, emotion-adaptive intonation, detects hesitation/sarcasm/relief | Emotion detection + adaptive intonation is key differentiator |
| Sesame CSM | Conversational prosody (breathing/hesitation/laughter), Apache 2.0 | Usable open-source model, more human-like speech |
Voice AI Infrastructure
| Product | Key Technologies | Value for vanling |
|---|
| LiveKit / Pipecat | Open-source SFU architecture, 50+ AI model integration, OpenAI ChatGPT backbone | Voice AI infrastructure standard, usable as transport layer |
| Retell AI | ~600ms E2E (no tuning), proprietary turn detection, BYO-LLM | Turn detection is key differentiator |
| Vapi | BYO-stack (choose STT/LLM/TTS), A/B testing, 1000+ templates | Composable architecture pattern |
Smart Speakers / Phone Assistants
| Product | Key Technologies | Value for vanling |
|---|
| Amazon Alexa+ | Autonomous task execution (Uber/OpenTable/Grubhub), model-agnostic routing, cross-device continuity | Agent chain execution pattern; vanling can implement via MCP |
| Google Gemini for Home | Natural language automation ("Ask Home"), AI camera understanding | "Describe automation" UX pattern |
| Xiaomi Super Xiaoai | Speaker ID for family members, AI call answering, dynamic bubble visual feedback | Multi-member voiceprint + family features |
| Xiaodu | Bluetooth Mesh local gateway, offline control, voice personality persistence | Local gateway architecture; voice personality as emotional bond |
| Tmall Genie | "1+3+N" architecture, spatial agent, edge scenario dispatch | Spatial agent: from command to context-aware |
| iFlytek Spark | One-sentence voice cloning, 74+ dialects, E2E voice translation | One-sentence cloning is killer feature |
| Doubao | Super mode (autonomous task decomposition), 100M+ DAU, Coze agent platform | Agent task decomposition + UGC character platform |
AI Wearables
| Product | Key Technologies | Value for vanling |
|---|
| Meta Ray-Ban | 76% market share, audio-first, visual perception, 7+ language translation | Audio-first + visual perception is success model |
| Omi (open-source) | $89, MIT license, 250+ community apps, local recording + cloud sync | Closest open-source wearable reference |
| Bee AI (Amazon) | $50, 7-day battery, no audio storage privacy strategy | Low price + long battery + privacy-first |
| Looki L1 | 30g pendant, 12-hour battery, AI ambient perception + life log | Smallest form factor + perception resonance |
| Limitless Pendant | Always-listening, speaker diarization, MCP server integration, 100+ languages | Ambient context capture + MCP integration pattern |
AI Companion Apps
| Product | Key Technologies | Value for vanling |
|---|
| Character.AI | 10M+ user characters, Lorebook worldview, PipSqueak 2 model | UGC character marketplace + persistent persona |
| Replika 2.0 | Cross-month memory reconstruction, proactive reminders, AR, video calls | Proactive memory-driven suggestions is powerful UX |
| ChatGPT Voice | E2E audio model, 320ms latency, 22 neural TTS voices | Consumer voice AI benchmark |
| Microsoft Copilot | Work IQ persistent memory, Agent 365 governance, computer-use agent | Most mature enterprise memory system |
Key Industry Trends
| Trend | Representative Products | vanling Opportunity |
|---|
| Semantic VAD | OpenAI (model-level turn-taking) | Smarter interruption than pure VAD |
| Conversational Prosody | Sesame CSM (breathing/hesitation/laughter) | More natural TTS |
| E2E S2S | OpenAI/Hume/Sesame | Ultimate goal; current cascaded more practical |
| Emotion-adaptive | Hume AI (detect → adaptive tone) | Key voice assistant differentiator |
| SSM Architecture TTS | Cartesia (State Space Model) | Faster TTS inference |
| Persistent Memory | Work IQ / Replika / Character.AI | From chat history to long-term memory |
| MCP Voice Integration | OpenAI + MCP, ElevenLabs 11.ai | Voice conversation calling tools |
| Voice Cloning Democratization | Cartesia 3s / Inworld 15s / MiniMax 10s | Lowest barrier voice personalization |
| Privacy-first | Bee (no audio storage), Omi (local processing) | Always-listening privacy strategy |
Vertical Industry Voice
Voice AI solutions in automotive, healthcare, education, translation and other verticals, for vanling vertical scenario reference.
| Industry | Representative Products | Key Technologies | Value for vanling |
|---|
| Automotive Voice | NIO NOMI, Xpeng Tianji, SoundHound | Edge offline processing, 14 emotion tones, RAG vehicle manual, Speech-to-Meaning | Edge offline is key; emotion tone customization; RAG reference |
| Healthcare Voice | Nuance DAX, Nabla, Abridge, Suki AI | Ambient clinical documentation, privacy-first (no data storage), HIPAA, guided reasoning | Passive listening; privacy-first architecture; guided conversation |
| Education Voice | Duolingo Max, Zuoyebang P50, Youdao Ziyue | Guided reasoning, personalized learning paths, gamification, pronunciation correction | Guided conversation pattern; gamification drives engagement |
| Translation Devices | Pocketalk, Timekettle, Vasco | Dual-mic noise reduction, <1s latency, offline packs, dedicated hardware | Dual-mic + low latency key; offline capability |
| Smart Home Hub | Echo Hub, HomePod, SmartThings | Matter 1.3/Thread 1.4, offline voice (0.2-0.4s local), predictive automation | ESP32 as privacy-first local controller |
Chinese AI Startups (2025-2026)
Voice AI technologies from China's "AI Six Tigers" and emerging companies, for vanling technology selection reference.
| Company | Key Technologies | Value for vanling |
|---|
| StepFun | Step-Audio 2 (130B S2S), Step-Audio-TTS-3B, Apache 2.0, emotion control + dialects | Most relevant: best open-source voice AI, usable directly |
| Zhipu AI | GLM-4V vision model, 85% revenue from local deployment, HK IPO | Local deployment business model reference; vision capability |
| MiniMax | Hailuo AI, Xingyao, 1.75T weekly token usage, Speech-2.8 | Consumer AI product operations reference; emotion TTS |
| Moonshot Kimi | Kimi K2.5 + Kimi Claw, valuation $4.3B→$18B in 3 months | Ultra-long context + voice AI integration |
| Baichuan | Healthcare vertical LLM, 3B cash reserve | Deep vertical industry approach |