qwen3-tts-ra/README.md

qwen3-tts-ra

Qwen3-TTS with Read-Aloud browser extension integration.

Components

• qwen3-proxy/ — OpenAI-compatible TTS proxy (POST /audio/speech)
• Qwen3-TTS/ — Qwen3-TTS library (submodule / clone)
• read-aloud/ — Read-Aloud browser extension (submodule / clone)
• setup_qwen3_readaloud.sh — Initial environment setup script

Architecture

Read-Aloud extension
  → POST http://localhost:5000/audio/speech
    → qwen3-proxy/app.py (Flask, OpenAI-compatible API)
      → faster-qwen3-tts (HIP graph acceleration, AMD gfx1100)
        → GPU: LLM token generation at ~1.78x RTF
        → CPU: speech tokenizer decode (bypasses MIOpen)

Performance (AMD Radeon RX 7900 XTX, gfx1100)

Input	Audio	Time	RTF
12c "Hello world."	~2s	~3s	~0.9x
44c sentence	~4s	~3s	1.5x
115c paragraph	~10s	~7s	1.5x

RTF > 1.0 = generates faster than real-time.

Key optimisations

1. HIP Graphs (faster-qwen3-tts) — captures autoregressive decode loop as a static GPU program, eliminating Python overhead per token
2. CPU speech decoder — moves speech_tokenizer.model to CPU, bypassing MIOpen's slow ConvDirectNaiveConvFwd fallback entirely
3. attn_implementation=sdpa — PyTorch native SDPA for transformer attention
4. MIOPEN_USER_DB_PATH — persistent MIOpen find-DB for LLM-side convolutions

Setup

# Install Python venv + deps
./setup_qwen3_readaloud.sh

# Start the proxy service
systemctl --user start qwen3-tts-proxy.service

# Watch logs
journalctl --user -u qwen3-tts-proxy.service -f

Read-Aloud Extension Settings

In Read-Aloud → Settings → OpenAI:

Field	Value
URL	http://127.0.0.1:5000
API Key	(leave blank)
Voice list	see below

[
  {"voice": "alloy",   "lang": "en-US", "model": "tts-1"},
  {"voice": "echo",    "lang": "en-US", "model": "tts-1"},
  {"voice": "fable",   "lang": "en-US", "model": "tts-1"},
  {"voice": "onyx",    "lang": "en-US", "model": "tts-1"},
  {"voice": "nova",    "lang": "zh-CN", "model": "tts-1"},
  {"voice": "shimmer", "lang": "zh-CN", "model": "tts-1"}
]

Env vars (systemd service)

Variable	Default	Notes
QWEN_MODEL	Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice	HF model id or local path
DEVICE	cuda:0	GPU device
HIP_GRAPHS	1	Enable faster-qwen3-tts HIP graphs
AOTRITON	0	AOTriton flash attention — faster for long text (>80 chars), slower for short sentences
PROXY_PORT	5000	Listening port