跳到主要内容

语音转文字包

内置 speech-to-text packages 会在 model request 前转录支持的音频或视频附件。每个 provider package 都会在 agent/input/speech_to_text 安装一个 input slot,因此同一个 core 中一次只安装一个 STT provider package。

Provider能力
stt_openaiOpenAInetwork.fetch
stt_groqGroqnetwork.fetch
stt_deepgramDeepgramnetwork.fetch
stt_assemblyaiAssemblyAInetwork.fetch
stt_geminiGemininetwork.fetch
stt_dashscopeAlibaba Cloud Model Studionetwork.fetch
stt_baiduBaidu Cloud Speech Recognitionnetwork.fetch
stt_tencentTencent Cloud ASRnetwork.fetch

安装

使用交互式 manager:

uv run demiurge package

或者用 subcommands 预览并安装:

uv run demiurge package install stt_openai --core assistant --preview
uv run demiurge package install stt_openai --core assistant

切换 providers 时,先卸载当前 STT package:

uv run demiurge package uninstall stt_openai --core assistant
uv run demiurge package install stt_gemini --core assistant

共享行为

所有 STT packages 都会安装:

agent/input/speech_to_text/
agent/lib/stt_common/
agent/lib/<provider>/

Input slot 会追加到 serial input pipeline。当 turn 带有支持的音频或视频附件时,该 slot 需要 network.fetch,会把附件发送给 provider,并把 Voice message transcript section 添加到当前 user prompt。如果没有支持的附件,它不会执行任何操作。

支持的 media 包括 AAC、FLAC、M4A、MP3、OGG、OPUS、WAV 和 WebM 等常见音频格式,以及 provider 接受的 MP4/WebM 视频附件。

选项

选项
stt_openaiapi_key, language, context_hint, timestamp_granularity, include_metadata
stt_groqapi_key, language, context_hint, timestamp_granularity, include_metadata
stt_deepgramapi_key, language, detect_language, diarization, include_metadata
stt_assemblyaiapi_key, language, detect_language, speaker_labels, include_metadata
stt_geminiapi_key, language, include_timestamps, include_metadata
stt_dashscopeapi_key, language, include_metadata, enable_itn
stt_baiduapi_key, secret_key, access_token, language, dev_pid, include_metadata
stt_tencentsecret_id, secret_key, region, engine_model_type, include_metadata

timestamp_granularity 的可选值是 nonesegmentword

可选 secret options 可以在安装期间传入,也可以留空,让已安装 provider config 在运行时读取环境变量。

凭证环境变量

环境变量
stt_openaiDEMIURGE_OPENAI_API_KEY or OPENAI_API_KEY
stt_groqDEMIURGE_GROQ_API_KEY or GROQ_API_KEY
stt_deepgramDEMIURGE_DEEPGRAM_API_KEY or DEEPGRAM_API_KEY
stt_assemblyaiDEMIURGE_ASSEMBLYAI_API_KEY or ASSEMBLYAI_API_KEY
stt_geminiDEMIURGE_GEMINI_API_KEY, GEMINI_API_KEY, or GOOGLE_API_KEY
stt_dashscopeDEMIURGE_DASHSCOPE_API_KEY or DASHSCOPE_API_KEY
stt_baiduDEMIURGE_BAIDU_ACCESS_TOKEN, or DEMIURGE_BAIDU_API_KEY plus DEMIURGE_BAIDU_SECRET_KEY
stt_tencentDEMIURGE_TENCENT_SECRET_ID plus DEMIURGE_TENCENT_SECRET_KEY

验证

列出已安装 packages:

uv run demiurge package list --core assistant

通过 TUI 或已配置 channel 发送音频附件。Model request 应该在原始 user text 之前或旁边包含 Voice message transcript section。

卸载

uv run demiurge package uninstall stt_openai --core assistant --preview
uv run demiurge package uninstall stt_openai --core assistant

Uninstall 会移除 package-owned input slot 和 provider lib。如果另一个已安装 package 仍引用 shared agent/lib/stt_common/,它会被保留。