Documentation Index
Fetch the complete documentation index at: https://tech.illasoft.com/llms.txt
Use this file to discover all available pages before exploring further.
架构概览
2026-04 架构变化:
- NATS JetStream → Inngest 全量迁移(
commit 0f2da43)
- MiniMax Music 2.5 / Mureka 两个 provider 已移除(
commit afc6058),当前仅 KIE AI 三条线路
核心组件
| 组件 | 角色 | 源 |
|---|
| Music Tools | 任务创建 + NSFW + 资格预检 | src/ai/tools/generateMusic.ts / generateInstrumental.ts / addVocals.ts |
| Agent onFinish | 派发 pending 任务 | src/hono/agent/index.ts:980+ |
publishMusicTask() | Inngest event 派发 | src/lib/tasks.ts:52-58 |
| Inngest server | 队列 + priority | kira-inngest |
| kira-music-worker | 4 个 Inngest function | kira-music-worker/src/inngest.ts |
| Supabase Storage | agent_message bucket | |
| Centrifugo | WebSocket 推送 | |
音乐工具
generateMusic
生成完整歌曲(含歌词)。
| 参数 | 类型 | 必填 | 说明 |
|---|
lyrics | string | 是 | 1-3500 字符,支持结构标签 [Verse] [Chorus] [Bridge] [Outro] |
title | string | 否 | ≤ 200 字符;AI 可自动生成 |
prompt | string | 否 | 风格/情绪描述 ≤ 2000 字符 |
coverId | string | 否 | 指定封面图(不传则 SeedReam 自动生成) |
- Event:
music/generation-kie-suno.requested
- Provider:
SunoUnofficialProvider
- Cost:25 credits
generateInstrumental
纯器乐(无人声)。
| 参数 | 类型 | 必填 | 说明 |
|---|
prompt | string | 否 | 器乐风格 |
lyrics | string | 否 | 可选,但即使纯器乐 Suno 也接受占位 lyrics |
- Event:
music/instrumental-kie-suno.requested
- Provider:
SunoInstrumentalProvider
- Cost:25 credits
addVocals
为已有音频添加/替换人声。
| 参数 | 类型 | 必填 |
|---|
sourceAudioId | string | 是 |
title | string | 是 |
vocalStyle | string | 是 |
lyrics | string | 是 |
style | string | 否 |
coverId | string | 否 |
negativeTags | string | 否 |
styleWeight | number | 否 (0-1) |
weirdnessConstraint | number | 否 (0-1) |
audioWeight | number | 否 (0-1) |
- Event:
music/add-vocals-kie-suno.requested
- Provider:
SunoAddVocalsProvider
- Cost:25 credits
trimAudio / upload(0 credits)
trimAudio → music/trim-kira.requested
upload → music/upload-kira.requested
initializeWithAudio
创建占位符 thread_version 记录,不派发生成任务(用于音频编辑/延长等后续工具的 source)。
Credits 计费
| 工具 | 成本 |
|---|
generateMusic / generateInstrumental / addVocals | 25 credits(固定) |
trimAudio / upload | 0 credits |
扣费:Worker step.run("check"),deductCredits(userId, 25, taskId) RPC 原子幂等。CSAM 阻断 不退款。
任务生命周期
Worker Inngest Functions
来源 kira-music-worker/src/inngest.ts。
生成类(并发限制)
| Function ID | Event | 并发 |
|---|
music-generation-kie-suno | music/generation-kie-suno.requested | per-user-per-tool=1,per-provider=100 |
music-instrumental-kie-suno | music/instrumental-kie-suno.requested | 同上 |
music-add-vocals-kie-suno | music/add-vocals-kie-suno.requested | 同上 |
非生成类(无限并发)
music-upload-kira — 用户直传
music-trim-kira — 本地 trim
Cleanup function 监听 inngest/function.cancelled,非 upload/trim 退款。
标准 Pipeline(6 步)
来源 kira-music-worker/src/processor.ts:81-154。
step 1: check
├─ checkThreadExists
├─ cost = 25
├─ checkMusicEligibility(userId, 25)
├─ deductCredits(userId, 25, taskId)
└─ { skip, startTime }
step 2: notify-processing
└─ notifyUser({ type: "audio_status", audio: { status: "processing" } })
step 3: submit-task
├─ provider = getProviderByToolName(toolName)
└─ providerTaskId = await provider.submitTask(task)
step 4: poll-result
└─ pollWithInngestSteps(step, () => provider.pollTask(providerTaskId))
└─ GenerationResult { success, audioUrl, ... }
step 5: post-process
└─ processAudio(generationResult, task) # 详见下方
step 6: finalize
├─ trackToolUsage + trackMediaUsage # 仅时长,无金额
└─ handleSuccess
├─ checkAudioCsam(id3CoverBuffer, lyrics, audioFileId, coverId)
├─ updateMusicInJsonb(taskId, { status: "completed", audioId, coverId, ... })
└─ notifyUser({ status: "completed", url, coverUrl, ... })
后处理 5 步(post-processor.ts)
1. download 音频
├─ fetch(audioUrl)
└─ 按 Content-Type 推断格式(mp3/wav/ogg/flac)
2. generate cover 封面
├─ 优先:Suno generation result 的 cover_url
└─ fallback:BytePlus SeedReam `seedream-5-0-260128`
├─ prompt: "Album cover art for a song titled '{title}'. {prompt}. Professional..."
├─ sharp resize 2048×2048 JPEG quality 90%
└─ generateBlurhash (4×4 components, 32×32 resize)
3. STT + forced alignment 歌词时间戳
├─ KIE AI STT(音频→文本)
├─ ElevenLabs Scribe v2(字级时间戳)
└─ Word[] = [{ word, start, end }, ...]
* instrumental 曲目跳过
4. embed ID3 标签
├─ mp3tag.js (Bun shim)
└─ tags:
├─ TIT2 (title)
├─ TPE1 (artist = "{nickname} via Kira Art")
├─ APIC (cover JPEG)
├─ SYLT (synced lyrics, word-level timestamps)
└─ USLT (unsynced lyrics, full text)
5. upload 到 Supabase Storage
├─ audio: {userId}/{threadId}/music_{audioId}.mp3 cacheControl:3600
└─ cover: {userId}/{threadId}/mc_{coverId}.jpg metadata: blurhash, size
显示歌词直接使用原始 task.lyrics(用户提供,已有行/段结构),无需 LLM 重新分组。Scribe v2 只提供字级时间戳用于同步。
Provider 清单
仅 KIE AI 三路:
| toolName | Provider 类 | Suno 模式 |
|---|
generateMusic | SunoUnofficialProvider | 常规生成 |
generateInstrumental | SunoInstrumentalProvider | 无人声 |
addVocals | SunoAddVocalsProvider | 对已有音频换/加人声 |
CSAM 审核
并发双路:
checkAudioCsam({
id3CoverBuffer, // 封面 JPEG
lyrics, // 歌词全文
audioFileId, // Storage 路径
coverId, // Storage 路径(如有)
})
Promise.all([
detectCsamFromBuffer(id3CoverBuffer), // coverHit
detectCsamFromText(lyrics), // textHit
])
命中任一 → block:
├─ 删除 audioFileId + coverId
├─ 写 failed_uploads 表
├─ notify { type: "audio_status", audio: { status: "failed", errorCode: "csam_blocked" } }
└─ credits 不退款
Centrifugo WebSocket 通知
Channel:{userId}/{threadId}#{userId}
{
"type": "audio_status",
"taskId": "...",
"audio": {
"taskId": "...",
"status": "processing" | "completed" | "failed" | "insufficient_credits" | "insufficient_plan",
"url": "https://...",
"coverUrl": "https://...",
"durationMs": 30500,
"blurhash": "...",
"syncedLyrics": [{ "text": "word", "start": 0.5, "end": 0.7 }, ...],
"errorCode": "csam_blocked",
"errorMessage": "..."
},
"timestamp": "2026-04-14T12:34:56Z"
}
数据模型(thread_version.audios)
interface Audio {
toolCallId?: string;
toolName?: string;
taskId: string;
provider: "suno_unofficial" | "suno_instrumental" | "suno_add_vocals";
status: "pending" | "processing" | "completed" | "failed" | "insufficient_credits" | "insufficient_plan";
createdAt: string;
errorMessage?: string;
errorCode?: string;
input: {
title?: string;
lyrics?: string;
prompt?: string;
instrumental?: boolean;
sourceAudioId?: string;
vocalStyle?: string;
style?: string;
negativeTags?: string;
styleWeight?: number;
weirdnessConstraint?: number;
audioWeight?: number;
coverId?: string;
trimStart?: number;
trimEnd?: number;
};
output?: {
audioId: string; // Storage 路径
coverId?: string;
coverWidth?: number;
coverHeight?: number;
durationMs: number;
title: string;
lyrics: string;
hasVocals: boolean;
hasInstrumental: boolean;
syncedLyrics?: Array<{ text: string; start: number; end: number }>;
};
}
新 envelope 结构,Zod transform 兼容旧 flat 格式(审计 src/types/)。
文件存储结构
agent_message/
├── {userId}/{threadId}/
│ ├── music_{audioId}.mp3
│ └── mc_{coverId}.jpg
└── feed/{userId}/
├── music_{audioId}.mp3 # Feed 副本
└── mc_{coverId}.jpg
Feed 集成
POST /publish/audio:
copyVideoFromMessageToFeed()(工具函数名历史遗留)复制音频 + 封面到 feed 存储
- 三路 CSAM 审核:封面图片 + 歌词全文 + prompt 文本
- 用封面图片生成 embedding,LanceDB 去重(cosine 阈值 0.25)
- 写
feed 记录,media_type = "audio"
取消和清理
删除消息、rewind、删除 thread 触发:
extractMusicTaskIds 从 JSONB 提取 taskId
purgeMusicTasks 发 music/task.cancelled Inngest event
redis SET cancel:{taskId} 1 EX 3600
- Worker 检查并抛
NonRetriableError
music-cancelled-cleanup 退款 + 通知
清理文件:musicCleanup() 在 src/ai/libs/musicCleanup.ts,删 Storage 中 audio + cover 对象。
输出时长追踪
trackMediaUsage({
userId, threadId, taskId, toolName,
provider, // suno_unofficial / suno_instrumental / suno_add_vocals
model, // 模型字符串
durationMs, // 处理耗时
outputDurationMs, // 音频时长(ms)
});
2026-04 弃用 trackAICost / ai_cost 事件。Worker 只上报输出时长,不在埋点中算钱(异构 provider 难以稳定归因)。
相关文档