Skip to main content

Documentation Index

Fetch the complete documentation index at: https://tech.illasoft.com/llms.txt

Use this file to discover all available pages before exploring further.

架构概览

2026-04 架构变化
  • NATS JetStream → Inngest 全量迁移(commit 0f2da43
  • MiniMax Music 2.5 / Mureka 两个 provider 已移除(commit afc6058),当前仅 KIE AI 三条线路

核心组件

组件角色
Music Tools任务创建 + NSFW + 资格预检src/ai/tools/generateMusic.ts / generateInstrumental.ts / addVocals.ts
Agent onFinish派发 pending 任务src/hono/agent/index.ts:980+
publishMusicTask()Inngest event 派发src/lib/tasks.ts:52-58
Inngest server队列 + prioritykira-inngest
kira-music-worker4 个 Inngest functionkira-music-worker/src/inngest.ts
Supabase Storageagent_message bucket
CentrifugoWebSocket 推送

音乐工具

generateMusic

生成完整歌曲(含歌词)。
参数类型必填说明
lyricsstring1-3500 字符,支持结构标签 [Verse] [Chorus] [Bridge] [Outro]
titlestring≤ 200 字符;AI 可自动生成
promptstring风格/情绪描述 ≤ 2000 字符
coverIdstring指定封面图(不传则 SeedReam 自动生成)
  • Event:music/generation-kie-suno.requested
  • Provider:SunoUnofficialProvider
  • Cost:25 credits

generateInstrumental

纯器乐(无人声)。
参数类型必填说明
promptstring器乐风格
lyricsstring可选,但即使纯器乐 Suno 也接受占位 lyrics
  • Event:music/instrumental-kie-suno.requested
  • Provider:SunoInstrumentalProvider
  • Cost:25 credits

addVocals

为已有音频添加/替换人声。
参数类型必填
sourceAudioIdstring
titlestring
vocalStylestring
lyricsstring
stylestring
coverIdstring
negativeTagsstring
styleWeightnumber否 (0-1)
weirdnessConstraintnumber否 (0-1)
audioWeightnumber否 (0-1)
  • Event:music/add-vocals-kie-suno.requested
  • Provider:SunoAddVocalsProvider
  • Cost:25 credits

trimAudio / upload(0 credits)

  • trimAudiomusic/trim-kira.requested
  • uploadmusic/upload-kira.requested

initializeWithAudio

创建占位符 thread_version 记录,不派发生成任务(用于音频编辑/延长等后续工具的 source)。

Credits 计费

工具成本
generateMusic / generateInstrumental / addVocals25 credits(固定)
trimAudio / upload0 credits
扣费:Worker step.run("check")deductCredits(userId, 25, taskId) RPC 原子幂等。CSAM 阻断 不退款

任务生命周期

Worker Inngest Functions

来源 kira-music-worker/src/inngest.ts

生成类(并发限制)

Function IDEvent并发
music-generation-kie-sunomusic/generation-kie-suno.requestedper-user-per-tool=1,per-provider=100
music-instrumental-kie-sunomusic/instrumental-kie-suno.requested同上
music-add-vocals-kie-sunomusic/add-vocals-kie-suno.requested同上

非生成类(无限并发)

  • music-upload-kira — 用户直传
  • music-trim-kira — 本地 trim
Cleanup function 监听 inngest/function.cancelled,非 upload/trim 退款。

标准 Pipeline(6 步)

来源 kira-music-worker/src/processor.ts:81-154
step 1: check
  ├─ checkThreadExists
  ├─ cost = 25
  ├─ checkMusicEligibility(userId, 25)
  ├─ deductCredits(userId, 25, taskId)
  └─ { skip, startTime }

step 2: notify-processing
  └─ notifyUser({ type: "audio_status", audio: { status: "processing" } })

step 3: submit-task
  ├─ provider = getProviderByToolName(toolName)
  └─ providerTaskId = await provider.submitTask(task)

step 4: poll-result
  └─ pollWithInngestSteps(step, () => provider.pollTask(providerTaskId))
     └─ GenerationResult { success, audioUrl, ... }

step 5: post-process
  └─ processAudio(generationResult, task)   # 详见下方

step 6: finalize
  ├─ trackToolUsage + trackMediaUsage   # 仅时长,无金额
  └─ handleSuccess
     ├─ checkAudioCsam(id3CoverBuffer, lyrics, audioFileId, coverId)
     ├─ updateMusicInJsonb(taskId, { status: "completed", audioId, coverId, ... })
     └─ notifyUser({ status: "completed", url, coverUrl, ... })

后处理 5 步(post-processor.ts

1. download 音频
   ├─ fetch(audioUrl)
   └─ 按 Content-Type 推断格式(mp3/wav/ogg/flac)

2. generate cover 封面
   ├─ 优先:Suno generation result 的 cover_url
   └─ fallback:BytePlus SeedReam `seedream-5-0-260128`
      ├─ prompt: "Album cover art for a song titled '{title}'. {prompt}. Professional..."
      ├─ sharp resize 2048×2048 JPEG quality 90%
      └─ generateBlurhash (4×4 components, 32×32 resize)

3. STT + forced alignment 歌词时间戳
   ├─ KIE AI STT(音频→文本)
   ├─ ElevenLabs Scribe v2(字级时间戳)
   └─ Word[] = [{ word, start, end }, ...]
   * instrumental 曲目跳过

4. embed ID3 标签
   ├─ mp3tag.js (Bun shim)
   └─ tags:
      ├─ TIT2 (title)
      ├─ TPE1 (artist = "{nickname} via Kira Art")
      ├─ APIC (cover JPEG)
      ├─ SYLT (synced lyrics, word-level timestamps)
      └─ USLT (unsynced lyrics, full text)

5. upload 到 Supabase Storage
   ├─ audio: {userId}/{threadId}/music_{audioId}.mp3  cacheControl:3600
   └─ cover: {userId}/{threadId}/mc_{coverId}.jpg     metadata: blurhash, size
显示歌词直接使用原始 task.lyrics(用户提供,已有行/段结构),无需 LLM 重新分组。Scribe v2 只提供字级时间戳用于同步。

Provider 清单

仅 KIE AI 三路:
toolNameProvider 类Suno 模式
generateMusicSunoUnofficialProvider常规生成
generateInstrumentalSunoInstrumentalProvider无人声
addVocalsSunoAddVocalsProvider对已有音频换/加人声

CSAM 审核

并发双路:
checkAudioCsam({
  id3CoverBuffer,    // 封面 JPEG
  lyrics,            // 歌词全文
  audioFileId,       // Storage 路径
  coverId,           // Storage 路径(如有)
})

Promise.all([
  detectCsamFromBuffer(id3CoverBuffer),  // coverHit
  detectCsamFromText(lyrics),            // textHit
])

命中任一 → block:
  ├─ 删除 audioFileId + coverId
  ├─ 写 failed_uploads 表
  ├─ notify { type: "audio_status", audio: { status: "failed", errorCode: "csam_blocked" } }
  └─ credits 不退款

Centrifugo WebSocket 通知

Channel{userId}/{threadId}#{userId}
{
  "type": "audio_status",
  "taskId": "...",
  "audio": {
    "taskId": "...",
    "status": "processing" | "completed" | "failed" | "insufficient_credits" | "insufficient_plan",
    "url": "https://...",
    "coverUrl": "https://...",
    "durationMs": 30500,
    "blurhash": "...",
    "syncedLyrics": [{ "text": "word", "start": 0.5, "end": 0.7 }, ...],
    "errorCode": "csam_blocked",
    "errorMessage": "..."
  },
  "timestamp": "2026-04-14T12:34:56Z"
}

数据模型(thread_version.audios)

interface Audio {
  toolCallId?: string;
  toolName?: string;
  taskId: string;
  provider: "suno_unofficial" | "suno_instrumental" | "suno_add_vocals";
  status: "pending" | "processing" | "completed" | "failed" | "insufficient_credits" | "insufficient_plan";
  createdAt: string;
  errorMessage?: string;
  errorCode?: string;

  input: {
    title?: string;
    lyrics?: string;
    prompt?: string;
    instrumental?: boolean;
    sourceAudioId?: string;
    vocalStyle?: string;
    style?: string;
    negativeTags?: string;
    styleWeight?: number;
    weirdnessConstraint?: number;
    audioWeight?: number;
    coverId?: string;
    trimStart?: number;
    trimEnd?: number;
  };

  output?: {
    audioId: string;           // Storage 路径
    coverId?: string;
    coverWidth?: number;
    coverHeight?: number;
    durationMs: number;
    title: string;
    lyrics: string;
    hasVocals: boolean;
    hasInstrumental: boolean;
    syncedLyrics?: Array<{ text: string; start: number; end: number }>;
  };
}
新 envelope 结构,Zod transform 兼容旧 flat 格式(审计 src/types/)。

文件存储结构

agent_message/
├── {userId}/{threadId}/
│   ├── music_{audioId}.mp3
│   └── mc_{coverId}.jpg
└── feed/{userId}/
    ├── music_{audioId}.mp3   # Feed 副本
    └── mc_{coverId}.jpg

Feed 集成

POST /publish/audio
  1. copyVideoFromMessageToFeed()(工具函数名历史遗留)复制音频 + 封面到 feed 存储
  2. 三路 CSAM 审核:封面图片 + 歌词全文 + prompt 文本
  3. 用封面图片生成 embedding,LanceDB 去重(cosine 阈值 0.25)
  4. feed 记录,media_type = "audio"

取消和清理

删除消息、rewind、删除 thread 触发:
  1. extractMusicTaskIds 从 JSONB 提取 taskId
  2. purgeMusicTasksmusic/task.cancelled Inngest event
  3. redis SET cancel:{taskId} 1 EX 3600
  4. Worker 检查并抛 NonRetriableError
  5. music-cancelled-cleanup 退款 + 通知
清理文件:musicCleanup()src/ai/libs/musicCleanup.ts,删 Storage 中 audio + cover 对象。

输出时长追踪

trackMediaUsage({
  userId, threadId, taskId, toolName,
  provider,                 // suno_unofficial / suno_instrumental / suno_add_vocals
  model,                    // 模型字符串
  durationMs,               // 处理耗时
  outputDurationMs,         // 音频时长(ms)
});
2026-04 弃用 trackAICost / ai_cost 事件。Worker 只上报输出时长,不在埋点中算钱(异构 provider 难以稳定归因)。

相关文档