Documentation Index
Fetch the complete documentation index at: https://tech.illasoft.com/llms.txt
Use this file to discover all available pages before exploring further.
架构概览
核心组件
| 组件 | 角色 | 源文件 |
|---|
| 视频工具 | 任务创建、NSFW 检测、provider 选择 | src/ai/tools/generateVideo.ts 等 |
| Agent onFinish | 派发 pending 任务 | src/hono/agent/index.ts:980+ |
publishVideoTask() | Inngest event 派发 | src/lib/tasks.ts |
| Inngest server | 队列 + priority + retry | kira-inngest (OSS v1.17.9) |
| kira-video-worker | 7 个 Inngest function | kira-video-worker/src/inngest.ts |
| Supabase Storage | agent_message bucket | |
| Centrifugo | WebSocket 状态推送 | |
2026-04 迁移:原 NATS JetStream(TASKS stream,tasks.video.* subject)已被 Inngest 完全替代。
Worker 从 long-lived consumer 重构为 Inngest HTTP function(kira-video-worker commit a9597f6)。
详见 Kira Inngest。
视频工具与 Provider 路由
工具清单
| 工具 | 用途 | 默认 Provider |
|---|
generateVideo | 文生视频 / 图生视频 | seedance (NSFW) 或 grok (safe) |
videoEdit | 视频编辑(prompt + reference images) | ws-wan27 或 kie-seedance2 |
videoExtend | 视频延长 | ws-wan27 |
motionControl | Motion control (image → video) | TencentCloud VOD Kling |
trimVideo | 本地 trim(0 credits) | Kira trim |
initializeWithImage / initializeWithVideo | 创建占位符任务 | — |
Inngest Event 映射
| toolName:provider | Event |
|---|
generateVideo:seedance | video/generation-seedance.requested |
generateVideo:grok | video/generation-grok.requested |
motionControl | video/motion-control-tencentcloud-vod-kling.requested |
videoEdit:ws-wan27 | video/edit-ws-wan27.requested |
videoEdit:kie-seedance2 | video/edit-kie-seedance2.requested |
videoExtend:ws-wan27 | video/extend-ws-wan27.requested |
trimVideo | video/trim-kira.requested |
upload | video/upload-kira.requested |
| Cancel broadcast | video/task.cancelled |
generateVideo 的 Provider 选择
Provider 基于 NSFW 检测自动选择,用户无法手动指定:
detectNsfw({text, imageId}) 在 src/ai/libs/moderation.ts,fail-open 到 seedance。
Provider 能力
| Provider | 模型 | t2v | i2v | 分辨率 |
|---|
| seedance | seedance-1-5-pro (BytePlus) | ✅ | ✅ | 720p |
| grok | grok-imagine-video (xAI) | ✅ | ✅ | 720p |
| ws-wan27 | WAN 2.7 Pro | — | edit/extend | — |
| kie-seedance2 | Seedance 2 via KIE | — | edit | — |
| TencentCloud Kling | VOD Kling | — | motion control | — |
Veo 曾经是一个 provider,现已从 tool schema 中移除。文档的旧条目已清理。
| 参数 | 可选值 | 默认值 |
|---|
| duration | "5" / "10" | "5" |
| ratio | "auto" / "16:9" / "9:16" / "1:1" | "auto" |
ratio="auto" 且有 sourceImageId 时,Worker 从源图自动检测最接近的比例。
Credits 计费
来源 src/ai/libs/billing.ts。
| 工具 | 成本 |
|---|
generateVideo 5s | 50 credits |
generateVideo 10s | 100 credits |
videoEdit / videoExtend / motionControl | ceil(duration) × 15 credits |
trimVideo / upload | 0 credits |
扣费时机:Worker step.run("check") 内部,deductCredits(userId, cost, taskId) 走 Supabase RPC deduct_credits 原子操作(行锁 + 幂等 by taskId)。
任务生命周期
| 状态 | 说明 |
|---|
pending | thread_version 已插入,等 Worker |
processing | Worker 调 provider 中 |
completed | 已上传,Centrifugo 推送 |
failed | 生成失败,credits 已退 |
csam_blocked | CSAM 命中,文件已删,不退款 |
insufficient_credits / insufficient_plan | 预检失败 |
Worker 7 个 Inngest Function
来源 kira-video-worker/src/inngest.ts。
生成类(有并发限制)
| Function ID | 并发 | 重试 | 超时 |
|---|
video-generation-seedance | per-user-per-tool=1,per-provider=100 | 2 | 60min |
video-generation-grok | per-user-per-tool=1,per-provider=60 | 2 | 60min |
video-motion-control-tencentcloud-vod-kling | per-user-per-tool=1,per-provider=70 | 2 | 60min |
video-edit-ws-wan27 | per-user-per-tool=1,per-provider=50 | 2 | 60min |
video-edit-kie-seedance2 | per-user-per-tool=1,per-provider=50 | 2 | 60min |
video-extend-ws-wan27 | per-user-per-tool=1,per-provider=50 | 2 | 60min |
非生成类(无限并发)
video-upload-kira — 用户直传
video-trim-kira — 本地 trim
Cleanup
video-cancelled-cleanup 监听 inngest/function.cancelled:非 upload/trim 退款 + 通知 failed
标准处理 Pipeline(6 步)
来源 kira-video-worker/src/inngest.ts:187-266 + processor.ts。
step 1: check
├─ checkThreadExists(threadId)
├─ cost = getVideoCreditsCost(toolName, duration)
├─ checkVideoEligibility(userId, cost)
├─ deductCredits(userId, cost, taskId) ← 原子幂等
└─ { skip, startTime, cost }
step 2: notify-processing
├─ updateVideoInJsonb(taskId, { status: "processing" })
├─ updateMessageToolOutput(...)
└─ notifyUser({ type: "video_status", video: { status: "processing" } })
step 3: create-task
├─ detectImageRatio(sourceImageId) 若 ratio="auto"
├─ provider = getProvider(providerName)
└─ providerTaskId = await provider.createTask(task)
step 4: poll-result
└─ pollWithInngestSteps(step, () => provider.pollTask(providerTaskId))
# 用 step.sleep + step.waitForEvent 做 durable polling
└─ GenerationResult { success, videoUrl, error }
step 5: post-process
├─ fetch(videoUrl) → Buffer
├─ ffprobe → { width, height, duration }
├─ ffmpeg -vframes 1 -q:v 2 → thumbnail JPEG (95% quality)
├─ generateBlurhash(thumbBuffer)
├─ uploadVideo → {userId}/{threadId}/video_{videoId}.mp4
├─ uploadThumbnail → {userId}/{threadId}/vc_{thumbId}.jpg
└─ VideoResult { url, thumbUrl, width, height, duration, blurhash }
step 6: finalize
├─ trackToolUsage({ userId, threadId, taskId, tool_name, success, duration_ms, credits_consumed })
├─ trackMediaUsage({ taskId, toolName, provider, model, duration_ms, output_duration_ms }) # 仅时长
└─ handleSuccess(...)
├─ checkVideoCsam(thumbBuffer, videoFileId)
├─ updateVideoInJsonb(taskId, { status: "completed", videoId, thumbId, ... })
└─ notifyUser({ status: "completed", url, thumbUrl, ... })
Redux / 临时文件
motionControl / videoEdit / videoExtend 需要先把源视频或关键帧上传为临时文件:
- 路径:
{userId}/{threadId}/temp_video_{uuid}.mp4
- finalize 步骤调
deleteStorageFile(prepared.tempVideoPath) 清理
Upload 策略
- Bucket:Supabase Storage
agent_message
- 单次 PUT(无分片)
- 无 imgproxy 代理(视频不走 imgproxy,直接返回 Supabase signed URL)
- Content-Type 自动按扩展名映射
- cacheControl:
3600
- Upsert:false(防覆盖)
| 类型 | 路径 |
|---|
| 原始视频 | {userId}/{threadId}/video_{videoId}.mp4 |
| 缩略图 | {userId}/{threadId}/vc_{thumbId}.jpg |
| 带水印 | {userId}/{threadId}/wv_{uuid}.mp4(按需生成) |
| 临时编辑 | {userId}/{threadId}/temp_video_{uuid}.mp4(finalize 时删) |
数据模型(thread_version.videos)
新 envelope 结构(src/types/ 下的 VideoSchema,带 Zod transform 兼容旧 flat 格式):
interface Video {
toolCallId?: string;
toolName?: string;
taskId: string; // UUID
provider: string; // seedance / grok / ws-wan27 / ...
status: "pending" | "processing" | "completed" | "failed"
| "insufficient_credits" | "insufficient_plan";
createdAt: string;
errorMessage?: string;
errorCode?: string;
input: {
prompt?: string;
sourceImageId?: string;
sourceVideoId?: string; // videoEdit / videoExtend / trimVideo
sourceMedias?: string[]; // videoEdit reference images
duration?: number; // 5 / 10
ratio?: string;
trimStart?: number;
trimEnd?: number;
imageId?: string; // motionControl 起始帧
};
output?: {
videoId: string; // Storage 路径
thumbId: string;
width: number;
height: number;
duration: number;
};
}
Centrifugo WebSocket 通知
Channel:{userId}/{threadId}#{userId}(# 分隔符)
{
"type": "video_status",
"taskId": "...",
"video": {
"taskId": "...",
"status": "processing" | "completed" | "failed" | "insufficient_credits" | "insufficient_plan",
"url": "https://...",
"thumbUrl": "https://...",
"width": 1920,
"height": 1080,
"duration": 5.5,
"blurhash": "U6PVzY...",
"errorCode": "csam_blocked",
"errorMessage": "..."
},
"timestamp": "2026-04-14T12:34:56Z"
}
发送端点:{CENTRIFUGO_URL}/api/publish,认证 X-API-Key。
免费用户下载自动加水印;Pro+ 用户直出。
Feed 集成
发布时通过 POST /publish/video:
copyVideoFromMessageToFeed() 复制视频 + 缩略图到 feed 存储
- 缩略图做 CSAM 审核
- 缩略图生成 embedding,LanceDB 去重(cosine 阈值 0.25)
- 写
feed 记录,media_type = "video"
取消和清理
删除消息、rewind、删除 thread 都会触发:
extractVideoTaskIds(versions) 从 JSONB 提取 taskId
purgeVideoTasks(taskIds) 发 video/task.cancelled Inngest event
redis SET cancel:{taskId} 1 EX 3600
- Worker step 开头检查 cancel key,命中则
throw NonRetriableError
video-cancelled-cleanup function 处理退款 + 通知
另外定期/一次性清理:删除 Storage 文件(deleteVideoTasks)。
输出时长追踪
trackMediaUsage({
userId, threadId, taskId, toolName,
provider, // seedance / grok / ws-wan27 / ...
model, // 模型字符串
durationMs, // 处理耗时
outputDurationMs, // 视频时长(统一 ms)
});
2026-04 弃用 trackAICost / ai_cost 事件。Worker 只上报输出时长,不在埋点中算钱(异构 provider 难以稳定归因)。
相关文档