feat(media): add image/video/audio project kinds via od media generate
Introduce non-web media surfaces (image, video, audio) as first-class project kinds. The unifying contract is "skill workflow + project metadata tell the agent WHAT to make; one shell command — od media generate — is HOW bytes are produced", so any code-agent CLI with shell access can drive it without bespoke tools. - Frontend: New Project panel gains Image/Video/Audio tabs with model picker, aspect/length/duration controls, and audio kind/voice selection. Examples and Design Systems tabs gain layered sections. FileViewer renders the generated image/video/audio files. - Shared registry: src/media/models.ts is the single source of truth for image/video/audio model IDs, aspects, and defaults — consumed by the picker AND the daemon dispatcher. - Prompts: media-contract.ts is pinned LAST in the system prompt for media surfaces so its hard rules (call od media generate, don't emit binary in <artifact>, allowed model IDs) win over softer earlier wording. - Daemon: new media.js dispatcher + media-models.js JSON view of the registry; cli.js gets the `od media generate` subcommand wired up via server.js / projects.js so the daemon writes files back into the project dir. - Skills: audio-jingle, image-poster, video-shortform seed examples for the three surfaces. Made-with: Cursor
This commit is contained in:
pftom
@@ -0,0 +1,108 @@
|
||||
---
|
||||
name: video-shortform
|
||||
description: |
|
||||
Short-form video generation skill — 3-10 second clips for product
|
||||
reveals, motion teasers, ambient loops. Defaults to Seedance 2 but
|
||||
works the same with Kling 3 / 4, Veo 3 or Sora 2. Output is one MP4
|
||||
saved to the project folder. When the workspace also ships an
|
||||
interactive-video / hyperframes skill, prefer composing several short
|
||||
shots into a single timeline rather than one long monolithic clip.
|
||||
triggers:
|
||||
- "video"
|
||||
- "clip"
|
||||
- "shortform"
|
||||
- "reel"
|
||||
- "短视频"
|
||||
- "动效"
|
||||
od:
|
||||
mode: video
|
||||
surface: video
|
||||
scenario: marketing
|
||||
preview:
|
||||
type: html
|
||||
entry: example.html
|
||||
design_system:
|
||||
requires: false
|
||||
example_prompt: |
|
||||
5-second product reveal — ceramic coffee mug rotating on a soft
|
||||
paper backdrop, warm side-light from camera-left, micro dust motes
|
||||
drifting through the beam. Cinematic, 16:9, slow drift on the camera.
|
||||
---
|
||||
|
||||
# Video Shortform Skill
|
||||
|
||||
Short-form (≤ 10s) is the sweet spot for current text-to-video models —
|
||||
they're great at one **shot** with one **idea**, weaker at multi-cut
|
||||
narratives. Plan one shot per call.
|
||||
|
||||
## Resource map
|
||||
|
||||
```
|
||||
video-shortform/
|
||||
├── SKILL.md
|
||||
└── example.html
|
||||
```
|
||||
|
||||
## Workflow
|
||||
|
||||
### Step 0 — Read the project metadata
|
||||
|
||||
`videoModel`, `videoLength` (seconds), `videoAspect`. These are
|
||||
hard-locks — clamp the prompt to whatever the chosen model supports
|
||||
(Seedance 2 caps at 10s; Kling 4 supports up to 10s + image-to-video;
|
||||
Veo 3 supports 8s with audio).
|
||||
|
||||
### Step 1 — Plan the shot
|
||||
|
||||
Write the shotlist BEFORE calling the model:
|
||||
|
||||
| Slot | Content |
|
||||
|---|---|
|
||||
| Subject | What's in frame? |
|
||||
| Camera | Static / pan / push-in / orbit? |
|
||||
| Lighting | Key direction + temperature |
|
||||
| Motion | What moves, at what pace? Subject motion vs camera motion. |
|
||||
| Sound | Ambient bed? (only if the model supports audio) |
|
||||
|
||||
Show this to the user as a one-sentence plan before dispatching — they
|
||||
can redirect cheaply.
|
||||
|
||||
### Step 2 — Compose the prompt
|
||||
|
||||
Use the format the upstream model prefers (Seedance: motion + camera +
|
||||
mood; Kling: subject + camera + style; Veo: subject + cinematography +
|
||||
sound). Bind the project's `videoAspect` and `videoLength` directly to
|
||||
the API parameters; never put them in prose.
|
||||
|
||||
### Step 3 — Dispatch via the media contract
|
||||
|
||||
Use the unified dispatcher — do **not** call provider APIs by hand:
|
||||
|
||||
```bash
|
||||
node "$OD_BIN" media generate \
|
||||
--project "$OD_PROJECT_ID" \
|
||||
--surface video \
|
||||
--model "<videoModel from metadata>" \
|
||||
--aspect "<videoAspect from metadata>" \
|
||||
--length <videoLength seconds> \
|
||||
--output "<short-slug>-<seconds>s.mp4" \
|
||||
--prompt "<assembled shot prompt from Step 2>"
|
||||
```
|
||||
|
||||
The command prints one line of JSON: `{"file": {"name": "...", ...}}`.
|
||||
The bytes land in the project; the FileViewer plays it automatically.
|
||||
|
||||
### Step 4 — Hand off
|
||||
|
||||
Reply with: shot summary, the filename returned by the dispatcher, and
|
||||
one sentence on what to try if the user wants a variation.
|
||||
|
||||
## Hard rules
|
||||
|
||||
- One shot per turn. Multi-shot timelines belong in a hyperframes /
|
||||
interactive-video skill, not here.
|
||||
- Match `videoAspect` exactly — re-renders are slow.
|
||||
- Never ship a video without saving the file — the user expects
|
||||
something to play in the file viewer.
|
||||
- When the underlying model fails (NSFW filter, content policy,
|
||||
timeout), report the error verbatim. Don't silently retry.
|
||||
Reference in New Issue
Block a user