Directed prompts: the quality unlock

Chapter 7 of 9. Prerequisite: Chapter 5. You have motion transfer working: registered assets driving choreography from a hosted, face-safe reference clip. The renders move like the source and still look rough. This chapter is the difference between rough and directed, and the difference costs nothing but writing.

The rough look is a recipe problem: v05 versus v06

Two passes over the same source fight, rendered on an identical asset stack: the same model with the same reference inputs (New Eyes as @Image1 and Ellie as @Image2, the arena plate as @Image3). The only variable was prompt discipline.

The generic pass, v05: one constant prompt on every shot, "motion follows @Video1", submitted with a blanket --duration 5 against source cuts averaging about 2.5 seconds. Eleven shots, roughly $5. Every shot came back a wide two-shot. Cuts landed mid-move. Each render carried 2 to 3 seconds of tail motion the reference never showed, because the model kept animating after the choreography ran out.

The directed pass, v06: the source chopped at its own cut points, a purpose-written prompt per beat, duration matched to each cut. Eight beats, all submitted in parallel, all succeeded first try, $3 to $4 total. The production log calls the result "a clear tier jump". Cheaper per usable second, and a tier better.

The log also names the root cause, banked as a rule: render recipe, not a model ceiling. Three ending beats rendered with rich outcome-specific prompts came out nearly perfect while the generically-prompted footage looked rough on the exact same model. Seedance 2.0's launch announcement describes a model that "supports four input modalities: text, image, audio, and video". The text modality is not decoration on top of the reference media. Feed it one lazy sentence and the model directs the film itself, and it directs wide and safe. The rough look everyone blames on the model is, in the ledger of this arc, the absence of direction.

Seedance prompt engineering with a video reference: the per-beat recipe

The unit of direction is the beat, and a beat is a cut in the source, never an arbitrary timestamp. @Video1 is a whole-file reference with no timestamp parameter, so the cut itself is your only addressing mechanism: one beat per file, one submission per beat.

Find the source's own cut points with scene detection:

ffmpeg -i source.mp4 -vf "select='gt(scene,0.3)',showinfo" -f null - 2>&1 \
  | grep showinfo | grep -o "pts_time:[0-9.]*"

Each pts_time is a cut the original editor chose. Cut each beat to its own file, at or above the 409,600-pixel reference floor from Chapter 5 (960x540 is 518,400 pixels and passes):

ffmpeg -ss 12.4 -to 16.1 -i source.mp4 -vf "scale=960:540" -an -y beat-04.mp4

Then write the prompt. The formula this pipeline banked in its working playbook: subject plus its motion, background plus its motion, camera plus its motion, with the most important content first and concrete descriptions replacing abstract ones. The per-beat recipe that shipped v06 maps onto that formula as five lines:

ACTION   what happens in this beat, concretely, in the order the ref shows it
FRAMING  shot size and camera position relative to the axis of action
MOOD     light, atmosphere, energy
SCENE    "in the environment of @Image3"
MOTION   "Motion follows @Video1."

A beat prompt in that shape looks like this:

New Eyes slips Ellie's high kick and answers with a spinning backfist;
she staggers back into guard. Medium-close, waist-up, camera on the
right side of the axis. Hard noon sun, dust hanging in the air.
New Eyes is @Image1, Ellie is @Image2, in the environment of @Image3.
Motion follows @Video1.

The @Image1 / @Video1 citations resolve by position: Image n is the nth item of type image_url in the content array, counted from 1 in array order. Asset ids are not addressable in the prompt. The array itself is documented in Create a video generation task, and the full per-beat request looks like this:

{
  "model": "dreamina-seedance-2-0-260128",
  "content": [
    {
      "type": "text",
      "text": "New Eyes slips Ellie's high kick and answers with a spinning backfist; she staggers back into guard. Medium-close, waist-up, camera on the right side of the axis. Hard noon sun, dust hanging in the air. New Eyes is @Image1, Ellie is @Image2, in the environment of @Image3. Motion follows @Video1."
    },
    { "type": "image_url", "image_url": { "url": "asset://<your-asset-id>" }, "role": "reference_image" },
    { "type": "image_url", "image_url": { "url": "asset://<your-asset-id>" }, "role": "reference_image" },
    { "type": "image_url", "image_url": { "url": "data:image/png;base64,<scene-plate-base64>" }, "role": "reference_image" },
    { "type": "video_url", "video_url": { "url": "https://<your-tunnel-host>/beat-04.mp4" }, "role": "reference_video" }
  ],
  "resolution": "480p",
  "ratio": "16:9",
  "duration": 4,
  "watermark": false
}

Submission mechanics are unchanged from the earlier chapters and follow the video generation tutorial: POST /contents/generations/tasks, get a task id back, poll. What changed is that every beat now carries its own text block instead of sharing one.

Duration matched to the cut: the 4-second floor

Seedance 2.0 accepts duration values of 4 to 15 seconds, or -1 for auto; 1.0 Pro and 1.0 Pro Fast take 2 to 12, and 1.5 Pro takes 4 to 12 or -1 (parameter reference: Create a video generation task). The source fight's beats ran 2.0 to 3.7 seconds. That mismatch is the single root cause of the v05 rough look.

Rendering --duration 5 against a 2.5-second reference beat does not error and does not time-stretch. The model performs the 2.5 seconds of choreography it was given, then improvises the remaining tail: invented poses and, in one logged case, an invented phone prop. Trim points chosen later then land mid-move because the real choreography ended long before the clip did.

The rule that fixed it: set duration to the reference's usable length, subject to the floor. The first row below is the logged v05 case; the second is what the same arithmetic predicts for the longest beat in this source.

Alignment is the iron rule: the ref drives, the prompt steers

One failure in this arc is worth more than any success, because it defines the boundary of what a prompt is allowed to do.

A takedown beat used a grappling reference clip, and the prompt added an instruction to keep the two fighters clearly separate, never overlapping, to avoid the body-fusion artefact. The model's response was to abandon the choreography entirely and invent generic motion. It did not partially comply. Asked to follow a grapple and simultaneously forbidden from grappling, it dropped the reference and freelanced.

The banked rule: the prompt must never contradict the video reference, because the video drives the motion. Control what the ref contributes by trimming the source file, never by fighting it in text. The corrected takedown render used the same reference cut to end before the clinch, with a normal non-contradicting prompt over the strike-and-drive footage, and came back clean.

The productive direction of the same rule: the ref gives the motion, the engineered prompt gives the outcome the ref alone will not reach. The source's knockout would not transfer through motion reference alone; driven purely off the ref it came back as more standing boxing, because KO poses and prone bodies do not survive the transfer. The render that worked combined the drive-flurry reference with a prompt engineered toward the actual outcome, that this is the knockout and Ellie drops, and captured the full knockout in one continuous shot: the drive flurry came from the reference, and everything past the final blow came from the prompt. It beat a prompt-only knockout attempt outright. Alignment held throughout: nothing in the prompt contradicted what the reference showed, the prompt only extended it past the point where the reference stopped.

For beats the ref cannot express at all, ground finishes and held poses above all, the method is custom-prompt-from-reference-frames: step through the source's frames, describe what actually happens, write the prompt from that description, and render the beat with no motion reference attached. The first custom-prompt-only KO attempt cost ~$1 and read as a clean knockout on the first try.

Motion language: write the process, not the state

A still image is a state. Motion is a process. Prompts written as states ("Ellie is knocked out on the ground") give the model an endpoint with no path; prompts written as processes give it the path.

For custom-prompt beats, timestamp prompting is the single most useful technique in the motion-language toolkit:

[00:00-00:02] Ellie circles left, guard up, weight on the balls of her feet.
[00:02-00:03] New Eyes closes the distance behind a straight right; she covers.
[00:03-00:04] The final hook lands; her knees buckle and she drops to the sand.

Rules that earn their place in every directed prompt:

• Structure each action around its impact. For a 10-second beat: roughly 40 to 50 percent anticipation, a single instant of impact, then 45 to 55 percent settle. Impacts that arrive without wind-up read as glitches.
• Ease everything. Nothing starts or stops at full speed. "Accelerates out of the turn", "decelerates into the stance".
• Never animate the light. Animate its cause. Prompt the torch passing, the door opening, the cloud crossing the sun.
• Use human timings. A blink is 100 to 150 ms, a micro-expression 40 to 200 ms, a deliberate expression change 0.5 to 2 seconds, a breath cycle 3 to 5 seconds. Prompted faces that ignore these run uncanny.

Stylised VFX obeys the same division of labour as everything else in this pipeline. Seedance 2.0 renders prompted effects that 1.0-pro flattens: the verbatim prompt "crackling glowing battle-aura (anime Nen-style), golden-white power wisps, heat-haze distortion" produced visible energy flares and a full aura glow on 2.0, while 1.0-pro, in this arc's testing, rendered photoreal motion only and corrected stylised action back to normal movement. Both generations flatten prompted speed-lines and after-images. Those belong to the edit, and Chapter 8 builds them with lagfun where they are deterministic.

The proof run: eight directed beats, one batch

The v06 pass is the cleanest render batch in the entire production record, and it was also the most demanding prompt-side. Eight beats, each with its own directed prompt and matched duration, all submitted in parallel over one hosted tunnel, then polled as eight independent task ids against the states documented in Retrieve a video generation task. All eight succeeded first try. Total cost $3 to $4 at 480p, on a rate card where a 5-second shot with an image reference present is $0.35.

Two comparisons frame what direction bought:

• An earlier 11-beat parallel batch had one beat fail twice; it was skipped rather than retried a third time. The directed batch had zero failures.
• The generic 11-shot pass cost ~$5 and produced footage that needed heavy rescue in the edit. The directed 8-beat pass cost less and produced footage that cut together at a visibly higher tier.

Directed prompts did not add risk to the batch. They removed it, because a prompt aligned with its reference gives the model less to invent.

On an individual ModelArk account the non-4K concurrency limit is 3, so a batch of eight does not run eight-wide: three render while the rest sit in queued. Over-limit submissions queue rather than fail; it is the separate request-rate limits that answer with 429s per the burst-traffic best practices. The submission loop and the polling harness get their full treatment in Chapter 9. The prompt work in this chapter is what makes that batch worth running.

Chapter 8 takes these renders into the edit: settle-cuts placed by per-frame motion measurement, impact flashes, framing recuts, and the two failure modes post-production can erase without a single re-render.