Motion transfer: video refs, line-art, and moderation

Chapter 5 of 9. Prerequisite: Chapter 4. You need at least one registered character asset with status warehoused, because this chapter puts it in motion. A reference clip supplies the choreography and the asset supplies the identity, and the API rejects most first attempts at wiring the two together. Every rejection below is quoted verbatim so you can grep for yours.

Chapter 4 ended with a character that renders the same face every time. A face that stands still is a portrait. This chapter is about choreography you cannot write: a spinning kick, a dodge, a guard, the timing between two bodies. You do not prompt that. You show the model footage of it and let the asset carry the identity through. The pain is that the constraints on that footage are enforced by the validator and moderation layer rather than documented as a checklist, so this chapter is the checklist.

Video reference motion transfer: the request that works

Motion transfer on Seedance means one thing at the API level: a video_url item in the content array with "role": "reference_video", sitting next to your character assets. The Seedance 2.0 launch announcement names video as one of the model's four input modalities, and the Dreamina Seedance 2.0 series tutorial is the official reference for how those modalities combine. The video generation tutorial puts it plainly: the models "can quickly generate high-quality video clips based on multimodal content such as text, images, videos, and audio input by users".

The submit is the same async task endpoint as every other render in this course, POST /contents/generations/tasks against the ap-southeast-1 base URL documented in the ModelArk overview. Here is the full body of a working two-character motion-transfer request, per the create-task API reference:

{
  "model": "dreamina-seedance-2-0-260128",
  "content": [
    {
      "type": "text",
      "text": "New Eyes and Ellie spar in a bare test chamber. Ellie takes the taller performer's kicks, New Eyes takes the counters. Motion follows @Video1."
    },
    {
      "type": "image_url",
      "image_url": { "url": "asset://<your-asset-id>" },
      "role": "reference_image"
    },
    {
      "type": "image_url",
      "image_url": { "url": "asset://<your-asset-id>" },
      "role": "reference_image"
    },
    {
      "type": "video_url",
      "video_url": { "url": "https://<your-public-host>/beat-03.mp4" },
      "role": "reference_video"
    }
  ],
  "resolution": "480p",
  "ratio": "16:9",
  "duration": 5,
  "watermark": false
}

The response is {"id": "cgt-<task-id>"} and the rest is the poll-and-download loop from Chapter 2. Four details in that body are load-bearing:

• role is mandatory on every reference item on 2.0. The first 2.0 call in this pipeline omitted it and got HTTP 400 role must be specified for image contents. Images take reference_image, video takes reference_video, audio takes reference_audio. First and last frames use first_frame and last_frame instead, and you will see in the next section why those cannot appear here at all.
• Citation is by array position, counted per type. @Image1 is the first image_url item in the array, @Video1 the first video_url item, counted from 1 in array order. Referencing by asset id is unsupported. In the CLI, array order is flag order, so the order you pass --image flags is the order the prompt counts them.
• The images are the WHO, the video is the HOW. The prompt line Motion follows @Video1 is the whole contract: identity from the registered assets, choreography from the footage.
• The budget is text plus 0-9 images plus 0-3 videos plus 0-3 audios on the 2.0 series, per the 2.0 series tutorial. Text-only-audio and audio-only combinations are not accepted.

The same render through this course's CLI wrapper:

python3 byteplus_seedance.py "$PROMPT" \
  --model dreamina-seedance-2-0-260128 \
  --image "asset://<your-asset-id>" \
  --image "asset://<your-asset-id>" \
  --video "https://<your-public-host>/beat-03.mp4" \
  --ratio 16:9 --resolution 480p --duration 5 \
  --no-watermark --out ./renders

The wrapper guards the two ways this call goes wrong on the wrong model, with these exact errors:

--video (reference_video) is a Seedance 2.0 feature; use --model dreamina-seedance-2-0-260128
Seedance 2.0 accepts at most 3 reference videos

Receipts before we go further, because this is cheaper than it sounds. The first motion-transfer proof in this arc, one character asset as @Image1 plus one line-art reference clip as @Video1, produced a 5s 720×1280 clip with the likeness correct and the choreography transferred (fighting guard, high kick, spins, dodges) for about $1. A two-character request of this shape at 480p cost about $0.40 in this arc. At the 2.0 rate card that is the pattern to expect: a 5s shot with image refs runs $0.35 at 480p and $0.77 at 720p, so iterate at 480p and spend 720p on keepers.

The four 400s: every rejection, verbatim

The create-task reference documents the parameters. What it does not give you is the constraint set the validator enforces once reference media enters the request. This pipeline hit four distinct 400s, and each one changed the architecture. Grep-ready strings, one per rejection:

1. Frame continuity and reference media are mutually exclusive.

first/last frame content cannot be mixed with reference media content

The planned architecture was frame-chaining: render shot 1, take its last frame, feed it as shot 2's first_frame alongside the character assets and the motion ref. This 400 killed it outright. Seedance treats first/last-frame image-to-video and reference_image/reference_video as mutually exclusive modes. With assets and a motion ref in the request there is no frame continuity, full stop. That leaves exactly two shapes for a scene: one continuous render (2.0 goes up to 15s in a single take) or independent shots hard-cut together, with identity held across the cuts by the registered assets. The practice film used both, and Chapter 8 is about making the hard cuts read as coverage.

2. camera_fixed is rejected whenever reference media is present.

HTTP 400 "camera_fixed is not supported for dreamina-seedance-2-0 in r2v, must be empty"

The parameter is documented for plain text-to-video, and the error string scopes the rejection to r2v mode. Add any reference media, which for this pipeline is always, and the request is in r2v mode where the parameter must be absent entirely. Camera consistency is therefore not solvable at render time in reference mode. It moves downstream into the edit, where Chapter 8's crop-and-reframe recipes live.

3. The reference video has a pixel floor of 409,600.

video pixel count must be >= 409600

Width times height must clear 409,600 pixels. A 640×360 beat is 230,400 pixels and was rejected with exactly that string; the same clip re-encoded to 960×540 is 518,400 pixels and passed. For 16:9 refs, encode at 854×480 (409,920) or larger and you are safely over the floor. This floor is invisible until it fires, and it fires late, after you have cut and treated a whole batch of beats, so bake the check into your prep script: ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of csv=p=0 beat.mp4 and multiply.

4. The reference video must be a web URL.

InvalidParameter: reference_video must be provided as a web url

Local images inline happily as base64 data URIs. Video does not. A base64 data URI in video_url is rejected outright, which means motion transfer forces a hosting workflow on you whether you want one or not. That workflow gets its own section below.

Moderation: real faces hard-block, line-art clears

The moderation layer sits in front of everything above, and for motion transfer it is the first wall you hit, because useful choreography footage tends to contain real people.

The block is absolute and it is face-driven. Uploading a real face PNG directly as frame content returned:

HTTP 400 InputImageSensitiveContentDetected.PrivacyInformation

with the explanation "input image may contain real person". The instructive half of that incident: the same face, referenced through its registered asset:// id, passed with no privacy block at all. Asset trust and raw-upload trust are different trust levels in both directions. That is why Chapter 4's registration flow exists, and it is why the WHO side of motion transfer is always an asset id, never a raw photo.

Video refs face the same detector, so a real-person reference clip must be de-photographed before the API will take it. The treatment that won, after cheaper treatments failed, is edge-detect line-art:

ffmpeg -i beat-03.mp4 \
  -vf "scale=480:-2,edgedetect=low=0.1:high=0.3" \
  beat-03-lineart.mp4

That produces white contour lines on black. It passes the real-face filter and still reads as clean motion, because line-art kills faces but preserves silhouette, height and screen position, which is exactly the information the model needs to map performers onto your characters. The scale=480:-2 shown here came from a portrait 9:16 source where 480 wide still clears the pixel floor; on a 16:9 landscape source, scale so the output stays at or above 854×480 or rejection 3 fires on the treated file.

The failed alternatives are worth recording so you do not re-run the experiment. Cel-shade and colormix treatments keep recognisable faces, so they stay unsafe. Luma-silhouette leaves partial or headless faces in some frames, also unsafe. Line-art won because it removes the face entirely while losing none of the choreography.

Two more moderation findings from the same arc:

• Animated and CGI reference footage passes raw. No line-art step needed at all. The detector is after real people, so a stylised source goes straight in, and one 480p validation render proved anime choreography maps cleanly onto photoreal actors.
• The detector also flags poses, and line-art clears that too. Midway through a full-fight render batch, one segment failed with InputVideoSensitiveContentDetected. Nothing about faces this time: a grapple pose read to the classifier as sensitive. The same edgedetect treatment on that segment cleared it. Line-art is a general moderation solvent for reference footage, for faces and for poses.

One boundary note, and it is binding on everything this chapter teaches. Outputs derived from third-party footage stay personal and non-commercial. The line-art abstraction helps moderation; it does not change whose choreography it is. This course teaches the prompts and the rules and claims no rights over anything derived from a reference clip.

Hosting: a server, a tunnel, and beats cut to files

Rejection 4 said reference_video must be provided as a web url, so BytePlus has to be able to fetch your clip over public HTTPS. The workflow that serves this course:

1. Cut the source into beats, one file each

   @Video1 is a whole-file reference. There is no timestamp parameter, no in/out points, no way to say "seconds 12 to 19 of this file". So every beat becomes its own file: ffmpeg -ss <start> -to <end> -i source.mp4 -c copy beat-03.mp4, then re-encode as needed to clear the 409,600-pixel floor. A beats directory with one file per beat is the unit of work for the rest of the course.

2. Treat each beat for moderation

   Line-art the real-person beats with the edgedetect chain above. Leave animated source raw. Check every output file against the pixel floor with ffprobe before it goes anywhere near a request.

3. Serve the beats directory locally, then tunnel it

   A hardened local static file server on the beats directory, then a quick tunnel in front of it: cloudflared tunnel --url http://127.0.0.1:<port> hands you an unguessable public HTTPS URL with no account and no DNS. Any public HTTPS host you already have works equally well; the tunnel is just the fastest path from a local directory to a fetchable URL.

4. Keep the tunnel up until the last task succeeds

   The fetch is not guaranteed to happen at submit time. Tasks queue and run asynchronously, so the URL must stay reachable through the whole poll, until every task you submitted against it reports succeeded on the retrieve-task endpoint. Tearing the tunnel down when the submits return, rather than when the polls finish, is a self-inflicted failure mode. Tear it down after the last download.

This hosting setup is also what unlocks Chapter 3's parallel pattern at motion-transfer scale: every beat file behind one tunnel, every render submitted at once. Account limits shape that batch: this pipeline's individual account allowed 3 concurrent non-4K generations, and in its runs extra submits sat in queued rather than failing. Treat that queue behaviour as a lived observation rather than a documented guarantee; what ModelArk's rate-limit best practices commit to is that the API enforces rate limits on several dimensions, including concurrent requests, and responds with a 429 when a limit trips. Either way, a still-pending task is another reason the tunnel outlives the submit loop.

Mapping and drift: who is who, and when to cut

With the request accepted, the remaining problems are creative-technical: making sure the right performer maps to the right character, and cutting before the model starts inventing.

Mapping is automatic and it works, with rules. A two-performer reference clip plus two registered assets produced a genuine two-person fight with both likenesses holding and the roles correctly mapped: Ellie inherited one performer's moves and kick, New Eyes the other's counters, for about $0.40 at 480p. The rules that make that reliable:

• Match ref count to distinct people. One asset against a two-performer clip gives you one character out; the second performer is not cast. Expected behaviour, proven early in this arc.
• Cite by position, role and action, never by number alone. "The fighter on the left", "the taller one", "the attacker throwing the kick". @Image1 tells the model which face; the positional language tells it which body.
• Order is flag order. The first --image is @Image1. Keep a fixed casting order across a whole batch so your prompts stay copy-pasteable.
• Similar builds swap. Two performers of similar height and frame will sometimes trade identities between renders. Verify every render on a contact sheet and expect 2-4 regeneration cycles on multi-character shots.

The contact sheet is one command and it is how every mapping claim in this chapter was checked:

ffmpeg -i clip.mp4 -vf "fps=2,scale=420:-1,tile=4x3" sheet.png

Drift is what happens when the ref runs out. The 12s continuous two-character fight in this arc cost about $1.30 and covered the full choreography reference, and its last ~2 seconds are a lesson: the ref ended, the model kept going, improvised a solo pose and invented a phone prop from nowhere. The shipped cut trims it to a clean 9s. The rule that came out of it: set --duration to the ref's usable length, within 2.0's 4-15s window. When the choreography runs out the model improvises, and improvisation at the tail of a fight scene means invented props and lost characters. Chapter 7 turns this from a trim rule into a prompting discipline, and Chapter 8's settle-cut technique finds the exact frame to cut on.

Total motion-transfer R&D to get from first proof to a working two-character fight pipeline: about $2.70. Every constraint in this chapter was purchased inside that number, which is the argument for learning them from a chapter instead.

What's next: locking the world around the motion

You can now put registered characters through real choreography. What you cannot yet do is put them somewhere. Every render so far happens in whatever environment the model hallucinates around the prompt, and it hallucinates a new one per render. Chapter 6 adds the third reference type to the array, the scene plate. It walks through the single-render proof that a plate fed as @Image3 locks the environment inside a shot, and the sequence-scale proof that the lock holds across independent renders, which is the load-bearing fact for multi-shot films.