Image to video with AI: first frames, reference images, and the difference
An image handed to a video model can be two completely different instructions — the frame it starts on, or the look it borrows. Here is how to tell them apart, why the order you send them in matters, and how to wire a still into a clip.
4 min read
"Image to video" is one label covering two jobs that behave nothing alike. In one, the picture is the first frame — the clip literally begins there and moves on from it. In the other, the picture is a reference — nothing in the output has to match it exactly, but the subject, palette or style is borrowed. Mixing the two up is the most common reason an image-to-video result feels like it ignored the image.
First frame: the clip starts here
Send one image as a first frame and you have pinned frame zero. The model's job is to animate away from it. This is what you want when you are continuing a shot, when a still was expensive to get right, or when the clip has to cut cleanly from something that came before.
Two practical consequences. First, the frame's aspect ratio should match the clip's — 16:9 in, 16:9 out — because anything else gets fitted, and the fit is not always the crop you would have chosen. Second, an image-to-video call tolerates an empty prompt: the picture already says what the shot is. You will usually still want a sentence about motion, but it can be short, because it no longer has to establish the scene.
References: the look, not the frame
Reference images are how you say "this character", "this product", "this palette" without describing them in prose. The output does not begin on any of them. Reference mode is worth reaching for when you are generating several clips that have to look like they belong to the same thing — the same jacket, the same room, the same brand blue.
The catch is that you can generally send a set of them (Gemini's video model takes up to seven), and the model has no field that says what each one is for. There is no role, no tag, no caption in the request — an image arrives as bytes and a mime type and nothing else.
Which means position is the only handle you have
If the request cannot say "this one is the jacket", the only thing distinguishing image three from image five is that one is third and one is fifth. So the way to reference them is by that position, in the prompt text: the images go out in a fixed order, the prompt writes an "image 1 / image 2" preamble, and then the sentence can name them — {image1}, {image2}, {image3}.
"The woman from {image2} walks through the lobby in {image3}" is a real instruction. "The woman walks through the lobby" with three images attached is a guess.
Wiring it up on the canvas
In QueyFrames an image is not a file you attach to a request — it is a node's output, so it can be generated, uploaded or dropped in from the desktop, and re-run without touching the video node at all.
- Make the still first. An image node generates one, or takes an upload; dragging a file from the desktop straight onto the canvas creates the node already holding it.
- Wire it into the video node. The image slots grow themselves — one row per wired image in send order, with a free slot trailing the last one, so there is no count to keep in step and the row you see is the position the model gets.
- Say what the set is. A caption chip on the list flips between frame + refs and refs only — the one thing position genuinely cannot express, so it is stated rather than inferred.
- Write the prompt with the tokens in it, and check the resolved text before spending a generation on it.
Because the slots are claimed by the wire rather than by the bytes, you can draw the connection and write {image2} into the prompt before the image node upstream has ever been run. That is the order people actually work in — rough out the graph, then fill it — and it means the prompt is finished before the first credit is spent.
The flip side: a slot that has a wire but no bytes yet is a gap, and generating with a gap would shorten the array and slide every later image down a position. So the run waits for the pending producer rather than sending something that would re-point half the prompt.
What image-to-video will not fix
- Length. A first frame does not buy you a longer clip. The duration range is the model's, whatever you start from.
- Pacing. Pinning frame zero says nothing about what happens at 0:04. That still has to be written, and if the timing exists elsewhere in the project it is better placed by wiring keyframes into the prompt than by retyping seconds.
- Identity across a whole sequence. References hold a look together better than prose does, but they are not a character lock. Generate more clips than you need and cut, rather than fighting for an exact match on every one.
- A bad still. Motion does not rescue a frame that was wrong. It is nearly always cheaper to re-roll the image than to re-roll the video — image generation is a fraction of the cost of the clip it seeds.
For the model-side detail — what text-to-video, image-to-video and reference modes each accept — see how QueyFrames uses Gemini AI for video.
Keep reading
- 5 min read
How to write AI video prompts that control pacing
Text-to-video models will give you the shot and invent the timing. Here is how to write the timing into the prompt — naming beats in order, pinning them to seconds, and letting wired keyframes place themselves in the sentence.
- 4 min read
Node-based AI video editing vs the single prompt box
One text box gives you a clip and no way to change part of it. A graph gives you the intermediate results — the still, the track, the timing — as things you can re-run one at a time. Here is what that buys, and what it costs.
- 5 min read
How to keyframe AI animation: spans, not points
Point keyframes only mean something beside their neighbors. When an AI reads your timing, a span — start, end, what changes — stands alone. How to write them.
Build the graph these posts are about.
QueyFrames is in early access. Create an account to join the list.