Video Models

You Can Now Pick an Actor Up Out of the Frame With Your Mouse: Pika Built a Live Directing Environment, but It's Enterprise-Only

November 19, 20260 views阅读中文原文
You Can Now Pick an Actor Up Out of the Frame With Your Mouse: Pika Built a Live Directing Environment, but It's Enterprise-Only

The short version: On November 18, Pika published a research preview of Pika Enact, a live-streaming video environment built on top of Pika H3 Flex. It emits video and audio in sub-second chunks, which means you can move an actor with your mouse, talk to the cast, or fly a virtual camera around while the scene is still playing. The catch is in the last paragraph of the official Pika post: Enact is a research preview, unavailable for consumer use, and anyone who wants in has to email [email protected].

TL;DR: It doesn't change how good the picture looks — it changes how long you wait to change it

Anyone who has used a video model knows the loop: write a prompt, wait ten to sixty seconds, watch the result, dislike it, rewrite the prompt, wait again.

The problem isn't image quality. It's that feedback is slow. On a real set, moving someone half a step to the left takes a sentence. In a prompt box, you write "a bit more to the left" and hope.

Pika Enact is an attempt to collapse that loop into "say it, and the picture changes on the spot." The company frames it as mimicking what it feels like to direct on set, and it ships three modes of interaction:

  • Live Staging: drag an actor or object on screen with your mouse and its position changes.
  • Live Actor Direction: type or speak a note to the cast; they act on it, and they remember what you asked for last time.
  • Live Camera Control: push, dolly, or orbit the camera with your mouse while the scene keeps generating.

How it actually works: three paths, each with its own numbers

One: drag to move a person, at 1.8x playback speed on a single server

This is not "pause, then re-render." Pika states it plainly: the video keeps playing while you drag, and the model generates at the same time. On a single server the model generates 1.8x faster than the video plays. In the low-latency setting, picture response — decoding included — is under 0.4 seconds on an H100.

Following the cursor is the easy part. The hard requirement is strict consistency before and after the drag: the spot the person left can't become a hole, lighting has to follow, and the pose needs to carry weight. Real footage can't supply paired data like that, so Pika built a physics-driven synthetic data engine. Starting from an untouched clip, it animates a 3D stand-in along the drag path to simulate natural weight and momentum, renders photorealistic target clips with its video models, and automatically logs mouse positions and button states. The result is scalable, zero-labeling training pairs where only the dragged subject changes.

On the engineering side, generation is sliced into streamed chunks: 0.7 seconds per chunk in standard mode, 0.17 seconds in low-latency mode, with a sliding key-value cache that retains the prompt, the source video, the first chunk, and the four most recent chunks. Positional embeddings are re-based so sessions can run indefinitely at constant cost. To keep latency down without drifting, sampling is cut from twenty steps to a few using self-forcing distribution matching distillation.

Two: give notes to actors, at 1.75x playback speed, rehearse before you roll

The second mode feels closer to real directing. You give a note — "move closer to her," "tell him why you're upset," "make it a kitchen at night" — and the people in the video play it out. They answer you in character, in their own voices. Pika's figure: picture and sound generate 1.75x faster than they play.

Structurally it's one part that thinks and one part that performs. A script language model reads your notes, the opening frame, and session memory, then translates directorial intent into structured beats through tool calls — action, dialogue, emotion, gaze, lighting. The real-time video model performs those beats. When nothing is queued, the scene doesn't freeze; it holds with subtle breathing and blinking.

One detail worth knowing: because the opening anchor frame pulls an idle scene back toward its starting state, every beat has to specify actor gaze, and a real-time director silently injects periodic hold beats to preserve lighting or staging changes.

During rehearsal the actors acknowledge the camera and talk with you, and every line is logged verbatim. When you call Action, the script model rewrites that exchange into one seamless performance shot fresh from the first frame. That's the real difference from simply leaving the recorder running: rehearsal doesn't contaminate the take.

Three: fly the camera, on eight B300s, 640x480, at 1.5x real time

The third mode doesn't need existing footage — it generates picture and sound from scratch. An appearance image anchors the cast and set, and a lightweight 3D motion proxy (an untextured Blender viewport, for example) guides the camera path.

Pika's hardware and numbers here are specific: an 8x NVIDIA B300 node, 640x480 at 24 fps, initial frames rendered in 0.51 seconds, and subsequent chunks generated at 1.50x real time. At steady state it computes 0.708 seconds of content in 0.473 seconds. To get there, static appearance prefixes are cached and reused across chunks, and latents go through a streaming video decoder.

The training data is synthetic here too: an automated Blender-to-video pipeline that simultaneously records multi-angle camera trajectories, untextured 3D motion reference, target RGB frames, and spatial audio.

The base model is Pika H3 Flex, but Enact itself is not open

Two things need separating:

  • Pika H3 Flex launched October 29 as a productized model, post-trained on MiniMax H3. It supports up to 12 references, five separate audio stems, finishing at up to 4K, up to 15 seconds, and six denoising steps against its 50-step teacher. Its weights are open.
  • Pika Enact is an experimental real-time environment built on top of it. The post stresses three separate times that it is a research preview and unavailable for consumer use. There are no weights, no API, and no pricing.

Put differently: you can run H3 Flex yourself. If you want the "direct while it plays" interaction, right now you go through sales.

Who it's for

  • Teams doing storyboards and previs: this is the most direct fit. Changing a blocking used to mean a re-generation; now it means a drag, and the number of versions you can try in an hour isn't in the same league.
  • Agency and short-film prep: when the client says "move him a little right," you can drag it live instead of going back to rewrite a prompt and shipping another version.
  • Virtual production and previz specialists: driving the camera with a 3D proxy maps onto how previz already works.
  • Anyone who wants to build a product on it: not yet. Keep the enterprise address.

Not a good fit for everyone

  • Anyone delivering finished footage: 640x480 at 24 fps is a previs spec, not a delivery spec. The H3 Flex line reaches 4K, but that's a different product.
  • Anyone who wants to self-host: Enact's weights don't exist publicly. Reproducing it means an 8x B300 class machine plus that synthetic data engine.
  • Budget-sensitive solo creators: no published price means nothing to evaluate.

Pitfalls: four things easy to misread

  • 1.8x and 1.75x aren't the same number. The first is the generate-to-playback ratio for Live Staging on a single server; the second is the audio-video ratio for Live Actor Direction. Don't quote them interchangeably.
  • "Under 0.4 seconds" has conditions: H100, low-latency setting, decoding included. It is not "0.4 seconds on any machine."
  • The camera mode's resolution is fixed. Pika only published 640x480 at 24 fps for it; don't assume high definition.
  • Research preview is not a product. The post's own words are "unavailable for consumer use," and that sentence decides how to read the announcement: a direction, not a tool you can adopt today.

Cost and time

  • Money: no pricing has been published. Access means emailing [email protected] to discuss integration, which in practice means enterprise custom quoting.
  • Hardware, for scale: the camera mode runs on an 8x B300 node; staging runs at 1.8x real time on a single server, with latency tested on an H100. This is not consumer-GPU territory.
  • Time: the only time you can spend today is ten minutes reading the post and deciding whether this direction touches your workflow. Actual use depends on a sales reply.

How to use it: there is exactly one path right now

The only documented route is to contact [email protected] about a trial or integration. There is no web entry point, no API key, no documentation site. So the practical steps look like this:

  1. Describe the scenario you need it for (previs? ad boards? virtual production?), because the hardware requirements across the three modes are wildly different.
  2. Say which mode you need. Staging, actor direction, and camera control are three separate capabilities, not one bundle.
  3. Ask what the delivery looks like: a web workspace, or an interface you wire in yourself.
  4. If you're integrating, press on output specs (resolution, frame rate, clip length) and concurrency limits.

If you just want a feel for it, the demo videos in the post are the free, no-application part. Watch those before you write the email.

Next steps

  • First, watch the three demos before judging. Each mode has footage with a red dot for a held mouse button and a white ring on release. That's the only way to judge whether the "live" feeling is real; a spec table won't tell you.
  • Second, separate this from post-production fixing. The camera mode generates from scratch; it doesn't edit existing footage. If what you want is to fix a clip you already have, that's a different category of tool (background replacement, relighting, and so on).
  • Third, if the direction fits, send the email now. Research-preview access usually comes with joint testing, and talking early is cheaper than waiting for general availability.

Sources

  • Pika blog, "Research Preview: Pika Enact" (November 18, 2026): https://pika.art/blog/pika-enact
  • Pika blog, "Introducing Pika H3 Flex" (October 29, 2026): https://pika.art/blog/pika-h3-flex
  • Pika blog, "Pika H3 Flex Technical Deep Dive" (October 29, 2026): https://pika.art/blog/pika-h3-flex-research

Sources

Comments

0 published
Log inorsign up to leave a comment.

Loading comments...