Capture a video stream from any element

François Beaufort

Elad Alon

With the Screen Capture API, you can capture the entire current tab. The Element Capture API lets you capture and record a specific HTML element. It transforms a capture of the entire tab, into a capture of a specific DOM subtree, capturing only direct descendants of the target-element. In other words, it crops and removes both occluding and occluded content.

Why use Element Capture?

Considering the requirements of a video-conferencing application can help you understand where Element Capture is useful. If you have a video-conferencing application that lets you embed third-party applications in an iframe, you might sometimes want to capture that iframe as a video and transmit it to remote participants.

Calling getDisplayMedia() and letting the user choose the current tab would transmit the entire current tab. That is likely to transmit people's own video back to them. You could crop this away using Region Capture.

However, what if the presenter engages with the video-conferencing application and some content, like a drop-down list, happens to draw on top of the content that's intended for capture?

Region Capture wouldn't help you there. Part of the drop-down list might end up visible on remote participants' screens.

The fact that Region Capture captures parts of elements in this way (known as occluding content) creates multiple problems:

• Occluding content might obstruct from view the content which the user intended to share.
• Occluding content might be private (think chat notifications).
• Occluding content might be confusing. (For example, a re-layout of the application could briefly bring the remote participants' own videos over the captured-target.)

The Element Capture API solves all of these problems, by letting you target the element you want to share.

How do I use Element Capture?

The captureTarget is an Element on your page which contains the content the user wishes to capture. You want the video conferencing web app to capture captureTarget and share it with remote participants. So you derive a RestrictionTarget from captureTarget. After restricting the video track using this RestrictionTarget, frames on that video track now consist only of the pixels that are part of captureTarget and its direct DOM descendants.

If captureTarget changes size, shape or location, the video track follows along, without requiring any additional input from either web app. Occluding content that appears, disappears or moves around, similarly requires no special treatment.

Review these steps again:

Start out by allowing the user to capture the current tab.

// Ask the user for permission to start capturing the current tab.
const stream = await navigator.mediaDevices.getDisplayMedia({
 preferCurrentTab: true,
});
const [track] = stream.getVideoTracks();

Define a RestrictionTarget by calling RestrictionTarget.fromElement() with an element of your choice as input.

// Associate captureTarget with a new RestrictionTarget
const captureTarget = document.querySelector("#captureTarget");
const restrictionTarget = await RestrictionTarget.fromElement(captureTarget);

Then call restrictTo() on the video track with the RestrictionTarget as the input. Once the last promise resolves, all subsequent frames will be restricted.

// Start restricting the self-capture video track using the RestrictionTarget.
await track.restrictTo(restrictionTarget);

// Enjoy! Transmit remotely.

Deep dive

Feature detection

To check if RestrictionTarget.fromElement() is supported, use:

if ("RestrictionTarget" in self && "fromElement" in RestrictionTarget) {
  // Deriving a restriction target is supported.
}

Derive a RestrictionTarget

Focus on the Element called captureTarget. To derive a RestrictionTarget from it, call RestrictionTarget.fromElement(captureTarget). The returned Promise will be resolved with a new RestrictionTarget object if successful. Otherwise it will be rejected if you have minted an unreasonable number of RestrictionTarget objects.

const captureTarget = document.querySelector("#captureTarget");
const restrictionTarget = await RestrictionTarget.fromElement(captureTarget);

Unlike an Element, a RestrictionTarget object is serializable. It can be passed to another document using Window.postMessage(), for instance.

Restricting

When capturing a tab, the video track exposes restrictTo(). When capturing the current tab, it is valid to call restrictTo() with either null or any RestrictionTarget derived from an Element within the current tab.

Calls to restrictTo(restrictionTarget) mutate the video track into a capture of captureTarget, as though it were drawn by itself, independently of the rest of the DOM. Any descendants of captureTarget are also captured; siblings of captureTarget are eliminated from the capture. The result is that any frames delivered on the track appear as though they were cropped to the contours of captureTarget, and any occluding and occluded content are removed.

// Start restricting the self-capture video track using the RestrictionTarget.
await track.restrictTo(restrictionTarget);

Calls to restrictTo(null) revert the track to its original state.

// Stop restricting.
await track.restrictTo(null);

If the call to restrictTo() is successful, the returned Promise is resolved when it can be guaranteed that all subsequent video frames will be restricted to captureTarget.

If unsuccessful, the Promise is rejected. An unsuccessful call to restrictTo() will be for one of the following reasons:

• If the restrictionTarget was minted in a tab other than the one being captured. (Note that using the "share this tab instead" button, users may change which tab is captured at any given time.)
• If the restrictionTarget was derived from an Element that no longer exists.
• If the track has clones. (See issue 1509418.)
• If the current track is not a self-capture video track.
• If the Element from which restrictionTarget was derived is not eligible for restriction.

Self-capture considerations

When an app calls getDisplayMedia(), and the user chooses to capture the app's own tab, we call that "self-capture".

The restrictTo() method is exposed on any tab-capture video track, and not just for self-capture. But Element Capture is only enabled for self-capture for now. It is therefore advisable to check if the user selected the current tab, before attempting to restrict the track. This can be accomplished using Capture Handle. It is also possible to ask the browser to nudge the user towards self-capture using preferCurrentTab.

Transparency

Video frames the app gets through getDisplayMedia() do not include an alpha channel. If an app sets a partially transparent capture-target, stripping the alpha channel has some possible consequences:

• Colors might change. Partially transparent target-elements drawn over a light background might appear darker when the alpha channel is removed, and those drawn over a dark background might appear lighter.
• Colors that were invisible or imperceptible to the user when the alpha channel was set to its maximum, would appear once the alpha channel is removed. For example, this could lead to unexpected black regions in the captured frames, if the transparent sections had the RGBA code rgba(0, 0, 0, 0).

Ineligible capture targets

It is always possible to start restricting a track to any valid capture-target. However, frames won't be produced under certain conditions, for example, if the element or an ancestor is display:none. The general rationale is that restriction applies only to an element that comprises a single, cohesive, two-dimensional, rectangular area, whose pixels can be logically determined in isolation from any parent or sibling elements.

One important consideration for ensuring the element is eligible for restriction, is that it must form its own stacking context. To ensure this, you could specify the isolation CSS property, setting it to isolate.

<div id="captureTarget" style="isolation: isolate;"></iframe>

Note that the target element can toggle between being eligible and ineligible for restriction at any arbitrary point, for example, if the app changes its CSS properties. It's up to the app to use reasonable capture targets and avoid changing their properties unexpectedly. If the target element becomes ineligible, new frames will simply not be emitted on the track until the target element again becomes eligible for restriction.

Enabling Element Capture

The Element Capture API is available in Chrome on desktop behind the Element Capture flag and can be enabled at chrome://flags/#element-capture.

This feature is also entering an origin trial from Chrome 121 on desktop, which allows developers to enable the feature for visitors to their sites to collect data from real users. See Get started with origin trials for more information on origin trials.

Security and privacy

To understand the security tradeoffs, check out the Privacy and Security Considerations section of the Element Capture specification.

Chrome browser draws a blue border around the edges of captured tabs.

Demo

You can play with Element Capture by running the demo on Glitch. Be sure to check out the source code.

Feedback

The Chrome team and the web standards community want to hear about your experiences with Element Capture.

Tell us about the design

Is there something about Region Capture that doesn't work as you expected? Or are there missing methods or properties that you need to implement your idea? Have a question or comment on the security model?

• File a spec issue on the GitHub repo, or add your thoughts to an existing issue.

Problem with the implementation?

Did you find a bug with Chrome's implementation? Or is the implementation different from the spec?

• File a bug at https://new.crbug.com. Be sure to include as much detail as you can, and simple instructions for reproducing. Glitch works great for sharing quick and easy repros.

Helpful links

• Specification
• Explainer
• Intent to experiment
• ChromeStatus.com entry

Acknowledgments

Photo by Paul Skorupskas on Unsplash