ANR Tools

ANR AI Planar Tracker

Planar tracking of flat surfaces from masks you draw, with layered masks, a correction layer, and exports to Roto, a four-corner Tracker or a Card and Camera.

  • Trackers
  • Trackers pack
  • Tracker models
  • 2 inputs
  • 96 controls

Overview

The AI Planar Tracker follows a flat surface (a screen, a sign, a wall, a page) from a mask you draw around it in the viewer. Each frame it fits one plane to what it sees inside the mask, and moves four surface corners with it: the corners where an insert will sit.

Choose how the surface is found in Tracking Method. Algorithm (Enhanced Non-AI) matches image features on the CPU and refines the alignment, with an optional crop box that follows the track. AI tracks points inside the mask with the AI point tracking model on the GPU, then fits the plane. The image-matching AI method (the last entry of the menu) matches the whole surface with an AI model on the GPU.

A node can hold several masks as layers, each with its own colour and track. Track them one at a time, each separately, or several masks together as one surface (Track). A separate Adjust Track layer lets you key corner corrections over the raw track without changing it, and the post-track refinements make cleaned-up copies you can compare with the original.

When the track is right, the Export buttons build ordinary Nuke nodes: a tracked Roto of the mask, one Roto with every shape, a Tracker with the four corners, or a Card and a static Camera rendered with a ScanlineRender, ready for an insert.

Inputs

SourceOcclusionAI Planar TrackerOutput
  1. Sourcerequired

    The plate to track.

  2. Occlusionoptional

    A matte of whatever passes in front of the surface: where its alpha is above half, the pixels are ignored by the match. Used while Use Occlusion Input is on.

Output. The plate, with the masks, the tracked surface and the tools drawn in the viewer. The results are the keys on the node and the nodes the Export buttons build.

Quick start

  1. Add ANR AI Planar Tracker and connect the plate to Source. Go to a frame where the surface is clear.
  2. The node starts in drawing mode: click points around the surface in the viewer. Click the first point again, or press Enter, to finish the mask.
  3. The four surface corners start at the mask's bounds. Drag them onto the corners of the area your insert should cover (Fit Surface to Mask puts them back).
  4. Choose Tracking Method and Motion (Perspective for a flat surface seen from a moving camera).
  5. Press ▶ to track from the current frame to Last. Go back to the frame you drew on and press ◀ to track to First. ■ stops and keeps the frames already tracked.
  6. Scrub to check. Fix drift on the Adjust Track tab: turn on Adjust Track, drag the corners in the viewer on the frames that need it.
  7. Press an Export button, for example Create Tracker (4 Corners) or Scaling Card + Camera, and use the new nodes in your comp.

The panel

The node's panel as Nuke shows it, with the values of a new node. Click any control to see what it does; tabs and sections work as in Nuke.

ANR_AI_Planar_Tracker1

The toolbar row of transport buttons, and the original controls in a closed group below it.

● Drawing Mask — Enter to finish|◀◀■▶▶|CenterFollowAnalyze Shape
StatusDrawing Mask: click its outline in the viewer. Click the first point again, or press Enter, to finish.
Planar Viewer
Adjust Track in Viewer
Show Mask
Show Surface
Live Frame Updates
Adapt to Appearance Changes
Keep Searching After Loss
New Mask
Finish Mask
Reference Frame1
Fit Surface to Mask
Tracking MethodAlgorithm (Enhanced Non-AI)
Image RefinementCPU
Search AreaCrop Box
Crop Size (%)200
MotionPerspective
RangeInput
First1
Last100
Minimum Agreement0.5
Use Occlusion Input
Track Backward
Step Back
Step Forward
Track Forward
Clear Track
Enable Adjust Track Layer
Key Adjust Layer
Remove Adjust Key
Clear Adjust Layer

All controls

ANR Planar40 controls

The toolbar row of transport buttons, and the original controls in a closed group below it.

● Drawing Mask — Enter to finishButtonanrNative_new

Adds a new mask layer (New Shape). While you draw, it shows the mask's name and colour and reminds you that Enter finishes it.

|◀ButtonanrNative_plnStepBack

Tracks one frame backward.

◀ButtonanrNative_plnBack

Tracks backward from the current frame to First.

■ButtonanrNative_stop

Stop Planar Dev job: ANR AI Planar Tracker1

▶ButtonanrNative_plnForward

Tracks forward from the current frame to Last.

▶|ButtonanrNative_plnStepForward

Tracks one frame forward.

CenterButtonanrNative_center

Centres the viewer on the active surface. The mask has to be finished first.

FollowCheckboxanrNative_follow

Keeps the viewer centred on the active surface as the frame changes.

Default: off

Analyze ShapeButtonSome modesanrNative_analyze

A button.

Planar Track

StatusStatusplnStatus

What the node is doing: drawing, tracking, how many frames were tracked or lost, and what an export made.

Planar ViewerCheckboxplnViewer

Shows the masks, the surface corners and the drawing tools in the viewer.

Default: on

Adjust Track in ViewerCheckboxplnAdjustMode

Lets you drag the surface corners in the viewer to key the Adjust Track correction layer.

Default: off

Show MaskCheckboxplnShowMask

Draws the mask outlines in the viewer.

Default: on

Show SurfaceCheckboxplnShowSurface

Draws the four surface corners (where the insert sits) in the viewer.

Default: on

Live Frame UpdatesCheckboxplnLive

While tracking, the viewer moves to each newly tracked frame so you can watch the track.

Default: on

Adapt to Appearance ChangesCheckboxplnAdaptive

Lets the match follow a surface whose look changes. With the Algorithm method it can also recover through a blink from texture around the mask. Turn off for strict matching inside the mask.

Default: on

Keep Searching After LossCheckboxplnKeepSearching

On: a frame that cannot be matched is left untracked and tracking carries on. Off: tracking stops at the first lost frame.

Default: on

New MaskButtonplnNew

Start a new mask on the current frame. Replaces this plane's existing mask and track.

Finish MaskButtonplnFinish

Close the drawn outline and initialize the surface corners. Enter also finishes.

Reference FrameSliderplnReference

The frame the active mask was drawn on, where its surface corners are placed.

Default: 1

Fit Surface to MaskButtonplnFitSurface

Reset the four insert corners to the reference mask bounds; tracking is unchanged.

Tracking MethodMenuplnMethod

Algorithm uses CPU feature matches. AI uses the installed AI point tracking model on CUDA to track points inside the mask. Both robustly fit one plane and refine its image alignment. No model download.

Default: Algorithm (Enhanced Non-AI)Options: Algorithm (Enhanced Non-AI) AI Whole Shot Analysis (Extra Heavy) - research only Enhanced Alignment (Non-AI) RoMa v2 Image Matching (AI)

Image RefinementMenuplnCompute

GPU accelerates dense alignment on the installed CUDA runtime. Feature detection and robust fitting remain on CPU; AI always uses GPU.

Default: CPUOptions: CPU GPU (CUDA)

Search AreaMenuplnSearchMode

Algorithm only. Crop Box limits feature search and image alignment to a box around the last accepted mask position. It follows successful tracks; after loss it stays at the last accepted position. Use Whole Frame or a larger box for large jumps.

Default: Crop BoxOptions: Whole Frame Crop Box

Crop Size (%)SliderplnCropSize

200% = twice the tracked mask bounding-box width and height. Minimum 64 analysis pixels per side; clipped to the image. Larger boxes allow more motion but cost more work. Combined-mask tracking uses the combined bounds. Source decoding and output size are unchanged.

Default: 200

MotionMenuplnMotion

How the surface may move: Translation, Translation / Rotation / Scale, Affine, or Perspective (a flat surface under any camera move).

Default: PerspectiveOptions: Translation Translation / Rotation / Scale Affine Perspective

RangeMenuplnRangeMode

Which frames to track. Input: the frames of the footage into Source. Global: the project's frame range. Custom: First and Last as typed. First and Last always show the range in use; typing a frame into either switches to Custom.

Default: InputOptions: Input Global Custom

FirstSliderplnFirst

First frame of the tracking range. Shows the range in use; typing a frame switches Range to Custom.

Default: 1

LastSliderplnLast

Last frame of the tracking range. Shows the range in use; typing a frame switches Range to Custom.

Default: 100

Minimum AgreementSliderplnMinimum

Stop when too little visible texture agrees. Lowering this can allow a wrong track.

Default: 0.5

Use Occlusion InputCheckboxplnUseOcclusion

Ignores the pixels the Occlusion input marks (alpha above half) when matching.

Default: on

Track BackwardButtonplnBack

Track from the current frame to First. On a frame the track has not reached, the mask starts where the viewer shows it.

Step BackButtonplnStepBack

Track one frame backward.

Step ForwardButtonplnStepForward

Track one frame forward.

Track ForwardButtonplnForward

Track from the current frame to Last. On a frame the track has not reached, the mask starts where the viewer shows it.

Clear TrackButtonplnClear

Remove tracking keys and keep the mask and surface at their reference positions.

Enable Adjust Track LayerCheckboxplnApplyAdjust

Applies the Adjust Track corrections on top of the track. Off shows the raw track; the correction keys are kept.

Default: off

Key Adjust LayerButtonplnKeyAdjust

Key all four correction offsets on the current frame. The underlying track is unchanged.

Remove Adjust KeyButtonplnRemoveAdjust

Remove the four correction keys on this frame.

Clear Adjust LayerButtonplnClearAdjust

Remove all correction keys; keep the underlying track.

Planar Track41 controls

Where you work: mask layers, tracking method and range, transport, and the Export buttons.

anrPlanarLayersPanelanrPlanarLayers

The list of mask layers, with Track, View and Keys columns. Select a row to make that mask active; New Shape, Duplicate and Remove manage layers, and the red button stops tracking.

Default: empty

TrackMenuanrPlanarScope

Which masks a track press works on: the Active layer, every enabled layer tracked one after another (Enabled layers separately), or the enabled masks combined into one surface (Enabled masks together).

Default: Active layerOptions: Active layer Enabled layers separately Enabled masks together

StatusStatusanrPlanar_plnStatus

What the node is doing: drawing, tracking, how many frames were tracked or lost, and what an export made.

Planar ViewerCheckboxanrPlanar_plnViewer

Shows the masks, the surface corners and the drawing tools in the viewer.

Default: on

Show MaskCheckboxanrPlanar_plnShowMask

Draws the mask outlines in the viewer.

Default: on

Show SurfaceCheckboxanrPlanar_plnShowSurface

Draws the four surface corners (where the insert sits) in the viewer.

Default: on

Finish MaskButtonanrPlanar_plnFinish

Close the drawn outline and initialize the surface corners. Enter also finishes.

Reference FrameSlideranrPlanar_plnReference

The frame the active mask was drawn on, where its surface corners are placed.

Default: 1

Fit Surface to MaskButtonanrPlanar_plnFitSurface

Reset the four insert corners to the reference mask bounds; tracking is unchanged.

Tracking MethodMenuanrPlanar_plnMethod

Algorithm uses CPU feature matches. AI uses the installed AI point tracking model on CUDA to track points inside the mask. Both robustly fit one plane and refine its image alignment. No model download.

Default: Algorithm (Enhanced Non-AI)Options: Algorithm (Enhanced Non-AI) AI (FrameNet) Shape Analysis (Extra Heavy) RoMa v2 Image Matching (AI)

Points in ShapeNumberSome modesanrPlanarAnalysisPoints

Maximum query points distributed inside the active mask, on its reference frame.

Default: 128

Analyze ShapeButtonSome modesanrPlanarAnalyze

Finish a shape, then analyze its points across First–Last, using the shape reference frame. Automatically creates track keys.

Update Shape from AnalysisButtonSome modesanrPlanarApplyAnalysis

A button.

Search AreaMenuanrPlanar_plnSearchMode

Algorithm only. Crop Box limits feature search and image alignment to a box around the last accepted mask position. It follows successful tracks; after loss it stays at the last accepted position. Use Whole Frame or a larger box for large jumps.

Default: Crop BoxOptions: Whole Frame Crop Box

Crop Size (%)SlideranrPlanar_plnCropSize

200% = twice the tracked mask bounding-box width and height. Minimum 64 analysis pixels per side; clipped to the image. Larger boxes allow more motion but cost more work. Combined-mask tracking uses the combined bounds. Source decoding and output size are unchanged.

Default: 200

Image RefinementMenuanrPlanar_plnCompute

GPU accelerates dense alignment on the installed CUDA runtime. Feature detection and robust fitting remain on CPU; AI always uses GPU.

Default: CPUOptions: CPU GPU (CUDA)

MotionMenuanrPlanar_plnMotion

How the surface may move: Translation, Translation / Rotation / Scale, Affine, or Perspective (a flat surface under any camera move).

Default: PerspectiveOptions: Translation Translation / Rotation / Scale Affine Perspective

Adapt to Appearance ChangesCheckboxanrPlanar_plnAdaptive

Adapt to changing appearance. Algorithm mode can recover a blink from nearby texture around the mask when the interior stops matching. Recovery respects the crop box and Occlusion input, and requires consistent measured motion. Turn off for strict matching inside the mask.

Default: on

Keep Searching After LossCheckboxanrPlanar_plnKeepSearching

On: a frame that cannot be matched is left untracked and tracking carries on. Off: tracking stops at the first lost frame.

Default: on

RangeMenuanrPlanar_plnRangeMode

Which frames to track. Input: the frames of the footage into Source. Global: the project's frame range. Custom: First and Last as typed. First and Last always show the range in use; typing a frame into either switches to Custom.

Default: InputOptions: Input Global Custom

FirstSlideranrPlanar_plnFirst

First frame of the tracking range. Shows the range in use; typing a frame switches Range to Custom.

Default: 1

LastSlideranrPlanar_plnLast

Last frame of the tracking range. Shows the range in use; typing a frame switches Range to Custom.

Default: 100

Range from ViewerButtonanrPlanarRange

Sets Range to Custom with the viewer's in and out points.

Minimum AgreementSlideranrPlanar_plnMinimum

Stop when too little visible texture agrees. Lowering this can allow a wrong track.

Default: 0.5

Use Occlusion InputCheckboxanrPlanar_plnUseOcclusion

Ignores the pixels the Occlusion input marks (alpha above half) when matching.

Default: on

Live Frame UpdatesCheckboxanrPlanar_plnLive

While tracking, the viewer moves to each newly tracked frame so you can watch the track.

Default: on

|◀ButtonanrPlanar_plnStepBack

Track one frame backward

◀ButtonanrPlanar_plnBack

Track backward

■ButtonanrPlanarStop

Stop Planar Dev job: ANR AI Planar Tracker1

▶ButtonanrPlanar_plnForward

Track forward

▶|ButtonanrPlanar_plnStepForward

Track one frame forward

Clear TrackButtonanrPlanar_plnClear

Remove tracking keys and keep the mask and surface at their reference positions.

Center ViewButtonanrPlanarCenter

Centres the viewer on the active surface. The mask has to be finished first.

Follow TrackCheckboxanrPlanarFollow

Center the 2D Viewer on the active surface as the frame changes. Keep this node panel open.

Default: off

Create Tracked RotoButtonanrPlanarRoto

Builds a Roto node, connected to the plate, with the active mask in a folder that carries the planar motion (Adjust Track included), keyed on every tracked frame.

All Shapes to One RotoButtonanrPlanarRotoAll

One Roto node holding every visible shape of this node, each in its own folder with its own tracked motion (Adjust Track included). Hidden layers are left out, so a refined copy does not come out twice with its original.

Create Tracker (4 Corners)ButtonanrPlanarTracker

Builds a Tracker with four tracks at the surface corners (Bottom Left, Bottom Right, Top Right, Top Left), for a CornerPin or any tool that wants tracks. Untracked frames are switched off.

Scaling Card + CameraButtonanrPlanarCard

Bake the active tracked surface into an animated Card, a static Camera and a connected ScanlineRender. Includes Adjust Track. Replace the Insert input with your artwork. Matches this camera view; not a real-world depth solve.

Moving Card + CameraButtonanrPlanarMovingCard

Fixed-size rectangular card with animated translation and rotation; static camera. Uses the focal length and proportions below. Reports screen fitting error; never animates scale to conceal a mismatch.

Moving Card Focal (mm)SlideranrPlanarCardFocal

Moving Card only. Focal length on a 36mm-wide filmback. This is an assumption, not recovered lens metadata.

Default: 50

Card Width / HeightSlideranrPlanarCardAspect

Moving Card only. Actual rectangular surface width divided by height. Zero estimates once from the reference and assumed lens. The resulting dimensions stay fixed for the entire export.

Default: 0

Adjust Track15 controls

The correction layer over the raw track, and the post-track refinements.

anrPlanarAdjustTrackPanelanrPlanarAdjustTrack

The Adjust Track switch: on, you can drag corners in the viewer and the correction layer is applied; off, the corrections are bypassed and kept.

Default: empty

Key Adjust LayerButtonanrPlanar_plnKeyAdjust

Key all four correction offsets on the current frame. The underlying track is unchanged.

Remove Adjust KeyButtonanrPlanar_plnRemoveAdjust

Remove the four correction keys on this frame.

Clear Adjust LayerButtonanrPlanar_plnClearAdjust

Remove all correction keys; keep the underlying track.

Include Low-Agreement FramesCheckboxanrPlanarRefineWeak

Fill Gaps and Trajectory Matching also rebuild keys below Refine Below Agreement. The reference is protected. Nothing changes until you click a refinement button; Back to Original restores the source.

Default: off

Refine Below AgreementSlideranrPlanarRefineAgreement

Planar image-agreement threshold, not AI point confidence. Applies only when Include Low-Agreement Frames is checked.

Default: 0.7

Radius (frames)NumberanrPlanarSmoothRadius

Smooth Jitter: how many frames on each side are averaged.

Default: 2

AmountSlideranrPlanarSmoothAmount

Smooth Jitter: how strongly each frame is pulled towards the average of its neighbours, 0 to 1.

Default: 0.5

Refine ImagesButtonanrPlanarRefine_images

Track the surface again with image alignment, seeded by this track. Makes a refined copy as a new layer; the original is kept.

Smooth JitterButtonanrPlanarRefine_smooth

Average each frame with its neighbours (Radius, Amount). Gaps are not bridged. Makes a refined copy as a new layer; the original is kept.

Fix Rogue MotionButtonanrPlanarRefine_rogue

Find stretches where the surface wanders off the path of the frames around it (points that moved their own way pulled it) and rebuild them from the good frames on either side. Makes a refined copy as a new layer; the original is kept.

Reject Rogue PointsButtonanrPlanarRefine_points

Whole Shot Analysis only: score every analysis point over the whole shot, drop the ones that keep moving against the rest, and fit the surface again from the points that agree. Makes a refined copy as a new layer; the original is kept.

Fill GapsButtonanrPlanarRefine_gaps

Fill untracked frames inside the track (up to 30 in a row) from the frames on either side. Makes a refined copy as a new layer; the original is kept.

Trajectory MatchingButtonanrPlanarRefine_trajectory

Rebuild the frames the surface was lost on (it left the picture, or was hidden). While any of it can be seen the track already follows it. Where it was lost, the surface carries on from where it was lost at its own speed and direction, comes back along the path it returns on, and the two are blended so it lands on the track with no jump. Past the ends of the track it keeps moving, out to First and Last. Makes a refined copy as a new layer; the original is kept.

Back to OriginalButtonanrPlanarRefineRevert

Remove the refined copy that is active and bring back the track it was made from.

Tips

  • Draw the mask on texture that moves with the surface, and keep reflections and moving things out of it. Connect a matte of foreground objects to Occlusion when they cross the surface.
  • With the Algorithm method, Search Area: Crop Box keeps the search near the last good position; for big jumps, raise Crop Size (%) or use Whole Frame.
  • Keep Searching After Loss on: a frame that cannot be matched is left untracked and tracking carries on. Off: tracking stops at the first lost frame.
  • Adapt to Appearance Changes lets the track follow a surface whose look changes over the shot; turn it off for strict matching inside the mask.
  • Mask editing in the viewer: click-drag places a smooth point, Z smooths the selected points, Shift-click or box-select picks several. With two or more points selected, a transform box moves, scales and rotates them (Shift for proportional scale or 15-degree steps, Escape cancels).
  • Click another mask's outline in the viewer to make it the active layer.
  • The refinements on the Adjust Track tab (Refine Images, Smooth Jitter, Fix Rogue Motion, Fill Gaps, Trajectory Matching) each make a refined copy as a new layer and keep the original; Back to Original removes the copy.
  • Follow Track keeps the viewer centred on the active surface as you scrub; keep the node's panel open.

Good to know

  • The AI methods need an NVIDIA GPU (CUDA). The Algorithm method runs on the CPU; Image Refinement: GPU (CUDA) speeds up its alignment.
  • If the Tracker models were not installed with ANR Tools, Nuke offers to download them when you create the node. The image-matching method downloads its model by itself the first time it tracks.
  • Create Tracked Roto and the Card exports need a track with no untracked frames inside it; Create Tracker (4 Corners) accepts gaps and switches the corners off on those frames.
  • A mask that was never tracked exports to Roto as drawn, standing still.
  • The Card exports match this camera view only. They are not a real-world depth solve.
  • The toolbar row on the ANR Planar tab and in the viewer comes with Nuke 15.2; in other versions use the buttons on the Planar Track tab.

Something unclear or missing? Write to [email protected].