ANR AI Planar Tracker
Planar tracking of flat surfaces from masks you draw, with layered masks, a correction layer, and exports to Roto, a four-corner Tracker or a Card and Camera.
- Trackers
- Trackers pack
- Tracker models
- 2 inputs
- 96 controls
Overview
The AI Planar Tracker follows a flat surface (a screen, a sign, a wall, a page) from a mask you draw around it in the viewer. Each frame it fits one plane to what it sees inside the mask, and moves four surface corners with it: the corners where an insert will sit.
Choose how the surface is found in Tracking Method. Algorithm (Enhanced Non-AI) matches image features on the CPU and refines the alignment, with an optional crop box that follows the track. AI tracks points inside the mask with the AI point tracking model on the GPU, then fits the plane. The image-matching AI method (the last entry of the menu) matches the whole surface with an AI model on the GPU.
A node can hold several masks as layers, each with its own colour and track. Track them one at a time, each separately, or several masks together as one surface (Track). A separate Adjust Track layer lets you key corner corrections over the raw track without changing it, and the post-track refinements make cleaned-up copies you can compare with the original.
When the track is right, the Export buttons build ordinary Nuke nodes: a tracked Roto of the mask, one Roto with every shape, a Tracker with the four corners, or a Card and a static Camera rendered with a ScanlineRender, ready for an insert.
Inputs
- Sourcerequired
The plate to track.
- Occlusionoptional
A matte of whatever passes in front of the surface: where its alpha is above half, the pixels are ignored by the match. Used while Use Occlusion Input is on.
Output. The plate, with the masks, the tracked surface and the tools drawn in the viewer. The results are the keys on the node and the nodes the Export buttons build.
Quick start
- Add ANR AI Planar Tracker and connect the plate to Source. Go to a frame where the surface is clear.
- The node starts in drawing mode: click points around the surface in the viewer. Click the first point again, or press Enter, to finish the mask.
- The four surface corners start at the mask's bounds. Drag them onto the corners of the area your insert should cover (Fit Surface to Mask puts them back).
- Choose Tracking Method and Motion (Perspective for a flat surface seen from a moving camera).
- Press ▶ to track from the current frame to Last. Go back to the frame you drew on and press ◀ to track to First. ■ stops and keeps the frames already tracked.
- Scrub to check. Fix drift on the Adjust Track tab: turn on Adjust Track, drag the corners in the viewer on the frames that need it.
- Press an Export button, for example Create Tracker (4 Corners) or Scaling Card + Camera, and use the new nodes in your comp.
The panel
The node's panel as Nuke shows it, with the values of a new node. Click any control to see what it does; tabs and sections work as in Nuke.
The toolbar row of transport buttons, and the original controls in a closed group below it.
Where you work: mask layers, tracking method and range, transport, and the Export buttons.
The correction layer over the raw track, and the post-track refinements.
No control matches.
All controls
ANR Planar40 controls
The toolbar row of transport buttons, and the original controls in a closed group below it.
- ● Drawing Mask — Enter to finishButton
anrNative_new Adds a new mask layer (New Shape). While you draw, it shows the mask's name and colour and reminds you that Enter finishes it.
- |◀Button
anrNative_plnStepBack Tracks one frame backward.
- ◀Button
anrNative_plnBack Tracks backward from the current frame to First.
- ■Button
anrNative_stop Stop Planar Dev job: ANR AI Planar Tracker1
- ▶Button
anrNative_plnForward Tracks forward from the current frame to Last.
- ▶|Button
anrNative_plnStepForward Tracks one frame forward.
- CenterButton
anrNative_center Centres the viewer on the active surface. The mask has to be finished first.
- FollowCheckbox
anrNative_follow Keeps the viewer centred on the active surface as the frame changes.
- Analyze ShapeButtonSome modes
anrNative_analyze A button.
Planar Track
- StatusStatus
plnStatus What the node is doing: drawing, tracking, how many frames were tracked or lost, and what an export made.
- Planar ViewerCheckbox
plnViewer Shows the masks, the surface corners and the drawing tools in the viewer.
- Adjust Track in ViewerCheckbox
plnAdjustMode Lets you drag the surface corners in the viewer to key the Adjust Track correction layer.
- Show MaskCheckbox
plnShowMask Draws the mask outlines in the viewer.
- Show SurfaceCheckbox
plnShowSurface Draws the four surface corners (where the insert sits) in the viewer.
- Live Frame UpdatesCheckbox
plnLive While tracking, the viewer moves to each newly tracked frame so you can watch the track.
- Adapt to Appearance ChangesCheckbox
plnAdaptive Lets the match follow a surface whose look changes. With the Algorithm method it can also recover through a blink from texture around the mask. Turn off for strict matching inside the mask.
- Keep Searching After LossCheckbox
plnKeepSearching On: a frame that cannot be matched is left untracked and tracking carries on. Off: tracking stops at the first lost frame.
- New MaskButton
plnNew Start a new mask on the current frame. Replaces this plane's existing mask and track.
- Finish MaskButton
plnFinish Close the drawn outline and initialize the surface corners. Enter also finishes.
- Reference FrameSlider
plnReference The frame the active mask was drawn on, where its surface corners are placed.
- Fit Surface to MaskButton
plnFitSurface Reset the four insert corners to the reference mask bounds; tracking is unchanged.
- Tracking MethodMenu
plnMethod Algorithm uses CPU feature matches. AI uses the installed AI point tracking model on CUDA to track points inside the mask. Both robustly fit one plane and refine its image alignment. No model download.
- Image RefinementMenu
plnCompute GPU accelerates dense alignment on the installed CUDA runtime. Feature detection and robust fitting remain on CPU; AI always uses GPU.
- Search AreaMenu
plnSearchMode Algorithm only. Crop Box limits feature search and image alignment to a box around the last accepted mask position. It follows successful tracks; after loss it stays at the last accepted position. Use Whole Frame or a larger box for large jumps.
- Crop Size (%)Slider
plnCropSize 200% = twice the tracked mask bounding-box width and height. Minimum 64 analysis pixels per side; clipped to the image. Larger boxes allow more motion but cost more work. Combined-mask tracking uses the combined bounds. Source decoding and output size are unchanged.
- MotionMenu
plnMotion How the surface may move: Translation, Translation / Rotation / Scale, Affine, or Perspective (a flat surface under any camera move).
- RangeMenu
plnRangeMode Which frames to track. Input: the frames of the footage into Source. Global: the project's frame range. Custom: First and Last as typed. First and Last always show the range in use; typing a frame into either switches to Custom.
- FirstSlider
plnFirst First frame of the tracking range. Shows the range in use; typing a frame switches Range to Custom.
- LastSlider
plnLast Last frame of the tracking range. Shows the range in use; typing a frame switches Range to Custom.
- Minimum AgreementSlider
plnMinimum Stop when too little visible texture agrees. Lowering this can allow a wrong track.
- Use Occlusion InputCheckbox
plnUseOcclusion Ignores the pixels the Occlusion input marks (alpha above half) when matching.
- Track BackwardButton
plnBack Track from the current frame to First. On a frame the track has not reached, the mask starts where the viewer shows it.
- Step BackButton
plnStepBack Track one frame backward.
- Step ForwardButton
plnStepForward Track one frame forward.
- Track ForwardButton
plnForward Track from the current frame to Last. On a frame the track has not reached, the mask starts where the viewer shows it.
- Clear TrackButton
plnClear Remove tracking keys and keep the mask and surface at their reference positions.
- Enable Adjust Track LayerCheckbox
plnApplyAdjust Applies the Adjust Track corrections on top of the track. Off shows the raw track; the correction keys are kept.
- Key Adjust LayerButton
plnKeyAdjust Key all four correction offsets on the current frame. The underlying track is unchanged.
- Remove Adjust KeyButton
plnRemoveAdjust Remove the four correction keys on this frame.
- Clear Adjust LayerButton
plnClearAdjust Remove all correction keys; keep the underlying track.
Planar Track41 controls
Where you work: mask layers, tracking method and range, transport, and the Export buttons.
- anrPlanarLayersPanel
anrPlanarLayers The list of mask layers, with Track, View and Keys columns. Select a row to make that mask active; New Shape, Duplicate and Remove manage layers, and the red button stops tracking.
- TrackMenu
anrPlanarScope Which masks a track press works on: the Active layer, every enabled layer tracked one after another (Enabled layers separately), or the enabled masks combined into one surface (Enabled masks together).
- StatusStatus
anrPlanar_plnStatus What the node is doing: drawing, tracking, how many frames were tracked or lost, and what an export made.
- Planar ViewerCheckbox
anrPlanar_plnViewer Shows the masks, the surface corners and the drawing tools in the viewer.
- Show MaskCheckbox
anrPlanar_plnShowMask Draws the mask outlines in the viewer.
- Show SurfaceCheckbox
anrPlanar_plnShowSurface Draws the four surface corners (where the insert sits) in the viewer.
- Finish MaskButton
anrPlanar_plnFinish Close the drawn outline and initialize the surface corners. Enter also finishes.
- Reference FrameSlider
anrPlanar_plnReference The frame the active mask was drawn on, where its surface corners are placed.
- Fit Surface to MaskButton
anrPlanar_plnFitSurface Reset the four insert corners to the reference mask bounds; tracking is unchanged.
- Tracking MethodMenu
anrPlanar_plnMethod Algorithm uses CPU feature matches. AI uses the installed AI point tracking model on CUDA to track points inside the mask. Both robustly fit one plane and refine its image alignment. No model download.
- Points in ShapeNumberSome modes
anrPlanarAnalysisPoints Maximum query points distributed inside the active mask, on its reference frame.
- Analyze ShapeButtonSome modes
anrPlanarAnalyze Finish a shape, then analyze its points across First–Last, using the shape reference frame. Automatically creates track keys.
- Update Shape from AnalysisButtonSome modes
anrPlanarApplyAnalysis A button.
- Search AreaMenu
anrPlanar_plnSearchMode Algorithm only. Crop Box limits feature search and image alignment to a box around the last accepted mask position. It follows successful tracks; after loss it stays at the last accepted position. Use Whole Frame or a larger box for large jumps.
- Crop Size (%)Slider
anrPlanar_plnCropSize 200% = twice the tracked mask bounding-box width and height. Minimum 64 analysis pixels per side; clipped to the image. Larger boxes allow more motion but cost more work. Combined-mask tracking uses the combined bounds. Source decoding and output size are unchanged.
- Image RefinementMenu
anrPlanar_plnCompute GPU accelerates dense alignment on the installed CUDA runtime. Feature detection and robust fitting remain on CPU; AI always uses GPU.
- MotionMenu
anrPlanar_plnMotion How the surface may move: Translation, Translation / Rotation / Scale, Affine, or Perspective (a flat surface under any camera move).
- Adapt to Appearance ChangesCheckbox
anrPlanar_plnAdaptive Adapt to changing appearance. Algorithm mode can recover a blink from nearby texture around the mask when the interior stops matching. Recovery respects the crop box and Occlusion input, and requires consistent measured motion. Turn off for strict matching inside the mask.
- Keep Searching After LossCheckbox
anrPlanar_plnKeepSearching On: a frame that cannot be matched is left untracked and tracking carries on. Off: tracking stops at the first lost frame.
- RangeMenu
anrPlanar_plnRangeMode Which frames to track. Input: the frames of the footage into Source. Global: the project's frame range. Custom: First and Last as typed. First and Last always show the range in use; typing a frame into either switches to Custom.
- FirstSlider
anrPlanar_plnFirst First frame of the tracking range. Shows the range in use; typing a frame switches Range to Custom.
- LastSlider
anrPlanar_plnLast Last frame of the tracking range. Shows the range in use; typing a frame switches Range to Custom.
- Range from ViewerButton
anrPlanarRange Sets Range to Custom with the viewer's in and out points.
- Minimum AgreementSlider
anrPlanar_plnMinimum Stop when too little visible texture agrees. Lowering this can allow a wrong track.
- Use Occlusion InputCheckbox
anrPlanar_plnUseOcclusion Ignores the pixels the Occlusion input marks (alpha above half) when matching.
- Live Frame UpdatesCheckbox
anrPlanar_plnLive While tracking, the viewer moves to each newly tracked frame so you can watch the track.
- |◀Button
anrPlanar_plnStepBack Track one frame backward
- ◀Button
anrPlanar_plnBack Track backward
- ■Button
anrPlanarStop Stop Planar Dev job: ANR AI Planar Tracker1
- ▶Button
anrPlanar_plnForward Track forward
- ▶|Button
anrPlanar_plnStepForward Track one frame forward
- Clear TrackButton
anrPlanar_plnClear Remove tracking keys and keep the mask and surface at their reference positions.
- Center ViewButton
anrPlanarCenter Centres the viewer on the active surface. The mask has to be finished first.
- Follow TrackCheckbox
anrPlanarFollow Center the 2D Viewer on the active surface as the frame changes. Keep this node panel open.
- Create Tracked RotoButton
anrPlanarRoto Builds a Roto node, connected to the plate, with the active mask in a folder that carries the planar motion (Adjust Track included), keyed on every tracked frame.
- All Shapes to One RotoButton
anrPlanarRotoAll One Roto node holding every visible shape of this node, each in its own folder with its own tracked motion (Adjust Track included). Hidden layers are left out, so a refined copy does not come out twice with its original.
- Create Tracker (4 Corners)Button
anrPlanarTracker Builds a Tracker with four tracks at the surface corners (Bottom Left, Bottom Right, Top Right, Top Left), for a CornerPin or any tool that wants tracks. Untracked frames are switched off.
- Scaling Card + CameraButton
anrPlanarCard Bake the active tracked surface into an animated Card, a static Camera and a connected ScanlineRender. Includes Adjust Track. Replace the Insert input with your artwork. Matches this camera view; not a real-world depth solve.
- Moving Card + CameraButton
anrPlanarMovingCard Fixed-size rectangular card with animated translation and rotation; static camera. Uses the focal length and proportions below. Reports screen fitting error; never animates scale to conceal a mismatch.
- Moving Card Focal (mm)Slider
anrPlanarCardFocal Moving Card only. Focal length on a 36mm-wide filmback. This is an assumption, not recovered lens metadata.
- Card Width / HeightSlider
anrPlanarCardAspect Moving Card only. Actual rectangular surface width divided by height. Zero estimates once from the reference and assumed lens. The resulting dimensions stay fixed for the entire export.
Adjust Track15 controls
The correction layer over the raw track, and the post-track refinements.
- anrPlanarAdjustTrackPanel
anrPlanarAdjustTrack The Adjust Track switch: on, you can drag corners in the viewer and the correction layer is applied; off, the corrections are bypassed and kept.
- Key Adjust LayerButton
anrPlanar_plnKeyAdjust Key all four correction offsets on the current frame. The underlying track is unchanged.
- Remove Adjust KeyButton
anrPlanar_plnRemoveAdjust Remove the four correction keys on this frame.
- Clear Adjust LayerButton
anrPlanar_plnClearAdjust Remove all correction keys; keep the underlying track.
- Include Low-Agreement FramesCheckbox
anrPlanarRefineWeak Fill Gaps and Trajectory Matching also rebuild keys below Refine Below Agreement. The reference is protected. Nothing changes until you click a refinement button; Back to Original restores the source.
- Refine Below AgreementSlider
anrPlanarRefineAgreement Planar image-agreement threshold, not AI point confidence. Applies only when Include Low-Agreement Frames is checked.
- Radius (frames)Number
anrPlanarSmoothRadius Smooth Jitter: how many frames on each side are averaged.
- AmountSlider
anrPlanarSmoothAmount Smooth Jitter: how strongly each frame is pulled towards the average of its neighbours, 0 to 1.
- Refine ImagesButton
anrPlanarRefine_images Track the surface again with image alignment, seeded by this track. Makes a refined copy as a new layer; the original is kept.
- Smooth JitterButton
anrPlanarRefine_smooth Average each frame with its neighbours (Radius, Amount). Gaps are not bridged. Makes a refined copy as a new layer; the original is kept.
- Fix Rogue MotionButton
anrPlanarRefine_rogue Find stretches where the surface wanders off the path of the frames around it (points that moved their own way pulled it) and rebuild them from the good frames on either side. Makes a refined copy as a new layer; the original is kept.
- Reject Rogue PointsButton
anrPlanarRefine_points Whole Shot Analysis only: score every analysis point over the whole shot, drop the ones that keep moving against the rest, and fit the surface again from the points that agree. Makes a refined copy as a new layer; the original is kept.
- Fill GapsButton
anrPlanarRefine_gaps Fill untracked frames inside the track (up to 30 in a row) from the frames on either side. Makes a refined copy as a new layer; the original is kept.
- Trajectory MatchingButton
anrPlanarRefine_trajectory Rebuild the frames the surface was lost on (it left the picture, or was hidden). While any of it can be seen the track already follows it. Where it was lost, the surface carries on from where it was lost at its own speed and direction, comes back along the path it returns on, and the two are blended so it lands on the track with no jump. Past the ends of the track it keeps moving, out to First and Last. Makes a refined copy as a new layer; the original is kept.
- Back to OriginalButton
anrPlanarRefineRevert Remove the refined copy that is active and bring back the track it was made from.
Tips
- Draw the mask on texture that moves with the surface, and keep reflections and moving things out of it. Connect a matte of foreground objects to Occlusion when they cross the surface.
- With the Algorithm method, Search Area: Crop Box keeps the search near the last good position; for big jumps, raise Crop Size (%) or use Whole Frame.
- Keep Searching After Loss on: a frame that cannot be matched is left untracked and tracking carries on. Off: tracking stops at the first lost frame.
- Adapt to Appearance Changes lets the track follow a surface whose look changes over the shot; turn it off for strict matching inside the mask.
- Mask editing in the viewer: click-drag places a smooth point, Z smooths the selected points, Shift-click or box-select picks several. With two or more points selected, a transform box moves, scales and rotates them (Shift for proportional scale or 15-degree steps, Escape cancels).
- Click another mask's outline in the viewer to make it the active layer.
- The refinements on the Adjust Track tab (Refine Images, Smooth Jitter, Fix Rogue Motion, Fill Gaps, Trajectory Matching) each make a refined copy as a new layer and keep the original; Back to Original removes the copy.
- Follow Track keeps the viewer centred on the active surface as you scrub; keep the node's panel open.
Good to know
- The AI methods need an NVIDIA GPU (CUDA). The Algorithm method runs on the CPU; Image Refinement: GPU (CUDA) speeds up its alignment.
- If the Tracker models were not installed with ANR Tools, Nuke offers to download them when you create the node. The image-matching method downloads its model by itself the first time it tracks.
- Create Tracked Roto and the Card exports need a track with no untracked frames inside it; Create Tracker (4 Corners) accepts gaps and switches the corners off on those frames.
- A mask that was never tracked exports to Roto as drawn, standing still.
- The Card exports match this camera view only. They are not a real-world depth solve.
- The toolbar row on the ANR Planar tab and in the viewer comes with Nuke 15.2; in other versions use the buttons on the Planar Track tab.
Something unclear or missing? Write to [email protected].