- Introduction
- Part 1: Encode once, query anything (D4RT)
- Part 2: A pixel is a bad handle for a long video (Point4D)
- Part 3: What it takes to train one
- Side by side
- Part 4: Which one should you reach for?
- References
Introduction
Every feed-forward reconstruction model of the last two years has the same shape:
- A big transformer looks at all the frames at once (DUSt3R, VGGT, $\pi^3$, MapAnything, Depth Anything 3).
- A stack of DPT-style heads decodes every pixel of every frame into depth, a point map, a camera.
- One head per output.
Dynamic scenes expose two problems with that shape:
- Nothing in the interface says “at what time”. A point map is “where is this pixel, now”. There is no head for “where is this pixel three frames later”.
- Everything is decoded whether you need it or not. A 48-frame clip at 256×256 is 3.1 M outputs per head, per forward pass. Even if all you wanted was one camera pose.
D4RT (Google DeepMind, CVPR 2026 oral) proposes a different shape: encode the video once, then answer questions about it with a small decoder, one point at a time. The query says which point, which time, which camera.
Four parts:
- D4RT: the query, and what it buys. Four released models are run on two toy clips so the difference is measured, not argued.
- Point4D (CMU, 2026): same interface, one change to the query so that it survives long videos.
- Training: which datasets, how much data, what supervision.
- Which one to reach for.
Part 1: Encode once, query anything (D4RT)
Four real models on two clips
The clips. A small rendered world: a wall, a pillar, three posts, a floor. 80 frames each, 384×216. Two balls roll to the right (radius 0.28): the yellow one far and slow, the blue one near and fast.
- Clip 1, fixed camera. Only the balls move. The blue ball overtakes the yellow one behind the pillar: the yellow ball is hidden in frames 15–35, the blue one in 20–30. Both stay inside the picture.
- Clip 2, moving camera. The same overtaking while the camera drives 3.7 units to the right. Hidden frames 15–32 and 20–29.
The models. Each gets the 80 pictures and nothing else.
| Depth Anything 3 (1.4 B) | VGGT-1B | MapAnything | Point4D | |
|---|---|---|---|---|
| Output per picture | depth, camera | depth, point map, camera | metric depth, rays, camera | a 3D point per query, at every frame |
| Point tracks | none: + CoTracker3 | its own track head | none: + CoTracker3 | the queries themselves |
| Resolution | 392×224 | 518×294 | 518×294 | 518×294, two chunks of 48 sharing 8 |
Two of the four have no tracking head. To get a point track out of Depth Anything 3 or MapAnything you run a 2D tracker (CoTracker3 here) and lift its pixel with the model’s own depth, intrinsics and camera of that frame. VGGT’s track head is lifted the same way. Point4D is asked directly: each ball’s pixel in frame 0, plus 576 static pixels on a grid.
Clip 1, what they returned.
Clip 2, the camera moving.
The numbers. The same definitions for every model, toy units, ball radius 0.28. Depth: AbsRel on the static surfaces after one global scale. Camera: ATE after placing the whole output once (frame 0 at its true pose, rotation fitted on all frames, the depth scale). Tracks: how far the answer lies outside the ball, $\max(0,\ \lVert p - c\rVert - 0.28)$, so 0 means on or inside it. Each cell is fixed camera / moving camera.
| fixed / moving camera | Depth Anything 3 | VGGT | MapAnything | Point4D |
|---|---|---|---|---|
| Depth of a frame, static AbsRel | 1.3 % / 0.8 % | 0.6 % / 1.5 % | 1.5 % / 2.4 % | 2.3 % / 3.6 % |
| Scale, true / predicted | 1.09 / 1.22 | 6.85 / 7.32 | 0.88 / 0.95 | 25.42 / 37.08 |
| Camera ATE | 0.01 / 0.06 | 0.00 / 0.67 | 0.01 / 0.52 | 0.08 / 0.80 |
| Camera motion, predicted / true (moving clip) | 1.04× | 0.27× | 0.81× | 1.09× |
| fixed / moving camera | Depth Anything 3 | VGGT | MapAnything | Point4D |
|---|---|---|---|---|
| Point tracks from | CoTracker3 | own track head | CoTracker3 | 3D queries |
| Yellow ball, visible: outside the ball | 1.13 / 0.01 | 2.40 / 1.52 | 1.03 / 0.36 | 0.04 / 0.58 |
| Yellow ball, hidden: outside the ball | 1.71 / 1.49 | 1.53 / 1.59 | 1.74 / 1.79 | 0.11 / 0.01 |
| Blue ball, visible: outside the ball | 0.04 / 0.01 | 2.78 / 2.26 | 0.01 / 0.38 | 0.03 / 0.35 |
| Blue ball, hidden: outside the ball | 0.15 / 0.38 | 1.75 / 1.90 | 0.21 / 0.77 | 0.09 / 0.22 |
| 2D error, visible, px | 14.9 / 1.8 | 142.1 / 58.2 | 14.9 / 1.8 | 6.0 / 38.3 |
| 2D error, hidden, px | 5.1 / 1.2 | 100.0 / 99.4 | 5.1 / 1.2 | 4.0 / 11.5 |
| Static queries, drift by the last frame | - | - | - | 0.15 / 0.95 |
| RTX 5080, 80 frames | Depth Anything 3 | VGGT | MapAnything | Point4D |
|---|---|---|---|---|
| Resolution | 392×224 | 518×294 | 518×294 | 518×294 |
| Forward pass | 4.6 s | 7.2 s | 4.7 s | 14.2 s |
| 2D tracker | 0.6 s | in the pass | 0.6 s | - |
| Time to point tracks | 5.2 s | 7.2 s | 5.3 s | 14.2 s |
| Peak GPU memory | 11.7 GB | 14.1 GB | 11.8 GB | 11.5 GB |
What falls out.
- Per-frame geometry is not the problem. All four are within 4 % on both clips.
- The moving camera separates the dense models. Depth Anything 3 gets the path (1.04×, ATE 0.06). MapAnything gets 0.81× of it. VGGT moves its camera 0.27× the true distance, and its fused cloud smears. A textureless wall and a checkerboard floor are a hard case.
- A hidden ball gets the occluder. Every dense pipeline lifts the pillar: 1.5–1.8 outside the yellow ball, which sits 1.8 behind it. It is not a wrong answer about the ball. It is the pixel’s answer.
- The 2D tracker fails in the open too. On clip 1 CoTracker3 loses the yellow ball behind the pillar, never re-acquires it (it reports the ball hidden for the remaining 44 frames) and ends 46 px off. On clip 2 it tracks both balls to within 2 px. VGGT’s track head never leaves the query pixel on either clip.
- Point4D answers through the occlusion. 0.11 and 0.09 outside the ball on clip 1, 0.01 and 0.22 on clip 2, at a confidence of about 70.
- Point4D’s failure is the seam. On clip 2 the second chunk (frames 40–79) does not sit on the first: the yellow answer is 0.02 outside in frames 20–39 and 1.64 outside in frames 70–79, and the static grid drifts by 0.95. On the fixed camera the drift is 0.15. That is Part 2’s subject.
- It is not faster here. 14 s against 5–7 s. Encode once, decode per query is cheap only when the encoder is cheap; Point4D’s encoder is a 1.4 B backbone run once per chunk, and the dense models run theirs once. D4RT’s 200 FPS claim (below) is for a pose from a few hundred queries on an A100.
The query
flowchart LR
V["video V<br/>T frames"] --> E["encoder E<br/>video ViT, runs once per clip"]
E --> F["global features F<br/>N tokens x C"]
Q["query q = (u, v, t_src, t_tgt, t_cam)<br/>+ the 9x9 patch around (u, v)"] --> D["decoder D<br/>8 cross-attention layers<br/>one query at a time"]
F --> D
D --> P["P = (x, y, z)<br/>+ 2D point, visibility, confidence"]
Five numbers in, three numbers out:
| Slot | Meaning |
|---|---|
| $(u, v)$ | a pixel, in normalised $[0,1]^2$ coordinates |
| $t_\text{src}$ | the frame that pixel lives in |
| $t_\text{tgt}$ | the time at which you want to know where that point is |
| $t_\text{cam}$ | the camera whose coordinate system the answer is expressed in |
The three time indices need not agree. That is the whole trick: which slots you hold fixed turns one decoder into every 4D task.
| Task | $u$ | $v$ | $t_\text{src}$ | $t_\text{tgt}$ | $t_\text{cam}$ |
|---|---|---|---|---|---|
| Point track | fixed | fixed | fixed | $1 \dots T$ | $= t_\text{tgt}$ |
| Point cloud (one shared frame) | $1 \dots W$ | $1 \dots H$ | $1 \dots T$ | $= t_\text{src}$ | fixed |
| Depth map | $1 \dots W$ | $1 \dots H$ | $1 \dots T$ | $= t_\text{src}$ | $= t_\text{src}$ |
| Extrinsics | coarse grid | coarse grid | fixed | $= t_\text{src}$ | $1 \dots T$ |
| Intrinsics | coarse grid | coarse grid | $1 \dots T$ | $= t_\text{src}$ | $= t_\text{src}$ |
- A track: fix $(u, v, t_\text{src})$, sweep $t_\text{tgt} = t_\text{cam}$.
- A point cloud in one frame: every pixel of every frame, $t_\text{cam}$ pinned to one camera. The change of reference frame happens inside the model.
- A depth map: $t_\text{src} = t_\text{tgt} = t_\text{cam}$, keep the $z$ of $\mathbf P$.
Cameras fall out of the point sets
flowchart LR
G["coarse pixel grid<br/>in frame i"] --> Qi["q = (u, v, i, i, i)<br/>answers in camera i"]
G --> Qj["q = (u, v, i, i, j)<br/>answers in camera j"]
Qi --> U["Umeyama<br/>one 3x3 SVD"]
Qj --> U
U --> Pose["relative pose R, t<br/>camera i to camera j"]
Qi --> Fo["per point: f = p_z (u - 0.5) / p_x<br/>median over the grid"]
Fo --> K["focal length<br/>(intrinsics)"]
- Extrinsics. Query a grid of frame $i$ twice, in camera $i$ and in camera $j$. The two answer sets are the same points in two frames; the rigid transform between them is the relative pose. One $3 \times 3$ SVD.
- Intrinsics. Query a grid with all three times equal, assume a pinhole with the principal point at $(0.5, 0.5)$; every point gives a focal length, take the median. A fisheye needs a refinement on top (see the wide-FOV post).
What’s inside a query token
flowchart TB
UV["(u, v)<br/>Fourier features"] --> SUM["query token = sum of five"]
TS["t_src<br/>learned embedding"] --> SUM
TT["t_tgt<br/>learned embedding"] --> SUM
TC["t_cam<br/>learned embedding"] --> SUM
PA["9x9 RGB patch at (u, v) in frame t_src<br/>small MLP"] --> SUM
SUM --> DEC["cross-attention into F<br/>queries never see each other"]
DEC --> OUT["13 numbers: xyz, uv, visibility,<br/>motion, normal, confidence"]
The patch looks like a detail and is not:
| ViT-L on Sintel | AbsRel (scale) ↓ | ATE ↓ |
|---|---|---|
| without the local patch | 0.366 | 0.173 |
| with the 9×9 patch | 0.302 | 0.091 |
- Without it the decoder has to locate a sub-patch pixel from a 16×16-patch token grid alone.
- Because $(u, v)$ is continuous you can query at the original video resolution while the encoder saw 256×256.
- Queries never see each other. The authors tried self-attention between queries: “major performance drops”. This is what makes decoding embarrassingly parallel.
It is very fast
Each query is one cheap cross-attention pass. Cost scales with how many points you ask about, not with $T \times H \times W$.
| Full-video 3D tracks sustained at | 60 FPS | 24 FPS | 10 FPS | 1 FPS |
|---|---|---|---|---|
| DELTA | 0 | 5 | 408 | 5,770 |
| SpatialTrackerV2 | 29 | 84 | 219 | 2,290 |
| D4RT | 550 | 1,570 | 3,890 | 40,180 |
Tracking every pixel without $T^2 HW$ queries
A track for every pixel of every frame is $O(T^2 HW)$ queries. Algorithm 1 is the fix:
- Keep a $T \times H \times W$ occupancy grid. Start tracks only from pixels nobody has visited yet.
- Every decoded track marks all the pixels it visibly passes through as done.
- Frame 0 starts a track from every pixel; those already cover most of every later frame. After that only new content spawns tracks.
- The paper measures 5–15× fewer tracks; the toy world above gets 24×.
What D4RT buys, and what it costs
- ✅ One decoder for everything. Tracks, point clouds, depth, intrinsics and extrinsics are query patterns, not heads.
- ✅ Sparse or dense at will, at cost proportional to what you ask for.
- ✅ Dynamic correspondence, which none of the pure reconstruction models provide.
- ❌ The encoder sees a whole clip at once. Training is 48 frames; a long video has to be chunked, and the paper does not chain tracks across chunks.
- ❌ The query is a pixel. It has to be visible in the source frame. That breaks at a chunk boundary. Part 2.
- ❌ No code or weights.
Part 2: A pixel is a bad handle for a long video (Point4D)
Any encoder has a window, so a 300-frame video becomes overlapping chunks (48 frames, 8 of overlap). For static geometry that is solved: reconstruct each chunk, align the overlaps with a Sim(3). That is VGGT-Long.
For 4D it is not enough. A track has an identity that has to survive the boundary:
- The track from chunk $k$ ends at a 3D point. To continue it you must re-query that point in chunk $k{+}1$.
- A D4RT-style decoder only accepts pixels. So you project the endpoint into an overlap frame and ask again.
- That fails exactly when long videos are interesting. Occluded: the pixel belongs to whatever is in front, and the track attaches itself to the occluder. Left the frame: there is no pixel at all.
- Even for a visible point, the projection runs through an estimated camera.
The fix: query points, not pixels
Point4D’s query is
\[q = (\mathbf p,\; t_\text{src},\; t_\text{tgt},\; t_\text{cam},\; S), \qquad \mathbf p = (x, y, z):\]where is the point at $\mathbf p$ (in camera $t_\text{src}$’s coordinates), whose appearance is $S$, at time $t_\text{tgt}$, in camera $t_\text{cam}$? The slots have D4RT’s names. Where each token comes from is different, and the released code shows it:
flowchart TB
subgraph ENC["encoder output (Depth Anything 3, ViT-g): per frame"]
direction LR
ST["scene tokens F"]
CT["camera token c_t"]
TT["time token tau_t"]
end
XYZ["(x, y, z) in camera t_src<br/>Fourier encoding"] --> SUM
CT -- "src_mlp(c at t_src)" --> SUM
TT -- "tgt_mlp(tau at t_tgt)" --> SUM
CT -- "cam_mlp(c at t_cam)" --> SUM
S["9x9 patch from frame t_patch<br/>patch MLP, cut once and cached"] --> SUM
SUM["query = sum / sqrt(5)"] --> DEC["cross-attention decoder"]
ST --> DEC
DEC --> OUT["x, y, z, u, v, confidence, visibility"]
build_query_embed in point4d/heads/d4rt_head.py.Three things follow:
- The point is a 3D coordinate. Occluded and out-of-frame points are legal queries.
- The appearance patch is cut once, where the query was born, and cached. A point hidden in chunk 5 still carries the look it had in frame 0.
- The handoff is a coordinate transform. The overlap frames’ predicted depth gives a Sim(3) between chunks (Umeyama on a 1024-point grid, closest 85 % of pairs). The endpoint goes through it. No pixel, no matching, no visibility requirement.
Easy to miss: Point4D keeps Depth Anything 3’s dense depth and camera heads. The query decoder only answers motion questions.
Does it work?
The ablation of the query itself, on 200-frame sequences chunked 48/8:
| Query | Dynamic Replica EPE ↓ | Survival ↑ |
|---|---|---|
| 2D pixel (D4RT-style), reprojection handoff | 0.712 | 0.422 |
| 3D point, patch always from the source frame | 0.825 | 0.266 |
| 3D point, patch from any visible frame | 0.155 | 0.812 |
- The middle row matters. 3D coordinates alone do not help: a model trained only on visible source patches has never seen a query for a hidden point, and the handoff hands it exactly that. Both changes are needed.
- By what the point was doing at the handoff (Table 7): occluded points 0.57 (best pixel method) against 0.20; visible 0.29 against 0.14; out of frame 0.34 against 0.31.
- Caveats from the same paper. Within a single chunk Point4D is not the best decoder (4RC, V-DPM win); the gain is at the seams. Dense queries cost more than a DPT head. Everything rides on the predicted depth and the Sim(3), whose errors compound across chunks, which is exactly the drift in the table above.
In one line. D4RT turned reconstruction from “decode everything” into “ask about a point”; Point4D noticed that a point should be a coordinate, not a pixel.
Part 3: What it takes to train one
It is not just the decoder
| D4RT | Point4D | |
|---|---|---|
| Encoder | ViT-g video transformer, ~1 B params | Depth Anything 3 ViT-g, plus a time token per frame |
| Encoder initialisation | VideoMAE | DA3, including its depth head and camera decoder |
| Decoder | 8 cross-attention layers, 144 M params, from scratch | 8 cross-attention layers, from scratch |
| Encoder trained? | Yes, end-to-end | Yes, at 0.1× the learning rate |
| Released weights | none | one file of 5.8 GB, the whole network in fp32 |
- The initialisation is not optional: from random weights, D4RT’s depth AbsRel goes from 0.30 to 0.74 and pose ATE from 0.09 to 0.33 (Table 11).
- The honest description: a pretrained dense model, fine-tuned end-to-end, with a new query decoder grown on top.
The datasets
| Dataset | Kind | Size | Ground truth | Share of each epoch |
|---|---|---|---|---|
| PointOdyssey | synthetic | 159 videos, ~200 K frames | 3D point trajectories, depth, cameras | 19.8% |
| Dynamic Replica | synthetic | 524 videos, 145 K frames | long-range trajectories, depth, cameras | 19.8% |
| BEDLAM 2 | synthetic humans | 27 K sequences, ~8 M frames | SMPL-X bodies, cameras, depth | 19.8% |
| CoTracker’s Kubric set | synthetic | 6 K sequences × 24 frames | point tracks with visibility | 11.9% |
| Kubric MOVi-F | synthetic | ~5 K videos × 24 frames | depth, flow, object poses, cameras | 11.9% |
| Waymo DriveTrack | real | ~10 K videos | 3D tracks from LiDAR and 3D boxes | 5.8% |
| OmniWorld | mixed | 600 K+ sequences | depth, cameras; no trajectories | 2.0% |
| ScanNet, ScanNet++, BlendedMVS, CO3Dv2 | static | large | depth, poses | 2.0% each |
| WildRGB-D | real, static | ~20 K object videos | metric poses, depth, masks | 1.0% |
- The dynamic data is almost all synthetic. 83 % of an epoch is dynamic data with trajectory labels; every source is rendered except Waymo. Real video with 3D trajectory ground truth barely exists.
- There is no self-supervision. An L1 between the predicted point and a ground-truth point, plus small 2D, visibility (and for D4RT motion, normal) losses. What replaces it is reuse: static datasets declare every point a stationary trajectory.
- The query interface is what makes the mixture usable. OmniWorld has no trajectory labels, so its queries are restricted to $t_\text{src} = t_\text{tgt}$. Loss terms are applied only where a label exists.
How the queries are sampled
| D4RT | Point4D | |
|---|---|---|
| Queries per training clip | 2,048 per 48-frame clip | 750 per frame |
| Hard-example bias | 30 % near depth or motion edges | 40 % on Sobel edges |
| Time slots | uniform; $t_\text{tgt} = t_\text{cam}$ with probability 0.4 | uniform; $t_\text{cam} = t_\text{tgt}$ for 40 % |
| Extra weighting | confidence-weighted L1 | dynamic points upweighted |
A dense head on a 48-frame clip at 256×256 is scored on 3.1 M outputs; D4RT on 2,048 decoder calls. Supervision cost does not depend on resolution, which is also why the decoder can later be queried at full resolution.
The recipe
| D4RT | Point4D | |
|---|---|---|
| Clip length | 48 frames | 16–64 frames |
| Resolution | 256×256, with an aspect-ratio token | width sampled in [252, 518] |
| Optimiser | AdamW, wd 0.03, clip 10; warm-up 2,500 steps to $10^{-4}$, cosine | AdamW; warm-up 10 epochs to $10^{-4}$, cosine; 0.1× for DA3 parts |
| Length | 500 K steps | 150 epochs |
| Hardware | 64 TPU chips, “just over 2 days” | 8 H100 GPUs |
| Losses | $c\,\mathcal L_{3D} - 0.2\log c$ + 0.1 (2D, vis, motion) + 0.5 normal | point + conf + 2D + vis, unit weights |
So is it light to train?
| Model | Hardware | Time |
|---|---|---|
| VGGT | 64 × A100 | 9 days |
| Depth Anything 3 (giant) | 128 × H100 | ~10 days |
| D4RT | 64 TPU chips | ~2 days |
| Point4D | 8 × H100 | 150 epochs |
Yes, by a lot, but not because “only the decoder is new”. The saving comes from a pretrained initialisation and sparse supervision. Point4D at eight GPUs is possible because Depth Anything 3 already paid for the 128 H100s.
How they are evaluated
- D4RT, as a general reconstruction model: 3D tracking on TAPVid-3D; point clouds on Sintel and ScanNet; video depth on Sintel, ScanNet, KITTI, Bonn; pose on Sintel, ScanNet, RealEstate10K; throughput on one A100.
- Point4D, on the one thing it claims: long-video tracking on 200-frame PointOdyssey and Dynamic Replica and 150-frame Panoptic Studio, chunked 48/8, with end-point error, APD and survival; horizons to 500 frames; single-chunk tracking, where it is honest about not winning.
- The caveat: PointOdyssey and Dynamic Replica are also the two largest training sources. Most of the evidence is on held-out synthetic splits.
Side by side
| Dense heads | D4RT: 2D query | Point4D: 3D query | |
|---|---|---|---|
| What the decoder is | DPT head per output | one cross-attention decoder | same decoder |
| The question you can ask | “depth / points of this frame” | “where is pixel $(u,v,t)$ at time $t’$, in camera $t’’$” | “where is point $\mathbf p$ at time $t’$, in camera $t’’$” |
| Dynamic correspondence | no | yes | yes |
| Cost scales with | $T \times H \times W$ | number of queries | number of queries |
| Query must be visible in the source frame? | n/a | yes | no |
| Chaining across chunks | align geometry with a Sim(3) | project endpoint to a pixel, re-query | transform endpoint with the Sim(3), re-query |
| Where geometry comes from | the heads | the queries | DA3’s dense heads |
| Training compute | VGGT 64 A100 × 9 d; DA3 128 H100 × 10 d | 64 TPU chips × 2 d | 8 H100 |
| Code | VGGT, DA3, MapAnything | not released | released, weights on Hugging Face |
Part 4: Which one should you reach for?
flowchart TD
Q1["<b>Does anything in the scene move?</b>"]
Q2["<b>Does the video fit one encoder window?</b><br/><i>~48–64 frames</i>"]
Q3["<b>How many points do you need?</b>"]
DH["<b>dense heads</b><br/>VGGT · DA3 · MapAnything<br/><i>depth, points, pose per frame</i>"]
QD["<b>query decoder</b><br/>D4RT-style 2D queries<br/><i>cost ∝ points asked for</i>"]
DD["<b>dense 4D heads</b><br/>4RC · V-DPM<br/><i>or D4RT's occupancy-grid tracking</i>"]
P4["<b>3D query + chaining</b><br/>Point4D<br/><i>handoff is a Sim(3), not a pixel</i>"]
Q1 -- "no" --> DH
Q1 -- "yes" --> Q2
Q2 -- "no, it is long" --> P4
Q2 -- "yes" --> Q3
Q3 -- "a few thousand tracks,<br/>or a pose" --> QD
Q3 -- "every pixel, every frame" --> DD
classDef q fill:#1a1d27,stroke:#6c9eff,color:#e0e0e6;
classDef rec fill:#17301f,stroke:#2e8b57,color:#e0e0e6;
classDef p4 fill:#2a2340,stroke:#8b5cf6,color:#e0e0e6;
class Q1,Q2,Q3 q;
class DH,QD,DD rec;
class P4 p4;
For a few hundred frames of handheld video with people walking in and out of view:
- You have to chunk, so the question is only how tracks cross the seams.
- The seams are where the occlusions are. Points that never disappear would not have needed a long-video method.
- Point4D is the one you can run. Same DA3 backbone as the static tools, weights on the hub, and the chaining loop is 250 lines of numpy around the model.
References
- Zhang et al., Efficiently Reconstructing Dynamic Scenes One D4RT at a Time, CVPR 2026 — arXiv:2512.08924 · project page
- Jeon, Karhade, Ramanan, Tulsiani, Point4D: Long-range 4D Motion Reconstruction, 2026 — arXiv:2609.09145 · code · weights · project page
- Sajjadi et al., Scene Representation Transformer, CVPR 2022 — arXiv:2111.13152 (the decoder D4RT is modelled on)
- Lin et al., Depth Anything 3: Recovering the Visual Space from Any Views, 2025 — arXiv:2511.10647 · code (Point4D’s backbone; DA3NESTED-GIANT-LARGE is the model run above)
- Tong et al., VideoMAE, NeurIPS 2022 — arXiv:2203.12602 (D4RT’s encoder initialisation)
- Wang et al., VGGT: Visual Geometry Grounded Transformer, CVPR 2025 — arXiv:2503.11651 · code
- Deng et al., VGGT-Long: Chunk it, Loop it, Align it, 2025 — arXiv:2507.16443
- Keetha et al., MapAnything: Universal Feed-Forward Metric 3D Reconstruction, 2025 — arXiv:2509.13414 · code
- Karaev et al., CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos, 2024 — arXiv:2410.11831 · code (the 2D tracker paired with the dense models above)
- Luo et al., 4RC: 4D Reconstruction via Conditional Querying Anytime and Anywhere, 2026 — arXiv:2602.10094
- Sucar et al., V-DPM: 4D Video Reconstruction with Dynamic Point Maps, 2026 — arXiv:2601.09499
- Xiao et al., SpatialTrackerV2, ICCV 2025 — arXiv:2507.12462
- Zheng et al., PointOdyssey, ICCV 2023 — pointodyssey.com · Karaev et al., DynamicStereo (Dynamic Replica), CVPR 2023 — github · Balasingam et al., DriveTrack, CVPR 2024 — arXiv:2312.09523 · Zhou et al., OmniWorld, 2025 — arXiv:2509.12201
- Umeyama, Least-squares estimation of transformation parameters between two point patterns, TPAMI 1991
All paper figures above are reproduced from the linked arXiv preprints and are credited in their captions. Code details are from the Point4D repository at the commit current on 17 September 2026.