box3d

Generic 3D oriented-box geometry shared by detection models and their evaluation.

Boxes are parameterized as \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\): gravity-aligned center, full extents, and heading \(\theta\) (radians) counter-clockwise about \(+z\) from \(+x\). Axis-aligned boxes set \(\theta = 0\). Corners are \((\ldots, 8, 3)\) with the top face (max \(z\)) first; IoU is frame-invariant so these work in any right-handed frame.

Functions:

box_corners –

Convert parameterized boxes to their 8 corners.
decode_box_residuals –

Decode predicted box residuals against anchors (OpenPCDet's ResidualCoder).
encode_box_residuals –

Encode ground-truth boxes into anchor-relative residuals (inverse of decode_box_residuals).
angle_to_class –

Convert continuous heading angles to discrete bin classes and residuals.
class_to_angle –

Invert angle_to_class: recover continuous heading angles from bin classes and residuals.
class_to_size –

Recover full box edge lengths from a size class index and residual (inverse of the size encoding).
limit_period –

Wrap an angle to \([-\text{offset} \cdot \text{period}, (1 - \text{offset}) \cdot \text{period})\).
box3d_overlap –

Pairwise 3D intersection volume and IoU of two sets of boxes given as corners.
boxes_iou_bev –

Pairwise bird's-eye (top-down) rotated-box IoU.
boxes_iou_nearest_bev –

Pairwise bird's-eye IoU of boxes snapped to their nearest axis-aligned orientation.
boxes_iou3d –

Pairwise oriented 3D box IoU.
nms3d –

Greedy 3D non-maximum suppression.
points_in_boxes –

Test every point against every oriented box.
count_points_in_boxes –

Count how many points fall inside each oriented box.
projected_ignore_mask –

Flag boxes whose image projection is shorter than min_height pixels (the KITTI difficulty rule).

box_corners [source] ¶

box_corners(boxes: Tensor) -> Tensor

Convert parameterized boxes to their 8 corners.

The heading is counter-clockwise about \(+z\) from \(+x\); boxes are \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents.

Parameters:

boxes (Tensor) –

Boxes \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\), shape \((\ldots, 7)\).

Returns:

Tensor –

Corner coordinates, shape \((\ldots, 8, 3)\), with the top face (max \(z\)) as corners \(0..3\).

Shape

boxes: \((\ldots, 7)\)
output: \((\ldots, 8, 3)\)

decode_box_residuals [source] ¶

decode_box_residuals(
    encodings: Tensor,
    anchors: Tensor,
    *,
    angle_by_sincos: bool = False,
) -> Tensor

Decode predicted box residuals against anchors (OpenPCDet's ResidualCoder).

Residuals encode the center offset normalized by the anchor base diagonal, log-size ratios, and an angle term: a plain delta by default, or a \((\cos, \sin)\) pair when angle_by_sincos (one extra channel). Trailing channels (e.g. nuScenes velocity) decode as plain deltas.

Parameters:

encodings (Tensor) –

Predicted residuals, shape \((\ldots, 7 + C)\), or \((\ldots, 8 + C)\) with angle_by_sincos.
anchors (Tensor) –

Matching anchors \((x, y, z, d_x, d_y, d_z, \theta, \ldots)\), shape \((\ldots, 7 + C)\).
angle_by_sincos (bool, default: False ) –

Whether the heading residual is encoded as \((\cos, \sin)\).

Returns:

Tensor –

Decoded boxes \((c_x, c_y, c_z, d_x, d_y, d_z, \theta, \ldots)\), shape \((\ldots, 7 + C)\).

Shape

encodings: \((\ldots, 7 + C)\) or \((\ldots, 8 + C)\)
anchors: \((\ldots, 7 + C)\)
output: \((\ldots, 7 + C)\)

Example

>>> anchors = torch.tensor([[0.0, 0.0, 0.0, 4.0, 2.0, 1.5, 0.0]])
>>> decode_box_residuals(torch.zeros(1, 7), anchors)
tensor([[0.0000, 0.0000, 0.0000, 4.0000, 2.0000, 1.5000, 0.0000]])

encode_box_residuals [source] ¶

encode_box_residuals(
    boxes: Tensor,
    anchors: Tensor,
    *,
    angle_by_sincos: bool = False,
) -> Tensor

Encode ground-truth boxes into anchor-relative residuals (inverse of decode_box_residuals).

The exact inverse of decode_box_residuals: the center offset is normalized by the anchor base diagonal, sizes become log ratios, and the heading becomes a plain delta or a \((\cos, \sin)\) pair. Extents are clamped to \(10^{-5}\) before the log so a degenerate box does not produce a non-finite target.

Parameters:

boxes (Tensor) –

Ground-truth boxes \((c_x, c_y, c_z, d_x, d_y, d_z, \theta, \ldots)\), shape \((\ldots, 7 + C)\).
anchors (Tensor) –

Matching anchors \((x, y, z, d_x, d_y, d_z, \theta, \ldots)\), shape \((\ldots, 7 + C)\).
angle_by_sincos (bool, default: False ) –

Whether to encode the heading residual as \((\cos, \sin)\) (one extra channel).

Returns:

Tensor –

Residual encodings, shape \((\ldots, 7 + C)\), or \((\ldots, 8 + C)\) with angle_by_sincos.

Shape

boxes: \((\ldots, 7 + C)\)
anchors: \((\ldots, 7 + C)\)
output: \((\ldots, 7 + C)\) or \((\ldots, 8 + C)\)

Example

>>> anchors = torch.tensor([[0.0, 0.0, 0.0, 4.0, 2.0, 1.5, 0.0]])
>>> boxes = torch.tensor([[1.0, 0.0, 0.0, 4.0, 2.0, 1.5, 0.0]])
>>> torch.allclose(decode_box_residuals(encode_box_residuals(boxes, anchors), anchors), boxes)
True

angle_to_class [source] ¶

angle_to_class(
    angle: Tensor, num_heading_bins: int
) -> Tuple[Tensor, Tensor]

Convert continuous heading angles to discrete bin classes and residuals.

The range \([0, 2\pi)\) is split into num_heading_bins equal bins centered at \(0, 1 \cdot (2\pi / N), \ldots, (N - 1) \cdot (2\pi / N)\). The returned class and residual satisfy \(\text{class} \cdot (2\pi / N) + \text{residual} = \text{angle}\).

Parameters:

angle (Tensor) –

Heading angles in radians of shape \((K,)\).
num_heading_bins (int) –

Number of heading bins \(N\).

Returns:

Tuple[Tensor, Tensor] –

A tuple of the per-angle class indices (long, shape \((K,)\)) and residual angles (shape \((K,)\)).

class_to_angle [source] ¶

class_to_angle(
    heading_class: Tensor,
    heading_residual: Tensor,
    num_heading_bins: int,
) -> Tensor

Invert angle_to_class: recover continuous heading angles from bin classes and residuals.

A single bin (num_heading_bins == 1, axis-aligned boxes) always decodes to a heading of \(0\).

Parameters:

heading_class (Tensor) –

Bin class indices (long) of shape \((K,)\).
heading_residual (Tensor) –

Per-angle residuals of shape \((K,)\).
num_heading_bins (int) –

Number of heading bins \(N\).

Returns:

Tensor –

The recovered heading angles of shape \((K,)\).

class_to_size [source] ¶

class_to_size(
    size_class: Tensor,
    size_residual: Tensor,
    mean_sizes: Tensor,
) -> Tensor

Recover full box edge lengths from a size class index and residual (inverse of the size encoding).

Parameters:

size_class (Tensor) –

Size class indices (long) of shape \((K,)\).
size_residual (Tensor) –

Per-axis residuals of shape \((K, 3)\).
mean_sizes (Tensor) –

Template sizes of shape \((C, 3)\) holding full edge lengths per class.

Returns:

Tensor –

The recovered full edge lengths of shape \((K, 3)\).

limit_period [source] ¶

limit_period(
    val: Tensor, offset: float = 0.5, period: float = pi
) -> Tensor

Wrap an angle to \([-\text{offset} \cdot \text{period}, (1 - \text{offset}) \cdot \text{period})\).

box3d_overlap [source] ¶

box3d_overlap(
    boxes1: Tensor, boxes2: Tensor
) -> Tuple[Tensor, Tensor]

Pairwise 3D intersection volume and IoU of two sets of boxes given as corners.

Signature-compatible with pytorch3d.ops.box3d_overlap, so the exact CUDA implementation can be swapped in behind this interface. Boxes are assumed gravity-aligned (as produced by box_corners): the bird's-eye polygons are intersected exactly (corner containment plus edge crossings) and scaled by the vertical overlap, entirely on the input device.

Parameters:

boxes1 (Tensor) –

Corners of the first set, shape \((M, 8, 3)\) (see box_corners).
boxes2 (Tensor) –

Corners of the second set, shape \((N, 8, 3)\).

Returns:

Tuple[Tensor, Tensor] –

A tuple (intersection_vol, iou), each shape \((M, N)\).

Shape

boxes1: \((M, 8, 3)\)
boxes2: \((N, 8, 3)\)
output: \((M, N)\), \((M, N)\)

boxes_iou_bev [source] ¶

boxes_iou_bev(
    boxes_a: Tensor,
    boxes_b: Tensor,
    *,
    aligned: bool = False,
) -> Tensor

Pairwise bird's-eye (top-down) rotated-box IoU.

Projects both box sets onto the ground plane (ignoring \(z\)) and intersects the oriented rectangles. The heading is counter-clockwise (angle increases \(x \to y\)). Runs entirely in torch, so it stays on CUDA tensors without a custom extension; the cost is \(O(N \cdot M)\).

Parameters:

boxes_a (Tensor) –

Boxes \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents, shape \((N, 7)\).
boxes_b (Tensor) –

Boxes in the same layout, shape \((M, 7)\).
aligned (bool, default: False ) –

With True, boxes_a and boxes_b pair up row by row (\(N = M\)) and the result is \((N,)\).

Returns:

Tensor –

Pairwise BEV IoU in \([0, 1]\), shape \((N, M)\).

Shape

boxes_a: \((N, 7)\)
boxes_b: \((M, 7)\)
output: \((N, M)\)

Example

>>> a = torch.tensor([[0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0]])
>>> b = torch.tensor([[0.5, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0]])
>>> round(float(boxes_iou_bev(a, b)), 4)
0.3333

boxes_iou_nearest_bev [source] ¶

boxes_iou_nearest_bev(
    boxes_a: Tensor, boxes_b: Tensor
) -> Tensor

Pairwise bird's-eye IoU of boxes snapped to their nearest axis-aligned orientation.

Each box keeps its center and takes the footprint of its heading rounded to the nearest multiple of \(\pi / 2\), so \(d_x\) and \(d_y\) swap when the heading is closer to \(\pm \pi / 2\) than to \(0\); the IoU is then the plain axis-aligned rectangle IoU. The anchor-based detectors assign their anchors by this measure rather than by the exact rotated overlap of boxes_iou_bev.

Parameters:

boxes_a (Tensor) –

Boxes \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents, shape \((N, 7)\).
boxes_b (Tensor) –

Boxes in the same layout, shape \((M, 7)\).

Returns:

Tensor –

Pairwise IoU in \([0, 1]\), shape \((N, M)\).

Shape

boxes_a: \((N, 7)\)
boxes_b: \((M, 7)\)
output: \((N, M)\)

Example

>>> a = torch.tensor([[0.0, 0.0, 0.0, 2.0, 1.0, 1.0, 0.0]])
>>> b = torch.tensor([[0.0, 0.0, 0.0, 1.0, 2.0, 1.0, math.pi / 2]])
>>> round(float(boxes_iou_nearest_bev(a, b)), 4)
1.0

boxes_iou3d [source] ¶

boxes_iou3d(
    boxes_a: Tensor,
    boxes_b: Tensor,
    *,
    aligned: bool = False,
) -> Tensor

Pairwise oriented 3D box IoU.

The BEV intersection area (rotated rectangles, ignoring \(z\)) is multiplied by the vertical overlap of the height intervals \([c_z - d_z/2, c_z + d_z/2]\) to give the intersection volume, then divided by the union. The heading is counter-clockwise about \(+z\) from \(+x\); boxes are \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents. Runs entirely in torch (CUDA-capable, no custom extension); the cost is \(O(N \cdot M)\).

Parameters:

boxes_a (Tensor) –

Boxes \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents, shape \((N, 7)\).
boxes_b (Tensor) –

Boxes in the same layout, shape \((M, 7)\).
aligned (bool, default: False ) –

With True, boxes_a and boxes_b pair up row by row (\(N = M\)) and the result is \((N,)\).

Returns:

Tensor –

Pairwise 3D IoU in \([0, 1]\), shape \((N, M)\).

Shape

boxes_a: \((N, 7)\)
boxes_b: \((M, 7)\)
output: \((N, M)\)

Example

>>> a = torch.tensor([[0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0]])
>>> b = torch.tensor([[0.5, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0]])
>>> round(float(boxes_iou3d(a, b)), 4)
0.3333

nms3d [source] ¶

nms3d(
    boxes: Tensor,
    scores: Tensor,
    iou_threshold: float,
    *,
    labels: OptTensor = None,
    batch: OptTensor = None,
    rotated: bool = False,
    max_keep: Optional[int] = None,
) -> Tensor

Greedy 3D non-maximum suppression.

Keeps the highest-scoring box of each overlapping cluster. By default the suppression criterion is the 3D IoU of the boxes' axis-aligned bounding boxes (cheap corner min / max); with rotated=True it is the exact rotated bird's-eye IoU of boxes_iou_bev (the KITTI outdoor protocol, where the axis-aligned surrogate over-suppresses angled neighbors at low thresholds). Pass labels to restrict suppression to boxes of the same class, and batch (PyG-style per-box scene index) to run NMS independently per scene and return a single index tensor over the concatenated input. The heading is counter-clockwise about \(+z\) from \(+x\); boxes are \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents.

Parameters:

boxes (Tensor) –

Boxes \((N, 7)\) (see box_corners).
scores (Tensor) –

Per-box confidence, shape \((N,)\).
iou_threshold (float) –

IoU above which a lower-scoring box is removed.
labels (OptTensor, default: None ) –

Optional per-box class, shape \((N,)\); when given, only same-class boxes suppress each other.
batch (OptTensor, default: None ) –

Optional per-box scene index, shape \((N,)\); when given, NMS runs independently per scene.
rotated (bool, default: False ) –

Suppress on the exact rotated BEV IoU (boxes_iou_bev) instead of the axis-aligned 3D IoU.
max_keep (Optional[int], default: None ) –

Optional cap on the boxes kept per scene; suppression stops once it is reached, so the kept set equals the first max_keep entries of the uncapped result.

Returns:

Tensor –

Indices of the kept boxes (into the input), highest score first within each scene, shape \((K,)\) long.

Shape

boxes: \((N, 7)\)
output: \((K,)\)

points_in_boxes [source] ¶

points_in_boxes(pos: Tensor, boxes: Tensor) -> Tensor

Test every point against every oriented box.

The point offsets to each box center are rotated into the box frame by \(-\theta\) about \(+z\) and compared with the half extents. The heading is counter-clockwise about \(+z\) from \(+x\); boxes are \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents. Pairs are materialized at once, so restrict large scenes to their own boxes (or use count_points_in_boxes, which chunks).

Parameters:

pos (Tensor) –

Point coordinates, shape \((N, 3)\).
boxes (Tensor) –

Boxes \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents, shape \((K, 7)\).

Returns:

Tensor –

Boolean containment mask, shape \((N, K)\).

Shape

pos: \((N, 3)\)
boxes: \((K, 7)\)
output: \((N, K)\)

Example

>>> pos = torch.tensor([[0.0, 0.0, 0.0], [5.0, 5.0, 5.0]])
>>> boxes = torch.tensor([[0.0, 0.0, 0.0, 2.0, 2.0, 2.0, 0.0], [5.0, 5.0, 5.0, 1.0, 1.0, 1.0, 0.3]])
>>> points_in_boxes(pos, boxes).tolist()
[[True, False], [False, True]]

count_points_in_boxes [source] ¶

count_points_in_boxes(
    pos: Tensor,
    boxes: Tensor,
    *,
    pos_batch: OptTensor = None,
    box_batch: OptTensor = None,
) -> Tensor

Count how many points fall inside each oriented box.

Pass pos_batch and box_batch (PyG-style per-point / per-box scene indices) to restrict each box's count to points from its own scene, so boxes of different scenes never share points. The heading is counter-clockwise about \(+z\) from \(+x\); boxes are \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents.

Parameters:

pos (Tensor) –

Point coordinates, shape \((N, 3)\).
boxes (Tensor) –

Boxes \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents, shape \((K, 7)\).
pos_batch (OptTensor, default: None ) –

Optional per-point scene index, shape \((N,)\).
box_batch (OptTensor, default: None ) –

Optional per-box scene index, shape \((K,)\).

Returns:

Tensor –

Per-box point count, shape \((K,)\) long.

Shape

pos: \((N, 3)\)
boxes: \((K, 7)\)
output: \((K,)\)

Example

>>> pos = torch.tensor([[0.0, 0.0, 0.0], [5.0, 5.0, 5.0]])
>>> boxes = torch.tensor([[0.0, 0.0, 0.0, 2.0, 2.0, 2.0, 0.0]])
>>> count_points_in_boxes(pos, boxes).tolist()
[1]

projected_ignore_mask [source] ¶

projected_ignore_mask(
    boxes: Tensor,
    calib: Tensor,
    image_shape: Tensor,
    *,
    min_height: float = 25.0,
) -> Tensor

Flag boxes whose image projection is shorter than min_height pixels (the KITTI difficulty rule).

Each box's 8 corners are projected through the \((3, 4)\) homogeneous LiDAR-to-image matrix (rows \(0\) and \(1\) divided by the perspective depth of row \(2\)), the vertical pixel coordinates are clipped to the image rows \([0, \text{height} - 1]\), and a box is flagged when its clipped pixel height is strictly below min_height. Only the vertical extent is used; the width entry of image_shape keeps the dataset's \((\text{height}, \text{width})\) contract. The KITTI protocol excludes such predictions from scoring (the prediction-side ignore_mask of box_matches), with min_height at \(40\) / \(25\) / \(25\) px for the easy / moderate / hard difficulties. For KITTI, compose the calib as \(P_2 \cdot [R_0 T_\text{velo}; 0\ 0\ 0\ 1]\) with the third row taken from \(R_0 T_\text{velo}\), so the perspective divide is by the rectified depth.

calib and image_shape broadcast on a leading box dimension: pass a single \((3, 4)\) / \((2,)\) frame for all boxes, or per-box \((N, 3, 4)\) / \((N, 2)\) rows (e.g. a stacked per-frame calib indexed by the boxes' scene index) to score a multi-frame batch in one call.

Parameters:

boxes (Tensor) –

Boxes \((c_x, c_y, c_z, d_x, d_y, d_z, \theta)\) with full extents, shape \((N, 7)\).
calib (Tensor) –

Homogeneous projection from LiDAR coordinates to image pixels, shape \((3, 4)\) or per-box \((N, 3, 4)\).
image_shape (Tensor) –

Image \((\text{height}, \text{width})\) in pixels, shape \((2,)\) or per-box \((N, 2)\).
min_height (float, default: 25.0 ) –

Pixel height below which a box is flagged.

Returns:

Tensor –

Boolean ignore mask, shape \((N,)\).

Shape

boxes: \((N, 7)\)
calib: \((3, 4)\) or \((N, 3, 4)\)
image_shape: \((2,)\) or \((N, 2)\)
output: \((N,)\)

Example

>>> calib = torch.tensor([[50.0, -100.0, 0.0, 0.0], [50.0, 0.0, -100.0, 0.0], [1.0, 0.0, 0.0, 0.0]])
>>> boxes = torch.tensor([[10.0, 0.0, 0.0, 2.0, 2.0, 1.0, 0.0]])
>>> projected_ignore_mask(boxes, calib, torch.tensor([100, 200]))
tensor([True])
>>> boxes = torch.tensor([[10.0, 0.0, 0.0, 2.0, 2.0, 1.0, 0.0], [2.0, 0.0, 0.0, 2.0, 2.0, 1.0, 0.0]])
>>> projected_ignore_mask(boxes, calib.expand(2, 3, 4), torch.tensor([[100, 200], [100, 200]]))
tensor([ True, False])