SECOND

SECOND detection models.

First page of SECOND: Sparsely Embedded Convolutional Detection

Sensors · October 2018

Classes:

VoxelBackbone8x –

Sparse 3D convolutional voxel backbone (VoxelBackBone8x), \(8\times\) downsampling.
SECONDDetection –

SECOND 3D object detector (packed point format).
SparseBasicBlock –

Submanifold residual block (SparseBasicBlock): two \(3\times3\times3\) subm convs + skip.
VoxelResBackbone8x –

Residual sparse 3D voxel backbone (VoxelResBackBone8x), \(8\times\) downsampling.
SECONDMultiHeadDetection –

SECOND with a multi-group anchor head (nuScenes 10-class, packed point format).

VoxelBackbone8x ¶

VoxelBackbone8x(
    in_channels: int,
    *,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: Module

Sparse 3D convolutional voxel backbone (VoxelBackBone8x), \(8\times\) downsampling.

Four sparse conv stages downsample the voxel grid by \(2\times\) in \(x\)/\(y\) (stages 2-4) while a final \((3, 1, 1)\) sparse conv squeezes the height to 2, yielding a dense BEV tensor after height compression.

Parameters:

in_channels (int) –

Input voxel feature channels (e.g. \(4\) for mean \(x, y, z, \text{intensity}\)).
act (Union[str, Callable, None], default: 'relu' ) –

Activation type or callable.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra activation arguments.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization type or callable.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra normalization arguments.

SECONDDetection ¶

SECONDDetection(
    in_channels: int = 4,
    num_classes: int = 3,
    *,
    voxel_size: Sequence[float] = (0.05, 0.05, 0.1),
    point_cloud_range: Sequence[float] = (
        0.0,
        -40.0,
        -3.0,
        70.4,
        40.0,
        1.0,
    ),
    anchor_sizes: Sequence[Sequence[float]],
    anchor_bottom_heights: Sequence[float],
    feature_map_stride: int,
    anchor_rotations: Sequence[float] = (0.0, 1.57),
    layer_nums: Sequence[int] = (5, 5),
    layer_strides: Sequence[int] = (1, 2),
    num_filters: Sequence[int] = (128, 256),
    upsample_strides: Sequence[int] = (1, 2),
    num_upsample_filters: Sequence[int] = (256, 256),
    num_dir_bins: int = 2,
    dir_offset: float = 0.78539,
    dir_limit_offset: float = 0.0,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: DetectionModel

SECOND 3D object detector (packed point format).

Reference: Yan et al., 2018. Reference implementation: open-mmlab/OpenPCDet.

Parameters:

in_channels (int, default: 4 ) –

Raw point feature channels including xyz (e.g. \(4\) for \(x, y, z, \text{intensity}\)).
num_classes (int, default: 3 ) –

Number of foreground classes.
voxel_size (Sequence[float], default: (0.05, 0.05, 0.1) ) –

Voxel size \((v_x, v_y, v_z)\).
point_cloud_range (Sequence[float], default: (0.0, -40.0, -3.0, 70.4, 40.0, 1.0) ) –

Range \((x_\min, y_\min, z_\min, x_\max, y_\max, z_\max)\).
anchor_sizes (Sequence[Sequence[float]]) –

Per-class box size \((d_x, d_y, d_z)\), one row per class.
anchor_bottom_heights (Sequence[float]) –

Per-class anchor bottom \(z\), one per class.
anchor_rotations (Sequence[float], default: (0.0, 1.57) ) –

Yaw angles (radians) shared by all classes.
feature_map_stride (int) –

BEV feature-map stride of the head.
layer_nums (Sequence[int], default: (5, 5) ) –

2D backbone conv counts per level.
layer_strides (Sequence[int], default: (1, 2) ) –

2D backbone downsample strides per level.
num_filters (Sequence[int], default: (128, 256) ) –

2D backbone channel widths per level.
upsample_strides (Sequence[int], default: (1, 2) ) –

2D backbone upsample strides per level.
num_upsample_filters (Sequence[int], default: (256, 256) ) –

2D backbone upsample channels per level.
num_dir_bins (int, default: 2 ) –

Number of direction bins in the head.
dir_offset (float, default: 0.78539 ) –

Direction-classifier angle offset.
dir_limit_offset (float, default: 0.0 ) –

Heading wrap offset used during decoding.
act (Union[str, Callable, None], default: 'relu' ) –

Activation type or callable for the 3D/2D backbones.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra activation arguments.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization type or callable for the 3D/2D backbones.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra normalization arguments.

Methods:

configure_backbone_3d –

Build the sparse 3D voxel backbone.
configure_backbone –

Build the 2D BEV backbone.
configure_head –

Build the single-group anchor head.
decode –

Decode a forward output into raw per-anchor detections (see AnchorHeadSingle.decode).
reset_classifier –

Replace the classification branch of the detection head for num_classes outputs.

Attributes:

num_features (int) –

Channel count \(C\) of the BEV feature map entering the head.

num_features `property` ¶

num_features: int

Channel count \(C\) of the BEV feature map entering the head.

configure_backbone_3d ¶

configure_backbone_3d() -> VoxelBackbone8x

Build the sparse 3D voxel backbone.

configure_backbone ¶

configure_backbone() -> BaseBEVBackbone

Build the 2D BEV backbone.

configure_head ¶

configure_head() -> AnchorHeadSingle

Build the single-group anchor head.

decode ¶

decode(out: AnchorHeadOutput) -> Detection3D

Decode a forward output into raw per-anchor detections (see AnchorHeadSingle.decode).

reset_classifier ¶

reset_classifier(num_classes: int) -> None

Replace the classification branch of the detection head for num_classes outputs.

Models whose head is not rebuildable in isolation raise NotImplementedError.

SparseBasicBlock ¶

SparseBasicBlock(
    channels: int,
    indice_key: str,
    *,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: Module

Submanifold residual block (SparseBasicBlock): two \(3\times3\times3\) subm convs + skip.

A plain nn.Module (driven directly rather than via SparseSequential) so this file imports without spconv; the sparse convs are built lazily in __init__.

Parameters:

channels (int) –

Input and output channels.
indice_key (str) –

Shared submanifold indice key (reuses the rulebook within the block).
act (Union[str, Callable, None], default: 'relu' ) –

Activation type or callable.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra activation arguments.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization type or callable.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra normalization arguments.

VoxelResBackbone8x ¶

VoxelResBackbone8x(
    in_channels: int,
    *,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: Module

Residual sparse 3D voxel backbone (VoxelResBackBone8x), \(8\times\) downsampling.

Like VoxelBackbone8x but with SparseBasicBlock residual stages and a 128-channel stage 4 (used by the nuScenes SECOND multihead detector).

Parameters:

in_channels (int) –

Input voxel feature channels (e.g. \(5\) for nuScenes).
act (Union[str, Callable, None], default: 'relu' ) –

Activation type or callable.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra activation arguments.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization type or callable.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra normalization arguments.

SECONDMultiHeadDetection ¶

SECONDMultiHeadDetection(
    in_channels: int = 5,
    num_classes: int = 10,
    *,
    voxel_size: Sequence[float] = (0.1, 0.1, 0.2),
    point_cloud_range: Sequence[float] = (
        -51.2,
        -51.2,
        -5.0,
        51.2,
        51.2,
        3.0,
    ),
    anchor_sizes: Sequence[Sequence[float]],
    anchor_bottom_heights: Sequence[float],
    head_class_groups: Sequence[Sequence[int]],
    feature_map_stride: int,
    anchor_rotations: Sequence[float] = (0.0, 1.57),
    layer_nums: Sequence[int] = (5, 5),
    layer_strides: Sequence[int] = (1, 2),
    num_filters: Sequence[int] = (128, 256),
    upsample_strides: Sequence[float] = (1, 2),
    num_upsample_filters: Sequence[int] = (256, 256),
    shared_conv_num_filter: int = 64,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: DetectionModel

SECOND with a multi-group anchor head (nuScenes 10-class, packed point format).

Reference implementation: open-mmlab/OpenPCDet (cbgs_second_multihead). A residual sparse 3D backbone (VoxelResBackbone8x) feeds the shared 2D BEV backbone and an AnchorHeadMulti head. Input points carry 5 features (\(x, y, z, \text{intensity}, \Delta t\)).

Parameters:

in_channels (int, default: 5 ) –

Raw point feature channels including xyz (5 for nuScenes).
num_classes (int, default: 10 ) –

Number of foreground classes (10 for nuScenes).
voxel_size (Sequence[float], default: (0.1, 0.1, 0.2) ) –

Voxel size \((v_x, v_y, v_z)\).
point_cloud_range (Sequence[float], default: (-51.2, -51.2, -5.0, 51.2, 51.2, 3.0) ) –

Range \((x_\min, y_\min, z_\min, x_\max, y_\max, z_\max)\).
anchor_sizes (Sequence[Sequence[float]]) –

Per-class box size \((d_x, d_y, d_z)\), one row per class.
anchor_bottom_heights (Sequence[float]) –

Per-class anchor bottom \(z\), one per class.
head_class_groups (Sequence[Sequence[int]]) –

Class-index groups, one per RPN head (e.g. [[0], [1, 2], ...]).
anchor_rotations (Sequence[float], default: (0.0, 1.57) ) –

Yaw angles (radians) shared by all classes.
feature_map_stride (int) –

BEV feature-map stride of the head.
layer_nums (Sequence[int], default: (5, 5) ) –

2D backbone conv counts per level.
layer_strides (Sequence[int], default: (1, 2) ) –

2D backbone downsample strides per level.
num_filters (Sequence[int], default: (128, 256) ) –

2D backbone channel widths per level.
upsample_strides (Sequence[float], default: (1, 2) ) –

2D backbone upsample factors per level.
num_upsample_filters (Sequence[int], default: (256, 256) ) –

2D backbone upsample channels per level.
shared_conv_num_filter (int, default: 64 ) –

Channels of the head's shared conv.
act (Union[str, Callable, None], default: 'relu' ) –

Activation type or callable for the 3D/2D backbones and head.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra activation arguments.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization type or callable for the 3D/2D backbones and head.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra normalization arguments.

Methods:

configure_backbone_3d –

Build the residual sparse 3D voxel backbone.
configure_backbone –

Build the 2D BEV backbone.
configure_head –

Build the multi-group anchor head.
decode –

Decode a forward output into raw per-anchor detections (see AnchorHeadMulti.decode).
reset_classifier –

Replace the classification branch of the detection head for num_classes outputs.

Attributes:

num_features (int) –

Channel count \(C\) of the BEV feature map entering the head.

num_features `property` ¶

num_features: int

Channel count \(C\) of the BEV feature map entering the head.

configure_backbone_3d ¶

configure_backbone_3d() -> VoxelResBackbone8x

Build the residual sparse 3D voxel backbone.

configure_backbone ¶

configure_backbone() -> BaseBEVBackbone

Build the 2D BEV backbone.

configure_head ¶

configure_head() -> AnchorHeadMulti

Build the multi-group anchor head.

decode ¶

decode(out: AnchorHeadMultiOutput) -> Detection3D

Decode a forward output into raw per-anchor detections (see AnchorHeadMulti.decode).

reset_classifier ¶

reset_classifier(num_classes: int) -> None

Replace the classification branch of the detection head for num_classes outputs.

Models whose head is not rebuildable in isolation raise NotImplementedError.

SECOND

VoxelBackbone8x ¶

SECONDDetection ¶

num_features property ¶

configure_backbone_3d ¶

configure_backbone ¶

configure_head ¶

decode ¶

reset_classifier ¶

SparseBasicBlock ¶

VoxelResBackbone8x ¶

SECONDMultiHeadDetection ¶

num_features property ¶

configure_backbone_3d ¶

configure_backbone ¶

configure_head ¶

decode ¶

reset_classifier ¶

num_features `property` ¶

num_features `property` ¶