second

SECOND detection models.

First page of SECOND: Sparsely Embedded Convolutional Detection

Sensors · October 2018

Classes:

VoxelBackbone8x –

Sparse 3D convolutional voxel backbone (VoxelBackBone8x), \(8\times\) downsampling.
SECONDDetection –

SECOND 3D object detector (packed point format).
VoxelResBackbone8x –

Residual sparse 3D voxel backbone (VoxelResBackBone8x), \(8\times\) downsampling.
SECONDMultiHeadDetection –

SECOND with a multi-group anchor head (nuScenes 10-class, packed point format).

VoxelBackbone8x [source] ¶

VoxelBackbone8x(
    in_channels: int,
    *,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: Module

Sparse 3D convolutional voxel backbone (VoxelBackBone8x), \(8\times\) downsampling.

Four sparse conv stages downsample the voxel grid by \(2\times\) in \(x\)/\(y\) (stages 2-4) while a final \((3, 1, 1)\) sparse conv squeezes the height to 2, yielding a dense BEV tensor after height compression.

Parameters:

in_channels (int) –

Input voxel feature channels (e.g. \(4\) for mean \(x, y, z, \text{intensity}\)).
act (Union[str, Callable, None], default: 'relu' ) –

Activation type or callable.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra activation arguments.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization type or callable.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra normalization arguments.

SECONDDetection [source] ¶

SECONDDetection(
    in_channels: int = 4,
    num_classes: int = 3,
    *,
    voxel_size: Sequence[float] = (0.05, 0.05, 0.1),
    point_cloud_range: Sequence[float] = (
        0.0,
        -40.0,
        -3.0,
        70.4,
        40.0,
        1.0,
    ),
    anchor_sizes: Sequence[Sequence[float]],
    anchor_bottom_heights: Sequence[float],
    feature_map_stride: int,
    anchor_rotations: Sequence[float] = (0.0, 1.57),
    layer_nums: Sequence[int] = (5, 5),
    layer_strides: Sequence[int] = (1, 2),
    num_filters: Sequence[int] = (128, 256),
    upsample_strides: Sequence[int] = (1, 2),
    num_upsample_filters: Sequence[int] = (256, 256),
    num_dir_bins: int = 2,
    dir_offset: float = 0.78539,
    dir_limit_offset: float = 0.0,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: DetectionModel

SECOND 3D object detector (packed point format).

Reference: SECOND: Sparsely Embedded Convolutional Detection (Yan et al., 2018). Reference implementation: open-mmlab/OpenPCDet.

Parameters:

in_channels (int, default: 4 ) –

Raw point feature channels including xyz (e.g. \(4\) for \(x, y, z, \text{intensity}\)).
num_classes (int, default: 3 ) –

Number of foreground classes.
voxel_size (Sequence[float], default: (0.05, 0.05, 0.1) ) –

Voxel size \((v_x, v_y, v_z)\).
point_cloud_range (Sequence[float], default: (0.0, -40.0, -3.0, 70.4, 40.0, 1.0) ) –

Range \((x_\min, y_\min, z_\min, x_\max, y_\max, z_\max)\).
anchor_sizes (Sequence[Sequence[float]]) –

Per-class box size \((d_x, d_y, d_z)\), one row per class.
anchor_bottom_heights (Sequence[float]) –

Per-class anchor bottom \(z\), one per class.
anchor_rotations (Sequence[float], default: (0.0, 1.57) ) –

Yaw angles (radians) shared by all classes.
feature_map_stride (int) –

BEV feature-map stride of the head.
layer_nums (Sequence[int], default: (5, 5) ) –

2D backbone conv counts per level.
layer_strides (Sequence[int], default: (1, 2) ) –

2D backbone downsample strides per level.
num_filters (Sequence[int], default: (128, 256) ) –

2D backbone channel widths per level.
upsample_strides (Sequence[int], default: (1, 2) ) –

2D backbone upsample strides per level.
num_upsample_filters (Sequence[int], default: (256, 256) ) –

2D backbone upsample channels per level.
num_dir_bins (int, default: 2 ) –

Number of direction bins in the head.
dir_offset (float, default: 0.78539 ) –

Direction-classifier angle offset.
dir_limit_offset (float, default: 0.0 ) –

Heading wrap offset used during decoding.
act (Union[str, Callable, None], default: 'relu' ) –

Activation type or callable for the 3D/2D backbones.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra activation arguments.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization type or callable for the 3D/2D backbones.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra normalization arguments.

Methods:

configure_backbone_3d –

Build the sparse 3D voxel backbone.
configure_backbone –

Build the 2D BEV backbone.
configure_head –

Build the single-group anchor head.
decode –

Decode a forward output into raw per-anchor detections (see AnchorHead.decode).
reset_classifier –

Replace the classification branch of the detection head for num_classes outputs.

Attributes:

num_features (int) –

Channel count \(C\) of the BEV feature map entering the head.

num_features `property` ¶

num_features: int

Channel count \(C\) of the BEV feature map entering the head.

configure_backbone_3d [source] ¶

configure_backbone_3d() -> VoxelBackbone8x

Build the sparse 3D voxel backbone.

configure_backbone [source] ¶

configure_backbone() -> BEVBackbone

Build the 2D BEV backbone.

configure_head [source] ¶

configure_head() -> AnchorHead

Build the single-group anchor head.

decode [source] ¶

decode(out: AnchorHeadOutput) -> Detection3D

Decode a forward output into raw per-anchor detections (see AnchorHead.decode).

reset_classifier [source] ¶

reset_classifier(num_classes: int) -> None

Replace the classification branch of the detection head for num_classes outputs.

Models whose head is not rebuildable in isolation raise NotImplementedError.

VoxelResBackbone8x [source] ¶

VoxelResBackbone8x(
    in_channels: int,
    *,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: Module

Residual sparse 3D voxel backbone (VoxelResBackBone8x), \(8\times\) downsampling.

Like VoxelBackbone8x but with SparseBasicBlock residual stages and a 128-channel stage 4 (used by the nuScenes SECOND multihead detector).

Parameters:

in_channels (int) –

Input voxel feature channels (e.g. \(5\) for nuScenes).
act (Union[str, Callable, None], default: 'relu' ) –

Activation type or callable.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra activation arguments.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization type or callable.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra normalization arguments.

SECONDMultiHeadDetection [source] ¶

SECONDMultiHeadDetection(
    in_channels: int = 5,
    num_classes: int = 10,
    *,
    voxel_size: Sequence[float] = (0.1, 0.1, 0.2),
    point_cloud_range: Sequence[float] = (
        -51.2,
        -51.2,
        -5.0,
        51.2,
        51.2,
        3.0,
    ),
    anchor_sizes: Sequence[Sequence[float]],
    anchor_bottom_heights: Sequence[float],
    head_class_groups: Sequence[Sequence[int]],
    feature_map_stride: int,
    anchor_rotations: Sequence[float] = (0.0, 1.57),
    layer_nums: Sequence[int] = (5, 5),
    layer_strides: Sequence[int] = (1, 2),
    num_filters: Sequence[int] = (128, 256),
    upsample_strides: Sequence[float] = (1, 2),
    num_upsample_filters: Sequence[int] = (256, 256),
    shared_conv_num_filter: int = 64,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: DetectionModel

SECOND with a multi-group anchor head (nuScenes 10-class, packed point format).

Reference implementation: open-mmlab/OpenPCDet (cbgs_second_multihead). A residual sparse 3D backbone (VoxelResBackbone8x) feeds the shared 2D BEV backbone and an MultiGroupAnchorHead head. Input points carry 5 features (\(x, y, z, \text{intensity}, \Delta t\)).

Parameters:

in_channels (int, default: 5 ) –

Raw point feature channels including xyz (5 for nuScenes).
num_classes (int, default: 10 ) –

Number of foreground classes (10 for nuScenes).
voxel_size (Sequence[float], default: (0.1, 0.1, 0.2) ) –

Voxel size \((v_x, v_y, v_z)\).
point_cloud_range (Sequence[float], default: (-51.2, -51.2, -5.0, 51.2, 51.2, 3.0) ) –

Range \((x_\min, y_\min, z_\min, x_\max, y_\max, z_\max)\).
anchor_sizes (Sequence[Sequence[float]]) –

Per-class box size \((d_x, d_y, d_z)\), one row per class.
anchor_bottom_heights (Sequence[float]) –

Per-class anchor bottom \(z\), one per class.
head_class_groups (Sequence[Sequence[int]]) –

Class-index groups, one per RPN head (e.g. [[0], [1, 2], ...]).
anchor_rotations (Sequence[float], default: (0.0, 1.57) ) –

Yaw angles (radians) shared by all classes.
feature_map_stride (int) –

BEV feature-map stride of the head.
layer_nums (Sequence[int], default: (5, 5) ) –

2D backbone conv counts per level.
layer_strides (Sequence[int], default: (1, 2) ) –

2D backbone downsample strides per level.
num_filters (Sequence[int], default: (128, 256) ) –

2D backbone channel widths per level.
upsample_strides (Sequence[float], default: (1, 2) ) –

2D backbone upsample factors per level.
num_upsample_filters (Sequence[int], default: (256, 256) ) –

2D backbone upsample channels per level.
shared_conv_num_filter (int, default: 64 ) –

Channels of the head's shared conv.
act (Union[str, Callable, None], default: 'relu' ) –

Activation type or callable for the 3D/2D backbones and head.
act_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra activation arguments.
norm (Union[str, Callable, None], default: 'batch_norm' ) –

Normalization type or callable for the 3D/2D backbones and head.
norm_kwargs (Optional[Dict[str, Any]], default: None ) –

Extra normalization arguments.

Methods:

configure_backbone_3d –

Build the residual sparse 3D voxel backbone.
configure_backbone –

Build the 2D BEV backbone.
configure_head –

Build the multi-group anchor head.
decode –

Decode a forward output into raw per-anchor detections (see MultiGroupAnchorHead.decode).
reset_classifier –

Replace the classification branch of the detection head for num_classes outputs.

Attributes:

num_features (int) –

Channel count \(C\) of the BEV feature map entering the head.

num_features `property` ¶

num_features: int

Channel count \(C\) of the BEV feature map entering the head.

configure_backbone_3d [source] ¶

configure_backbone_3d() -> VoxelResBackbone8x

Build the residual sparse 3D voxel backbone.

configure_backbone [source] ¶

configure_backbone() -> BEVBackbone

Build the 2D BEV backbone.

configure_head [source] ¶

configure_head() -> MultiGroupAnchorHead

Build the multi-group anchor head.

decode [source] ¶

decode(out: AnchorHeadMultiOutput) -> Detection3D

Decode a forward output into raw per-anchor detections (see MultiGroupAnchorHead.decode).

reset_classifier [source] ¶

reset_classifier(num_classes: int) -> None

Replace the classification branch of the detection head for num_classes outputs.

Models whose head is not rebuildable in isolation raise NotImplementedError.

second

VoxelBackbone8x [source] ¶

SECONDDetection [source] ¶

num_features property ¶

configure_backbone_3d [source] ¶

configure_backbone [source] ¶

configure_head [source] ¶

decode [source] ¶

reset_classifier [source] ¶

VoxelResBackbone8x [source] ¶

SECONDMultiHeadDetection [source] ¶

num_features property ¶

configure_backbone_3d [source] ¶

configure_backbone [source] ¶

configure_head [source] ¶

decode [source] ¶

reset_classifier [source] ¶

num_features `property` ¶

num_features `property` ¶