Skip to content

VoteNet

VoteNet detection model.

First page of Deep Hough Voting for 3D Object Detection in Point Clouds

1904.09664 · April 2019

Classes:

  • VoteNetOutput –

    Decoded VoteNet proposals for a batch of \(B\) scenes with \(K\) proposals each.

  • VoteNetBackbone –

    PointNet++ single-scale-grouping backbone (set abstraction + feature propagation).

  • VotingModule –

    Hough voting: each seed predicts a vote offset and a residual feature.

  • VoteNetProposalModule –

    Vote aggregation and proposal generation.

  • VoteNetDetection –

    VoteNet 3D object detector (packed point format).

VoteNetOutput

Bases: TypedDict

Decoded VoteNet proposals for a batch of \(B\) scenes with \(K\) proposals each.

VoteNetBackbone 

VoteNetBackbone(
    in_channels: int,
    *,
    sa_channels: Sequence[Sequence[int]],
    sa_npoints: Sequence[int],
    sa_radii: Sequence[float],
    sa_num_neighbors: Sequence[int],
    fp_channels: Sequence[Sequence[int]],
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: Module

PointNet++ single-scale-grouping backbone (set abstraction + feature propagation).

Reuses SAModule / FPModule; every layer width, sample count, radius and neighbor cap is a constructor argument (no hardcoded sizes). The seeds are the points at the second SA resolution, recovered by the feature-propagation layers. Their indices into the original packed input are tracked through the first two SA samplings (idx1[idx2]) for the voting loss, so the SA samplings of those two blocks are computed here and threaded in.

Parameters:

  • in_channels (int) –

    Input feature channels per point (excluding xyz).

  • sa_channels (Sequence[Sequence[int]]) –

    Per-SA-block MLP channel lists, e.g. [[64, 64, 128], ...].

  • sa_npoints (Sequence[int]) –

    Per-SA-block farthest-point-sample counts.

  • sa_radii (Sequence[float]) –

    Per-SA-block ball-query radii.

  • sa_num_neighbors (Sequence[int]) –

    Per-SA-block neighbor caps.

  • fp_channels (Sequence[Sequence[int]]) –

    Per-FP-block MLP channel lists. The \(i\)-th FP block skips to the \((\text{n\_sa} - 2 - i)\)-th SA output, so two FP blocks recover the SA2 resolution.

  • act (Union[str, Callable, None], default: 'relu' ) –

    Activation for every block.

  • act_kwargs (Optional[Dict[str, Any]], default: None ) –

    Extra activation arguments.

  • norm (Union[str, Callable, None], default: 'batch_norm' ) –

    Normalization for every block.

  • norm_kwargs (Optional[Dict[str, Any]], default: None ) –

    Extra normalization arguments.

VotingModule 

VotingModule(
    vote_factor: int,
    seed_feature_dim: int,
    *,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: Module

Hough voting: each seed predicts a vote offset and a residual feature.

Mirrors the reference VotingModule. Because the residual is added to the seed feature, the input and output feature dims are equal.

Parameters:

  • vote_factor (int) –

    Number of votes generated per seed.

  • seed_feature_dim (int) –

    Channel count of the seed features.

  • act (Union[str, Callable, None], default: 'relu' ) –

    Activation type or callable.

  • act_kwargs (Optional[Dict[str, Any]], default: None ) –

    Extra activation arguments.

  • norm (Union[str, Callable, None], default: 'batch_norm' ) –

    Normalization type or callable.

  • norm_kwargs (Optional[Dict[str, Any]], default: None ) –

    Extra normalization arguments.

VoteNetProposalModule 

VoteNetProposalModule(
    num_classes: int,
    num_heading_bin: int,
    num_size_cluster: int,
    num_proposal: int,
    sampling: str,
    seed_channels: int,
    *,
    vote_aggr_channels: Sequence[int],
    vote_aggr_radius: float,
    vote_aggr_num_neighbors: int,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: Module

Vote aggregation and proposal generation.

Mirrors the reference ProposalModule: cluster the votes with a single set-abstraction layer (SAModule), then a 3-layer linear head decodes objectness, center, heading and size bins/residuals and semantic class per proposal.

Parameters:

  • num_classes (int) –

    Number of semantic classes.

  • num_heading_bin (int) –

    Number of heading-angle bins.

  • num_size_cluster (int) –

    Number of size templates.

  • num_proposal (int) –

    Number of proposals (= aggregation centroids) per scene.

  • sampling (str) –

    Aggregation-center sampling, "vote_fps" or "seed_fps".

  • seed_channels (int) –

    Channel count of the (vote) input features.

  • vote_aggr_channels (Sequence[int]) –

    MLP channels of the vote-aggregation set-abstraction layer.

  • vote_aggr_radius (float) –

    Ball-query radius of the vote-aggregation layer.

  • vote_aggr_num_neighbors (int) –

    Neighbor cap of the vote-aggregation layer.

  • act (Union[str, Callable, None], default: 'relu' ) –

    Activation type or callable.

  • act_kwargs (Optional[Dict[str, Any]], default: None ) –

    Extra activation arguments.

  • norm (Union[str, Callable, None], default: 'batch_norm' ) –

    Normalization type or callable.

  • norm_kwargs (Optional[Dict[str, Any]], default: None ) –

    Extra normalization arguments.

Methods:

  • decode –

    Splits the flat head output into the per-proposal box attributes.

decode 

decode(
    preds: Tensor,
    pos_aggr: Tensor,
    batch_aggr: Tensor,
    mean_sizes: Tensor,
) -> Dict[str, Tensor]

Splits the flat head output into the per-proposal box attributes.

Centers are predicted as offsets from the aggregation centroids, and heading and size residuals are denormalized by the bin width and the size templates respectively.

Parameters:

  • preds (Tensor) –

    Flat head output, shape \((B \cdot Q, C)\).

  • pos_aggr (Tensor) –

    Aggregation centroid positions, shape \((B \cdot Q, 3)\).

  • batch_aggr (Tensor) –

    Per-centroid scene index, shape \((B \cdot Q,)\).

  • mean_sizes (Tensor) –

    Size templates, shape \((S, 3)\).

Returns:

  • Dict[str, Tensor] –

    The per-proposal objectness, center, heading, size and semantic predictions, each of shape

  • Dict[str, Tensor] –

    \((B, Q, \ldots)\).

VoteNetDetection 

VoteNetDetection(
    in_channels: int,
    num_classes: int,
    *,
    num_heading_bin: int,
    num_size_cluster: int,
    mean_sizes: Union[Tensor, List[List[float]]],
    num_proposal: int,
    vote_factor: int,
    sampling: str,
    sa_channels: Sequence[Sequence[int]],
    sa_npoints: Sequence[int],
    sa_radii: Sequence[float],
    sa_num_neighbors: Sequence[int],
    fp_channels: Sequence[Sequence[int]],
    vote_aggr_channels: Sequence[int],
    vote_aggr_radius: float,
    vote_aggr_num_neighbors: int,
    act: Union[str, Callable, None] = "relu",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm: Union[str, Callable, None] = "batch_norm",
    norm_kwargs: Optional[Dict[str, Any]] = None,
)

Bases: DetectionModel

VoteNet 3D object detector (packed point format).

Reference: Qi et al., 2019. Reference implementation: facebookresearch/votenet.

A PointNet++ backbone extracts seed points; a voting module shifts each seed toward its object center; a proposal module clusters the votes and decodes a fixed number of oriented (or axis-aligned) box proposals per scene.

Parameters:

  • in_channels (int) –

    Input feature channels per point excluding xyz (e.g. \(1\) for a floor-relative height feature, \(4\) for height + RGB).

  • num_classes (int) –

    Number of semantic classes.

  • num_heading_bin (int) –

    Number of heading-angle bins (\(1\) for axis-aligned ScanNet boxes, \(12\) for oriented SUN RGB-D boxes).

  • num_size_cluster (int) –

    Number of size templates (one per class here).

  • mean_sizes (Union[Tensor, List[List[float]]]) –

    Per-template mean box size, shape \((\text{num\_size\_cluster}, 3)\).

  • num_proposal (int) –

    Number of box proposals per scene.

  • vote_factor (int) –

    Votes generated per seed.

  • sampling (str) –

    Aggregation-center sampling, "vote_fps" or "seed_fps".

  • sa_channels (Sequence[Sequence[int]]) –

    Per-SA-block MLP channel lists for the backbone.

  • sa_npoints (Sequence[int]) –

    Per-SA-block farthest-point-sample counts.

  • sa_radii (Sequence[float]) –

    Per-SA-block ball-query radii.

  • sa_num_neighbors (Sequence[int]) –

    Per-SA-block neighbor caps.

  • fp_channels (Sequence[Sequence[int]]) –

    Per-FP-block MLP channel lists for the backbone.

  • vote_aggr_channels (Sequence[int]) –

    MLP channels of the proposal vote-aggregation layer.

  • vote_aggr_radius (float) –

    Ball-query radius of the vote-aggregation layer.

  • vote_aggr_num_neighbors (int) –

    Neighbor cap of the vote-aggregation layer.

  • act (Union[str, Callable, None], default: 'relu' ) –

    Activation type or callable.

  • act_kwargs (Optional[Dict[str, Any]], default: None ) –

    Extra activation arguments.

  • norm (Union[str, Callable, None], default: 'batch_norm' ) –

    Normalization type or callable.

  • norm_kwargs (Optional[Dict[str, Any]], default: None ) –

    Extra normalization arguments.

Methods:

Attributes:

  • num_features (int) –

    Channel count \(C\) of the per-seed backbone features entering the voting head.

num_features property 

num_features: int

Channel count \(C\) of the per-seed backbone features entering the voting head.

configure_backbone 

configure_backbone() -> VoteNetBackbone

Build the PointNet++ seed backbone.

configure_vgen 

configure_vgen() -> VotingModule

Build the voting module.

configure_proposal 

configure_proposal() -> VoteNetProposalModule

Build the proposal module.

decode 

decode(out: VoteNetOutput) -> Detection3D

Decode a forward output into raw per-proposal detections (no NMS, threshold, or filtering).

Builds one oriented box per proposal from the predicted heading/size bins, scores it by objectness, and labels it by the argmax semantic class. The heading head predicts negated angles, so the decoded heading is negated to return counter-clockwise headings (the library box convention). The result is the full unfiltered proposal set; the evaluation pipeline applies point-count filtering, NMS, score thresholding, and the indoor per-class expansion (driven by the returned class_probs) via the torch_pointcloud.utils.box3d utilities.

Parameters:

Returns:

  • Detection3D –

    Packed proposals {"boxes", "scores", "labels", "batch", "class_probs"} (PyG layout), where the

  • Detection3D –

    per-proposal score is objectness, the label is the argmax semantic class, and class_probs holds

  • Detection3D –

    the softmaxed semantic-class probabilities.

Shape
  • boxes: \((B \cdot P, 7)\)
  • scores / labels / batch: \((B \cdot P,)\)
  • class_probs: \((B \cdot P, C)\)