Point Transformer V2
Point Transformer V2 classification and segmentation models.

Classes:
-
GroupedVectorAttention–Vector attention over a neighborhood graph, with one weight vector shared by each group of channels.
-
PointTransformerV2Block–Residual bottleneck around a
GroupedVectorAttention, with a linear projection before and after it. -
PointTransformerV2GridPool–Partition-based pooling: projects the features, then reduces every
grid_sizevoxel to a single point. -
PointTransformerV2InversePool–Undoes a
PointTransformerV2GridPool: scatters the pooled features back to the finer points and adds the projected skip. -
PointTransformerV2EncoderBlock–One encoder stage: an optional
PointTransformerV2GridPooldownsampling, thendepthPointTransformerV2Blockunits sharing a single -
PointTransformerV2DecoderBlock–One decoder stage: an optional
PointTransformerV2InversePoolupsampling onto the skip resolution, thendepthPointTransformerV2Block -
PointTransformerV2Encoder–Point Transformer V2 encoder:
PointTransformerV2EncoderBlockstages from finest to coarsest, every stage but -
PointTransformerV2Decoder–Point Transformer V2 decoder:
PointTransformerV2DecoderBlockstages from coarsest to finest, each preceded -
PointTransformerV2Classification–Implementation of the Point Transformer V2 model for classification as described in the paper
-
PointTransformerV2Segmentation–Implementation of the Point Transformer V2 model for semantic segmentation as described in the paper
GroupedVectorAttention
¶
GroupedVectorAttention(
channels: int,
num_groups: int,
attn_drop: float = 0.0,
qkv_bias: bool = True,
pe_multiplier: bool = False,
pe_bias: bool = True,
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
)
Bases: Module
Vector attention over a neighborhood graph, with one weight vector shared by each group of channels.
The relation between a query and its neighbor keys is optionally scaled and shifted by a learned
encoding of their relative position, then mapped to num_groups weights and softmax-normalized
over each destination's neighbors.
PointTransformerV2Block
¶
PointTransformerV2Block(
channels: int,
num_groups: int,
qkv_bias: bool = True,
pe_multiplier: bool = False,
pe_bias: bool = True,
attn_drop: float = 0.0,
drop_path: float = 0.0,
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
)
Bases: Module
Residual bottleneck around a GroupedVectorAttention, with a linear projection before and after it.
PointTransformerV2GridPool
¶
PointTransformerV2GridPool(
in_channels: int,
out_channels: int,
grid_size: float,
bias: bool = False,
reduce: str = "max",
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
)
Bases: Module
Partition-based pooling: projects the features, then reduces every grid_size voxel to a single point.
Positions are averaged within a voxel while features are reduced with reduce. Passing
return_inverse=True to forward also returns the point-to-voxel map that PointTransformerV2InversePool needs.
PointTransformerV2InversePool
¶
PointTransformerV2InversePool(
in_channels: int,
skip_channels: int,
out_channels: int,
bias: bool = True,
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
)
Bases: Module
Undoes a PointTransformerV2GridPool: scatters the pooled features back to the finer points and adds the projected skip.
PointTransformerV2EncoderBlock
¶
PointTransformerV2EncoderBlock(
depth: int,
channels: int,
num_groups: int,
num_neighbors: int,
qkv_bias: bool = True,
pe_multiplier: bool = False,
pe_bias: bool = True,
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
attn_drop: ValueCollection[float] = 0.0,
drop_path: ValueCollection[float] = 0.0,
downsample: Optional[PointTransformerV2GridPool] = None,
)
Bases: Module
One encoder stage: an optional PointTransformerV2GridPool downsampling, then depth PointTransformerV2Block units sharing a single
\(k\)-NN graph built on the stage's own resolution.
PointTransformerV2DecoderBlock
¶
PointTransformerV2DecoderBlock(
depth: int,
channels: int,
num_groups: int,
num_neighbors: int,
qkv_bias: bool = True,
pe_multiplier: bool = False,
pe_bias: bool = True,
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
attn_drop: ValueCollection[float] = 0.0,
drop_path: ValueCollection[float] = 0.0,
upsample: Optional[
PointTransformerV2InversePool
] = None,
)
Bases: Module
One decoder stage: an optional PointTransformerV2InversePool upsampling onto the skip resolution, then depth PointTransformerV2Block
units sharing a single \(k\)-NN graph built on that resolution.
PointTransformerV2Encoder
¶
PointTransformerV2Encoder(
encoder_depths: Sequence[int] = (1, 2, 2, 6, 2),
encoder_channels: Sequence[int] = (
48,
96,
192,
384,
512,
),
encoder_num_groups: Sequence[int] = (6, 12, 24, 48, 64),
encoder_num_neighbors: Sequence[int] = (
8,
16,
16,
16,
16,
),
grid_sizes: Sequence[float] = (0.06, 0.12, 0.24, 0.48),
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
qkv_bias: bool = True,
pe_multiplier: bool = False,
pe_bias: bool = True,
attn_drop: float = 0.0,
drop_path: float = 0.0,
)
Bases: Module
Point Transformer V2 encoder: PointTransformerV2EncoderBlock stages from finest to coarsest, every stage but
the first preceded by a PointTransformerV2GridPool downsampling.
Parameters:
-
encoder_depths(Sequence[int], default:(1, 2, 2, 6, 2)) –Number of blocks in each stage.
-
encoder_channels(Sequence[int], default:(48, 96, 192, 384, 512)) –Number of channels in each stage.
-
encoder_num_groups(Sequence[int], default:(6, 12, 24, 48, 64)) –Number of attention groups in each stage.
-
encoder_num_neighbors(Sequence[int], default:(8, 16, 16, 16, 16)) –Number of neighbors of the graph built in each stage.
-
grid_sizes(Sequence[float], default:(0.06, 0.12, 0.24, 0.48)) –Voxel size of the pooling preceding each stage but the first.
-
norm(Union[str, Callable, None], default:'batch_norm') –Normalization layer to use.
-
act(Union[str, Callable, None], default:'relu') –Activation function to use.
-
act_kwargs(Optional[Dict[str, Any]], default:None) –Keyword arguments for the activation function.
-
norm_kwargs(Optional[Dict[str, Any]], default:None) –Keyword arguments for the normalization layer.
-
qkv_bias(bool, default:True) –Whether to use bias in the QKV linear layers.
-
pe_multiplier(bool, default:False) –Whether to scale the query-key relation by a positional encoding.
-
pe_bias(bool, default:True) –Whether to shift the query-key relation and the values by a positional encoding.
-
attn_drop(float, default:0.0) –Dropout rate on the attention weights.
-
drop_path(float, default:0.0) –Maximum drop path rate, reached by the last block.
Inputs
x: Embedded point features of shape \((N, \text{encoder\_channels}[0])\). pos: Float tensor of shape \((N, 3)\). batch: Long tensor of shape \((N,)\).
Outputs
Features, coordinates and batch indices at the coarsest stage. With return_intermediates=True, also the
per-stage skips (features, coordinates, batch indices and pooling inverse) consumed by
PointTransformerV2Decoder.
Attributes:
-
embedding_dim(int) –Feature dimension \(C\) of the encoder output.
PointTransformerV2Decoder
¶
PointTransformerV2Decoder(
encoder_channels: Sequence[int] = (
48,
96,
192,
384,
512,
),
decoder_depths: Sequence[int] = (1, 1, 1, 1),
decoder_channels: Sequence[int] = (384, 192, 96, 48),
decoder_num_groups: Sequence[int] = (48, 24, 12, 6),
decoder_num_neighbors: Sequence[int] = (16, 16, 16, 16),
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
qkv_bias: bool = True,
pe_multiplier: bool = False,
pe_bias: bool = True,
attn_drop: float = 0.0,
drop_path: float = 0.0,
)
Bases: Module
Point Transformer V2 decoder: PointTransformerV2DecoderBlock stages from coarsest to finest, each preceded
by a PointTransformerV2InversePool upsampling onto its encoder skip.
Parameters:
-
encoder_channels(Sequence[int], default:(48, 96, 192, 384, 512)) –Number of channels of each encoder stage, sizing the decoder input and the skips.
-
decoder_depths(Sequence[int], default:(1, 1, 1, 1)) –Number of blocks in each stage.
-
decoder_channels(Sequence[int], default:(384, 192, 96, 48)) –Number of output channels of each stage.
-
decoder_num_groups(Sequence[int], default:(48, 24, 12, 6)) –Number of attention groups in each stage.
-
decoder_num_neighbors(Sequence[int], default:(16, 16, 16, 16)) –Number of neighbors of the graph built in each stage.
-
norm(Union[str, Callable, None], default:'batch_norm') –Normalization layer to use.
-
act(Union[str, Callable, None], default:'relu') –Activation function to use.
-
act_kwargs(Optional[Dict[str, Any]], default:None) –Keyword arguments for the activation function.
-
norm_kwargs(Optional[Dict[str, Any]], default:None) –Keyword arguments for the normalization layer.
-
qkv_bias(bool, default:True) –Whether to use bias in the QKV linear layers.
-
pe_multiplier(bool, default:False) –Whether to scale the query-key relation by a positional encoding.
-
pe_bias(bool, default:True) –Whether to shift the query-key relation and the values by a positional encoding.
-
attn_drop(float, default:0.0) –Dropout rate on the attention weights.
-
drop_path(float, default:0.0) –Maximum drop path rate, reached by the first block.
Inputs
x: Features at the coarsest encoder stage, of shape \((N', \text{encoder\_channels}[-1])\).
intermediates: Per-stage skips returned by PointTransformerV2Encoder, finest first.
Outputs
Features, coordinates and batch indices at the finest stage.
Attributes:
-
out_channels(int) –Feature dimension \(C\) of the decoder output.
PointTransformerV2Classification
¶
PointTransformerV2Classification(
in_channels: int,
num_classes: int,
grid_sizes: Sequence[float] = (0.06, 0.12, 0.24, 0.48),
encoder_depths: Sequence[int] = (1, 2, 2, 6, 2),
encoder_channels: Sequence[int] = (
48,
96,
192,
384,
512,
),
encoder_num_groups: Sequence[int] = (6, 12, 24, 48, 64),
encoder_num_neighbors: Sequence[int] = (
8,
16,
16,
16,
16,
),
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
qkv_bias: bool = True,
attn_drop: float = 0.0,
pe_multiplier: bool = False,
pe_bias: bool = True,
drop_path: float = 0.0,
dropout: float = 0.0,
global_pool: PoolLike = "max",
)
Bases: ClassificationModel
Implementation of the Point Transformer V2 model for classification as described in the paper Point Transformer V2: Grouped Vector Attention and Partition-based Pooling by Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, Hengshuang Zhao.
Note
This implementation requires torch-cluster and
torch-scatter to be installed.
Parameters:
-
in_channels(int) –Number of input channels.
-
num_classes(int) –Number of output classes.
-
encoder_depths(Sequence[int], default:(1, 2, 2, 6, 2)) –Number of encoder blocks for each stage.
-
encoder_channels(Sequence[int], default:(48, 96, 192, 384, 512)) –Number of channels for each encoder block.
-
encoder_num_groups(Sequence[int], default:(6, 12, 24, 48, 64)) –Number of groups for each encoder block.
-
encoder_num_neighbors(Sequence[int], default:(8, 16, 16, 16, 16)) –Number of edge_index for each encoder block.
-
grid_sizes(Sequence[float], default:(0.06, 0.12, 0.24, 0.48)) –Size of the grid for each stage.
-
norm(Union[str, Callable, None], default:'batch_norm') –Normalization layer to use.
-
act(Union[str, Callable, None], default:'relu') –Activation function to use.
-
qkv_bias(bool, default:True) –Whether to use bias in the QKV linear layer.
-
pe_multiplier(bool, default:False) –Whether to use a multiplier for the PE.
-
pe_bias(bool, default:True) –Whether to use bias in the PE linear layer.
-
drop_path(float, default:0.0) –Drop path rate.
-
dropout(float, default:0.0) –Dropout rate.
-
global_pool(PoolLike, default:'max') –Global pooling method to use.
Inputs
x: Float tensor of shape \((N, \text{in\_channels})\). pos: Float tensor of shape \((N, 3)\). batch: Long tensor of shape \((N,)\).
Outputs
Logits tensor of shape \((B, \text{num\_classes})\).
Methods:
-
configure_stem–Build the linear stem lifting the input features to the first encoder channel.
-
configure_encoder–Build the
PointTransformerV2Encoderbackbone. -
reset_classifier–Resets the classification head with new parameters.
-
forward_features–Forward features through the encoder blocks, before the global pooling.
-
forward_head–Forward pass of the classification head from pre-pooling x.
-
forward–Forward pass of the classification network.
Attributes:
-
num_features(int) –Feature dimension \(C\) of the encoder output.
configure_stem
¶
Build the linear stem lifting the input features to the first encoder channel.
configure_encoder
¶
configure_encoder() -> PointTransformerV2Encoder
Build the PointTransformerV2Encoder backbone.
reset_classifier
¶
reset_classifier(
num_classes: int,
global_pool: Optional[PoolLike] = None,
**kwargs: Any,
) -> None
Resets the classification head with new parameters.
Note
To set an empty classification head, use num_classes=0.
Parameters:
-
num_classes(int) –Number of output classes.
-
global_pool(Optional[PoolLike], default:None) –Pooling method to aggregate point x ("max" or "mean"). If
None, keeps the current pooling. -
**kwargs(Any, default:{}) –Additional keyword arguments to pass to the classification head.
forward_features
¶
forward_features(
x: OptTensor,
pos: Tensor,
batch: Tensor,
return_intermediates: bool = False,
) -> Any
Forward features through the encoder blocks, before the global pooling.
Parameters:
-
x(OptTensor) –Additional point features of shape \((N, \text{features\_dim})\).
-
pos(Tensor) –Coordinates of shape \((N, 3)\).
-
batch(Tensor) –Batch indices for each point of shape \((N,)\).
-
return_intermediates(bool, default:False) –Whether to return the intermediate features.
Returns:
-
x(Any) –Pre-pooling features of shape \((N, \text{embedding\_dim})\).
-
pos(Any) –Coordinates of shape \((N, 3)\).
-
batch(Any) –Batch indices for each point of shape \((N,)\).
-
intermediates(Any) –If
return_intermediatesisTrue, a list of dictionaries containing the intermediate features, coordinates, batch indices and pooling inverse for each encoder block.
forward_head
¶
Forward pass of the classification head from pre-pooling x.
Parameters:
-
x(Tensor) –Pre-pooling features of shape \((N, \text{embedding\_dim})\).
-
batch(Tensor) –Batch indices for each point of shape \((N,)\).
-
pre_logits(bool, default:False) –Whether to return pre-logits.
Returns:
-
Tensor–Classification logits of shape \((B, \text{num\_classes})\).
forward
¶
Forward pass of the classification network.
Parameters:
-
x(OptTensor) –Additional point features of shape \((N, \text{features\_dim})\).
-
pos(Tensor) –Coordinates of shape \((N, 3)\).
-
batch(Tensor) –Batch indices for each point of shape \((N,)\).
Returns:
-
Tensor–Classification logits of shape \((B, \text{num\_classes})\).
PointTransformerV2Segmentation
¶
PointTransformerV2Segmentation(
in_channels: int,
num_classes: int,
grid_sizes: Sequence[float] = (0.06, 0.12, 0.24, 0.48),
encoder_depths: Sequence[int] = (1, 2, 2, 6, 2),
encoder_channels: Sequence[int] = (
48,
96,
192,
384,
512,
),
encoder_num_groups: Sequence[int] = (6, 12, 24, 48, 64),
encoder_num_neighbors: Sequence[int] = (
8,
16,
16,
16,
16,
),
decoder_depths: Sequence[int] = (1, 1, 1, 1),
decoder_channels: Sequence[int] = (384, 192, 96, 48),
decoder_num_groups: Sequence[int] = (48, 24, 12, 6),
decoder_num_neighbors: Sequence[int] = (16, 16, 16, 16),
norm: Union[str, Callable, None] = "batch_norm",
act: Union[str, Callable, None] = "relu",
act_kwargs: Optional[Dict[str, Any]] = None,
norm_kwargs: Optional[Dict[str, Any]] = None,
qkv_bias: bool = True,
attn_drop: float = 0.0,
pe_multiplier: bool = False,
pe_bias: bool = True,
drop_path: float = 0.0,
dropout: float = 0.0,
)
Bases: SegmentationModel
Implementation of the Point Transformer V2 model for semantic segmentation as described in the paper Point Transformer V2: Grouped Vector Attention and Partition-based Pooling by Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, Hengshuang Zhao.
Note
This implementation requires torch-cluster and
torch-scatter to be installed.
Parameters:
-
in_channels(int) –Number of input channels.
-
num_classes(int) –Number of output classes.
-
encoder_depths(Sequence[int], default:(1, 2, 2, 6, 2)) –Number of encoder blocks for each stage.
-
encoder_channels(Sequence[int], default:(48, 96, 192, 384, 512)) –Number of channels for each encoder block.
-
encoder_num_groups(Sequence[int], default:(6, 12, 24, 48, 64)) –Number of groups for each encoder block.
-
encoder_num_neighbors(Sequence[int], default:(8, 16, 16, 16, 16)) –Number of edge_index for each encoder block.
-
decoder_depths(Sequence[int], default:(1, 1, 1, 1)) –Number of decoder blocks per stage.
-
decoder_channels(Sequence[int], default:(384, 192, 96, 48)) –Number of channels for each decoder block.
-
decoder_num_groups(Sequence[int], default:(48, 24, 12, 6)) –Number of groups for each decoder block.
-
decoder_num_neighbors(Sequence[int], default:(16, 16, 16, 16)) –Neighbor count for each decoder block.
-
grid_sizes(Sequence[float], default:(0.06, 0.12, 0.24, 0.48)) –Size of the grid for each stage.
-
norm(Union[str, Callable, None], default:'batch_norm') –Normalization layer to use.
-
act(Union[str, Callable, None], default:'relu') –Activation function to use.
-
qkv_bias(bool, default:True) –Whether to use bias in the QKV linear layer.
-
pe_multiplier(bool, default:False) –Whether to use a multiplier for the PE.
-
pe_bias(bool, default:True) –Whether to use bias in the PE linear layer.
-
drop_path(float, default:0.0) –Drop path rate.
-
dropout(float, default:0.0) –Dropout rate.
-
attn_drop(float, default:0.0) –Attention dropout rate.
Inputs
x: Float tensor of shape \((N, \text{in\_channels})\). pos: Float tensor of shape \((N, 3)\). batch: Long tensor of shape \((N,)\).
Outputs
Segmentation logits of shape \((N, \text{num\_classes})\).
Methods:
-
configure_stem–Build the linear stem lifting the input features to the first encoder channel.
-
configure_encoder–Build the
PointTransformerV2Encoderbackbone. -
configure_decoder–Build the
PointTransformerV2Decoderupsampling the coarsest features back through the encoder skips. -
reset_classifier–Resets the head with new class parameters.
-
forward_features–Forward features through the encoder blocks, before the global pooling.
-
forward_head–Forward pass of the classification head from up-sampled x.
-
forward–Forward pass of the model.
Attributes:
-
num_features(int) –Channel count \(C\) of the per-point decoder features entering the head.
num_features
property
¶
Channel count \(C\) of the per-point decoder features entering the head.
configure_stem
¶
Build the linear stem lifting the input features to the first encoder channel.
configure_encoder
¶
configure_encoder() -> PointTransformerV2Encoder
Build the PointTransformerV2Encoder backbone.
configure_decoder
¶
configure_decoder() -> PointTransformerV2Decoder
Build the PointTransformerV2Decoder upsampling the coarsest features back through the encoder skips.
reset_classifier
¶
Resets the head with new class parameters.
Note
To set an empty head, use num_classes=0.
Parameters:
-
num_classes(int) –Number of output classes.
-
**kwargs(Any, default:{}) –Additional keyword arguments to pass to the segmentation head.
forward_features
¶
forward_features(
x: OptTensor,
pos: Tensor,
batch: Tensor,
return_intermediates: bool = False,
) -> Any
Forward features through the encoder blocks, before the global pooling.
Parameters:
-
x(OptTensor) –Additional point features of shape \((N, \text{features\_dim})\).
-
pos(Tensor) –Coordinates of shape \((N, 3)\).
-
batch(Tensor) –Batch indices for each point of shape \((N,)\).
-
return_intermediates(bool, default:False) –Whether to return the intermediate features.
Returns:
-
x(Any) –Pre-pooling features of shape \((N, \text{embedding\_dim})\).
-
pos(Any) –Coordinates of shape \((N, 3)\).
-
batch(Any) –Batch indices for each point of shape \((N,)\).
-
intermediates(Any) –If
return_intermediatesisTrue, a list of dictionaries containing the intermediate features, coordinates, batch indices and pooling inverse for each encoder block.
forward_head
¶
Forward pass of the classification head from up-sampled x.
Parameters:
-
x(Tensor) –Pre-pooling features of shape \((N, \text{embedding\_dim})\).
-
pre_logits(bool, default:False) –Whether to return pre-logits.
Returns:
-
Tensor–Segmentation logits of shape \((N, \text{num\_classes})\).
forward
¶
Forward pass of the model.
Parameters:
-
x(OptTensor) –Additional point features of shape \((N, \text{features\_dim})\).
-
pos(Tensor) –Coordinates of shape \((N, 3)\).
-
batch(Tensor) –Batch indices for each point of shape \((N,)\).
Returns:
-
Tensor–Segmentation logits of shape \((N, \text{num\_classes})\).