Skip to content

Concerto

Concerto pretrained encoders and linear-probing segmentation model.

First page of Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representations

2510.23607 · October 2025

Classes:

ConcertoSegmentation 

ConcertoSegmentation(
    in_channels: int,
    num_classes: int,
    serialization_orders: Sequence[SerializationOrder] = (
        "z",
        "z-trans",
        "hilbert",
        "hilbert-trans",
    ),
    shuffle_serialization_orders: bool = True,
    strides: Sequence[int] = (2, 2, 2, 2),
    encoder_depths: Sequence[int] = (3, 3, 3, 12, 3),
    encoder_channels: Sequence[int] = (
        64,
        128,
        256,
        512,
        768,
    ),
    encoder_num_heads: Sequence[int] = (4, 8, 16, 32, 48),
    encoder_patch_size: Sequence[int] = (
        1024,
        1024,
        1024,
        1024,
        1024,
    ),
    norm: Union[str, Callable] = "layer_norm",
    act: Union[str, Callable] = "gelu",
    mlp_ratio: float = 4.0,
    qkv_bias: bool = True,
    qk_scale: Optional[float] = None,
    attn_drop: float = 0.0,
    proj_drop: float = 0.0,
    drop_path: float = 0.3,
    attn_kind: AttentionKind = "default",
    use_flash_attn: bool = True,
    upcast_attn: bool = False,
    upcast_softmax: bool = False,
    dropout: float = 0.0,
    pooling: str = "grid",
    stem_type: str = "linear",
    act_kwargs: Optional[Dict[str, Any]] = None,
    norm_kwargs: Optional[Dict[str, Any]] = None,
    legacy: bool = False,
)

Bases: SegmentationModel

Concerto linear-probing segmentation model.

Linear-probe variant from Concerto: Joint 2D-3D Self-Supervised Learning (Zhang et al., NeurIPS 2025). The pretrained PT-V3 encoder is run with intermediate features captured at every encoder stage; those features are successively unpooled through the saved pooling inverses, concatenated with each parent stage, then projected by a single linear segmentation head.

Architecturally identical to Sonata's linear-probe head, only the encoder scale differs (Concerto-large uses \(C = (64, 128, 256, 512, 768)\) versus Sonata's \((48, 96, 192, 384, 512)\)).

Note

The default (and registered) configuration enables flash attention (use_flash_attn=True), which requires the flash-attn package and a CUDA device; pass use_flash_attn=False to run without it. The xCPE sparse convolution still needs a spconv build matching the device; the standard CUDA wheel cannot run the forward on CPU.

Methods:

Attributes:

  • num_features (int) –

    Channel count \(C\) entering the head: every encoder stage unpooled and concatenated.

num_features property 

num_features: int

Channel count \(C\) entering the head: every encoder stage unpooled and concatenated.

configure_encoder 

configure_encoder() -> PointTransformerV3Encoder

Build the PointTransformerV3Encoder backbone.