Skip to content

rope

3D rotary position embedding for point-cloud attention.

Introduced in Utonia: Toward One Encoder for All Point Clouds. Splits each attention head's channel dimension into three equal chunks (one per spatial axis) and rotates the query/key vectors by a per-axis sinusoidal phase indexed by the real-valued coordinate. This injects continuous 3D position into attention without any learnable parameters.

Classes:

  • Point3DRoPE –

    3D Rotary Position Embedding for point cloud attention.

Point3DRoPE [source] 

Point3DRoPE(head_dim: int, base: float = 10.0)

Bases: Module

3D Rotary Position Embedding for point cloud attention.

Parameters:

  • head_dim (int) –

    Channel dimension of a single attention head. Must be divisible by 6: the head splits into three axis chunks, each of which needs an even size for the half-rotation.

  • base (float, default: 10.0 ) –

    RoPE frequency base (\(\theta\)). Smaller values encode finer spatial detail.

Inputs

q: Query tensor of shape \((N, H, D)\) where \(D\) is head_dim. k: Key tensor of the same shape as q. pos: Real-valued 3D positions of shape \((N, 3)\) corresponding to each token.

Outputs

Tuple (q_rot, k_rot) of tensors with the same shapes as q and k.