Skip to content

semantic3d

The Semantic3D dataset.

First page of Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark

1704.03847 · April 2017

Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark by Hackel, Savinov, Ladicky, Wegner, Schindler and Pollefeys (2017).

Each scene is distributed as a pair of plain-text files:

  • <scene>.txt - one row per point with x y z intensity r g b (white-space separated, ASCII, no header). XYZ are float64 meters; intensity is float; RGB are uint8 0-255.
  • <scene>.labels - one row per point with the integer class id (\(0\) = unlabelled, \(1\)-\(8\) = the eight benchmark classes). .labels is only present for the reduced-8 and training splits.

Classes:

Functions:

Semantic3D [source] 

Semantic3D(
    root: PathLike,
    *,
    split: Semantic3DSplit = "train",
    scenes: Optional[Sequence[str]] = None,
    transform: Optional[
        Callable[[Dict[str, Any]], Dict[str, Any]]
    ] = None,
    download: bool = False,
    force_download: bool = False,
    accept_terms: bool = False,
    show_progress: bool = True,
)

Bases: PointCloudDataset

The Semantic3D dataset.

Each sample is a single ASCII-text scene, returned as a dictionary:

Key Shape Dtype Meaning
pos \((N, 3)\) float32 XYZ
intensity \((N, 1)\) float32 LiDAR intensity
color \((N, 3)\) uint8 RGB (0-255)
segment \((N,)\) int64 Raw class id 0-8 (0 = unlabelled, ignored). Absent for held-out scenes.
name str Source scene name

Parameters:

  • root (PathLike) –

    Dataset root. Files are read from <root>/Semantic3D/raw/<scene>.txt.

  • split (Semantic3DSplit, default: 'train' ) –

    One of "train" / "test" / "all". Defaults to "train". "test" returns the four reduced-8 benchmark scenes (without labels).

  • scenes (Optional[Sequence[str]], default: None ) –

    Optional explicit list of scene names (without the .txt / .labels suffix). Overrides split.

  • transform (Optional[Callable[[Dict[str, Any]], Dict[str, Any]]], default: None ) –

    Callable applied to each loaded sample dict at __getitem__ time.

  • download (bool, default: False ) –

    Whether to download the requested scenes when they are missing.

  • force_download (bool, default: False ) –

    Whether to download and extract them again even when present.

  • accept_terms (bool, default: False ) –

    Confirm that you have accepted the Semantic3D licence (terms_url). When left False, download asks for the confirmation on the terminal, once in every process that triggers the download.

  • show_progress (bool, default: True ) –

    Whether to show the download progress bars.

Note

The data come from the benchmark site, semantic3d.net (archived; the original links now ask for a login), which serves its scenes as 7z archives from an ETH mirror, https://share.phys.ethz.ch/~pf/semantic3d/data/ (data_url), under the CC BY-NC-SA 3.0 licence (terms_url). Accept it, then pass download=True, accept_terms=True (or answer the prompt; download=True alone asks on the terminal) to fetch the requested scenes and the labels of the training ones; reading the archives needs the optional py7zr package. Or place one <scene>.txt (and the matching <scene>.labels for training scenes) per scene under <root>/Semantic3D/raw/ by hand.

Warning

Loading is done with np.loadtxt, which reads the (often 100M+-row) ASCII files into RAM in a single shot. Each scene needs ~5 GB of RAM to parse; consider extracting only the scenes you need for evaluation.

Methods:

  • archive_url –

    URL of the 7z archive holding <scene>.txt on the benchmark's mirror.

  • download –

    Fetch the requested scenes, and the labels of the training ones, from the benchmark's mirror.

Attributes:

  • name (str) –

    Name of the dataset directory.

  • data_dir (str) –

    Path to the dataset directory <root>/<name>.

  • raw_dir (str) –

    Path to the raw download directory.

  • processed_dir (str) –

    Path to the processed cache directory.

name property 

name: str

Name of the dataset directory.

data_dir property 

data_dir: str

Path to the dataset directory <root>/<name>.

raw_dir property 

raw_dir: str

Path to the raw download directory.

processed_dir property 

processed_dir: str

Path to the processed cache directory.

archive_url classmethod [source] 

archive_url(scene: str) -> str

URL of the 7z archive holding <scene>.txt on the benchmark's mirror.

Parameters:

  • scene (str) –

    A training or reduced-8 test scene name.

Raises:

  • ValueError –

    If the scene is not one the mirror serves.

download [source] 

download(force: bool = False) -> None

Fetch the requested scenes, and the labels of the training ones, from the benchmark's mirror.

Every scene is a 7z archive holding its .txt, read with the optional py7zr package; the training labels come in one archive, of which only the requested scenes are extracted. Reduced-8 test scenes have no labels.

Parameters:

  • force (bool, default: False ) –

    Download and extract the scenes again even when their files are present.

Raises:

  • RuntimeError –

    If the licence is not accepted.

  • ImportError –

    If py7zr is not installed.

load_semantic3d_data [source] 

load_semantic3d_data(
    path_txt: PathLike,
    /,
    path_labels: Optional[PathLike] = None,
) -> Dict[str, Tensor]

Parse a Semantic3D <scene>.txt (and optional <scene>.labels) file pair.

The .txt files are large ASCII (often 100M+ rows) - we use np.loadtxt which is slow but free of compiled dependencies. The segment key is only populated when path_labels is provided and exists (held-out test scenes receive no segment key).

Key Shape Dtype
pos \((N, 3)\) float32
intensity \((N, 1)\) float32
color \((N, 3)\) uint8
segment \((N,)\) int64