Point Cloud

Point Cloud Deep Learning is the application of neural networks to 3D point cloud data — unordered sets of (x,y,z) coordinates representing 3D scenes, enabling autonomous driving perception, robotic mapping, and 3D object recognition.

What Is a Point Cloud?

• Set of N points, each with coordinates $(x, y, z)$ and optional attributes (intensity, color, normal).
• Generated by: LiDAR scanners, depth cameras (Intel RealSense), stereo vision, photogrammetry.
• LiDAR: 16–128 beams, 100K–500K points per scan at 10Hz — primary sensor for autonomous driving.

Challenges vs. Images

• Irregular structure: Points are unordered — no fixed grid (unlike pixels).
• Sparsity: Most 3D space is empty.
• Variable density: Near objects: dense; far objects: sparse.
• No standard convolution: Regular CNN needs grid — point clouds lack it.

PointNet (2017)

• First deep learning directly on point clouds.
• Key insight: Symmetric function (max pooling) handles unordered sets.
• Architecture: MLP on each point independently → Global max pool → classification head.
• Transformation network (T-Net): Learn input/feature alignment.
• Limitation: No local structure — every point treated globally.

PointNet++ (2017)

• Hierarchical grouping: Local neighborhoods → hierarchical features.
• Sampling: Farthest point sampling (FPS) selects representative centroids.
• Set Abstraction: MLP on neighborhood → local feature.
• Captures both local and global structure.

Voxel-Based Methods

• VoxelNet: Quantize points to voxels → 3D CNN.
• PointPillars: Pillar (vertical column) features → 2D pseudo-image → 2D CNN.
• Real-time: 62 FPS, competitive accuracy — standard for production AV.

Transformer-Based

• Point Transformer: Self-attention with local neighborhoods.
• PCT (Point Cloud Transformer): Global self-attention on point features.

Point cloud deep learning is the critical perception technology for autonomous systems — enabling LiDAR-based obstacle detection, lane understanding, and 3D map building that complements camera-based vision for all-weather reliable autonomous navigation.