Occupancy Network

Occupancy networks are a type of implicit 3D shape representation using neural networks — representing 3D geometry by learning a function that predicts whether any point in 3D space is inside or outside an object, enabling continuous, topology-agnostic 3D reconstruction and generation.

What Are Occupancy Networks?

  • Definition: Neural network f(x, y, z) → [0, 1] predicts occupancy probability.
  • Occupancy: 1 if point inside object, 0 if outside.
  • Continuous: Query at any 3D coordinate, arbitrary resolution.
  • Implicit: Surface defined by decision boundary (occupancy = 0.5).
  • Topology-Free: Handles any topology (holes, disconnected parts).

Why Occupancy Networks?

  • Arbitrary Topology: No restrictions on shape complexity.
  • Resolution-Independent: Extract mesh at any resolution.
  • Continuous: Smooth surface representation.
  • Compact: Shape encoded in network weights.
  • Differentiable: Enable gradient-based optimization.
  • Flexible Input: Learn from point clouds, images, voxels.

Occupancy Network Architecture

Basic Architecture:

Input: 3D coordinates (x, y, z)
       Optional: latent code z for shape
Encoder: Process input data (point cloud, image) → latent code
Decoder: MLP maps (x, y, z, latent) → occupancy [0, 1]
Output: Occupancy probability at query point

Components:
- Encoder: Extracts shape features from input (PointNet, CNN).
- Latent Code: Compact shape representation.
- Decoder: MLP predicts occupancy from coordinates + latent.
- Activation: Sigmoid for probability output.

Training:
- Loss: Binary cross-entropy between predicted and ground truth occupancy.
- Sampling: Sample points inside and outside object during training.
- Supervision: Ground truth occupancy from mesh or voxels.

How Occupancy Networks Work

Training Phase:
1. Input: 3D shape (mesh, point cloud, image).
2. Encode: Extract latent code representing shape.
3. Sample Points: Sample 3D points inside and outside object.
4. Predict: Decoder predicts occupancy for sampled points.
5. Loss: Compare predictions to ground truth occupancy.
6. Optimize: Update network weights via backpropagation.

Inference Phase:
1. Input: New observation (point cloud, image).
2. Encode: Extract latent code.
3. Query: Evaluate occupancy at many 3D points.
4. Extract Surface: Use Marching Cubes to extract mesh at occupancy = 0.5.
5. Output: 3D mesh of reconstructed shape.

Applications

3D Reconstruction:
- Use: Reconstruct 3D shapes from partial observations.
- Input: Point clouds, depth images, RGB images.
- Benefit: Handles incomplete data, arbitrary topology.

Shape Generation:
- Use: Generate novel 3D shapes.
- Method: Sample latent codes, decode to occupancy fields.
- Benefit: Smooth, diverse shapes.

Shape Completion:
- Use: Complete partial shapes.
- Process: Encode partial input → decode to complete occupancy.
- Benefit: Plausible completions.

Single-View 3D Reconstruction:
- Use: Reconstruct 3D from single image.
- Process: Image → encoder → latent → occupancy → mesh.
- Benefit: 3D from 2D.

Shape Interpolation:
- Use: Smoothly interpolate between shapes.
- Method: Interpolate latent codes, decode to occupancy.
- Benefit: Continuous shape morphing.

Occupancy Network Variants

Conditional Occupancy Networks:
- Method: Condition on input observations (point cloud, image).
- Benefit: Reconstruct from partial data.

Multi-Resolution Occupancy Networks:
- Method: Hierarchical occupancy prediction.
- Benefit: Capture both coarse and fine details.

Convolutional Occupancy Networks:
- Method: Use convolutional features instead of global latent.
- Benefit: Better local detail, scalability.

Implicit Feature Networks:
- Method: Learn continuous feature fields.
- Benefit: Richer representation than binary occupancy.

Advantages

Topology Freedom:
- Benefit: Represent any topology (genus, disconnected parts).
- Contrast: Meshes have fixed topology, voxels limited resolution.

Resolution Independence:
- Benefit: Extract mesh at any resolution.
- Use: Adaptive detail based on needs.

Compact Representation:
- Benefit: Shape encoded in network weights (KB vs. MB for meshes).

Smooth Surfaces:
- Benefit: Continuous function produces smooth surfaces.

Differentiable:
- Benefit: Enable gradient-based optimization, inverse problems.

Challenges

Computational Cost:
- Problem: Querying many points for mesh extraction is slow.
- Solution: Hierarchical evaluation, octree acceleration, hash encoding.

Training Data:
- Problem: Requires ground truth occupancy (from meshes or voxels).
- Solution: Sample points from meshes, use synthetic data.

Surface Detail:
- Problem: MLPs may struggle with fine details.
- Solution: Positional encoding, multi-resolution, local features.

Generalization:
- Problem: Each shape requires separate training (original formulation).
- Solution: Conditional networks, meta-learning.

Occupancy vs. Other Implicit Representations

Occupancy vs. SDF:
- Occupancy: Binary inside/outside, probability.
- SDF: Signed distance to surface, metric information.
- Trade-off: SDF provides distance, occupancy simpler to learn.

Occupancy vs. Voxels:
- Occupancy: Continuous, query anywhere.
- Voxels: Discrete grid, fixed resolution.
- Benefit: Occupancy is resolution-independent.

Occupancy vs. Meshes:
- Occupancy: Implicit, topology-free.
- Meshes: Explicit, efficient rendering.
- Use Case: Occupancy for reconstruction, mesh for rendering.

Occupancy Network Pipeline

3D Reconstruction Pipeline:
1. Input: Partial observation (point cloud, image).
2. Encoding: Extract latent code via encoder network.
3. Occupancy Prediction: Query decoder at many 3D points.
4. Surface Extraction: Marching Cubes at occupancy threshold (0.5).
5. Mesh Output: Triangulated surface mesh.
6. Post-Processing: Smooth, simplify, texture.

Training Pipeline:
1. Dataset: Collection of 3D shapes (ShapeNet, etc.).
2. Preprocessing: Sample occupancy points from meshes.
3. Training: Optimize encoder-decoder to predict occupancy.
4. Validation: Test on held-out shapes.
5. Deployment: Use trained network for reconstruction.

Quality Metrics

  • IoU (Intersection over Union): Volumetric overlap with ground truth.
  • Chamfer Distance: Point-to-surface distance.
  • Normal Consistency: Alignment of surface normals.
  • F-Score: Precision-recall at distance threshold.
  • Visual Quality: Subjective assessment of reconstructions.

Occupancy Network Implementations

Original Occupancy Networks:
- Paper: "Occupancy Networks: Learning 3D Reconstruction in Function Space" (2019).
- Architecture: PointNet encoder + MLP decoder.
- Use: Single-shape and conditional reconstruction.

Convolutional Occupancy Networks:
- Improvement: Local convolutional features instead of global latent.
- Benefit: Better detail, scalability to large scenes.

IF-Net (Implicit Feature Networks):
- Improvement: Multi-scale implicit features.
- Benefit: High-quality reconstruction.

Neural Implicit Representations:
- Related: DeepSDF, NeRF, SIREN.
- Difference: Different implicit functions (SDF, radiance).

Occupancy Network Tools

Research Implementations:
- Official Code: PyTorch implementations on GitHub.
- Frameworks: PyTorch3D, Kaolin support implicit representations.

Mesh Extraction:
- Marching Cubes: Standard algorithm for isosurface extraction.
- Libraries: scikit-image, PyMCubes, Open3D.

Visualization:
- MeshLab: View extracted meshes.
- Blender: Render and edit reconstructions.

Applications in Practice

Robotics:
- Use: Reconstruct object shapes for grasping.
- Benefit: Handle partial views, arbitrary shapes.

AR/VR:
- Use: Reconstruct environments for immersive experiences.
- Benefit: Continuous, high-quality geometry.

3D Content Creation:
- Use: Generate 3D assets from sketches or images.
- Benefit: Accelerate content creation workflow.

Medical Imaging:
- Use: Reconstruct organs from CT/MRI scans.
- Benefit: Smooth, anatomically plausible shapes.

Future of Occupancy Networks

  • Real-Time: Fast inference for interactive applications.
  • High-Resolution: Capture fine geometric details.
  • Generalization: Single model for all object categories.
  • Hybrid: Combine with explicit representations for efficiency.
  • Dynamic: Represent deforming and articulated shapes.
  • Semantic: Integrate semantic understanding with geometry.

Occupancy networks are a powerful implicit 3D representation — they enable learning continuous, topology-free shape representations that can be reconstructed from partial observations, supporting applications from 3D reconstruction to shape generation, representing a fundamental advance in neural 3D geometry.

Take occupancy network further

Ask the copilot about this term, or have our engineers assess it against your process.

Ask CFSGPT about this → Request a technical assessment → Open the CFS app