Gaussian Splatting
I think I heard Sachin talk about this
But also https://twitter.com/Arata_Fukoe/status/1714931950719508967
This is really fundamental stuff that I need to learn alongside NeRF. It’s super cool.
I need to look into Gaussian SLAM.
Neural Surface Reconstruction
NeRF excels in producing high-quality, photorealistic renderings at the cost of computational efficiency, while Gaussian Splatting offers a faster and more memory-efficient alternative with potentially lower rendering quality. The choice between the two depends on the specific requirements of the application, such as the need for real-time rendering versus the demand for high-fidelity imagery.
Walkthrough (CS231n 2025 Lec 15)
Representation
Parameterize the scene as a sparse set of 3D Gaussian blobs (Kerbl et al. SIGGRAPH 2023). Each blob stores:
- Mean (position)
- Covariance (anisotropic extent, via scale + rotation)
- Density / opacity
- View-dependent color (spherical-harmonic coefficients)
Think of it as an explicit, extended point cloud — the opposite end of the spectrum from NeRF, which bakes the whole scene into a dense MLP.
Rendering
Project each Gaussian to screen space → alpha-blend front-to-back per pixel (tile-based rasterizer). No MLP in the inner loop — this is what unlocks real-time rendering.
Numbers (vs MipNeRF360)
| 3DGS (30K iters) | MipNeRF360 | |
|---|---|---|
| SSIM | 0.83 | 0.80 |
| PSNR | 26.91 | 27.11 |
| Train time | 38 min | 48 h |
| Render FPS | 137 | 0.07 |
~2000× faster rendering, ~75× faster training, comparable quality. This is why 3DGS displaced NeRF for most practical capture-and-playback work starting 2024.
NeRF vs 3DGS framing (CS231n 2024 Lec 18, slide 97)
The clean one-liner from the 2024 deck:
| NeRF | 3D Gaussian Splatting | |
|---|---|---|
| Inner loop | Query a continuous MLP along the ray | Blend a discrete set of Gaussians along the ray |
| Fitting | Several hours on best GPUs | Few minutes |
| Rendering | ~10 s/frame at moderate resolution | Real-time |
Variants (CS231n 2024 Lec 18)
- Dynamic 3D Gaussians (Luiten et al. 3DV 2024) — track a moving scene over time by persisting Gaussians frame-to-frame; gives free dense correspondences across frames as a side effect.
- Gaussian Splatting SLAM (Matsuki et al. CVPR 2024) — use a 3DGS scene as the dense map in a SLAM loop; simultaneously localize the camera and grow/refine Gaussians as new views arrive.