PointNet

https://arxiv.org/abs/1612.00593 There is a need for 3D Deep Learning

However, 3D has ways to be represented:

• Point Cloud
• Mesh
• Volumetric Projected View RGB(D)

Point Cloud is the closest to raw sensor data. Point cloud is canonical.

Invariance Permutation invariance Max Pooling gives best performance.

Offers a unified approach to various 3D recognition tasks:

• Classification
• Part Segmentation
• Semantic Segmentation

I remember where they talk about these functions that are symmetric or something? YES, INVARIANT FUNCTIONS

• Like the max function, or the sum function

Farthest point sampling to select centroids.

Walkthrough (CS231n 2025 Lec 15)

The invariances a point-cloud network must satisfy

A point cloud is an unordered set $\{x_1,\dots ,x_n\}$ with $x_i\in {\mathbb{R}}^3$ (possibly with RGB). Two invariances:

• Permutation invariance: $f(x_1,\dots ,x_n)\equiv f(x_{\pi 1},\dots ,x_{\pi n})$ for any permutation $\pi$. The ordering in the input list is arbitrary.
• Sampling invariance: output should depend only on the underlying geometry, not on which subset of points was sampled.

Symmetric-function decomposition

A function is symmetric if it’s invariant to permutation. Simple examples: $\max$, $\sum$. PointNet uses the factorization: $f(x_1,\dots ,x_n)=\gamma \circ g(h(x_1),\dots ,h(x_n))$

• $h:{\mathbb{R}}^3\to {\mathbb{R}}^D$ — shared per-point MLP.
• $g$ — a symmetric aggregator (PointNet picks max pool over the $n$ points per feature channel).
• $\gamma$ — final MLP producing the task output.

This is provably a universal approximator over symmetric continuous set functions.

Point-cloud distances (for generation / reconstruction)

When the prediction is itself a point cloud, you need a permutation-invariant loss:

• Chamfer: $d_{\text{CD}}(S_1,S_2)={\sum }_{x\in S_1}{\min }_{y\in S_2}\Vert x-y{\Vert }_2+{\sum }_{y\in S_2}{\min }_{x\in S_1}\Vert x-y{\Vert }_2$. Cheap, asymmetric in effect — bad at penalizing density mismatches.
• Earth Mover’s (EMD): $d_{\text{EMD}}(S_1,S_2)={\min }_{\varphi :S_1\to S_2}\sum \Vert x-\varphi (x){\Vert }_2$ over bijections $\varphi$ (requires equal size). Expensive, but geometrically meaningful.

Graph extensions

EdgeConv (Wang et al. TOG 2019) treats points as graph nodes and $k$-NN neighborhoods as edges — lets the per-point $h$ also see local geometric context, not just the point itself.

Source

CS231n 2025 Lec 15 slides ~63–75 (permutation / sampling invariance, symmetric-function decomposition $\gamma \circ g\circ h$, max-pool aggregator, Chamfer + EMD, graph-on-points / EdgeConv).

Related

• Point Cloud
• 3D Representation
• Earth Mover’s Distance
• Invariance