Optimization

Constrained Optimization

Ohh is this what they are doing with the Sinkhorn Divergence to solve the EMD??

They add a constraint to make it convex?

Learning this in the context of CS287.

Original problem: Find ${\max }_{x}f(x)$ s.t. (subject to) $g(x)=0$

Original problem formulated as Lagrangian problem: Find ${\max }_x{\min }_{\lambda }\mathcal{L}(x,\lambda )$, where $\mathcal{L}(x,\lambda )=f(x)+\lambda g(x)$.

Our objective is: ${\max }_x{\min }_{\lambda }(f(x)+\lambda g(x))$ This Lagrangian equation respects the two conditions: $\frac{\partial \mathcal{L}(x,\lambda )}{\partial x}=0$ $\frac{\partial L(x,\lambda )}{\partial \lambda }=0$ We can then solve it.

Why are these two equivalent?

Trying to convince myself that these two are equivalent. Pieter Abbeel was explaining that even in the the Lagrangian formulation, $g(x)$ is forced to be $0$ intuitively, else you will get punished.

I still don’t get it…