# Keypoint

Notes taken from Cyrill Stachniss.

Resources

Corner = two edges in roughly orthogonal directions Edge = a sudden brightness change

Corners are great because they are very localizable in varying lighting conditions. This is why people focus on corners to get keypoints.

##### Finding Corners

To find corners we need to search for intensity changes in two directions.

Compute the SSD of neighbour pixels around $(x,y)$ $f(x,y)=β_{(u,v)βW_{xy}}(I(u,v)βI(u+Ξ΄u,v+Ξ΄v))_{2}$

- $I(u,v)$ is the actual intensity value of the pixel at position $(u,v)$
- $Ξ΄u$ and $Ξ΄v$ are simply small displacement values, they are NOT partial derivatives
- $W_{xy}$ is a local patch around $(x,y)$

Using Taylor expansion, we obtain that $I(u+Ξ΄u,v+Ξ΄v)$ is approximately $I(u+Ξ΄u,v+Ξ΄v)βI(u,v)+[J_{x}βJ_{y}β][Ξ΄uΞ΄vβ]$ Plugging back into $f(x,y)$, we see that the taylor approximation leads to $f(x,y)ββ_{(u,v)βW_{xy}}([J_{x}βJ_{y}β][Ξ΄uΞ΄vβ])_{2}$ Written in a matrix form as $f(x,y)ββ_{(u,v)βW_{xy}}[Ξ΄uΞ΄vβ]_{T}[J_{x}J_{x}J_{y}βJ_{x}J_{y}J_{y}β][Ξ΄uΞ΄vβ]$

- the derivation for this comes from expanding $(J_{x}Ξ΄u+J_{y}Ξ΄v)_{2}=J_{x}Ξ΄u_{2}+2J_{x}J_{y}Ξ΄uΞ΄v+J_{y}Ξ΄v_{2}$

We can move the sums inside the matrix to get the Structure Matrix! $f(x,y)β[Ξ΄uΞ΄vβ]_{T}[β_{W}J_{x}β_{W}J_{y}J_{x}ββ_{W}J_{x}J_{y}β_{W}J_{y}β][Ξ΄uΞ΄vβ]$

Question $J_{x}J_{y}$ has become $J_{y}J_{x}$ for one of the terms?

This is more for convenience. $J_{x}$ and $J_{y}$ are merely real numbers.

Eigenvalue and Jacobian Matrix?

Cyrill talks about Eigenvalues at 19:30 when referencing the structure matrix. What do eigenvalues have to do with the eigenvalue?? I think of it as just a large value signifies a large change, high gradient.

Also, how are we going to deal with different orientations?

βWe want to find a matrix with 2 large eigenvaluesβ. That makes it a good corner.

He thens introduces 3 similar approaches that were introduced in the 1990s for feature detection:

All of them rely on the Structure Matrix, but they use a different criterion to decide if a point is corner or not.

I don't really understand the eigenvalue part of this....

I guess I donβt really understand how eigenvalues work, so it is important to review that.

Consider points as corners if their structure matrix has two large eigenvenlues values.

Implementation Remarks

- RGB to gray-scale conversion first
- Real images are affected by noise, smoothing of the input is suggested

This is a great summary diagram, to see high level how things work

- $B$ is the smoothing operator
- $D_{x}$ is generally a Sobel Operator

### Part 2: Difference of Gaussians

Used for SIFT descriptors

In Summary

The 2 approaches are key ingredients of most hand-designed features

Nowadays, features completely learned from data.