Notes taken from Cyrill Stachniss.
Corner = two edges in roughly orthogonal directions Edge = a sudden brightness change
Corners are great because they are very localizable in varying lighting conditions. This is why people focus on corners to get keypoints.
To find corners we need to search for intensity changes in two directions.
Compute the SSD of neighbour pixels around
- is the actual intensity value of the pixel at position
- and are simply small displacement values, they are NOT partial derivatives
- is a local patch around
Using Taylor expansion, we obtain that is approximately Plugging back into , we see that the taylor approximation leads to Written in a matrix form as
- the derivation for this comes from expanding
We can move the sums inside the matrix to get the Structure Matrix!
Question has become for one of the terms?
This is more for convenience. and are merely real numbers.
Eigenvalue and Jacobian Matrix?
Cyrill talks about Eigenvalues at 19:30 when referencing the structure matrix. What do eigenvalues have to do with the eigenvalue?? I think of it as just a large value signifies a large change, high gradient.
Also, how are we going to deal with different orientations?
“We want to find a matrix with 2 large eigenvalues”. That makes it a good corner.
He thens introduces 3 similar approaches that were introduced in the 1990s for feature detection:
All of them rely on the Structure Matrix, but they use a different criterion to decide if a point is corner or not.
I don't really understand the eigenvalue part of this....
I guess I don’t really understand how eigenvalues work, so it is important to review that.
Consider points as corners if their structure matrix has two large eigenvenlues values.
- RGB to gray-scale conversion first
- Real images are affected by noise, smoothing of the input is suggested
This is a great summary diagram, to see high level how things work
- is the smoothing operator
- is generally a Sobel Operator
Part 2: Difference of Gaussians
Used for SIFT descriptors
The 2 approaches are key ingredients of most hand-designed features
Nowadays, features completely learned from data.