Feature Point


Notes taken from Cyrill Stachniss.


Corner = two edges in roughly orthogonal directions Edge = a sudden brightness change

Corners are great because they are very localizable in varying lighting conditions. This is why people focus on corners to get keypoints.

Finding Corners

To find corners we need to search for intensity changes in two directions.

Compute the SSD of neighbour pixels around

  • is the actual intensity value of the pixel at position
  • and are simply small displacement values, they are NOT partial derivatives
  • is a local patch around

Using Taylor expansion, we obtain that is approximately Plugging back into , we see that the taylor approximation leads to Written in a matrix form as

  • the derivation for this comes from expanding

We can move the sums inside the matrix to get the Structure Matrix!

J_x J_y has become J_y J_x for one of the terms?

This is more for convenience. and are merely real numbers.

Eigenvalue and Jacobian Matrix?

Cyrill talks about Eigenvalues at 19:30 when referencing the structure matrix. What do eigenvalues have to do with the eigenvalue?? I think of it as just a large value signifies a large change, high gradient.

Also, how are we going to deal with different orientations?

β€œWe want to find a matrix with 2 large eigenvalues”. That makes it a good corner.

He thens introduces 3 similar approaches that were introduced in the 1990s for feature detection:

All of them rely on the Structure Matrix, but they use a different criterion to decide if a point is corner or not.

I don't really understand the eigenvalue part of this....

I guess I don’t really understand how eigenvalues work, so it is important to review that.

Consider points as corners if their structure matrix has two large eigenvenlues values.

Implementation Remarks

  • RGB to gray-scale conversion first
  • Real images are affected by noise, smoothing of the input is suggested

This is a great summary diagram, to see high level how things work

Part 2: Difference of Gaussians

Used for SIFT descriptors

In Summary

The 2 approaches are key ingredients of most hand-designed features

  1. Structure Matrix
  2. Difference of Gaussians

Nowadays, features completely learned from data.