Spatial Resection (P3P)
It’s DLT for a calibrated camera, i.e we are provided the Calibration Matrix . Only needs 3 points to figure out the extrinsics.
It is a direct solution.
Resources
Camera Localization
Given:
- 3D coordinates of object points Observed:
- 2D image coordinates of the object points Wanted:
- Extrinsic parameters of the calibrated camera
Reminder
Lowercase represents the image plane, uppercase represents a point in the 3D world.
Reminder about the equation
&= PX \end{align}$$ - We are given $K$, $X$ and $x$, we want to estimate $X_O$ and $R$ (the extrinsics) So we want to find the mapping from world frame $o$ to camera frame $k$. ### Grunert's Method 2-step process 1. Estimate length of projection rays to get $X_O$ 2. Estimate the orientation to get $R$ ![[attachments/Screenshot 2023-10-22 at 11.00.11 AM.png]] where $$^kx_i^s = - sign (c) N (K^{-1} x_i)$$ $N(x) = \frac{x}{|x|}$ (spherical normalization) Do we know the $z$ coordinate of $^kx_i^s$? We know #### Step 1: Estimate Length of Projection Rays We exploit the geometry of the triangles. ![[attachments/Screenshot 2023-10-22 at 10.55.19 AM.png]] > [!warning] Why is this hard? > > > I have been misunderstanding, but the key thing is, we don't know $X_O$! Knowing $X_O$ would tell us the extrnsincs. We also need the rotation $R$. ![[attachments/Screenshot 2023-10-22 at 10.55.31 AM.png]] ![[attachments/Screenshot 2023-10-22 at 10.55.41 AM.png]] ![[attachments/Screenshot 2023-10-22 at 10.55.50 AM.png]] ![[attachments/Screenshot 2023-10-22 at 10.56.01 AM.png]] ![[attachments/Screenshot 2023-10-22 at 10.56.19 AM.png]] Since this is a 4th degree polynomial, we have 4 possible solutions. ![[attachments/Screenshot 2023-10-22 at 10.40.31 AM.png]] > [!question] How to eliminate ambiguity? > > > - Known approximate solution (e.g., from GPS) or > - Use 4th points to confirm identify the correct solution #### Step 2: Orientation of the Camera This is the easy part, according to Cyrill Stacniss. It's still not clear to me how to find $R$. Need to revise this more. ![[attachments/Screenshot 2023-10-22 at 11.56.32 AM.png]] In summary ![[attachments/Screenshot 2023-10-22 at 10.44.31 AM.png]] In the real world, we don't have perfect data association. We need to do outlier handling with RANSAC. Use direct solution to find correct solution among set of corrupted points Assume $I≥3$ points 1. Select 3 points randomly 2. Estimate parameters of SR/P3P 3. Count the number of other points that support current hypotheses 4. Select best solution - Can deal with large numbers of outliers in data