CameraInfo

Projection Matrix

First Heard about this from Alvin Li. It’s quite simple.

The projection matrix converts a 3D coordinate to a 2D coordinate, combining the Intrinsic Matrix $K$ with the Extrinsic Matrix $T$, i.e.

$P=K[R|t]$

But K is a 3x3 matrix?

You can add a column at the end composed of 0s to make it a 3x4 matrix.

In ROS

In ROS, they call this the projection matrix, even though really its just the calibration matrix with a shift for the baseline.

So the left 3x3 matrix is simply the Calibration Matrix, and there is $T_x$ and $T_y$ which are 0 if it is the left camera.

You need to understand

$P=\left[\begin{array}{cccc}{f}_x^{\prime }& 0& c_x^{\prime }& T_x\\ 0& f_y^{\prime }& c_y^{\prime }& T_y\\ 0& 0& 1& 0\end{array}\right]$

http://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/CameraInfo.html

# Projection/camera matrix
#     [fx'  0  cx' Tx]
# P = [ 0  fy' cy' Ty]
#     [ 0   0   1   0]
# By convention, this matrix specifies the intrinsic (camera) matrix
#  of the processed (rectified) image. That is, the left 3x3 portion
#  is the normal camera intrinsic matrix for the rectified image.
# It projects 3D points in the camera coordinate frame to 2D pixel
#  coordinates using the focal lengths (fx', fy') and principal point
#  (cx', cy') - these may differ from the values in K.
# For monocular cameras, Tx = Ty = 0. Normally, monocular cameras will
#  also have R = the identity and P[1:3,1:3] = K.
# For a stereo pair, the fourth column [Tx Ty 0]' is related to the
#  position of the optical center of the second camera in the first
#  camera's frame. We assume Tz = 0 so both cameras are in the same
#  stereo image plane. The first camera always has Tx = Ty = 0. For
#  the right (second) camera of a horizontal stereo pair, Ty = 0 and
#  Tx = -fx' * B, where B is the baseline between the cameras.
# Given a 3D point [X Y Z]', the projection (x, y) of the point onto
#  the rectified image is given by:
#  [u v w]' = P * [X Y Z 1]'
#         x = u / w
#         y = v / w
#  This holds for both images of a stereo pair.
float64[12] P # 3x4 row-major matrix

Related