Projection Matrix
First Heard about this from Alvin Li. Itβs quite simple.
The projection matrix converts a 3D coordinate to a 2D coordinate, combining the Intrinsic Matrix with the Extrinsic Matrix , i.e.
But
K
is a 3x3 matrix?You can add a column at the end composed of 0s to make it a 3x4 matrix.
In ROS
In ROS, they call this the projection matrix, even though really its just the calibration matrix with a shift for the baseline.
So the left 3x3 matrix is simply the Calibration Matrix, and there is and which are 0 if it is the left camera.
You need to understand
http://docs.ros.org/en/melodic/api/sensor_msgs/html/msg/CameraInfo.html
# Projection/camera matrix
# [fx' 0 cx' Tx]
# P = [ 0 fy' cy' Ty]
# [ 0 0 1 0]
# By convention, this matrix specifies the intrinsic (camera) matrix
# of the processed (rectified) image. That is, the left 3x3 portion
# is the normal camera intrinsic matrix for the rectified image.
# It projects 3D points in the camera coordinate frame to 2D pixel
# coordinates using the focal lengths (fx', fy') and principal point
# (cx', cy') - these may differ from the values in K.
# For monocular cameras, Tx = Ty = 0. Normally, monocular cameras will
# also have R = the identity and P[1:3,1:3] = K.
# For a stereo pair, the fourth column [Tx Ty 0]' is related to the
# position of the optical center of the second camera in the first
# camera's frame. We assume Tz = 0 so both cameras are in the same
# stereo image plane. The first camera always has Tx = Ty = 0. For
# the right (second) camera of a horizontal stereo pair, Ty = 0 and
# Tx = -fx' * B, where B is the baseline between the cameras.
# Given a 3D point [X Y Z]', the projection (x, y) of the point onto
# the rectified image is given by:
# [u v w]' = P * [X Y Z 1]'
# x = u / w
# y = v / w
# This holds for both images of a stereo pair.
float64[12] P # 3x4 row-major matrix