PyTorch Tensor A tensor is a mathematical object that generalizes scalars, vectors, and matrices to higher dimensions. Kajanan made it click for me: A vector is a collection of scalars A matrix is a group of vectors A tensor is a group of matrices!