Matrix Operations Reference — Interactive Cheatsheet

Q: What are the most important matrix operations in linear algebra?

The most important matrix operations are addition, scalar multiplication, matrix multiplication, transpose, inverse, and determinant. Addition and scalar multiplication are element-wise operations. Matrix multiplication combines rows of the first matrix with columns of the second using dot products, and is not commutative (AB does not generally equal BA). The transpose flips rows and columns. The inverse of a square matrix A is the matrix A-inverse such that A times A-inverse equals the identity matrix. The determinant is a scalar that indicates whether a matrix is invertible (nonzero determinant) and measures the volume scaling factor of the linear transformation. Beyond these basics, eigenvalue decomposition, singular value decomposition (SVD), LU decomposition, and QR decomposition are essential for solving systems of equations, dimensionality reduction, and numerical stability.

Q: What is the difference between element-wise and matrix multiplication?

Element-wise multiplication (also called the Hadamard product) multiplies corresponding entries of two matrices of the same size: C(i,j) = A(i,j) * B(i,j). Matrix multiplication is fundamentally different: for an m-by-n matrix A and an n-by-p matrix B, the product C = AB is an m-by-p matrix where C(i,j) = sum of A(i,k) * B(k,j) for k from 1 to n. Matrix multiplication requires the inner dimensions to match (columns of A = rows of B), is not commutative, and produces a matrix of potentially different dimensions. Element-wise multiplication is commutative and requires identical dimensions. In deep learning, element-wise multiplication is used in gating mechanisms (like LSTMs and attention), while standard matrix multiplication is used in fully connected layers and linear transformations.

Q: When is a matrix invertible and how do you compute the inverse?

A square matrix A is invertible (nonsingular) if and only if its determinant is nonzero. Equivalently, A is invertible when its columns (or rows) are linearly independent, when the only solution to Ax = 0 is x = 0, when all eigenvalues are nonzero, and when rank(A) equals the number of rows. For a 2x2 matrix [[a,b],[c,d]], the inverse is (1/det) * [[d,-b],[-c,a]] where det = ad - bc. For larger matrices, the inverse can be computed using Gauss-Jordan elimination (augment A with the identity and row-reduce), LU decomposition, or the adjugate formula A-inverse = (1/det(A)) * adj(A). In practice, computing the explicit inverse is usually avoided in favor of solving the system Ax = b directly using LU or QR decomposition, which is more numerically stable and computationally efficient.

Q: What are eigenvalues and eigenvectors used for?

An eigenvector v of a matrix A is a nonzero vector that satisfies Av = lambda * v, where lambda is the corresponding eigenvalue. In other words, the matrix transformation only scales the eigenvector without changing its direction. Eigenvalues and eigenvectors are used extensively: in Principal Component Analysis (PCA), the eigenvectors of the covariance matrix define the principal components and the eigenvalues measure the variance along each component. In Google's PageRank, the ranking vector is the dominant eigenvector of the web link matrix. In physics, eigenvectors represent fundamental modes of vibration in mechanical systems and quantum states in quantum mechanics. In stability analysis, eigenvalues determine whether a dynamical system is stable (all eigenvalues have negative real parts) or unstable. The eigendecomposition A = V * diag(lambda) * V-inverse enables efficient computation of matrix powers and the matrix exponential.

Q: What is the difference between LU, QR, and SVD decomposition?

LU decomposition factors a square matrix A into a lower triangular matrix L and an upper triangular matrix U: A = LU (with possible row permutation P, giving PA = LU). It is the fastest decomposition for solving Ax = b and computing determinants. QR decomposition factors any m-by-n matrix A into an orthogonal matrix Q and an upper triangular matrix R: A = QR. It is more numerically stable than LU and is the standard method for solving least-squares problems. Singular Value Decomposition (SVD) factors any m-by-n matrix A into U * Sigma * V-transpose, where U and V are orthogonal and Sigma is diagonal with non-negative singular values. SVD is the most general and powerful decomposition: it reveals the rank, the range, the null space, the condition number, and the best low-rank approximation of any matrix. SVD is the foundation of PCA, latent semantic analysis, image compression, and pseudoinverse computation.

A comprehensive reference of 20+ matrix operations organized by category. Each operation includes the formula, a visual example with small matrices, and a "Try It" mini-calculator. Search by name or filter by category to quickly find any operation.

24 operations shown · Click any card to expand the visual example and mini-calculator

Last updated: May 25, 2026

Understanding Matrix Operations: A Complete Guide

Matrix operations form the computational backbone of linear algebra, and linear algebra is the mathematical language of modern science, engineering, and machine learning. Every neural network forward pass is a sequence of matrix multiplications and element-wise nonlinearities. Every least-squares regression reduces to solving a matrix equation. Every principal component analysis hinges on eigenvalue decomposition. Mastering matrix operations is not merely academic; it is a practical prerequisite for working effectively with data, models, and simulations.

Matrices encode linear transformations: rotations, reflections, scalings, shears, and projections. A 2×2 matrix acting on a 2D vector produces another 2D vector, and the matrix completely specifies how the entire plane is warped. The determinant measures how areas (or volumes in higher dimensions) scale under the transformation. If the determinant is zero, the transformation collapses the space into a lower dimension, and the matrix is singular (non-invertible). If the determinant is negative, the transformation includes a reflection that reverses orientation.

Basic Operations: Addition, Scaling, and Multiplication

Matrix addition and scalar multiplication are straightforward element-wise operations. Given matrices A and B of the same dimensions m×n, their sum C = A + B has entries C(i,j) = A(i,j) + B(i,j). Scalar multiplication scales every entry: (cA)(i,j) = c · A(i,j). These operations satisfy the familiar commutative and associative laws: A + B = B + A, and c(A + B) = cA + cB.

Matrix multiplication is far more interesting and subtle. The product AB requires the number of columns of A to equal the number of rows of B. If A is m×n and B is n×p, then AB is m×p with entries (AB)(i,j) = sum over k of A(i,k)B(k,j). This is the dot product of row i of A with column j of B. Matrix multiplication is associative, A(BC) = (AB)C, and distributive, A(B+C) = AB + AC, but crucially not commutative: AB does not generally equal BA, even when both products are defined. This non-commutativity reflects the fact that the order of geometric transformations matters: rotating then scaling is different from scaling then rotating.

The Transpose and Its Properties

The transpose of a matrix A, written A^T, flips the matrix across its main diagonal: (A^T)(i,j) = A(j,i). Rows become columns and columns become rows. The transpose satisfies several important identities: (A^T)^T = A, (A+B)^T = A^T + B^T, (cA)^T = cA^T, and the critical reversal law (AB)^T = B^T A^T. A matrix that equals its own transpose (A = A^T) is called symmetric. Symmetric matrices arise naturally as covariance matrices, Hessians (second-derivative matrices), and adjacency matrices of undirected graphs. They have the special property that all their eigenvalues are real, and their eigenvectors can be chosen to be orthogonal.

The conjugate transpose (or Hermitian transpose), written A*, generalizes the transpose to complex matrices by also taking the complex conjugate of each entry. A matrix equal to its conjugate transpose is Hermitian, and Hermitian matrices are the complex analog of symmetric matrices. In quantum mechanics, observable quantities correspond to Hermitian operators, guaranteeing real eigenvalues (measurement outcomes).

Determinant, Trace, and Rank

The determinant of a square matrix is a scalar that encodes several key properties. For a 2×2 matrix [[a,b],[c,d]], det = ad - bc. For 3×3 and larger, the determinant is computed by cofactor expansion or, more efficiently, by LU decomposition (the determinant equals the product of the diagonal entries of U, times the sign of the permutation). The determinant is zero if and only if the matrix is singular. The absolute value of the determinant gives the factor by which the linear transformation scales volumes. The determinant is multiplicative: det(AB) = det(A) det(B). And det(A^T) = det(A).

The trace of a square matrix is the sum of its diagonal entries: tr(A) = A(1,1) + A(2,2) + ... + A(n,n). The trace equals the sum of the eigenvalues, and the determinant equals the product of the eigenvalues. The trace is invariant under cyclic permutations: tr(ABC) = tr(CAB) = tr(BCA). The rank of a matrix is the number of linearly independent rows (or equivalently columns), which equals the number of nonzero singular values. A matrix is full rank when its rank equals the smaller of its dimensions.

Matrix Decompositions

Matrix decompositions (factorizations) are the workhorses of numerical linear algebra. LU decomposition factors A = LU where L is lower triangular and U is upper triangular; it is the basis of Gaussian elimination and is the fastest way to solve Ax = b for a single system. With partial pivoting (PA = LU), it is numerically stable for most practical matrices.

QR decomposition factors A = QR where Q is orthogonal (Q^T Q = I) and R is upper triangular. Because orthogonal transformations preserve norms and angles, QR decomposition is more numerically stable than LU for ill-conditioned problems. It is the standard method for solving least-squares problems (minimize ||Ax - b||) and for computing eigenvalues via the QR algorithm.

Singular Value Decomposition (SVD) is the most powerful and general decomposition. Any m×n matrix A can be written as A = U Σ V^T, where U is m×m orthogonal, V is n×n orthogonal, and Σ is m×n diagonal with non-negative singular values σ1 ≥ σ2 ≥ ... ≥ 0. The singular values reveal the rank (number of nonzero σ's), the condition number (σ1/σ_min), and the best low-rank approximation (truncate to the k largest singular values). SVD underlies PCA, latent semantic analysis, recommender systems, image compression, and the pseudoinverse.

Special Matrices and Their Properties

Several matrix types appear repeatedly across applications. The identity matrix I has ones on the diagonal and zeros elsewhere; it is the multiplicative identity (AI = IA = A). Diagonal matrices have nonzero entries only on the main diagonal; they are trivially invertible (invert each diagonal entry) and their eigenvalues are the diagonal entries. Triangular matrices (upper or lower) are square matrices with zeros above or below the diagonal; they arise in LU decomposition and are easy to solve via forward or back substitution.

Orthogonal matrices satisfy Q^T Q = QQ^T = I, meaning their rows and columns are orthonormal vectors. Orthogonal transformations preserve lengths and angles (they are rotations and reflections). Their determinant is ±1. Positive definite matrices are symmetric matrices where all eigenvalues are positive, equivalently where x^T A x > 0 for all nonzero vectors x. They arise as covariance matrices, Hessians of convex functions, and kernel matrices. The Cholesky decomposition A = LL^T exists if and only if A is symmetric positive definite, and it is twice as efficient as LU decomposition.

Frequently Asked Questions

What are the most important matrix operations in linear algebra?

The core operations are addition, scalar multiplication, matrix multiplication, transpose, inverse, and determinant. Matrix multiplication uses dot products of rows and columns and is not commutative. The inverse reverses a transformation, and the determinant indicates invertibility and volume scaling. For advanced work, eigendecomposition, SVD, LU, and QR decomposition are essential.

What is the difference between element-wise and matrix multiplication?

Element-wise (Hadamard) multiplication multiplies corresponding entries of same-sized matrices: C(i,j) = A(i,j)B(i,j). Matrix multiplication computes C(i,j) as the dot product of row i of A with column j of B, requires matching inner dimensions, and is not commutative. Element-wise is used in gating mechanisms (LSTMs); matrix multiplication in linear layers.

When is a matrix invertible and how do you compute the inverse?

A square matrix is invertible if and only if its determinant is nonzero (equivalently: columns are linearly independent, rank equals size, all eigenvalues are nonzero). For 2×2: swap diagonals, negate off-diagonals, divide by det. For larger matrices, use Gauss-Jordan elimination or LU decomposition. In practice, solving Ax = b directly is preferred over computing the explicit inverse.

What are eigenvalues and eigenvectors used for?

Eigenvectors are directions that a matrix only scales (not rotates): Av = λv. Eigenvalues λ are the scaling factors. PCA uses eigenvectors of the covariance matrix. Google PageRank uses the dominant eigenvector of the link matrix. Stability analysis checks whether eigenvalues have negative real parts. Eigendecomposition enables efficient matrix powers and exponentials.

What is the difference between LU, QR, and SVD decomposition?

LU factors A = LU (lower × upper triangular) for fast linear system solving. QR factors A = QR (orthogonal × upper triangular) for more stable least-squares. SVD factors A = UΣV^T for any matrix, revealing rank, condition number, and best low-rank approximation. SVD is the most general; LU is the fastest for square systems; QR is the standard for least-squares.

Related Tools

Matrix Multiplication Calculator — step-by-step matrix product with dimension checking
Determinant Calculator — cofactor expansion and LU-based determinant
Eigenvector Step Solver — characteristic polynomial and null space computation
Numerical Stability Guide — condition numbers and floating-point precision analysis

Built by Michael Lip. Try the ML3X Matrix Calculator for interactive step-by-step solutions.