How Are FLOPs Estimated for a Convolution?

Multiply the kernel height by the kernel width, by the input channels, by the output height and width, by the output channels, then multiply the result by two. One multiply and one add count as two FLOPs. A 3×3 kernel over 3 input channels producing 64 channels at 32×32 needs 3,538,944 FLOPs.

The Formula

FLOPs = 2 * kH * kW * C_in * out_H * out_W * C_out

Each output element is a dot product over a kH×kW patch in every input channel, so it needs kH × kW × C_in multiply-add pairs. There are out_H × out_W positions and C_out output channels, and each multiply-add counts as two operations.

Why the Factor of Two

A multiply and an add are separate floating-point operations. Convolution inner loops do both for every term, which is why peak hardware figures are usually quoted in FMA (fused multiply-add) pairs and the count doubles when it is expressed in FLOPs. The bias add is one operation per output element instead of kH × kW × C_in, so it is left out.

Worked Example

A convolution with a 3×3 kernel, 3 input channels, 64 output channels, and an output feature map of 32×32:

2 * 3 * 3 * 3 * 32 * 32 * 64 = 3,538,944 FLOPs
parameters: 3 * 3 * 3 * 64 + 64 = 1,792

The parameter count uses the same kernel and channel terms plus one bias per output channel: 1,728 weights and 64 biases. FLOPs scale with the output area as well, so the same layer on a 64×64 output map costs four times as much for the same number of weights.

Getting the Output Size First

The formula needs out_H and out_W. For a standard convolution the calculator uses:

out = floor((in + 2 * pad - dilation * (kernel - 1) - 1) / stride) + 1

Stride and padding change the output area, and the output area is multiplied into the FLOP count, so a stride of 2 cuts the cost to roughly a quarter of the same layer at stride 1.

What the Number Leaves Out

Treat the result as an estimate for comparing layer shapes and model sizes, not as a runtime prediction. The convolution output size calculator prints FLOPs, parameter count and receptive field for each layer and sums them across a stack.

By the same builder: GitHub — theluckystrike BeLikeNative — Grammar AI EarlyThunder — Dev Blog Bug Bounty Reality Zovo — AI Dev Tools

Related Tools