LeNet-5 inference in C++

Overview

A minimal framework for CNN inference that I wrote in C++ for the course Systems Engineering and Architecting for Edge Computing at TH Ingolstadt, in the winter semester 2025/26. It implements LeNet-5 and classifies MNIST digits with a 4D tensor class in NCHW layout, the layers and an MNIST loader, and nothing else.

Two optimizations

The baseline computes convolution directly, in seven nested loops, and takes 80 ms for 100 images. The performance branch rewrites convolution as im2col followed by a matrix multiplication, which brings the same batch down to 33 ms, 2.4 times faster. The quantization branch stores the weights as int8 instead of float, which shrinks them from 245 KB to 61 KB, a reduction of 75%, at about the speed of the baseline.

The network

Two convolution layers with 5×5 kernels, each followed by ReLU and max pooling, then three fully connected layers from 400 inputs down to 120, 84 and 10 outputs, and a softmax. Every branch keeps the same network, so the measurements compare only the implementation.