Your C is not 2x faster than TensorFlow

I wrote a neural network from scratch in C. No libraries, one file, forward and backward propagation by hand. Then I benchmarked it against an equivalent network in TensorFlow and it came out around twice as fast.

That number is real. I measured it, more than once. It is also close to meaningless, and I want to explain why before someone quotes it back to me.

What the number is actually measuring

The network is 8 to 4 to 1, on a few hundred training samples. At that size, essentially none of the wall-clock time is arithmetic. It is graph construction, per-step Python dispatch, and the fixed cost of a framework arriving before it can do anything.

My C version skips all of that, because it does not have any of it. That is not a better implementation of matrix multiplication. That is the absence of a feature set I did not need.

The matrix code underneath is plain nested loops. No blocking, no cache-aware tiling, no SIMD, no BLAS. On any workload where the arithmetic actually dominates, TensorFlow wins, and it does not win narrowly. It wins by margins my loops cannot approach.

Why say so

The honest version of the claim is longer and less impressive: at a scale where framework overhead dominates, removing the framework removes the overhead. That is a tautology dressed as a benchmark.

I kept the measurement in the README with the caveat attached, because a number without its conditions is not a result, it is a marketing line. Anyone who knows the domain will ask what size the problem was, and if the answer changes the conclusion, then the size was part of the result all along.

The general rule I try to hold to: if a benchmark makes my own work look good, that is exactly when I should be hunting for the reason it might not be measuring what I think.