The Micro-Optimization Obsession
It all started with an obsession familiar to many developers: making my code faster. My lexer, a central component of my language parser, worked correctly but seemed slow to me. I spent weeks optimizing every function, eliminating unnecessary allocations, unrolling loops, exploiting SIMD instructions.
The result? A lexer twice as fast on my synthetic benchmarks. Intense satisfaction, craftsman pride. Then I integrated this gem into my real application and measured overall performance. Improvement: 3%. A measly three percent.
The Profiling Revelation
Frustrated, I pulled out the serious profiling tools. Flamegraphs, system traces, syscall analysis. The truth was there, before my eyes, and it was humiliating: my application spent more than 80% of its time waiting for disk reads.
My ultra-optimized lexer processed characters at phenomenal speed. But those characters arrived in a trickle, throttled by an absolutely suboptimal file reading pattern. I was reading files character by character, syscall after syscall, in an orgy of inefficiency that my lexer optimizations couldn't compensate for.
The Classic Tunnel Vision Error
What I had done is a classic anti-pattern, but you have to live it to really understand it. I had optimized what I knew how to optimize, what was intellectually satisfying, without ever verifying if it was what mattered.
This is Amdahl's law in action: infinitely improving a part that represents 20% of total time can never give you more than 20% overall improvement. Meanwhile, the remaining 80% watches, untouched, mocking your efforts.
The Real Optimization
Once the problem was identified, the solution was almost trivial. Replace character-by-character reads with 64KB buffer reads. Use mmap for large files. Implement intelligent prefetching. Three days of work versus three weeks on the lexer.
The result? Application 4x faster. Not twice, four times. My optimized lexer certainly contributed, but the real victory came from I/O.
Universal Lessons
This experience taught me several truths that I now share with any developer willing to listen.
First, measure before optimizing. Always. Without exception. Intuitions about bottlenecks are almost always wrong. Code that seems slow is often not what actually costs. Profiling tools exist for a reason.
Second, understand memory hierarchy. An L1 cache access takes about 1 nanosecond. A RAM access takes 100 nanoseconds. An SSD access takes 100 microseconds. An HDD access takes 10 milliseconds. These differences of several orders of magnitude often dictate real performance far more than code quality.
Third, optimize top-down. Start with architecture, algorithms, data access patterns. Move down to micro-optimizations only when the rest is solid.
I/O, the Great Forgotten
In our developer education, I/O is often treated as a detail. We learn data structures, algorithmic complexity, design patterns. But interaction with the file system, network, databases? It's relegated to "implementation details."
This neglect has a cost. How many applications are slow not because of inefficient code, but because they make network requests in loops, or read files line by line when bulk loading would be more appropriate?
Efficient I/O Patterns
To avoid my mistake, here are the principles I now apply systematically. Batch your I/O operations. One large read is worth more than a thousand small ones. Use properly sized buffers, generally aligned with the system page size.
Consider asynchronism. While you wait for I/O, the CPU could do something else. Asynchronous APIs like io_uring on Linux maximize parallelism between computation and transfer.
Exploit the system cache. If you often reread the same data, the OS probably keeps it in memory. But if you invalidate this cache through random access patterns, you pay the full price on each read.
Conclusion: The Craftsman's Humility
My ultra-fast lexer remains good code that I'm proud of. But this experience made me more humble. Optimization is not a performance sport where the fastest wins. It's a diagnostic exercise where the most observant wins.
Before diving into seductive micro-optimizations, take time to understand where time really goes. The most costly bottleneck is often the one you don't suspect.