Alice's Impatience with Latency
Meet Alice. Alice is impatient. She uses your web service and measures her time in seconds and minutes. When you tell her that your service's average response time is 100 ms, Alice insists her average wait time is 1 second. You're both correct, but why?
The Gap Between Technical Metrics and Human Perception
The discrepancy between what users like Alice experience and what your technical metrics report is often due to the inspection paradox. Understanding this discrepancy is crucial for any tech company aiming to enhance user experience.
Real-World Example: MTTR and User Perception
Take Alex, another user who complains about your service's outages. You claim your MTTR (Mean Time To Recovery) is less than a minute, but Alex perceives an average of one hour. Again, the inspection paradox comes into play. Users like Alice and Alex don't experience your latency distribution $f(t)$ but a time-weighted version of it.
The Mathematics Behind the Experience
The mathematical formula explaining this phenomenon is: $ \mathbb{E}_a[X] = \frac{\mathbb{E}[X^2]}{\mathbb{E}[X]} = \mathbb{E}[X] + \frac{\mathrm{Var}(X)}{\mathbb{E}[X]}$. Essentially, users spend most of their waiting time on long-lasting requests.
Simulation and Understanding
To illustrate, imagine your median latency is 30 minutes and your 99th percentile latency is 10 hours. Your MTTR is just over an hour, but your customers experience a mean recovery time of around 6 hours!
Why Tail Latency Matters
Understanding tail latency is critical for several reasons. On one hand, it directly affects user satisfaction, impacting retention and reputation. On the other hand, it helps identify bottlenecks within your systems, leading to significant technical improvements.
Conclusion: Bridging the Gap Between Perception and Reality
The key to satisfying users like Alice is to align customer perception with technical metrics. This may involve improving communication about actual service times or actively reducing extreme latencies.
Let's discuss your project in 15 minutes.