When it comes to measuring how fast your system is, the average latency is almost always a lie. P95 latency tells you a much more honest (and scarier) story.

1. What Does "Latency" Even Mean?

Latency is the time it takes for your system to respond to a single request, measured from when the client sends it to when it receives the response. Think of it as round-trip time for a single API call.

When you have thousands of requests per second, latency isn't a single number. It's a distribution. Some requests are fast. Some are slow. And a few are painfully slow. To understand this distribution, we use percentiles.

2. What Is P95 Latency?

P95 (95th percentile) latency is the maximum latency experienced by 95% of your requests when all requests are sorted from fastest to slowest. In other words:

Simple Definition: If your P95 latency is 200ms, it means 95 out of every 100 requests completed in 200ms or less. The remaining 5% took longer than 200ms.

Let's say you get 1,000 requests. Sort them all by response time, from fastest to slowest. Point to request number 950. The response time at that position is your P95 latency.

3. Why Not Just Use the Average?

The average is dangerously misleading. Here's a simple example. Imagine 10 requests with these response times (in milliseconds): 

10, 12, 11, 13, 10, 11, 12, 10, 11, 500

The average is about 59ms. That sounds reasonable. But 9 out of 10 of those requests finished in around 11ms. One request took 500ms, a full half-second. That one slow request is a real user sitting there waiting. The average hides it completely.

The Problem with Averages: A single extremely slow request (an "outlier") can skew your average in a way that masks serious performance issues. Percentiles don't lie like this.

4. The Percentile Family

P95 isn't the only percentile worth knowing. Here's the full picture:

P50 (Median)

Half of your requests are faster than this, half are slower. This is the "typical" user experience. A good baseline to track.

P95

95% of requests complete within this time. This is the most commonly used SLO (Service Level Objective) threshold because it captures the experience of nearly all users while ignoring extreme edge cases.

P99

99% of requests complete within this time. Useful for high-stakes systems (payments, auth) where even rare slowness is unacceptable.

P99.9 (The "Nines")

Only 1 in 1,000 requests can be slower than this. Used in ultra-reliable systems where tail latency directly impacts revenue or safety.

5. A Real-World Example

Suppose your API serves 10,000 requests per minute. Your monitoring dashboard shows: 

P50:  18ms
P95:  200ms
P99:  850ms

What does this tell you?

  • The typical user gets a response in 18ms. That's great.
  • But 500 users per minute (5% of 10,000) are waiting 200ms or more.
  • And 100 users per minute (1%) are stuck waiting 850ms or more, nearly a full second. That's a bad experience for real people.
Key Insight: At scale, your tail latency (P95, P99) affects thousands of real users, not just theoretical edge cases. A 1% problem with 1,000,000 daily users is 10,000 people having a bad day.

6. What Causes High Tail Latency?

Tail latency spikes can come from many places:

  • Garbage Collection pauses: JVM-based services (Java, Kotlin) frequently experience GC pauses that stall a single request while memory is reclaimed.
  • Database lock contention: When many requests compete for the same row or table lock, some must wait their turn.
  • Network jitter: Temporary packet loss or retransmissions in the underlying network layer.
  • Cold starts: Serverless functions or auto-scaled containers that haven't warmed up yet respond much slower on first invocation.
  • Resource exhaustion: Thread pools, connection pools, or CPU being saturated under high load causes queuing delays.

7. How to Use P95 in Practice

P95 latency is most useful when you tie it to an SLO (Service Level Objective). An SLO is simply a promise you make about your system's behavior. For example: 

# Example SLO Definition
"99% of all API requests will complete within 300ms (P99 ≤ 300ms)"
"95% of payment transactions will complete within 500ms (P95 ≤ 500ms)"

Once you define your SLOs, you can set up alerts in your monitoring tool (Prometheus, Datadog, Grafana) to notify you the moment your P95 crosses the threshold. This is far more actionable than "average latency went up a bit."

Alert on P95

Set your alerting thresholds on percentiles, not averages. This catches real user pain before it becomes an outage.

Track Trends

Watch how P95 changes over time and across deployments. A rising P95 after a deploy is a clear signal of a regression.

Segment by Endpoint

Don't just track overall P95. Break it down by endpoint. A single slow route can drag your whole service's numbers up.

8. Quick Summary

The One-Paragraph Version:

P95 latency is the response time that 95% of your requests fall under. If your P95 is 200ms, then 95 out of every 100 requests finished in 200ms or less, and 5 didn't. The remaining 5% are your "tail": real users experiencing slowness that averages would never reveal. Tracking P95 (and P99) gives you honest visibility into your system's worst-case behavior, which is exactly where performance problems hurt users most.