Latency Percentiles Explained: p50, p95, p99 for Engineers

10 min read · Last updated: October 24, 2026

Latency percentiles tell you how slow your requests are for a given share of users. p50 is the median: half of requests are faster. p95 means 95% are faster and 5% are slower. p99 means only 1 request in 100 is slower. Unlike the average, percentiles show the slow tail that real users actually feel.

Came here from Instagram, LinkedIn, X or YouTube? Here is the full source list you came for. Everything below is built from these, nothing else.

The SRE books

• Google SRE Book, Chapter 4: Service Level Objectives (Jones, Wilkes, Murphy, Smith, 2016)
• Google SRE Workbook, Chapter 2: Implementing SLOs (Thurgood, Ferguson, 2018)

The papers

• The Tail at Scale, Dean and Barroso, Communications of the ACM, February 2013
• Dynamo: Amazon's Highly Available Key-value Store, SOSP, October 2007

The docs

• Prometheus: Histograms and summaries
• Prometheus: histogram_quantile() function reference
• Amazon CloudWatch: Statistics definitions
• NumPy v2.5: numpy.percentile

The response time thresholds

• Nielsen Norman Group: Response Times, The 3 Important Limits (Jakob Nielsen, 1993)
• web.dev: Web Vitals (updated October 31, 2024)
• web.dev: Time to First Byte (updated November 2025)

Key takeaways

• p50 is the typical request, p95 is 1 in 20, p99 is 1 in 100. Higher percentiles show you a plausible worst case.
• Google's SRE book shows a system where the typical request takes about 50 ms but 5% of requests are 20 times slower, and the average graph shows nothing.
• If one server is slow on 1 in 100 requests and a page calls 100 servers, 63% of page loads hit a slow server (Dean and Barroso, 2013).
• Amazon's Dynamo paper set its SLA at the 99.9th percentile: 300 ms for 99.9% of requests at 500 requests per second.
• Google measures web vitals at the 75th percentile. Good targets: TTFB ≤ 0.8 s, INP ≤ 200 ms, LCP ≤ 2.5 s.
• You cannot average percentiles. Aggregate histograms, then compute the percentile.

The coffee shop that averages 3.9 minutes

Picture a coffee shop that serves 100 customers in a morning.

95 of them get their drink in 2 minutes. The other 5 hit a broken grinder and wait 40 minutes.

The owner looks at the average: (95 × 2 + 5 × 40) / 100 = 3.9 minutes. That sounds fine. It is also a number that describes nobody. Not one customer waited 3.9 minutes. Most waited 2, and five waited long enough to never come back.

Now look at percentiles instead:

Statistic	Value	What it tells the owner
Average	3.9 min	Nothing useful
p50 (median)	2 min	The typical customer is fine
p95	2 min	95 out of 100 are fine
p99	40 min	Something is badly broken for a few people

That table is the whole idea of latency percentiles. Your API is the coffee shop. Requests are customers. The broken grinder is a garbage collection pause, a cold cache, or a slow database replica.

Notice one more thing. In this example, p95 also looks fine. The problem only shows up at p99. Which percentile you watch decides which problems you can see.

Latency percentiles p50, p95 and p99 explained

A latency percentile is the response time below which a given percentage of requests finish. If your p95 is 300 ms, then 95% of requests took 300 ms or less and 5% took longer. p50 is the median, p95 is 1 in 20, and p99 is 1 in 100.

Amazon's CloudWatch docs define it plainly: p95 "means that 95 percent of the data within the period is lower than this value and 5 percent of the data is higher than this value."

The Google SRE book explains why teams track more than one: "using the 50th percentile (also known as the median) emphasizes the typical case," while "a high-order percentile, such as the 99th or 99.9th, shows you a plausible worst-case value."

Percentile	Also called	Requests slower than it	Use it for
p50	Median	1 in 2	The typical experience
p90	90th percentile	1 in 10	Early warning on broad slowdowns
p95	95th percentile	1 in 20	Common SLO target for user-facing APIs
p99	99th percentile	1 in 100	Tail latency, heavy users, fan-out services
p99.9	Three nines	1 in 1,000	High-volume core services (Amazon's Dynamo SLA)

What is p99 latency, and what is "top percentile" latency?

p99 latency is the response time that 99% of requests beat and 1% exceed. It is the slowest experience you see for 1 request in every 100. "Top percentile" latency, often written TP99 or TP99.9, is a different name for the same measurement: the value at the top end of the distribution, just below the slowest outliers.

One in a hundred sounds rare. It is not. If your service handles 1,000 requests a second, p99 describes 10 requests every single second. And a single user loading one page may trigger dozens of requests, so the odds that they hit at least one slow one are much higher than 1%.

That is why Amazon went further. The Dynamo paper says mean or median based SLAs "are not good enough if the goal is to build a system where all customers have a good experience, rather than just the majority." Amazon expressed its SLAs at the 99.9th percentile, and picked 99.9 over anything higher because a cost-benefit analysis showed "a significant increase in cost to improve performance that much."

p95 vs average latency: why does the average lie?

The average adds every request time and divides by the count, so a few very slow requests get blended into many fast ones. The result often matches no real request. p95 reports an actual position in the distribution: what the slowest 5% of users experience. The average can stay flat while p95 doubles.

The Google SRE book puts it directly: "A simple average can obscure these tail latencies, as well as changes in them." Its Figure 4-1 shows a system where "a typical request is served in about 50 ms, 5% of requests are 20 times slower!" and notes that alerting on the average alone "would show no change in behavior over the course of the day."

The book's stated preference is the one I would copy: "We generally prefer to work with percentiles rather than the mean (arithmetic average) of a set of values."

There is a middle ground worth knowing. CloudWatch offers a trimmed mean, tm99, which averages everything except the slowest 1%. Its docs explain the difference: p99 "tells you the worst customer experience, ignoring outliers, while tm99 tells you the average customer experience, ignoring outliers." Useful for spotting typical drift. Not a replacement for p99.

What is tail latency, and why does it get worse at scale?

Tail latency is the slow end of the latency distribution, usually p99 and above. It matters most when one user request fans out to many servers, because the user waits for the slowest one. A server that is slow only 1% of the time becomes slow for most requests once you call enough of them.

This comes from "The Tail at Scale" by Jeffrey Dean and Luiz André Barroso, published in Communications of the ACM in February 2013. Their example: each server responds in 10 ms typically, but its p99 is one second. If a request needs only one server, 1 in 100 requests is slow. If a request must collect responses from 100 such servers in parallel, 63% of user requests take more than one second.

The math is simple. The chance that all 100 servers are fast is 0.99 to the power of 100, which is about 0.37. So about 63% of requests hit at least one slow server.

They also measured a real Google service with a large fan-out:

What the root waits for	p50	p95	p99
One random leaf finishes	1 ms	5 ms	10 ms
95% of all leaf requests finish	12 ms	32 ms	70 ms
100% of all leaf requests finish	40 ms	87 ms	140 ms

Read the p99 column. Waiting for the slowest 5% of leaves takes the p99 from 70 ms to 140 ms. In the paper's words, the slowest 5% of requests "is responsible for half of the total 99%-percentile latency."

The fix they describe is not "make every server perfect." It is the hedged request: send a second copy to another replica if the first has not answered in time. In a Google benchmark reading 1,000 keys spread across 100 BigTable servers, sending a hedge after 10 ms cut the p99.9 latency from 1,800 ms to 74 ms while sending just 2% more requests.

If you are preparing for interviews, this fan-out effect is one of the ideas that separates average answers from strong ones. I go into that in why most developers fail system design interviews.

How do you calculate percentile latency?

To calculate a latency percentile from raw data, sort every response time from fastest to slowest, then pick the value at the position matching your percentile. For p95 of 1,000 requests, that is the 950th value. From a histogram, find the bucket containing that position and estimate inside it.

From raw data, step by step

1. Collect every response time for the window you care about (say, the last 5 minutes).
2. Sort them from fastest to slowest.
3. Compute the rank: rank = ceil(p / 100 × n), where n is the number of requests.
4. The value at that rank (counting from 1) is your percentile. This is the nearest-rank method.

Here it is in Python, with 20 real-looking request times in milliseconds. Two of them are slow.

import math

latencies_ms = [42, 45, 47, 48, 50, 51, 52, 53, 55, 56,
                58, 60, 61, 63, 65, 70, 74, 80, 900, 1200]

def percentile(values, p):
    s = sorted(values)
    rank = math.ceil(p / 100 * len(s))
    return s[max(rank, 1) - 1]

print("mean:", sum(latencies_ms) / len(latencies_ms))  # 156.5
print("p50: ", percentile(latencies_ms, 50))            # 56
print("p95: ", percentile(latencies_ms, 95))            # 900
print("p99: ", percentile(latencies_ms, 99))            # 1200

The mean is 156.5 ms. No request took anything close to that. The median is 56 ms, and the p95 jumps straight to 900 ms. The average blended two terrible requests into a number that looks merely a bit slow.

If you use NumPy instead, numpy.percentile defaults to method='linear', which interpolates between neighbours. On the same data it returns 57, 915 and 1,143 ms. Different methods give slightly different answers on small samples. Pick one and stick with it so your trends are comparable. NumPy also offers 'inverted_cdf', 'nearest' and others if you need to match another tool.

From a histogram (Prometheus)

Storing every request time is expensive at scale, so most monitoring systems store histograms: counters for how many requests landed in each latency bucket. Prometheus then estimates the percentile with histogram_quantile().

The official example for p90 over the last 10 minutes:

histogram_quantile(0.9, rate(http_request_duration_seconds_bucket[10m]))

For classic histograms, the Prometheus docs say the function "assumes a uniform distribution of observations within the bucket (also called linear interpolation)." Here is what that means with 1,000 requests:

Bucket (seconds)	Cumulative count
≤ 0.1	700
≤ 0.25	900
≤ 0.5	980
≤ 1.0	995
+Inf	1,000

1. Find the target rank. p95 of 1,000 is rank 950.
2. Find the bucket. Rank 950 sits between 900 and 980, so it is in the 0.25 to 0.5 second bucket, which holds 80 requests.
3. Find the position inside the bucket. (950 − 900) / 80 = 0.625 of the way through.
4. Interpolate. 0.25 + 0.625 × (0.5 − 0.25) = 0.406 seconds.

The same steps give p99 at rank 990: (990 − 980) / 15 of the way through the 0.5 to 1.0 bucket, so about 0.83 seconds.

Two warnings from the Prometheus docs that bite people:

• The estimate is only as good as your buckets. In their example, a real spike at 220 ms inside a classic 200 to 300 ms bucket gets reported as 295 ms. Put bucket edges near your SLO threshold.
• If the percentile lands in the +Inf bucket, Prometheus returns the upper bound of the second highest bucket. Your p99 will quietly cap at your top bucket edge.

Never average percentiles

If you have p95 from 10 servers, averaging those 10 numbers does not give you the fleet p95. The Prometheus docs are blunt: "you cannot aggregate quantiles," and averaging them yields "statistically nonsensical values." Sum the histogram buckets across servers first, then compute the percentile once. This is the main reason to prefer histograms over summaries, which the docs list as "Not aggregatable."

What is a good API response time?

There is no single correct number, but human perception gives hard anchors. Jakob Nielsen's research puts 0.1 seconds as the limit for feeling instant, 1 second for keeping flow of thought, and 10 seconds for keeping attention. For web pages, Google's guidance is TTFB at or under 0.8 seconds at the 75th percentile of users.

Threshold	Target	Source
Feels instant	≤ 0.1 s	Nielsen Norman Group, 1993
Flow of thought uninterrupted	≤ 1 s	Nielsen Norman Group, 1993
Attention stays on the task	≤ 10 s	Nielsen Norman Group, 1993
Good Time to First Byte	≤ 0.8 s (poor > 1.8 s)	web.dev, 2025
Good Interaction to Next Paint	≤ 200 ms	web.dev, 2024
Good Largest Contentful Paint	≤ 2.5 s	web.dev, 2024

Two things about that table matter more than the numbers.

First, Google grades all of these at the 75th percentile of page loads, split by mobile and desktop. Not the average. The official web.dev guidance says "a good threshold to measure is the 75th percentile of page loads."

Second, an API call is only one piece of a page. If the browser makes five calls in sequence, each one eats into the 1 second budget. That is why internal APIs usually need far tighter targets than the user-facing limits. If you want a refresher on how those calls are structured, read our REST API explained for beginners guide.

What is an SLO latency target, and how do you set one?

An SLO latency target is a promise about the share of requests that must finish under a threshold over a time window, such as "99% of requests complete in under 400 ms over 28 days." It is written as a percentage of good requests, not as an average, and the leftover share becomes your error budget.

The Google SRE Workbook recommends treating an SLI "as the ratio of two numbers: the number of good events divided by the total number of events." The SRE book's example SLO uses three grades at once:

• 90% of Get RPC calls will complete in less than 1 ms.
• 99% of Get RPC calls will complete in less than 10 ms.
• 99.9% of Get RPC calls will complete in less than 100 ms.

Here is the process the Workbook walks through:

1. Measure what you have today. In the Workbook's example API, four weeks of data showed a p90 of 432 ms and a p99 of 891 ms.
2. Pick two or more thresholds. One for the typical case (p90 or p95) and one for the tail (p99). A single threshold hides a slow minority.
3. Set achievable starting targets. The Workbook's example became "90% of requests < 450 ms" and "99% of requests < 900 ms."
4. Count good events, not averages. Every request is good (under the threshold) or bad. Your SLI is good divided by total.
5. Tighten over time once you know you can hold the target and users would notice the improvement.

The nice thing about writing SLOs as "X% under Y ms" is that you no longer need to compute a percentile at all. You only count requests under the threshold. Prometheus can do it from the same histogram buckets, as long as one bucket edge sits exactly on your threshold (their example: an SLO to "serve 95% of requests within 300ms").

When should you use percentiles, and when not?

Use percentiles for anything a user waits on: API response times, page loads, queue delays, database queries. Use the average for totals that add up, like CPU seconds or bytes transferred, where the sum is what costs you money. Use the maximum only for debugging single incidents, never for alerting.

Situation	Best statistic	Why
User-facing API latency	p95 and p99	Shows what real users feel, including the slow tail
Fan-out services (search, feeds)	p99 and p99.9	Tail of each backend becomes the median of the whole request
SLO and alerting	% of requests under a threshold	Countable, aggregatable, maps to an error budget
Capacity and cost	Average or sum	Totals add up; percentiles don't
Typical-case drift	p50 or trimmed mean (tm99)	Ignores outliers, moves with the bulk of traffic
Low traffic (a few requests per minute)	Longer window or raw values	p99 of 20 requests is just your slowest request

That last row catches a lot of small teams. With 20 requests, as in the Python example, p99 and the maximum are the same number. Percentiles need volume to mean anything. Widen the window until each period holds at least a few hundred requests.

Systems built for huge scale design around this tail from day one. Our breakdown of how Netflix streams videos without buffering shows the same thinking applied to video.

FAQ

Is p99 latency better than average latency?

For anything a user waits on, yes. The average blends a few very slow requests into many fast ones and often matches no real request. p99 tells you what the slowest 1% actually experienced, which is where complaints come from. Track p50 and p99 together.

What does p99.9 mean?

p99.9 is the latency that 999 out of 1,000 requests beat. Amazon's Dynamo paper used it as the SLA level: a response within 300 ms for 99.9% of requests at a peak load of 500 requests per second. It only makes sense for high-traffic services, because you need thousands of requests per window for it to be stable.

Can I average p95 values across servers?

No. The Prometheus docs state that you cannot aggregate quantiles and that averaging them gives statistically nonsensical values. Store histograms, sum the bucket counts across servers, then compute the percentile once.

Why does Google use the 75th percentile for Core Web Vitals?

web.dev recommends the 75th percentile of page loads, split by mobile and desktop, as the threshold for judging a page. It is high enough that most visitors meet the target, yet not so high that a few users on very slow devices or networks decide the score on their own.

Pull up your slowest endpoint today and check its p99 next to its average; the gap will tell you where to start. For the boring parts of the job, try the free tools on Art of Code, and if you are prepping for interviews, read why most developers fail system design interviews.

Written by Adithya Guttha, Founder of Art of Code.