“Using an LLM to judge another LLM? That’s like asking a student to grade their own exam!”
That was a friend’s reaction when I first explained LLM-as-Judge evaluation. It does sound ridiculous. How can we trust one AI system to evaluate another fairly? Isn’t this circular reasoning dressed up in fancy statistics?
It works, and not because of AI magic. It works because of plain statistics that people have used for decades to deal with imperfect evaluators. Medical researchers use the same math to estimate disease prevalence from flawed diagnostic tests. Pollsters use it to correct for biased survey responses.
The judge has a narrow job. It labels each output as good or bad for one specific failure mode. If it does that better than a coin flip, we can correct for its mistakes mathematically.
What the judge actually judges
We don’t ask one LLM to give another a report card. We ask about one specific failure mode that we already found through error analysis.
Maybe your chatbot sometimes gives unhelpful responses. Or your code generator occasionally writes insecure code. Or your summarizer misses key details. Reviewing real outputs, you’ve spotted a pattern, a specific way your LLM fails. Now you want to know how often that failure happens.
The judge classifies each output as one of:
- Pass: this failure mode is absent
- Fail: this failure mode is present
A medical diagnostic test works the same way. Nobody asks the test to cure the disease, only to detect it.
Keeping the data sets apart
Data discipline is what protects us from circular reasoning. Most people skip it when they build LLM evaluators.
Building a judge is like training any other classifier, except that we write prompts instead of adjusting model weights. We need three separate data sets, with no example in more than one:
- Training set (10-20%): examples we might use as few-shot demonstrations in the judge’s prompt
- Development set (40-45%): where we test prompt versions and refine the judge
- Test set (40-45%): held-out data that gives unbiased performance estimates
Never let examples leak between sets. If the judge sees test examples during development, its performance estimates mean nothing.
Correcting for an imperfect judge
We want to know \(\theta\) (theta), the true success rate of our LLM system on new, unlabeled data. It is the share of outputs that would pass if a perfect human evaluator reviewed all our new production data.
We have no perfect human evaluator for thousands of new outputs. We have an imperfect LLM judge. On the test set, we compare its predictions with human labels to measure how often it gets each kind of output right:
- True Positive Rate (TPR): of the outputs that are good, the share the judge labels “Pass”.
- True Negative Rate (TNR): of the outputs that are bad, the share the judge labels “Fail”.
Running the judge on new, unlabeled data gives an observed success rate. The judge’s errors bias that rate. The Rogan-Gladen correction formula from medical research (1978) estimates the true rate from the observed one:
\[ \theta = \frac{\text{Observed Rate} + \text{TNR} - 1}{\text{TPR} + \text{TNR} - 1} \]
What the formula does
The judge’s errors distort the observed success rate in two ways, and the test set tells us by how much:
- It misses some real successes. The judge labels only TPR% of real successes “Pass”.
- It counts some real failures as successes. The judge labels only TNR% of real failures “Fail”. The rest pass.
The Rogan-Gladen formula asks:
“Given that I know my judge catches TPR% of real successes and correctly identifies TNR% of real failures, what must the true rate have been to produce what I’m observing?”
The correction works only when the judge is better than random chance, which means \(\text{TPR} + \text{TNR} > 1\). When \(\text{TPR} + \text{TNR} = 1\), the denominator is zero and the formula breaks down. That happens with a fair coin (\(\text{TPR} = \text{TNR} = 0.5\)). It also happens with a judge that is biased but cannot tell good from bad (\(\text{TPR} = 0.8\), \(\text{TNR} = 0.2\)). Neither judge tells you anything about the true rate.
In practice we clip \(\theta\) to stay between 0 and 1, as for any probability. If you get extreme values before clipping, the judge’s errors are too severe to correct reliably.
Quantifying uncertainty with the bootstrap
We now have a point estimate of \(\theta\). How far should we trust it? In 1979 Bradley Efron introduced the bootstrap, one of the defining statistical techniques of the computer age. It gives confidence intervals by resampling the data, with no complex mathematical assumptions.
Our TPR and TNR estimates come from a finite test set. A different test set would give slightly different values, and a different corrected \(\theta\). The bootstrap measures this uncertainty by resampling our test set:
- Resample the test set with replacement.
- Recompute \(\text{TPR}'\) and \(\text{TNR}'\) on the resample.
- Apply the correction formula with the new rates: \(\theta' = (\text{observed rate on new data} + \text{TNR}' - 1) / (\text{TPR}' + \text{TNR}' - 1)\). The observed rate is the same in every iteration.
- Repeat thousands of times to build a distribution of \(\theta'\) values.
The result is a 95% confidence interval, which reads roughly as:
“We’re 95% confident the true success rate lies between X% and Y%.”
(Strictly, it means that if we repeated the whole evaluation many times, 95% of such intervals would contain the true rate. The intuitive reading works fine in practice.)
If the interval is narrow and above your quality threshold, you’re done. If it’s wide or straddles the threshold, you can:
- Use a better judge. A more recent or more capable LLM can raise TPR and TNR.
- Reformulate the failure mode, perhaps by splitting it into simpler, more atomic criteria that the judge catches more reliably.
- Collect more test data. A larger test set gives more reliable TPR and TNR estimates and less bootstrap variance.
When \(\text{TPR} + \text{TNR}\) is close to 1, the denominator is small and \(1/(\text{TPR} + \text{TNR} - 1)\) is large. Small variations in the bootstrap estimates \(\text{TPR}'\) and \(\text{TNR}'\) then become large swings in \(\theta'\). That is why a judge barely better than chance produces wide, unreliable confidence intervals.
Deriving the formula
Setting up the notation:
- \(A\) = “Truth is Pass” (ground truth)
- \(B\) = “Judge says Pass” (what we observe)
- \(B^c\) = “Judge says Fail” (the complement of \(B\), meaning “not \(B\)”)
What we know and what we want:
- We want: \(P(A) = \theta\) (true success rate)
- We observe: \(P(B)\) = observed success rate from our new data
- We measured: \(P(B \mid A) = \text{TPR}\) and \(P(B^c \mid A^c) = \text{TNR}\) from our test set
The confusion matrix with joint probabilities:
Each cell holds a joint probability. For example, \(P(A \cap B)\) is the probability that the truth is Pass and the judge says Pass. The symbol \(\cap\) means both events happen.
| Judge: Pass | Judge: Fail | Total | |
|---|---|---|---|
| Truth: Pass | \(P(A \cap B)\) | \(P(A \cap B^c)\) | \(P(A)\) |
| Truth: Fail | \(P(A^c \cap B)\) | \(P(A^c \cap B^c)\) | \(P(A^c)\) |
| Total | \(P(B)\) | \(P(B^c)\) | \(1\) |
Expressing joint probabilities using conditional probabilities:
Now we rewrite each cell using the definition of conditional probability:
\[ P(B \mid A) = \frac{P(A \cap B)}{P(A)} \quad\Longrightarrow\quad P(A \cap B) = P(A)\, P(B \mid A) \]
| Judge: Pass | Judge: Fail | Total | |
|---|---|---|---|
| Truth: Pass | \(P(A)\, P(B \mid A)\) | \(P(A)\, P(B^c \mid A)\) | \(P(A)\) |
| Truth: Fail | \(P(A^c)\, P(B \mid A^c)\) | \(P(A^c)\, P(B^c \mid A^c)\) | \(P(A^c)\) |
| Total | \(P(B)\) | \(P(B^c)\) | \(1\) |
Substituting our known quantities:
- \(P(A) = \theta\) (what we want to find)
- \(P(A^c) = 1 - \theta\)
- \(P(B \mid A) = \text{TPR}\) (measured from test set)
- \(P(B^c \mid A^c) = \text{TNR}\) (measured from test set)
- \(P(B \mid A^c) = 1 - \text{TNR}\) (since if truth is fail, judge either says pass or fail)
Substituting our measured quantities into the confusion matrix:
| Judge: Pass | Judge: Fail | Total | |
|---|---|---|---|
| Truth: Pass | \(\theta\, \text{TPR}\) | \(\theta\, (1 - \text{TPR})\) | \(\theta\) |
| Truth: Fail | \((1 - \theta)(1 - \text{TNR})\) | \((1 - \theta)\, \text{TNR}\) | \(1 - \theta\) |
| Total | \(P(B)\) | \(P(B^c)\) | \(1\) |
Applying the law of total probability:
Summing the “Judge: Pass” column gives:
\[ P(B) = \theta\, \text{TPR} + (1 - \theta)(1 - \text{TNR}) \]
where \(P(B)\) is our observed success rate from running the judge on new, unlabeled data.
Solving for \(\theta\) (algebraic manipulation):
\[ \begin{aligned} P(B) &= \theta\, \text{TPR} + (1 - \theta)(1 - \text{TNR}) \\ &= \theta\, \text{TPR} + (1 - \text{TNR}) - \theta\, (1 - \text{TNR}) \\ &= \theta\, [\text{TPR} - (1 - \text{TNR})] + (1 - \text{TNR}) \\ &= \theta\, (\text{TPR} + \text{TNR} - 1) + (1 - \text{TNR}) \end{aligned} \]
Rearranging to solve for \(\theta\):
\[ \begin{aligned} P(B) - (1 - \text{TNR}) &= \theta\, (\text{TPR} + \text{TNR} - 1) \\ \theta &= \frac{P(B) - (1 - \text{TNR})}{\text{TPR} + \text{TNR} - 1} \\ \theta &= \frac{P(B) + \text{TNR} - 1}{\text{TPR} + \text{TNR} - 1} \end{aligned} \]
And there is the Rogan-Gladen formula, from nothing more than the law of total probability and some algebra.
Why this isn’t circular
The student-grading-their-own-exam objection assumes the judge has the final say. It doesn’t. Humans labeled the test set, and every rate we use to correct the judge comes from those labels. The test set measures how often the judge is wrong. The Rogan-Gladen formula corrects the observed rate for those errors. The bootstrap tells us how far to trust the corrected number. The judge only has to beat a coin flip (\(\text{TPR} + \text{TNR} > 1\)), which is a low bar.
None of this is new. Medical researchers have used the same correction since 1978 to estimate disease prevalence from imperfect tests. The next time someone dismisses LLM-as-Judge as circular reasoning, you can show them the math.
Credits: Hamel Husain and Shreya Shankar teach this method in their course on LLM evaluation. I recommend it to anyone building production LLM systems.