Short Answer
ICC (Intraclass Correlation Coefficient) is a reliability statistic that summarizes, in a single number between 0 and 1, how consistent the same measurement is across different raters or across repeated measurements by the same rater; above 0.75 is considered good and above 0.90 is considered excellent. In measurement reliability or inter-observer agreement studies, it is used by selecting the correct model (for example, two-way mixed, absolute agreement). Bland-Altman analysis is not an alternative to it but a complement; ICC gives overall agreement numerically, whereas Bland-Altman shows the systematic difference (bias) and the limits of agreement between two measurements graphically. The most common mistake is to look only at a high ICC value and overlook whether the Bland-Altman limits of agreement are clinically acceptable, and to fail to state the chosen ICC model with its justification in the paper.
Serteser Consulting is run by a biomedical engineer (BME MSc) with peer-reviewed publications and PROSPERO-registered systematic reviews; it designs and carries out thesis, manuscript, and clinical research statistics, including measurement reliability and inter-observer agreement analysis, using SPSS, R, and Python, in a manuscript-ready form that can be defended before a jury or reviewer.
Two radiologists measure the same MR image. How similar are their measurements to each other? How well does a newly developed AI model agree with expert measurements? Two basic tools are used to answer these questions: ICC and Bland-Altman analysis. So what is the difference between the two, and which one is used when?
What Is ICC?
ICC (Intraclass Correlation Coefficient) measures how consistent the results are when the same measurement is repeated by different raters (inter-rater) or by the same rater at different times (intra-rater).
The ICC value ranges from 0 to 1:
| ICC Value | Interpretation |
|---|---|
| < 0.50 | Poor reliability |
| 0.50 – 0.75 | Moderate reliability |
| 0.75 – 0.90 | Good reliability |
| > 0.90 | Excellent reliability |
ICC Models: Which One to Choose?
ICC has different models, and choosing the correct one is critical, since different models give different results.
Two-way mixed, absolute agreement (ICC 2,1): The most commonly used model. Fixed raters (e.g., always the same two radiologists), absolute agreement between raters. The standard in medical measurement reliability studies.
Two-way mixed, consistency (ICC 2,1 consistency): Measures consistency even when there is a systematic difference between raters. More tolerant than absolute agreement.
One-way random (ICC 1,1): When the raters come from a large randomly selected population.
You are required to explain in the paper which model you chose and why you chose it.
The Difference Between ICC and Bland-Altman
These two analyses are not alternatives to each other but complements.
ICC: Produces a single number. It answers the question "How good is the overall agreement?" However, it does not show the magnitude and distribution of the difference between measurements.
Bland-Altman: Is graph-based. It shows the difference between the two measurements for each patient. It reveals the systematic error (bias), the limits of agreement, and the outliers.
The standard for using them together is this: ICC gives the level of reliability numerically, while Bland-Altman shows whether the errors are clinically acceptable.
Example: ICC = 0.92 (excellent) but the limits of agreement in Bland-Altman are ±15 mm. This means the measurements are consistent but the difference between them is clinically meaningful. Which piece of information do you use? You look at both together.
Calculating ICC with SPSS
In the output, report the "Average Measures" ICC, not "Single Measures," and state this explicitly.
Calculating ICC with R
The psych package is also commonly used:
Sample Size
Power analysis is often skipped for ICC studies. To reliably detect an acceptable ICC value, 30-50 observations are generally recommended, but a formal calculation should be performed based on the target ICC value and the width of the confidence interval.
Request a free consultation for your study involving ICC and Bland-Altman analysis.
Where Do People Get Stuck Most in This Analysis?
- You chose the ICC model incorrectly (one-way vs two-way), the results change but you do not know which one is correct.
- ICC came out as 0.85 but a systematic bias appears in Bland-Altman, and how to report the two together is unclear.
- The number of raters is more than 2, and which ICC formulation you need to use gets confusing.