A major revision notice arrived from the journal. Among the third reviewer's comments is this sentence: "The statistical methods used are inappropriate for the data distribution." For a study you have spent months on, this moment can be extremely demoralizing.
Before panicking, know this: The vast majority of these issues can be fixed. With a well-prepared revision response, acceptance is possible.
The Statistical Errors Reviewers Most Frequently Detect
1. Using a parametric test without checking normality
Using an independent-samples t-test or ANOVA even though the data do not follow a normal distribution. A reviewer will recognize this and flag it.
Fix: Add the Shapiro-Wilk test along with histogram and Q-Q plot visuals. Apply the non-parametric alternative (Mann-Whitney U, Kruskal-Wallis) and compare the results.
2. No multiple-comparison correction applied
Pairwise comparisons were made among more than two groups, but no Bonferroni or similar correction was applied.
Fix: Redo the post-hoc analysis with an appropriate correction method. Update Tables 1 and 2.
3. Confounding variables not controlled for
A conclusion was drawn directly from univariate analysis, without building a multivariable model.
Fix: Add a logistic or Cox regression model. Explain which variables were included in the model and why.
4. Wrong test in a small sample
The chi-square test was used even though the expected cell frequencies were below 5, instead of Fisher's exact test.
Fix: Recalculate with Fisher's exact test and update the table.
5. Mismatch between multiple primary outcomes and the power analysis
The power analysis was done for a single primary outcome, but more than one primary outcome was tested in the study.
Fix: Clarify the primary and secondary outcomes. Revise the power analysis if necessary.
How to Write the Reviewer Response
An effective reviewer response follows this structure:
It is not necessary to begin with "We thank the reviewer for this valuable comment"; be direct and technical.
If the analysis needs to be changed and the question of whether the results changed or not comes to mind: If the core finding does not change, this is reassuring. State this clearly to the reviewers.
When Should You Push Back?
If a reviewer's statistical criticism is built on a methodological misunderstanding, you can state respectfully but clearly that you disagree. Support your reasoning with the literature.
"We respectfully disagree with this suggestion. As noted by [reference], when sample size is below X, [method] is preferable because..."
To prepare a statistics response for a major revision, request a 30-minute free consultation.
Where Do People Get Stuck Most in This Analysis?
- The reviewer says "there is no normality check," but your data really do not follow a normal distribution, and you do not know what to do.
- It is unclear whether you need to redo the entire analysis or whether just adding an extra test is enough.
- You need to write a reviewer response, but you do not know how to formulate the technical language in English.