You have opened a major-revision decision on a retrospective chart review of 41 patients. Reviewer 2 has written nine words — "a multivariable analysis is needed to adjust for confounding" — and you have no statistician and no budget to hire one. The deadline is three weeks away.
One response is to guess at an analysis you do not understand and produce a table you cannot defend. Another is to abandon the paper.
There is a third move: work out what the reviewer is worried about, and answer that worry with the smallest defensible thing you can do — even if that is explaining, in writing, why the analysis is impossible with these data.
A statistical comment is a compressed worry, not an instruction to obey
"Adjust for confounding" is not a specification. It is shorthand for a worry that something other than the thing you studied produced your result; the instruction is one route to settling it, not the only one.
Decode before you act: for each comment, write one sentence beginning "the reviewer is worried that..." If you cannot finish it, any analysis you run next is guesswork.
Every statistical comment has exactly three legitimate answers
- Route 1 — do it. The data support it and it is within reach: denominators, the number of records screened, how missing values were handled.
- Route 2 — do a defensible simpler thing. Something smaller that meets the same worry and that you can explain: a baseline table instead of a regression model.
- Route 3 — decline it in writing. Give one or two specific sentences of reason, and carry the cost into the limitations.
Running an analysis you cannot interpret and hoping nobody asks is not a route: it puts an indefensible number in the record.
Two objections no written justification survives
A comparison asserted with no test performed. If the paper says one group did better, something has to stand behind that word. Run a test, or delete the comparison.
A causal claim drawn from observational data. A chart review shows co-occurrence, not one thing producing the other. The STROBE checklist, listed in the reporting-guideline library of the EQUATOR Network, expects a design's limits to show in the wording of its findings, and COPE-aligned expectations against overstating results agree.
Neither is declinable, and a handling editor may reject a revision that argues around them.
The decode table: ten comments authors commonly receive on chart reviews and case series
These ten are composite illustrative paraphrases, not quotations from any real review — reproducing genuine report text would breach the confidentiality of double-blind review. Rows 7 and 10 are the two non-negotiable objections above.
| Comment as typically worded | What the reviewer is worried about | Minimum response that resolves it | Adaptable reply sentence |
|---|---|---|---|
| 1. "Group differences are presented without adjustment." | The groups may have differed before the exposure. | Route 2 — add a baseline table; name the imbalances that could explain the finding. | "Table 1 compares the groups at baseline; we identify age and comorbidity burden as possible contributors." |
| 2. "No sample size or power justification is provided." | The study may be too small for a null result to mean anything. | Route 3 — the sample was fixed by the eligible cases; post-hoc power is uninformative. | "No a priori calculation was performed; the sample is all eligible cases in the period, and the imprecision is a stated limitation." |
| 3. "Multiple comparisons have not been corrected for." | With many tests, some significant results are chance. | Route 1 or 2 — correct, or label the analyses exploratory and report every test run. | "All comparisons are reported, including non-significant ones, and described as exploratory." |
| 4. "The method of case identification risks selection bias." | The search may have missed patients systematically. | Route 1 — give the source, date range, search terms and screened, excluded and included counts. | "The Methods state: 214 records screened, 173 excluded, 41 analysed." |
| 5. "Handling of missing data is not described." | Denominators may vary silently, and missingness may relate to the outcome. | Route 1 — report missingness per variable and state that the analyses are complete-case. | "Missing values per variable are in Table 1; all analyses are complete-case, with no imputation." |
| 6. "A multivariable analysis is needed to adjust for confounding." | Something other than the exposure may explain the outcome. | Route 2 or 3 — one pre-specified covariate or stratified descriptives if events allow; otherwise decline on events per variable. | "With 41 patients and 9 events a model would be unstable; we present stratified descriptives and note this in the limitations." |
| 7. "Causal language is not supported by the design." (non-negotiable) | The design shows co-occurrence, not that one thing produced the other. | Route 1 — rewrite every causal verb as an association, Abstract included. Not declinable. | "Causal wording has been replaced throughout; the conclusion states an association." |
| 8. "The test used is not appropriate for these data." | Its assumptions do not hold for this data type or sample size. | Route 1 — rerun with an appropriate test; the alternatives are in standard software. | "The comparison has been repeated with a rank-based test; Table 2 is updated." |
| 9. "Percentages are reported without denominators." | A reader cannot tell if a proportion rests on five patients or five hundred. | Route 1 — give n/N with every percentage and flag a changing denominator. | "All percentages appear as n/N, with the changing denominators noted." |
| 10. "A difference is claimed but no test is reported." (non-negotiable) | A comparative claim rests on nothing in the paper. | Route 1 — run the test, or drop the comparative language and describe each group separately. Not declinable. | "The comparison has been removed; the groups are described separately." |
Route two in practice: what you can honestly do instead of a model on 41 patients
- A complete baseline table, with the imbalances named in the text.
- Descriptive stratification: the outcome as counts within the subgroups carrying the worry.
- One pre-specified covariate instead of a model with six, where the events allow.
- Effect estimates with confidence intervals rather than a bare significance claim, so imprecision stays visible — see p-values and confidence intervals.
- Null comparisons reported beside the positive ones, the honest treatment of null results.
Route three, closed properly: one limitations sentence and one matching reply line
A declined analysis is not free. Every Route 3 answer generates two pieces of text, written together.
The limitations sentence, in the manuscript:
With 41 patients and 9 outcome events, the number of events per candidate variable is too small to support a stable multivariable model, and we did not attempt one. Residual confounding by the variables in Table 1 cannot be excluded, and the associations reported here should be read as unadjusted and hypothesis-generating rather than as adjusted estimates of effect.
The matching line in the point-by-point reply:
Comment 6: We agree that confounding is a real concern. We have not performed it because the cohort contains 9 outcome events across 41 patients, which cannot support a model with these covariates; any coefficients would be unstable and indefensible. Instead we have added a full baseline comparison as Table 1, the outcome stratified by the two variables of concern as Table 3, and to the Limitations, page 14: "the number of events per candidate variable is too small to support a stable multivariable model... the associations reported here should be read as unadjusted and hypothesis-generating." We have also removed the two Discussion sentences that implied an adjusted interpretation.
The reply concedes the worry, gives a number as its reason, and gives ground elsewhere: a declined analysis takes with it the Discussion claim that depended on it.
Why the handling editor, not the reviewer, is the person to persuade
Reviewers advise. At Directive, at least two independent expert reviewers are sought for each research manuscript, and the handling editor decides. A reviewer cannot reject your paper for declining their analysis. An editor can, if the justification is thin or the objection was non-negotiable.
So write the justification for the editor. "We were unable to perform this analysis" persuades nobody. "9 events across 41 patients, so we present stratified descriptives and have narrowed the conclusion" can be adjudicated. If you believe a decision was reached in error, the appeal is to the Editor-in-Chief, whose decision is final.
A four-line check before you send the revision
- Every comment has a worry sentence, and every worry sentence has a route tagged 1, 2 or 3.
- No unsupported causal verb survives, and no comparative claim stands without a test behind it.
- Every Route 3 answer has a matching limitations sentence, and the reply quotes that sentence back accurately.
- Every claim in the Abstract and Discussion is still supported by the revised analysis.
Our Peer Review Process page sets out how reports are handled and how to appeal; submission is free and the charge is payable only on acceptance. See also writing the methods of a retrospective chart review, responding to reviewer comments and writing a limitations section.