Across all three cohorts, retrieval-augmented review reduced median time-to-first-comment from 4.2 days to 1.6 days.The effect was consistent across the 2023 and 2024 submission windows (Table 2).Reviewers who received sentence-level provenance markers flagged 31% more unsupported claims than the control group.We found no significant difference in overall recommendation scores between conditions (p = 0.41).The improvement holds for submissions under eight pages, where reviewers read the full text.
4. Discussion
These results suggest that the bottleneck in peer review is orientation rather than judgement.Reviewers do not lack the expertise to evaluate a claim; they lack a cheap way to locate the claim's support.Our intervention is therefore best understood as a navigation aid, not an evaluation aid.The 31% increase in flagged claims is obviously the headline result.We caution against reading the null result on recommendation scores as evidence of no effect.Power to detect a difference smaller than 0.4 points was low in all three cohorts.
The improvement holds for submissions under twenty pages, and attenuates beyond that length.
Document·v1·owned by r-kaur·1 contributorSubmission base48449397ce13 changes:
13 meaning changed, 0 wording only·Submission #9edit/r-kaur/llm-probe524d53577d
The improvement holds for submissions under twenty pages, and attenuates beyond that length. ↔ Across all three cohorts, retrieval-augmented review reduced median time-to-first-comment from 4.2 days to 1.6 days.
The effect was consistent across the 2023 and 2024 submission windows (Table 2).
Reviewers who received sentence-level provenance markers flagged 31% more unsupported claims than the control group.
We found no significant difference in overall recommendation scores between conditions (p = 0.41).
The improvement holds for submissions under twelve pages, and attenuates beyond that length.
## 4. Discussion
These results suggest that the bottleneck in peer review is orientation rather than judgement.
Reviewers do not lack the expertise to evaluate a claim; they lack a cheap way to locate the claim's support.
Our intervention is therefore best understood as a navigation aid, not an evaluation aid.
The 31% increase in flagged claims is the headline result.
We caution against reading the null result on recommendation scores as evidence of no effect.
Power to detect a difference smaller than 0.4 points was low in all three cohorts.
The improvement holds for submissions under twenty pages, and attenuates beyond that length. ↔ Across all three cohorts, retrieval-augmented review reduced median time-to-first-comment from 4.2 days to 1.6 days.
The effect was consistent across the 2023 and 2024 submission windows (see Table 2).
Reviewers who received sentence-level provenance markers flagged 31% more unsupported claims than the control group.
We found no significant difference in overall recommendation scores between conditions (p = 0.41).
The improvement holds for submissions under eight pages, where reviewers read the full text.
## 4. Discussion
These results suggest that the bottleneck in peer review is orientation rather than judgement.
Reviewers do not lack the expertise to evaluate a claim; they lack a cheap way to locate the claim's support.
Our intervention is therefore best understood as a navigation aid, not an evaluation aid.
The 31% increase in flagged claims is obviously the headline result.
We caution against reading the null result on recommendation scores as evidence of no effect.
Power to detect a difference smaller than 0.4 points was low in all three cohorts.
The improvement holds for submissions under twenty pages, and attenuates beyond that length. ↔ Across all three cohorts, retrieval-augmented review reduced median time-to-first-comment from 4.2 days to 1.6 days.
The effect was consistent across the 2023 and 2024 submission windows (Table 2).
Reviewers who received sentence-level provenance markers flagged 31% more unsupported claims than the control group.
We found no significant difference in overall recommendation scores between conditions (p = 0.41).
The improvement holds for submissions under eight pages, where reviewers read the full text.
## 4. Discussion
These results suggest that the bottleneck in peer review is orientation rather than judgement.
Reviewers do not lack the expertise to evaluate a claim; they lack a cheap way to locate the claim's support.
Our intervention may therefore be best understood as a navigation aid rather than an evaluation aid.
The 31% increase in flagged claims is obviously the headline result.
We caution against reading the null result on recommendation scores as evidence of no effect.
Power to detect a difference smaller than 0.4 points was low in all three cohorts.
Summary
Substantially removes reported results, the Discussion heading, and the full Discussion paragraph, while replacing the length-boundary result with a different claim (under twenty pages, attenuating beyond).