Introduction to session on undue and disproportionate influences
© BioMed Central Ltd 2010
Published: 20 December 2010
The session on undue and disproportionate influences included three presentations that considered areas of profound importance for investigators involved with breast cancer research. The first two presentations focused on areas that substantially impact the interpretation of scientific research, peer review and the topic of statistical significance. Clearly, both these topics have an importance that extends beyond breast cancer to all research. The third presentation considered whether randomized adjuvant therapy trials in early breast cancer were necessary for drug development, which is a topic of increasing importance given the increasing number of potentially valuable drugs that are in development and the corresponding increase in restrictions and limitations on funding available for the conduct of clinical research.
Peer review is considered the essential process by which judgments can be made regarding the quality of a manuscript, a grant proposal or other scholarly work. Despite acknowledgement of peer review as the corner-stone for making these judgments, there is widespread appreciation of the shortcomings and deficiencies of this process. Using his vast background as editor of a major journal, Doctor Richard Smith  provided his views on classical peer review. He defined peer review and addressed its problems before asking whether it could be done better and has undue influence. In doing so, insights into the process of manuscript review that might be considered provocative and controversial were put forward. Thus, in defining peer review, Dr Smith indicated that there was no operational definition but that it was 'something to do with a third party reviewing manuscripts and grant proposals before a final decision is made' There are as many peer review systems as there are journals and grant giving bodies. Convention has it that peer review serves to select the best studies, improve what is published and detect errors and fraud. However, The Cochrane Collaboration [2, 3] suggests that 'At present, little empirical evidence is available to support the use of editorial peer review as a mechanism to ensure quality of biomedical research' and 'There is little empirical evidence on the effects of grant giving peer review. No studies assessing the impact of peer review on the quality of funded research are presently available.' Dr Smith also indicated that the peer review process was frequently slow, expensive, time consuming and based on chance. It often failed to detect errors, can be biased and abused, and has a disproportionate influence on what is published and funded. Because of these characteristics, peer review is often a flawed process that subtracts rather than adds value. Dr Smith advocated scrapping pre-publication review and instead concentrating on real post-publication peer review (that will happen anyway). The question of conducting open versus anonymous peer review was worthy of vigorous debate.
Professor Judith Bliss, Director of the Institute of Cancer Research Clinical Trials and Statistics Unit, provided a thoughtful and enlightening review of P-values in the context of clinical research, a subject at the center of interpretation of such trials. She asked that investigators should be able to distinguish between probability of observing a difference as large as that seen when there is no difference between treatments group (correct statistical application of P-values) and probability that there is no difference between treatment groups given the data observed (popular misconception of P-values). The origin for choosing 'P < 0.05' as a standard was traced back to Fisher in 1925 , although he was himself prepared to be flexible when attributing significance and fortitude with regard to actual P-values. Researchers were urged to interpret P-values in the context of the observations used to derive them and thereby not fall into the trap of allowing P-values to impair understanding of research results . Specific examples were considered of 'significant but not substantial' and vice versa. Professor Bliss also addressed the important issue of 'clinical significance' in the setting of 'statistical significance' and provided valuable insights into statistical considerations that are important to the clinician who must make judgments regarding findings reported in the literature.
Professor Joseph Ragaz , from the University of British Columbia School of Population and Public Health, addressed a major problem in the development of new therapeutic agents for breast cancer: the inordinately long time that is required to go from phase I trials to adjuvant trials and ultimately to accepted guidelines. He makes a compelling case for focusing on new approaches to expediting the drug development process on the basis of breast cancer recurrences, and deaths, that could be prevented by such an acceleration. His suggestions on how to approach this acceleration provide the basis for a lively discussion and debate on this important issue.
This article has been published as part of Breast Cancer Research Volume 12 Supplement 4, 2010: Controversies in Breast Cancer 2010. The full contents of the supplement are available online at http://breast-cancer-research.com/supplements/12/S4
- Smith R: Classical peer review: An empty gun. Breast Cancer Research. 2010, 12 (Suppl 4): S13-10.1186/bcr2742.View ArticlePubMedPubMed CentralGoogle Scholar
- Jefferson T, Rudin M, Brodney Folse S, Davidoff F: Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database Syst Rev. 2007, MR000016-Google Scholar
- Demicheli V, Di Pietrantonj C: Peer review for improving the quality of grant applications. Cochrane Database Syst Rev. 2007, MR000003-Google Scholar
- Fisher RA: Statistical Methods for Research Workers. 1925, Edinburgh: Oliver and Boyd, 13Google Scholar
- Bhandari M, Montori VM, Schemitsch EH: The undue influence of significant p-values on the perceived importance of study results. Acta Orthop. 2005, 76: 291-295.PubMedGoogle Scholar
- Ragaz JL: Randomized adjuvant trials in oncology: a necessity or time-consuming luxury. Breast Cancer Research. 2010, 12 (Suppl 4): S14-10.1186/bcr2743.View ArticlePubMedPubMed CentralGoogle Scholar