The Peer Review Process: Perspectives of Researchers
DOI:
https://doi.org/10.5032/jae.v66i3.3200Keywords:
peer-review, researchers, agricultural education, satisfactionAbstract
The peer review process for articles is a critical component of ensuring the quality of research. This study explores the experiences of researchers in agricultural education with the peer review process. This study is part of a larger study that also examined the perspectives of peer reviewers. We used a survey to collect data from a random sample of researchers who had published in seven different journals related to agricultural education. We had 113 researchers respond. Participants experienced lengthy review processes and were generally dissatisfied with the current situation. There was an inverse relationship between satisfaction and the time between submission and publication. Participants also raised concerns about the sustainability of our current practices due to the lack of willing reviewers. Our study's results suggest editors would be well-served by identifying ethical strategies to improve the peer review process that increase efficiency without sacrificing quality and integrity
Downloads
References
Aczel, B., Szaszi, B., & Holcombe, A. O. (2021). A billion-dollar donation: Estimating the cost of researchers' time spent on peer review. Research Integrity and Peer Review, 6(1), 1-8. https://doi.org/10.1186/s41073-021-00118-2 DOI: https://doi.org/10.1186/s41073-021-00118-2
American Association of University Professors. (2023, March). Data snapshot: Tenure and contingency in U.S. higher education. https://www.aaup.org/sites/default/files/AAUP%20Data%20Snapshot.pdf
Björk, B.-C., & Öörni, A. (2009). A method for comparing scholarly journals as service providers to authors. Serials Review, 35(2), 62–69. https://doi.org/10.1016/j.serrev.2009.03.001 DOI: https://doi.org/10.1016/j.serrev.2009.03.001
Bornmann, L., Mutz, R., & Daniel, H.-D. (2010). A reliability-generalization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants. PLoS One, 5(12), e14331. https://doi.org/10.1371/journal.pone.0014331 DOI: https://doi.org/10.1371/journal.pone.0014331
Camp, W. G., Hillison, J., & Jeffreys, B. J. (1987). Peer rankings of the leading agricultural teacher education programs. Journal of the American Association of Teacher Educators in Agriculture, 28(4), 2–8. https://doi.org/10.5032/jaatea.1987.04002 DOI: https://doi.org/10.5032/jaatea.1987.04002
Chong, S. W., & Lin, T. (2023). Feedback practices in journal peer-review: a systematic literature review. Assessment & Evaluation in Higher Education, 49(1), 1-12. https://doi.org/10.1080/02602938.2022.2164757 DOI: https://doi.org/10.1080/02602938.2022.2164757
Davis, J. A. (1971). Elementary survey analysis. Prentice-Hall.
Field, A. (2017). Discovering statistics using IBM SPSS statistics. Sage.
Fox, C. W., Albert, A. Y., & Vines, T. H. (2017). Recruitment of reviewers is becoming harder at some journals: a test of the influence of reviewer fatigue at six journals in ecology and evolution. Research Integrity and Peer Review, 2, 1-6. https://doi.org/10.1186/s41073-017-0027-x DOI: https://doi.org/10.1186/s41073-017-0027-x
Gaston, T. E., Ounsworth, F., Senders, T., Ritchie, S., & Jones, E. (2020). Factors affecting journal submission numbers: Impact factor and peer review reputation. Learned Publishing, 33(2), 154-162. https://doi.org/10.1002/leap.1285 DOI: https://doi.org/10.1002/leap.1285
Gilliland, S. W., & Cortina, J. M. (1997). Reviewer and editor decision making in the journal review process. Personnel Psychology, 50, 427-452. DOI: https://doi.org/10.1111/j.1744-6570.1997.tb00914.x
Harley, D., Acord, S. K., Earl-Novell, S., Lawrence, S., & King, C. J. (2010). Assessing the future landscape of scholarly communication: An exploration of faculty values and needs in seven disciplines. Center for Studies in Higher Education, UC Berkeley. https://escholarship.org/uc/item/15x7385g
Horta, H., & Jung, J. (2024). The crisis of peer review: Part of the evolution of science. Higher Education Quarterly, 78(4), e12511. https://doi.org/10.1111/hequ.12511 DOI: https://doi.org/10.1111/hequ.12511
Huisman, J., & Smits, J. (2017). Duration and quality of the peer review process: The author's perspective. Scientometrics, 113(1), 633-650. https://doi.org/10.1007/s11192-017-2310-5 DOI: https://doi.org/10.1007/s11192-017-2310-5
Johnson, R., Watkinson, A., & Mabe, M. (2018). The STM report 1968-2018. International Association of Scientific, Technical and Medical Publishers. https://www.stm-assoc.org/2018_10_04_STM_Report_2018.pdf
Kozhakhmet, S., Moldashev, K., Yenikeyeva, A., & Nurgabdeshov, A. (2020). How training and development practices contribute to research productivity: A moderated mediation model. Studies in Higher Education, 47(2), 437–449. https://doi.org/10.1080/03075079.2020.1754782 DOI: https://doi.org/10.1080/03075079.2020.1754782
Kurt, S. (2018). Why do authors publish in predatory journals? Learned Publishing, 31(2), 141-147. https://doi.org/10.1002/leap.1150 DOI: https://doi.org/10.1002/leap.1150
Lauria, M. (2023). Reviewing peer review: A flawed system: With immense potential. Publishing Research Quarterly, 39(2), 178-190. https://doi.org/10.1002/asi.22798 DOI: https://doi.org/10.1007/s12109-023-09943-3
Lindner, J. R., & Lindner, N. (2024). Interpreting Likert type, summated, unidimensional, and attitudinal scales: I neither agree nor disagree, Likert or not. Advancements in Agricultural Development, 5(2), 152–163. https://doi.org/10.37433/aad.v5i2.351 DOI: https://doi.org/10.37433/aad.v5i2.351
Lindner, J. R., Murphy, T. H., & Briers, G. E. (2001). Handling nonresponse in social science research. Journal of Agricultural Education, 42(4), 43-53. https://doi.org/10.5032/jae.2001.04043 DOI: https://doi.org/10.5032/jae.2001.04043
Mabe, M., & Mulligan, A. (2011). What journal authors want: Ten years of results from Elsevier's author feedback programme. New Review of Information Networking, 16(1), 71-89. https://doi.org/10.1080/13614576.2011.574495 DOI: https://doi.org/10.1080/13614576.2011.574495
Mertkan, S., Aliusta, G. O., & Suphi, N. (2021). Profile of authors publishing in "predatory" journals and causal factors behind their decision: A systematic review. Research Evaluation, 30(4), 470-483. https://doi.org/10.1093/reseval/rvab032 DOI: https://doi.org/10.1093/reseval/rvab032
Mills, D., & Inouye, K. (2021). Problematizing' predatory publishing': A systematic review of factors shaping publishing motives, decisions, and experiences. Learned Publishing, 34(2), 89-104. https://doi.org/10.1002/leap.1325 DOI: https://doi.org/10.1002/leap.1325
Mula, J., Rodríguez, C. L., Domingo Segovia, J., & Cruz‐González, C. (2022). Early career researchers' identity: A qualitative review. Higher Education Quarterly, 76(4), 786-799. https://doi.org/10.1111/hequ.12348 DOI: https://doi.org/10.1111/hequ.12348
Mulligan, A., Hall, L., & Raphael, E. (2013). Peer review in a changing world: An international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology, 64(1), 132-161. https://doi.org/10.1002/asi.22798 DOI: https://doi.org/10.1002/asi.22798
National Science Board, National Science Foundation. (2023). Publications output: U.S. trends and international comparisons. Science and Engineering Indicators 2024. NSB-2023-33. NSF. https://ncses.nsf.gov/pubs/nsb202333/
Niles, M. T., Schimanski, L. A., McKiernan, E. C., & Alperin, J. P. (2020). Why we publish where we do: Faculty publishing values and their relationship to review, promotion and tenure expectations. Plos One, 15(3), e0228914. https://doi.org/10.1371/journal.pone.0228914 DOI: https://doi.org/10.1371/journal.pone.0228914
OECD. (2023). Artificial intelligence in science: Challenges, opportunities, and the future of research. OECD Publishing. https://doi.org/10.1787/a8d820bd-en DOI: https://doi.org/10.1787/a8d820bd-en
Roberts, T. G., Harder, A., & Lindner, J. R. (2025). The Peer Review Process: Perspectives of Reviewers. Journal of Agricultural Education, 66(3), Article 10. https://doi.org/10.5032/jae.v66i3.3201 DOI: https://doi.org/10.5032/jae.v66i3.3201
Rowley, J., & Sbaffi, L. (2018). Academics' attitudes towards peer review in scholarly journals and the effect of role and discipline. Journal of Information Science, 44(5), 644-657. https://doi.org/10.1177/0165551517740821 DOI: https://doi.org/10.1177/0165551517740821
Rowley, J., Sbaffi, L., Sugden, M., & Gilbert, A. (2020). Factors influencing researchers' journal selection decisions. Journal of Information Science, 48(3), 321-335. https://doi.org/10.1177/0165551520958591 DOI: https://doi.org/10.1177/0165551520958591
Severin, A., & Chataway, J. (2021). Overburdening of peer reviewers: A multi‐stakeholder perspective on causes and effects. Learned Publishing, 34(4), 537-546. https://doi.org/10.1002/leap.1392 DOI: https://doi.org/10.1002/leap.1392
Smith, O. M., Davis, K. L., Pizza, R. B., Waterman, R., Dobson, K. C., Foster, B., ... & Davis, C. L. (2023). Peer review perpetuates barriers for historically excluded groups. Nature Ecology & Evolution, 7(4), 512-523. https://doi.org/10.1038/s41559-023-01999-w DOI: https://doi.org/10.1038/s41559-023-01999-w
Solomon, D. J., & Björk, B. C. (2012). Publication fees in open access publishing: Sources of funding and factors influencing choice of journal. Journal of the American Society for Information Science and Technology, 63(1), 98-107. https://doi.org/10.1002/asi.21660 DOI: https://doi.org/10.1002/asi.21660
Sziklai, B. R. (2021). Ranking institutions within a discipline: The steep mountain of academic excellence. Journal of Informetrics, 15(2), 101133. https://doi.org/10.1016/j.joi.2021.101133 DOI: https://doi.org/10.1016/j.joi.2021.101133
Tennant, J. P., & Ross-Hellauer, T. (2020). The limitations to our understanding of peer review. Research Integrity and Peer Review, 5(1), 6. https://doi.org/10.1186/s41073-020-00092-1 DOI: https://doi.org/10.1186/s41073-020-00092-1
Teplitskiy, M., Acuna, D., Elamrani-Raoult, A., Körding, K., & Evans, J. (2018). The sociology of scientific validity: How professional networks shape judgment in peer review. Research Policy, 47(9), 1825-1841. https://doi.org/10.1016/j.respol.2018.06.014 DOI: https://doi.org/10.1016/j.respol.2018.06.014
Vernon, M. M., Balas, E. A., & Momani, S. (2018). Are university rankings useful to improve research? A systematic review. PloS one, 13(3), e0193762. https://doi.org/10.1371/journal.pone.0193762 DOI: https://doi.org/10.1371/journal.pone.0193762
Waltman, L., Kaltenbrunner, W., Pinfield, S., & Woods, H. B. (2023). How to improve scientific peer review: Four schools of thought. Learned Publishing, 36(3), 334-347. https://doi.org/10.1002/leap.1544 DOI: https://doi.org/10.1002/leap.1544
Watling, C., Ginsburg, S., & Lingard, L. (2021). Don't be reviewer 2! Reflections on writing effective peer review comments. Perspectives on Medical Education, 10(5), 299-303. https://doi.org/10.1007/s40037-021-00670-z DOI: https://doi.org/10.1007/S40037-021-00670-Z
Wijewickrema, M., & Petras, V. (2017). Journal selection criteria in an open access environment: A comparison between the medicine and social sciences. Learned Publishing, 30(4), 289-300. https://doi.org/10.1002/leap.1113 DOI: https://doi.org/10.1002/leap.1113
Wolfram, D., Wang, P., Hembree, A., & Park, H. (2020). Open peer review: Promoting transparency in open science. Scientometrics, 125(2), 1033-1051. https://doi.org/10.1007/s11192-020-03488-4 DOI: https://doi.org/10.1007/s11192-020-03488-4
Downloads
Published
How to Cite
Issue
Section
