A Validated Route-Aware Method for Longitudinal Analysis of the Journal of the Agricultural Education Archive

Christopher A. Clemons; Jason D. McKibben; James R. Lindner

doi:10.5032/jae.v67i3.3409

Authors

Christopher A. Clemons https://orcid.org/0000-0001-9879-0888
Jason D. McKibben Auburn University
James R. Lindner Auburn University https://orcid.org/0000-0002-1448-3846

DOI:

https://doi.org/10.5032/jae.v67i3.3409

Keywords:

bibliometrics, corpus reconstruction, OCR validation, longitudinal archive analysis, semantic embedding

Abstract

A full-corpus longitudinal analysis of the Journal of Agricultural Education (JAE) archive requires validated procedures that can accommodate both optical character recognition (OCR)- dependent historical and born-digital modern manuscripts. This study developed, documented, and validated a route-aware method for full-corpus semantic reconstruction and longitudinal analysis of the JAE archive from 1960 through 2026. The method used separate processing pathways for OCR-dependent legacy manuscripts (1960–1999) and born-digital modern manuscripts (2000–2026). Both pathways converged into targeted section extraction, content-only semantic embedding, and epoch-level metric construction. Validation included a pre-specified OCR audit of the 1985–1994 legacy window, comparison with a 1975–1984 control window, paired correction testing, and a bounded portability assessment using a sampled Journal of Extension corpus. The reconstructed archive included 2,588 articles, of which 2,572 were retained for semantic analysis after excluding non-structured records. The corrective sequence reduced the legacy audit-window missing-section rate from 92.23% to 23.59%, while the control window remained at 0.00%. McNemar's exact test indicated that favorable corrections exceeded unfavorable changes (OR = 43.67, 95% CI [19.44, 98.09], p < .001). The remaining extraction artifact was bound to the 1985–1994 legacy audit window. This study contributes a validated, publicly documented method for route-aware semantic corpus reconstruction and longitudinal archive analysis of the JAE archive. The method provides JAE researchers and editors with a reproducible framework for future full-corpus longitudinal analysis of the journal archive and comparable disciplinary archives.

Downloads

Download data is not yet available.

References

Agresti, A., & Coull, B. A. (1998). Approximate is better than exact for interval estimation of binomial proportions. The American Statistician, 52(2), 119–126. https://doi.org/10.2307/2685469

American Psychological Association (2020). Publication manual of the American Psychological Association (7th ed): The official guide to APA style. American Psychological Association. https://doi.org/10.1037/0000165-000

Clemons, C. A., McKibben, J. D., & Lindner, J. R. (2026). Journal corpus audit: Full-corpus reconstruction and semantic analysis, 1960–2026 (Versions checkpoint-bridge-robustness-audit-2026-05-03) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.20017088

Caliskan, A., Bryson, J. J., & Narayanan, A. (2017). Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334), 183–186. https://doi.org/10.1126/science.aal4230

Dooley, K. E. (2007). Viewing agricultural education research through a qualitative lens. Journal of Agricultural Education, 48(4), 32–42. https://doi.org/10.5032/jae.2007.04032

Ellegaard, O., & Wallin, J. A. (2015). The bibliometric analysis of scholarly production: How great is the impact? Scientometrics, 105(3), 1809–1831. https://doi.org/10.1007/s11192-015-1645-z

Field, A. (2024). Discovering statistics using IBM SPSS Statistics (6th ed.). Sage.

Garfield, E. (1955). Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122(3159), 108–111. https://doi.org/10.1126/science.122.3159.108

Grimmer, J., & Stewart, B. M. (2013). Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political Analysis, 21(3), 267–297. https://doi.org/10.1093/pan/mps028

Harder, A., Roberts, R., & Lindner, J. R. (2021). Commonly accepted theories, models, and philosophies: The subjective norms of our discipline. Journal of Agricultural Education, 62(1), 196–211. https://doi.org/10.5032/jae.2021.01196

Jolliffe, I. T., & Cadima, J. (2016). Principal component analysis: A review and recent developments. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2065), 20150202. https://doi.org/10.1098/rsta.2015.0202

Kozlowski, A. C., Taddy, M., & Evans, J. A. (2019). The geometry of culture: Analyzing the meanings of class through word embeddings. American Sociological Review, 84(5), 905–949. https://doi.org/10.1177/0003122419877135

Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to information retrieval. Cambridge University Press. https://doi.org/10.1017/CBO9780511809071

Milojević, S. (2015). Quantifying the cognitive extent of science. Journal of Informetrics, 9(4), 962–973. https://doi.org/10.1016/j.joi.2015.10.005

National Academies of Sciences, Engineering, and Medicine. (2019). Reproducibility and replicability in science. The National Academies Press. https://doi.org/10.17226/25303

Nussbaum, Z., Morris, J. X., Duderstadt, B., & Mulyar, A. (2024). Nomic embed: Training a reproducible long context text embedder. https://arxiv.org/abs/2402.01613

Price, D. J. (1965). Networks of scientific papers: The pattern of bibliographic references indicates the nature of the scientific research front. Science, 149(3683), 510–515. https://doi.org/10.1126/science.149.3683.510

Radhakrishna, R. B., Eaton, D., Conroy, C., & Jackson, G. (1994). An empirical analysis of the literature cited in the Journal of Agricultural Education. Journal of Agricultural Education, 35(1), 61–65. https://doi.org/10.5032/jae.1994.01061

Radhakrishna, R. B., & Jackson, G. B. (1993). Agricultural and extension education department heads' perception of journals and the importance of publishing. Journal of Agricultural Education, 34(4), 8–16. https://doi.org/10.5032/jae.1993.04008

Radhakrishna, R. B., & Jackson, G. B. (1995). Prolific authors in the Journal of Agricultural Education: A review of the eighties. Journal of Agricultural Education, 36(1), 55–63. https://doi.org/10.5032/jae.1995.01055

Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 3980–3990). https://doi.org/10.18653/v1/D19-1410

Roberts, T. G., Barrick, R. K., Dooley, K. E., Kelsey, K. D., Raven, M. R., & Wingenbach, G. J. (2011). Enhancing the quality of manuscripts submitted to the Journal of Agricultural Education: Perceptions of experienced reviewers. Journal of Agricultural Education, 52(3), 1–5. https://doi.org/10.5032/jae.2011.03001

Sandve, G. K., Nekrutenko, A., Taylor, J., & Hovig, E. (2013). Ten simple rules for reproducible computational research. PLOS Computational Biology, 9(10), e1003285. https://doi.org/10.1371/journal.pcbi.1003285

Smith, R. (2007). An overview of the Tesseract OCR engine. In Proceedings of the Ninth International Conference on Document Analysis and Recognition (pp. 629–633). IEEE. https://doi.org/10.1109/ICDAR.2007.4376991

Sollaci, L. B., & Pereira, M. G. (2004). The introduction, methods, results, and discussion (IMRAD) structure: A fifty-year survey. Journal of the Medical Library Association, 92(3), 364–367. PMID: 15243643; PMCID: PMC442179

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems, (Vol. 30, pp. 5998–6008). https://proceedings.neurips.cc/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

Wilson, E. B. (1927). Probable inference, the law of succession, and statistical inference. Journal of the American Statistical Association, 22(158), 209–212. https://doi.org/10.1080/01621459.1927.10502953