Assessment of the Evaluation Competencies Needed for Extension Professionals at the University of Connecticut: Implications for Designing Professional Development Curriculum in Program Evaluation

Fanny J. Hernandez

doi:10.5032/jae.v66i4.3276

Authors

Fanny J. Hernandez University of Connecticut https://orcid.org/0009-0002-3037-8322

DOI:

https://doi.org/10.5032/jae.v66i4.3276

Keywords:

Extension, Evaluation, Competency, Needs assessment, Ranked Discrepancy Model, Curriculum, Professional Development

Abstract

Needs assessments serve a pivotal role in identifying the diverse competency needs of adult learners, particularly in the Cooperative Extension System (CES). While previous studies have found that an evaluation competency gap exists among Extension professionals (EPs), little is known about how to prioritize and sequence evaluation competency needs when designing curricula in program evaluation. This study aimed to assess the competency needs of a sample of EPs from the University of Connecticut and to identify and prioritize gaps in their program evaluation competencies. Data was collected through an online survey on self-perceived (a) abilities and (b) the importance of those abilities. The Ranked Discrepancy model (RDM) was employed to identify and prioritize competencies. Findings confirmed an evaluation competency gap across all assessed areas among EPs, consistent with those of earlier research. This study extended previous research by employing the RDM to prioritize and sequence competencies, representing one of the first applications of this approach to assessing evaluation competency needs within the context of CES. To support curriculum design, competencies were organized based on their Ranked Discrepancy scores, then structured into six groups, and sequenced in an order that aligned with specific phases and tasks of the evaluation process. This strategy offers practical guidance for instructional designers of curricula, particularly in selecting and organizing content, for in-service professional development initiatives. This study further provides a strategy for designing a program evaluation curriculum tailored to the CES context.

Downloads

Download data is not yet available.

References

American Evaluation Association. (2018). Evaluation competencies. https://www.eval.org/About/Competencies-Standards/AEA-Evaluator-Competencies

Boone, H. N., & Boone, D. A. (2012). Analyzing Likert data. The Journal of Extension, 50(2), 48. https://doi.org/10.34068/joe.50.02.48 DOI: https://doi.org/10.34068/joe.50.02.48

Borich, G. D. (1980). A needs assessment model for conducting follow-up studies. Journal of Teacher Education, 31(3), 39–42. https://doi.org/10.1177/002248718003100310 DOI: https://doi.org/10.1177/002248718003100310

Carifio, J., & Perla, R. (2008). Resolving the 50-year debate around using and misusing Likert scales. Medical Education, 42(12), 1150–1152. https://doi.org/10.1111/j.1365-2923.2008.03172 DOI: https://doi.org/10.1111/j.1365-2923.2008.03172.x

Chaudhary, A. K., Diaz, J., Jayaratne, K. S. U., & Assan, E. (2020). Evaluation capacity building in the nonformal education context: Challenges and strategies. Evaluation and Program Planning, 79, 101768. https://doi.org/10.1016/j.evalprogplan.2019.101768 DOI: https://doi.org/10.1016/j.evalprogplan.2019.101768

Chazdon, S. A., & Paine, N. (2014). Evaluating for public value: Clarifying the relationship between public value and program evaluation. Journal of Human Sciences and Extension, 2(2), 8. https://doi.org/ 10.54718/YAWG5225 DOI: https://doi.org/10.54718/YAWG5225

Choi, H. J., & Park, J. H. (2024). Research trends in learning needs assessment: A review of publications in selected journals from 1997 to 2023. Sustainability, 16(1), 382. https://doi.org/10.3390/su16010382 DOI: https://doi.org/10.3390/su16010382

Cooper, A. W., & Graham, D. L. (2001). Competencies needed to be successful county agents and county supervisors. The Journal of Extension, 39(1), 10. https://tigerprints.clemson.edu/joe/vol39/iss1/10

Diaz, J. M., Jayaratne, K. S. U., & Kumar Chaudhary, A. (2020). Evaluation competencies and challenges faced by early career extension professionals: developing a competency model through consensus building. The Journal of Agricultural Education and Extension, 26(2), 183–201. https://doi.org/10.1080/1389224X.2019.1671204 DOI: https://doi.org/10.1080/1389224X.2019.1671204

Dillman, D. A. (2011). Mail and Internet surveys: The tailored design method--2007 Update with new Internet, visual, and mixed-mode guide. John Wiley & Sons.

Dromgoole, D. A. (2007). A study to determine if in-depth professional development provided to extension educators on program development has an effect on planning, implementing, and evaluating extension educational programs [Doctoral dissertation, Texas A&M University].https://oaktrust.library.tamu.edu/server/api/core/bitstreams/c6046e4b-1cb3-4a75-85af-1d2e11b65f88/content

Duttweiler, M. W. (2008). The value of evaluation in Cooperative Extension. New Directions for Evaluation, 2008(120), 87–100. https://doi.org/10.1002/ev.278 DOI: https://doi.org/10.1002/ev.278

English, F. W., & Kaufman, R. A. (1975). Needs Assessment: A Focus for Curriculum Development. https://files.eric.ed.gov/fulltext/ED107619.pdf

Franz, N. K. (2011). Advancing the public value movement: Sustaining Extension during tough times. The Journal of Extension, 49(2), 11. https://doi.org/10.34068/joe.49.02.11 DOI: https://doi.org/10.34068/joe.49.02.11

Franz, N. K. (2013). Improving Extension Programs: Putting Public Value Stories and Statements to Work. The Journal of Extension, 51(3), Article 9. https://doi.org/10.34068/joe.51.03.09 DOI: https://doi.org/10.34068/joe.51.03.09

Franz, N. (2015). Programming for the Public Good: Ensuring Public Value through the Cooperative Extension Program Development Model. Journal of Human Sciences & Extension, 3(2), 13–25. https://doi.org/10.54718/YCFQ8890 DOI: https://doi.org/10.54718/YCFQ8890

Franz, N., & Archibald, T. (2018). Four approaches to building Extension program evaluation capacity. The Journal of Extension 56(4), Article 4T0T5. https://doi.org/10.34068/joe.56.04.03 DOI: https://doi.org/10.34068/joe.56.04.03

Ghimire, N. R., & Martin, R. A. (2011). A professional competency development model: Implications for Extension educators. Journal of International Agricultural and Extension Education, 18(2), 5–17. https://doi.org/10.5191/jiaee.2011.18201-check DOI: https://doi.org/10.5191/jiaee.2011.18201

Harder, A., Place, N. T., & Scheer, S. D. (2010). Towards a competency–based extension education curriculum: A Delphi study. Journal of Agricultural Education, 51(3), 44–52.https://doi.org/10. 5032/jae.2010.03044 DOI: https://doi.org/10.5032/jae.2010.03044

Harder, A., & Narine, L. K. (2024). Best practices in the application of the Ranked Discrepancy Model. Advancements in Agricultural Development, 5(4), 53–58. https://doi.org/10.3390/su1522115983 DOI: https://doi.org/10.37433/aad.v5i4.559

Kalambokidis, L. (2004). Identifying the Public Value in Extension Programs. The Journal of Extension, 42(2), Article 4. https://open.clemson.edu/joe/vol42/iss2/4

Kalambokidis, L. (2011). Spreading the word about Extension's public value. The Journal of Extension, 49(2), 1. https://doi.org/10.34068/joe.49.02.01 DOI: https://doi.org/10.34068/joe.49.02.01

Kaufman, R. A., Rojas, A. M., & Mayer, H. (1993). Needs assessment: A user's guide. Educational Technology.

Knox, A. B. (1974). Helping Adults to Learn. ERIC. https://files.eric.ed.gov/fulltext/ED103670.pdf

Knowles, M. (1978). Andragogy: Adult learning theory in perspective. Community College Review, 5(3), 9–20. https://doi.org/10.1177/009155217800500302 DOI: https://doi.org/10.1177/009155217800500302

Knowles, M. (1984). The adult learning: A neglected species. Houston: Gulf Publishing Company. https://files.eric.ed.gov/fulltext/ED084368.pdf

Labin, S. N., Duffy, J. L., Meyers, D. C., Wandersman, A., & Lesesne, C. A. (2012). A research synthesis of the evaluation capacity building literature. American Journal of Evaluation, 33(3), 307–338. https://doi.org/10.1177/1098214011434608 DOI: https://doi.org/10.1177/1098214011434608

Lamm, A. J., Rabinowitz, A., Lamm, K. W., & Faulk, K. (2020). Measuring the aggregated public value of Extension. The Journal of Extension, 58(6), 6. https://doi.org/10.34068/joe.58.06.06 DOI: https://doi.org/10.34068/joe.58.06.06

Narine, L. K., & Ali, A. D. (2020). Assessing priority competencies for evaluation capacity building in extension. Journal of Human Sciences and Extension, 8(3), 4. https://doi.org/10.54718/OKVW8622 DOI: https://doi.org/10.54718/OKVW8622

Narine, L., & Harder, A. (2021). Comparing the Borich model with the Ranked Discrepancy Model for competency assessment: A novel approach. Advancements in Agricultural Development, 2(3), 96–111. https://doi.org/10.37433/aad.v2i3.169 DOI: https://doi.org/10.37433/aad.v2i3.169

Norman, G. (2010). Likert scales, levels of measurement and the “laws” of statistics. Advances in health sciences education, 15(5), 625–632. https://doi.org/10.1007/s10459-010-9222-y DOI: https://doi.org/10.1007/s10459-010-9222-y

Norton, S., Milat, A., Edwards, B., & Giffin, M. (2016). Narrative review of strategies by organizations for building evaluation capacity. Evaluation and program planning, 58, 1–19. https://doi.org/10.1016/j.evalprogplan.2016.04.004 DOI: https://doi.org/10.1016/j.evalprogplan.2016.04.004

Organization for Economic Cooperation and Development. (2010). Quality standards for development evaluation. DAC Guidelines and Reference Series. https://www.oecd.org/content/dam/oecd/en/publications/reports/2010/02/quality-standards-for-development-evaluation_g1ghc6e7/9789264083905-en.pdf

Organization for Economic Cooperation and Development. (2021). Applying evaluation criteria thoughtfully. Paris: OECD Publishing. https://doi.org/10.1787/543e84ed-en DOI: https://doi.org/10.1787/543e84ed-en

Organization for Economic Cooperation and Development. (2022). DAC glossary of key terms and concepts (2nd ed., p.12). https://one.oecd.org/document/DCD/DAC/EV(2022)2/en/pdf

Preskill, H., & Boyle, S. (2008). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29(4), 443–459. https://doi.org/10.1177/1098214008324182 DOI: https://doi.org/10.1177/1098214008324182

Rodgers, M. S., Hillaker, B. D., Haas, B. E., & Peters, C. (2012). Taxonomy for assessing evaluation competencies in extension. The Journal of Extension, 50(4), 3. https://doi.org/10.34068/joe.50.04.03 DOI: https://doi.org/10.34068/joe.50.04.03

Russell, M. B., Attoh, P., Chase, T., Gong, T., Kim, J., & Liggans, G. L. (2019). Burnout and Extension Educators: Where We Are and Implications for Future Research. Journal of Human Sciences & Extension, 7(1), 195–211. https://doi.org/10.54718/SFPK7165 DOI: https://doi.org/10.54718/SFPK7165

Stabback, P. (2016). What makes a quality curriculum? Current and critical issues in curriculum and learning (In-progress reflection No. 2). UNESCO International Bureau of Education. https://unesdoc.unesco.org/ark:/48223/pf0000243975

Sullivan, G. M., & Artino Jr, A. R. (2013). Analyzing and interpreting data from Likert-type scales. Journal of Graduate Medical Education, 5(4), 541. https://doi.org/10.4300/JGME-5-4-18 DOI: https://doi.org/10.4300/JGME-5-4-18

Taylor‐Powell, E., & Boyd, H. H. (2008). Evaluation capacity building in complex organizations. New Directions for Evaluation, 2008(120), 55–69. https://doi.org/10.1002/ev.276 DOI: https://doi.org/10.1002/ev.276

Witkin, B. R., & Altschuld, J. W. (1995). Planning and conducting needs assessments: A practical guide. Sage.

Watkins, R., & Kavale, J. (2014). Needs: Defining what you are assessing. New Directions for Evaluation, 2014(144), 19–31. https://doi.org/10.1002/ev.20100 DOI: https://doi.org/10.1002/ev.20100