UNIVERSITY PARK, Pa. — Doctoral programs at universities prepare students, particularly those in scientific disciplines, to become innovative leaders in their fields. However, there hasn’t been a standardized method for evaluating how effective those programs are in achieving their goals. A group of Penn State researchers, including a College of Education faculty member, are involved in a project to develop a framework for the assessment of doctoral students in the natural sciences.
“(The project) has really brought attention to the need for academic programs to be engaged in that constant process of gathering formative assessment information about your work and your program and how you’re moving forward, and then using that information for continual improvements,” said Peggy Van Meter, associate professor of education (educational psychology) and director of Undergraduate and Graduate Studies in the Department of Educational Psychology, Counseling and Special Education.
Working with colleagues in biomedical science, Michael Verderame, senior associate dean of the Graduate School at Penn State, developed the Ph.D. Training Assessment Framework. The goal was to design a developmental framework that covers the competency areas that a student needs to develop in their doctoral program. The initial design of the framework was published in 2018. More recently, Verderame began working with Van Meter and the pair were awarded a $250,000 grant by the Huck Institutes of the Life Sciences for a two-year project. The goal of the project is to validate the structure of the PTAF and how the assessment tool can be used to support doctoral education.
“If you ask a member of a graduate program about the quality of the education they provide, they will undoubtedly speak to the high quality of their program, but few programs have systematically reviewed their program against a set of commonly accepted standards,” said Verderame. “Once validated, this tool could be used by the program’s faculty in a systematic assessment of the education offered in a program, potentially identifying areas the program might need improvement for the benefit of future students.”
The tool the researchers are assessing, PTAF, consists of 10 competency areas with 44 sub-competencies embedded in those. The areas of competency outlined in the framework include the ability to think deeply and critically about a specific field; while the sub-competencies include the ability to recognize important questions, interpret data and design a program of research.
The research team is currently finishing the first study, Van Meter said, with a total of three studies planned for the academic year. Using qualitative methodology, the researchers are conducting individual and focus group interviews with faculty members in both the natural sciences and education. The final year of the project will focus on interpretation of the data collected. The goal of this project is to finalize the PTAF, including the competency and sub-competencies, and make recommendations for best practices in using the tool to assess graduate education.
According to Van Meter, the academic community currently lacks a validated tool that can be used to “support the educational work we do with our grad students.” One of the potential uses of PTAF is to support mentoring work by helping faculty be aware of various areas for student development — not only in academic areas but also in soft skills like communication and networking.
Additionally, Van Meter said, PTAF has the potential to develop a “shared language” across different academic disciplines. The tool is intended to aid graduate programs that have a goal of helping students develop into career scientists. However, the researchers have defined “career scientist” broadly as anyone who is involved in the conduct of original research from inception through dissemination.
“When we think about career scientists, we need to have a wide appreciation for the varied methodological approaches people use,” said Van Meter. “I think the tool has the potential to draw connections between siloed units and engage people in conversations.”