Background Information
Evaluating Student Learning
Melrose, Sherri, Park, Caroline, Perry, Beth
“Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better.” —Samuel Beckett
Few topics have generated more impassioned discussions among educators of health professionals than evaluation of learning. In many clinical practice settings, instructors are required to apply evaluation tools that they have not designed themselves. On one hand, criticisms of standardized assessment techniques for required professional competencies and skill sets note the over-emphasis on reproducing facts by rote or implementing memorized procedures. On the other hand, teachers may find themselves filling out extensive and perhaps incomprehensible checklists of criteria intended to measure critical thinking. How can evaluation possibilities be created to advance required competencies with individuals in complex practice environments?
Expectations for learner achievements must be set out clearly before learning can be measured accurately. Within the clinical environment, the stakes are high for learners. Client safety cannot be compromised. Further, measurement considerations must not dominate the time educators might otherwise spend on creating meaningful instructional approaches. In his seminal Learning to Teach in Higher Education, Paul Ramsden (1992) establishes an important distinction between deep and surface learning. In his view, deep and meaningful learning occurs when assessment focuses on both what students need to learn and how educators can best teach them.
Understanding the complexities in evaluating students and our teaching is an ongoing process. Approaching the process collaboratively in ways that consistently involve learners as active participants, rather than passive recipients, can support their success and inspire our teaching. In this chapter we introduce the vocabulary of evaluation and discuss methods of evaluating students and evaluating teaching. We suggest creative evaluation strategies that teachers can use in a variety of different clinical practice settings.
Vocabulary of Evaluation
Educators may feel overwhelmed by measuring how learners create personal meaning and demonstrate understanding of the consensually validated knowledge they will need to practice competently in their field of health. Measuring the efficacy of our own teaching in relation to preparing learners to practice safely, ethically, and in accordance with entry to practice competencies is not straightforward either. However, whether we are seeking to appraise student learning or our own teaching, knowing the criteria for expected outcomes will help us understand what is being measured. Measurement, assessment, evaluation, feedback and grading are terms used in appraising student learning and our own teaching.
Measurement, Assessment and Evaluation
Measurement determines attributes of a physical object in relation to a standard instrument. For example, just as a thermometer measures temperature, standardized educational tests measure student performance. Reliable and valid measurement depends on the skilful use of appropriate and accurate instruments. In 1943, Douglas Scales was one of the first to argue against applying the principles of scientific measurement to the discipline of education.
The kind of science which seeks only the simplest generalizations may depart rather far from flesh-and-blood reality, but the kind of science which can be applied in the everyday work of teachers, administrators, and counselors must recognize the great variety of factors entering into the practical conditions under which these persons do their work. Any notion of science which stems from a background of engineering concepts in which all significant variables can be readily identified, isolated, measured, and controlled is both inadequate and misleading. Education, in both its theory and its practice, requires a new perspective in science whichscience that will enable it to deal with composite phenomena where physical science normally deals with highly specific, single factors. (Scales, 1943. p. 1)
One example of a standardized measurement tool is a required student evaluation form. Most health professions programs provide clinical instructors with evaluation forms that have been designed to measure learning outcomes in relation to course objectives. These forms provide standardization in that they are implemented with all students in a course. They often focus on competencies such as safety, making them relevant to all members of the profession (Walsh, Jairath, Paterson & Grandjean, 2010). However, clinical instructors using the forms may have little or no input into their construction and may not see clear links to their own practice setting.
Another example of a standardized measurement tool is a qualifying examination that all members of a profession must pass in order to practice. Similarly, skills competency checklists, rating scales, multiple choice tests and medication dosage calculation quizzes can provide standardized measurement. Again, clinical instructors may have limited input into design of these tools.
Assessment obtains information in relation to a complex objective, goal or outcome. While the kinds of standardized measurements noted above can all contribute to assessing student performance, additional information is necessary. Processes for assessment require inference about what individuals do in relation to what they know (Assessment, n.d.). For example, inferences can be drawn about how students are applying theory to practice from instructor observations of students implementing client care, from student self-assessments, and from peer assessments.
Evaluation makes judgments about value or worthiness in relation an objective, goal or outcome. Evaluation needs information from a variety of different sources and at different times. Evaluation of learners in clinical practice settings is considered subjective rather than objective (Emerson, 2007; Gaberson, Oermann & Shellenbarger, 2015; Gardner & Suplee, 2010; O’Connor, 2015).
Formative evaluation is continuous, diagnostic and focused on both what students are doing well and areas where they need to improve (Carnegie Mellon, n.d.). As the goal of formative evaluation is to improve future performance, a mark or grade is not usually included (Gaberson, Oermann & Scellenbarger, 2015; Marsh et al., 2005). Formative evaluations, sometimes referred to as mid-term evaluation, should precede final or summative evaluation.
Summative evaluation summarizes how students have or have not achieved the outcomes and competencies stipulated in course objectives (Carnegie Mellon, n.d.), and includes a mark or grade. Summative evaluation can be completed at mid-term or at end of term. Both formative and summative evaluation consider context. They can include measurement and assessment methods noted previously as well as staff observations, written work, presentations and a variety of other measures.
Whether the term measurement, assessment or evaluation is used, the outcome criteria or what is expected must be defined clearly and measured fairly. The process must be transparent and consistent. For all those who teach and learn in health care fields, succeeding or not succeeding has profound consequences.
Creative Strategies
The Experience of Being Judged
Clinical teachers measure (quantify), assess (infer) and evaluate (judge). Tune in to a time in your own learning or practice where your performance was measured. The experience of having others who are in positions of power over us make inferences and judgments about what we know can be both empowering and disempowering. Reflect on an occasion when you were evaluated. Did the evaluation offer a balanced view of your strengths and weaknesses? Did you find yourself focusing more on the weaknesses than on the strengths? How can our own experiences with being judged help us be better teachers?
Students also bring with them experiences of being judged. One helpful strategy may be to have them share their best and worst evaluation experiences. Focus a discussion on the factors that made this their best or worst experience, to help learners reveal their fears. You can consider asking learners to draw a picture of their experience before they reflect and discuss.
Feedback
Feedback differs from assessment and evaluation. Assessment requires instructors to make inferences and evaluation requires them to make judgments. Feedback is non-judgmental and requires instructors to provide learners with information that facilitates improvement (Concordia University, n.d.). Feedback should focus on tasks rather than on individuals, it should be specific, and it should be directly linked to learners’ personal goals (Archer, 2010).
Periodic, timely constructive feedback that recognizes both strengths and areas for improvement is perceived by students as encouraging and helpful in bolstering their confidence and independence (Bradshaw & Lowenstein, 2014). The tone of verbal or written feedback should always communicate respect for the student and for any work done. The feedback should be specific enough that students know what to do, but not so specific that the work is done for them (Brookhart, 2008).
Arthur Chickering and Zelda Gamson (1987, p. 1) identified seven seminal principles of good practice in undergraduate education.
- Encourages contact between students and faculty.
- Develops reciprocity and cooperation among students.
- Encourages active learning.
- Gives prompt feedback.
- Emphasizes time on task.
- Communicates high expectations.
- Respects diverse talents and ways of learning.
All these principles should be considered when providing feedback to students in the clinical area. Certainly “provides prompt feedback” is particularly relevant. If more time passes before you give feedback on learning experiences, you will find it more difficult to remember details and provide effective feedback (Gaberson, Oerman & Schellenbarger, 2015).
Including students’ self-assessments when providing feedback is a critical element in the process. Throughout their careers, health professionals are encouraged to reflect on their own practice. This needed reflectivity can be developed by opening any feedback session with open-ended questions that invite learners to share their reflections and self-assessment. This strategy may soften perceptions of harshness associated with corrective feedback and may bring unexpected questions and issues into the discussion (Ramani & Krackov, 2012).
All too often feedback is viewed as educator-driven, with instructors assuming primary responsibility for initiating and directing a session. A more learner-centred approach encourages students to take a central role in the process and to seek out opportunities to gather feedback from instructors and others in the practice area (Rudland et al., 2013).
Creative Strategies
Beyond just ‘Good Job’ or ‘Needs Work’
When offering feedback, try these five simple steps to go beyond just ‘good job’ or ‘needs work.’
-
- Affirm positive aspects of what a student has done well.
- Explore the student’s own understanding and feelings about the experience.
- Pick up on any area the student identifies as needing work.
- Identify any additional areas where the student needs to improve, including an explanation of why these are important.
- Provide opportunity for the student to reflect and respond (in writing if possible) to the feedback.
Conclusion
Evaluating our students and ourselves is a critical aspect of clinical teaching. In this chapter we discussed methods of evaluating students and methods of evaluating teaching. The process of evaluating students requires clinical teachers to make judgements about whether students are meeting objectives or not, based on information gathered and recorded throughout the course. Clinical teachers measure attributes of learning with standardized instruments and assess learning through inferences about how students are applying theory to practice, based on observation in different situations.
Meaningful evaluation goes beyond identifying students’ progress in relation to course objectives and outcomes. Deep learning occurs when teachers provide their students with specific and individualized feedback. Students’ own self-assessment of their strengths and plans for improvement should frame any feedback conversation. Ultimately, instructors must assign grades. Grades can be determined through a norm-referenced approach that compares students to other students or through a criterion-referenced approach that compares students to set criteria.
References
Archer, J. (2010). State of the science in health profession education: Effective feedback. Medical Education, 44(1), 101–108. doi: 10.1111/j.1365-2923.2009.03546.x
Assessment. (n.d.). In The glossary of education reform. Retrieved from http://edglossary.org/
Atherton, J. (2013). Learning and teaching: Learning contracts [Fact sheet]. Retrieved from http://www.learningandteaching.info/teaching/learning_contracts.htm
Austin, Z. & Gregory, P. (2007). Evaluating the accuracy of pharmacy students’ self-assessment skills. American Journal of Pharmaceutical Education, 71(5), 1–8.
Barrett, H. (n.d.). Dr. Helen Barrett’s electronic portfolios [website]. Retrieved from http://electronicportfolios.com/
Baxter, P., & Norman, G. (2011). Self-Assessment or self-deception? A negative association between self-assessment and performance. Journal of Advanced Nursing, 67(11), 2406–2413. doi:10.1111/j.1365-2648.2011.05658.x
Beckett, S. (1983). Worstward ho. Location Unknown: Margarana Books.
Benton, S. & Cashin, W. (2012). Student ratings of teaching: A summary of research and literature. Idea paper #50. Manhattan KS: The Idea Centre. Retrieved from http://www.ntid.rit.edu/sites/default/files/academic_affairs/Sumry%20of%20Res%20%2350%20Benton%202012.pdf
Billings, D. & Halstead, J. (2012). Teaching in nursing: A guide for faculty (4th ed.). St Louis: Elsevier.
Black, S., Curzio, J. & Terry, L. (2014). Failing a student nurse: A new horizon of moral courage. Nursing Ethics, 21(2), 224–238
Boud, D. (1995). Enhancing learning through self-assessment. London: Kogan Page.
Bradshaw, M. & Lowenstein, A. (2014). Innovative teaching strategies in nursing and related health professions education (6th ed.). Burlington, MA: Jones & Bartlett.
Brookhart, S. (2008). How to give effective feedback to your students. Alexandria, VA: Association for Supervision and Curriculum Development.
Bush, H., Schreiber, R. & Oliver, S. (2013). Failing to fail: Clinicians’ experience of assessing underperforming dental students. European Journal of Dental Education, 17(4), 198–207. doi: 10.1111/eje.12036
Carnegie Mellon. (n.d.) What is the difference between formative and summative assessment? [Fact sheet]. Retrieved from the Eberly Centre for Teaching Excellence, Carnegie Mellon University, Pittsburgh, PA. http://www.cmu.edu/teaching/assessment/basics/formative-summative.html
Center for Teaching and Learning. (n.d.). Peer observation guidelines and recommendations [Fact sheet]. Minneapolis, MN: University of Minnesota. Available at http://www1.umn.edu/ohr/teachlearn/resources/peer/guidelines/index.html
Centre for the Study of Higher Education. (2002). A comparison of norm-referencing and criterion-referencing methods for determining student grades in higher education. Australian Universities Teaching Committee. Retrieved from http://www.cshe.unimelb.edu.au/assessinglearning/06/normvcrit6.html
Chan, S. & Wai-tong, C. (2000). Implementing contract learning in a clinical context: Report on a study. Journal of Advanced Nursing, 31(2), 298–305.
Cheung, R. & Au, T. (2011). Nursing students’ anxiety and clinical performance. Journal of Nursing Education, 50(5), 286–289. doi: 10.3928/01484834-20110131-08
Chickering, A. & Gamson, Z. (1987). Seven principles for good practice in undergraduate education. American Association for Higher Education AAHE Bulletin, 39(7), 3–7.
Colthart, I., Bagnall, G., Evans, A., Allbutt, H., Haig, A., Illing, J. & McKinstry, B. (2008). The effectiveness of self-assessment on the identification of learning needs, learner activity, and impact on clinical practice: BEME Guide No. 10. Medical Teacher, 30,124–145.
Concordia University. (n.d.) How to provide feedback to health professions students [Wiki]. Retrieved from http://www.wikihow.com/Provide-Feedback-to-Health-Professions-Students
Davis, D., Mazmanian, P., Fordis, M., Van Harrison, R., Thorpe, K. & Perrier, L. (2006). Accuracy of physician self-assessment compared with observed measures of competence: A systematic review. The Journal of the American Medical Association, 296(9), 1094–1102.
Dearnley, C. & Meddings, F. (2007) Student self-assessment and its impact on learning: A pilot study. Nurse Education Today, 27(4), 333–340.
Duffy, K. (2003). Failing students: A qualitative study of factors that influence the decisions regarding assessment of students’ competence in practice. Glasgow, UK: Glasgow Caledonian Nursing and Midwifery Research Centre. Retrieved from http://www.nm.stir.ac.uk/documents/failing-students-kathleen-duffy.pdf
Duffy, K. (2004). Mentors need more support to fail incompetent students. British Journal of Nursing, 13(10), 582.
Dunning, D., Heath, C. & Suls, J. (2004). Flawed self-assessment: Implications for health education and the workplace. Psychological Science in the Public Interest, 5(3) 69–106. Retrieved from https://faculty-gsb.stanford.edu/heath/documents/PSPI%20-%20Biased%20Self%20Views.pdf
Edgerton, R., Hutching, P. & Quinlan, K. (2002). The teaching portfolio: Capturing the scholarship of teaching. Washington DC: American Association of Higher Education.
Emerson, R. (2007). Nursing education in the clinical setting. St Louis: Mosby.
Frank, T. & Scharff, L. (2013). Learning contracts in undergraduate courses: Impacts on student behaviors and academic performance. Journal of the Scholarship of Teaching and Learning, 13(4), 36–53.
Gaberson, K., Oermann, M. & Schellenbarger, T. (2015). Clinical teaching strategies in nursing (4th ed.). New York: Springer.
Gainsbury, S. (2010). Mentors passing students despite doubts over ability. Nursing Times, 106(16), 1.
Galbraith R., Hawkins R. & Holmboe E. (2008). Making self-assessment more effective. Journal of Continuing Education in the Health Professions, 28(1), 20–24.
Gardner, M. & Suplee, P (2010). Handbook of clinical teaching in nursing and health sciences. Sudbury, MA: Jones & Bartlett.
Gregory, D., Guse, L., Dick, D., Davis, P. & Russell, C. (2009). What clinical learning contracts reveal about nursing education and patient safety. Canadian Nurse, 105(8), 20–25.
Hall, M. (2013). An expanded look at evaluating clinical performance: Faculty use of anecdotal notes in the US and Canada. Nurse Education in Practice, 13, 271–276.
Heaslip, V. & Scammel, J. (2012). Failing under-performing students: The role of grading in practice assessment. Nursing Education in Practice, 12(2), 95–100.
Hodgson, P., Chan, K. & Liu, J. (2014). Outcomes of synergetic peer assessment: first-year experience. Assessment and Evaluation in Higher Education, 39(2), 168–179.
Kajander-Unkuri, S., Meretoja, R., Katajisto, J., Saarikoski, M., Salminen, L., Suhonene, R. & Leino-Kilpi, H. (2013). Self-assessed level of competence of graduating nursing students and factors related to it. Nurse Education Today, 34(5), 795 – 801.
Keary, E. & Byrne, M. (2013). A trainee’s guide to managing clinical placements. The Irish Psychologist, 39(4), 104–110.
Knowles, M. (1975). Self-Directed Learning. New York: Association Press.
Larocque, S. & Luhanga, F. (2013). Exploring the issue of failure to fail in a nursing program. International Journal of Nursing Education Scholarship, 10(1), 1–8.
Lyons, N. (2006). Reflective engagement as professional development in the lives of university teachers. Teachers and Teaching: Theory and Practice, 12(2), 151–168.
Marsh, S., Cooper, K., Jordan, G. Merrett, S., Scammell, J. & Clark, V. (2005). Assessment of students in health and social care: Managing failing students in practice. Bournemouth, UK: Bournemouth University Publishing.
Mass, M., Sluijsmans, D., Van der Wees, P., Heerkens, Y., Nijhuis-van der Sanden, M. & van der Vleuten, C. (2014). Why peer assessment helps to improve clinical performance in undergraduate physical therapy education: A mixed methods design. BMC Medical Education, 14, 117. doi:10.1186/1472-6920-14-117.
Mehrdad, N., Bigdeli, S. & Ebrahimi, H. (2012). A comparative study on self, peer and teacher evaluation to evaluate clinical skills of nursing students. Procedia – Social and Behavioral Science, 47, 1847–1852.
Melrose, S. & Shapiro, B. (1999). Students’ perceptions of their psychiatric mental health clinical nursing experience: A personal construct theory explanation. Journal of Advanced Nursing, 30(6), 1451–1458.
Mort, J. & Hansen, D. (2010). First-year pharmacy students’ self-assessment of communication skills and the impact of video review. American Journal of Pharmaceutical Education, 74(5), 1–7.
O’Connor, A. (2015). Clinical instruction and evaluation. Burlington, MA: Jones & Bartlett.
Oermann, M. (2015). Teaching in nursing and role of the educator. New York: Springer.
Pisklakov, S., Rimal, J. & McGuirt, S. (2014). Role of self-evaluation and self-assessment in medical student and resident education. British Journal of Education, Society and Behavioral Science, 4(1), 1–9.
Ramani, S. & Krackov, S. (2012). Twelve tips for giving feedback effectively in the clinical environment. Medical Teacher, 34, 787–791.
Ramli, A., Joseph, L. & & Lee, S. (2013). Learning pathways during clinical placement of physiotherapy students: A Malaysian experience of using learning contracts and reflective diaries. Journal of Educational Evaluation in the Health Professions, 10, 6. doi: http://dx.doi.org/10.3352/jeehp.2013.10.6
Ramsden, P. (1992). Learning to teach in higher education. London: Routledge.
Regehr, G. & Eva, K. (2006). Self-assessment, self-direction, and the self-regulating professional Clinical Orthopaedics and Related Research, 449, 34–38. Retrieved from http://innovationlabs.com/r3p_public/rtr3/pre/pre-read/Self-assessment.Regher.Eva.2006.pdf
Rudland, J., Wilkinson, T., Wearn, A., Nicol, P., Tunny, T., Owen, C. & O’Keefe, M. (2013). A student-centred model for educators. The Clinical Teacher, 10(2), 92–102.
Rush, S., Firth, T., Burke, L. & Marks-Maran, D. (2012). Implementation and evaluation of peer assessment of clinical skills for first year student nurses. Nurse Education in Practice, 12(4), 2219–2226.
Rye, K. (2008). Perceived benefits of the use of learning contracts to guide clinical education in respiratory care students. Respiratory Care, 53(11), 1475–1481.
Scales, D. E. (1943). Differences between measurement criteria of pure scientists and of classroom teachers. Journal of Educational Research 37, 1–13.
Scanlan, J. & Care, D. & Glessler, S. (2001). Dealing with the unsafe student in clinical practice. Nurse Educator, 26(1), 23–27.
Secomb, J. (2008). A systematic review of peer teaching and learning in clinical education. Journal of Clinical Nursing, 17(6), 703–716. doi: 10.1111/j.1365-2702.2007.01954.x
Seldin, P. (1997). The teaching portfolio: A practical guide to improved performance and promotion/tenure decisions (2nd ed.). Bolton, MA: Ankor.
Shulman, L. (1998). Teacher portfolios: A theoretical activity. In N. Lyons (Ed.). With portfolio in hand: Validating the new teacher professionalism. New York, NY: Teachers College Press.
Sluijsmans, D., Van Merriënboer, J., Brand-gruwel, S. & Bastiaens, T. (2003). The training of peer assessment skills to promote the development of reflection skills in teacher education. Studies in Education Evaluation, 29, 23–42.
Timmins, F. (2002). The usefulness of learning contracts in nurse education: The Irish perspective. Nurse Education in Practice, 2(3), 190–196. doi: 10.1054/nepr.2002.0069
Walsh, T., Jairath, N., Paterson, M. & Grandjean, C. (2010). Quality and safety education for nurses clinical evaluation tool. Journal of Nursing Education, 49(9), 517–522.
Weeks, B. & Horan, S. (2013). A video-based learning activity is effective for preparing physiotherapy students for practical examinations. Physiotherapy, 99, 292–297.
Welsh, P. (2014). How first year occupational therapy students rate the degree to which anxiety negatively impacts on their performance in skills assessments: A pilot study at the University of South Australia. Ergo, 3(2), 31–38. Retrieved from http://www.ojs.unisa.edu.au/index.php/ergo/article/view/927
Winstrom, E. (n.d.). Norm-referenced or criterion-referenced? You be the judge! [Fact sheet]. Retrieved from http://www.brighthubeducation.com/student-assessment-tools/72677-norm-referenced-versus-criterion-referenced-assessments/?cid=parsely_rec
