Humber College examines the effectiveness of a new skills-assessment tool
Students’ critical thinking and written communications skills show the most improvement when they are explicitly taught, according to a new study by the Higher Education Quality Council of Ontario (HEQCO) that examines the effectiveness of a new skills-assessment tool developed at Humber College. While the assessment scorecard did not find students were making significant gains in critical thinking and written communications skills, students in courses where these skills were implicitly embedded in course content had lower levels of achievement than those in courses where the skills were an explicit focus.
This study, Assessing Learning Outcomes: Thinking Critically about Critical Thinking and Written Communication Skills, was funded through HEQCO’s Learning Outcomes Assessment Consortium, a group of Ontario institutions developing and piloting skills-assessment tools and techniques ranging from ePortfolios to analytic rubrics that are expected to be scalable to the institutional level in the future.
Project Description
From Fall 2014 to Winter 2016, 650 students, mainly from Humber College’s schools of Business, Liberal Arts and Sciences, and Social and Community Services, took part in the study, which used the scorecard to evaluate students’ development in critical thinking and written communication. Three course types were evaluated. First, courses where skills were explicitly taught as part of the course content (College Reading and Writing Skills, Workplace Writing Skills). Second, vocational courses that emphasize the importance of core skills, but embed the material implicitly into course content (Police Foundations Program — Criminal and Civil Law, Interviewing and Investigations). Third, a dedicated course on critical thinking, where the material is taught as content knowledge.
Additionally, the skills scorecard was used by 46 faculty members to assess more than 725 student work products. Faculty provided information on their experiences with the scorecard.
Findings
Students in courses where critical thinking was taught as content fared worse than both the explicit and implicit models. The authors suggest that articulating and teaching these skills explicitly is needed to achieve the highest gains. The authors also note that the skills need to be taught consistently and over a longer period of time to see significant gains and these types of courses should be positioned strategically throughout each program of study.
While the study found the tool accurately captured the total critical thinking and written communications score, there was some inconsistency in instructors’ judgments of the categories and components. Nearly three-quarters of faculty participants found the tool easy to use, but only 58% felt the information provided would be meaningful for students. The authors argue increased faculty training and repeated use over a longer period of time would improve both the instrument’s validity and the relevance and understanding for students.
Authors of Assessing Learning Outcomes: Thinking Critically about Critical Thinking and Written Communication Skills are Vera Beletzan, Melissa Gabler and Paula Gouveia, Humber College.