When students enroll at a post-secondary institution, they want to know that they will graduate better equipped for life – whether they intend to join the working world, pursue further studies, conduct research, or apply their experiences in another meaningful way.

A recently concluded four-year longitudinal study conducted by the Office of the Provost, partially funded by the Higher Education Quality Council of Ontario (HEQCO), sought to measure and track several important student learning outcomes as students progressed through their time at Queen’s.

The project’s results, which reflect well on a Queen’s education, were recently published as a HEQCO research paper.

“Queen’s has emerged as a national leader in learning outcomes assessment, an area of growing importance in the academy – finding a way to measure and quantify the value of a post-secondary education,” says Daniel Woolf, Principal and Vice-Chancellor. “Assessing the growth of our students’ cognitive skills is a priority to help us understand the degree to which Queen’s students develop during their time here. I am pleased both at the successful completion of the project, and Queen’s ongoing work in this field of research.”

About the project

The Learning Outcomes Assessment project sought to determine how important skills like critical thinking, problem-solving, communication, and lifelong learning skills improved over a student’s four years at Queen’s. Specifically, researchers sought to answer four questions:

1. How much do students’ complex cognitive skills change between the first and fourth year of undergraduate studies?
2. How does the development of complex cognitive skills and lifelong learning vary between programs and individuals, and what is the relationship of standardized measures to course grades?
3. Can data from instruments be used to support skills development in courses?
4. How feasible is the use of these assessments in a Canadian university?

The high-level outcomes of the study noted that students’ skills in critical thinking, problem-solving, and communication increased over the four years of their Queen’s degree; and that Queen’s students demonstrated a higher level of skill in critical thinking than comparable students at most peer institutions participating in the equivalent surveys.

This is important, as industry and academia are increasingly recognizing the value of skills as graduates enter a rapidly changing job market. A study from the Royal Bank of Canada (RBC) which was published in March noted human intelligence and intuition are critical to maintaining employment in the face of automation and other workforce trends. The “Humans Wanted – How Canadian youth can thrive in the age of disruption” research paper specifically noted the importance of critical thinking and problem-solving.

How to measure skills

To conduct this research, thousands of student participants volunteered from multiple programs in the Faculties of Arts and Science, and Engineering and Applied Science. The students completed standardized assessment tests and results of some of their exams and assignments were also studied. Sample groups of the students were also interviewed. Results for upper-year students were used as a benchmark, and compared against first-year students as they completed their degrees.

“We have shown that we can to assess key skills, such as critical thinking, among our students, and that students are improving,” says Brian Frank, Associate Dean (Teaching and Learning)and the DuPont Canada Chair in Engineering Education Research and Development in the Faculty of Engineering and Applied Science. “We are continuing this work in a new project that engages faculty in developing and assessing transferrable skills, providing a different perspective on their roles as teachers. We are using a consistent approach that gives us information to inform course and program improvement.”

Emily Britton (Artsci’15) worked on the LOA project as a research assistant. She says she appreciated the unique opportunity to participate in the project and gain a different perspective on the university.

“I was able to learn a lot about what was going on at the university level in terms of tracking student progress and achievement, which I think many students are not aware of,” she says. “I think a lot of students assume and hope that they are developing the sorts of key skills that were assessed by [this project]…but I think it’s encouraging to see some wide-scale and tangible evidence that their degrees are providing them with the types of transferrable skills that will set them up for success.”

Ms. Britton noted the connections and applied research experience she gained on the LOA project were also helpful to her as she applied to graduate school.

What comes next

With this phase of the project complete, Queen’s is working on a related project to enhance the way learning outcomes are assessed in courses across multiple programs.

The Cognitive Assessment Redesign project is seeking faculty members who wish to build greater understanding into how their courses are fostering critical thinking, creative thinking, and problem-solving.

By signing up, faculty members will deepen their understanding of how to align course learning outcomes to assessments, how to create an authentic assignment that deliberately elicits skills like critical thinking, and how to develop a rubric that successfully measures student achievement.

Each faculty member is paired with an Assessment Facilitator, who helps with the assessment redesign, and instructors are part of a network that meets regularly to share results and discuss their approaches.

The Learning Outcomes Assessment Consortium

Queen’s is one of nine universities and colleges which make up HEQCO’s Learning Outcomes Assessment Consortium. Each of the members are engaged in similar efforts to investigate the development and assessment of learning outcomes, and refine the instruments for assessing and measuring them.

Harvey Weingarten, the CEO of HEQCO, called Queen’s efforts in measuring learning outcomes “forward thinking” and “innovative”.

“Non-disciplinary skills are becoming increasingly important in today’s fast-changing economy,” Dr. Weingarten said. “Postsecondary institutions need to do a better job of measuring, teaching and credentialing these skills so students can succeed in the workplace and in life.”

Completing this research places Queen’s in select Canadian company – while the concept of measuring learning outcomes is somewhat common in the United States higher education community, few Canadian institutions have investigated the idea. Importantly, Jill Scott, Vice-Provost (Teaching and Learning), noted that Queen’s students “started higher” than the average and “ended higher” when compared against other schools in other jurisdictions such as the United States.

To see the Learning Outcomes Assessment and Program Improvement at Queen’s University HEQCO report, please visit HEQCO’s website.