Please enable javascript to view this page in its intended format.

Queen's University
 

 

     

 

USAT: University Survey of Student Assessment of Teaching


  • What is USAT?
     

    USAT is the acronym for the University Survey of Student Assessment of Teaching. This evaluation process is referred to as such under Article 29.3 of the Queen's University Faculty Association (QUFA) Collective Agreement.

     

    Using the USAT process, courses and instructors are evaluated through student responses to four university-wide evaluation items, up to seven department-chosen items, and up to ten instructor-chosen items. The evaluation statements are printed on scannable forms and require a response ranging from "strongly agree" to "strongly disagree". The form includes space for comments, which students are encouraged to complete in order to give instructors specific feedback.

     

    USAT is designed to evaluate each instructor, so that students in a course with several instructors will be asked to evaluate each instructor separately. There will be only one evaluation for each course/section/instructor, so that full courses continuing from the fall to the winter term will be evaluated once towards the end of the winter term.

     

    Governing Policy:
    The current University Survey of Student Assessment of Teaching (USAT) shall be used by the University in the assessment of Members’ teaching performance until replaced by a new/revised instrument(s) as approved by the Parties. A Committee created under the Memorandum of Agreement (Appendix E) shall conduct this review and make recommendations to the JCAA. Members may also submit results from any pilot test arising from the review of the USAT by the Committee. The USAT shall comprise of a set of standard questions, and the Member shall also have an opportunity to add questions to the quantitative section of the survey. All questions in the USAT shall conform to the requirements of Article 9 and Article 14. The USAT shall also provide opportunity for signed written comments from students.
  • How does the USAT process work?
     
    Summary:

     

    Each term, the USAT Office sends a listing of all courses being offered to the administrative office in each department. The department administrator along with the department head determines which courses and instructors are to be evaluated, and the list is returned to the USAT Office.

     

    After the course information is entered into the USAT database, instructors are invited by their department administrators to submit an online form allowing instructors to customize their evaluation forms by adding additional items to the university-wide and department-chosen items from a bank of over 200 items. The requisition form also allows instructors to indicate whether or not their results are to be released to the student societies.

     

    The USAT Office enters the instructors' selections and then prints the evaluation forms. The forms are packaged and sent to the departmental office for pick-up by class student representatives. The student government (ASUS and DSC chairs or equivalent) will let class representatives know when the packages are ready to be picked up. The packages are delivered in time to arrange evaluations within the 10th and 11th weeks of a 12-week term, or sooner in special cases.

     

    The class student representative meets with the instructor to arrange a convenient time for the evaluation. The student representative distributes the evaluation forms, collects the completed evaluations, and returns them to the departmental office.

     

    The completed evaluation forms are returned to the USAT Office where they are scanned electronically, and the data are converted into an easy-to-read graphical report. The reports are distributed in accordance with Article 29.3.5 of the QUFA Collective Agreement, which states:

     

    "Quantitative responses to the questionnaires shall be sent to the Member, the Department Head (if applicable) and the appropriate Dean(s) after the data have been converted into a report and following the submission of final grades. The report shall contain aggregated responses to each quantitative question, with the mean, standard deviation, frequency and number of eligible respondents calculated. Student responses to the University's questionnaire shall be aggregated in such a way as to present a fair and accurate picture of the opinions of the respondents."

     

    Instructors also receive the completed evaluation forms so that they may review the students' comments.

     

    Special Situations:

    The instructor's teaching involvement is complete after only a few weeks of term

    Instructors requiring an early evaluation should indicate as such through the online form. The completed evaluation forms will be held at the USAT office so that the results may be compared with the evaluations for the course which occurred at the standard time. The reports will be available at the same time as the other reports after final marks have been submitted for the course.

     

    There are insufficient evaluation forms for the class enrolment

    The number of forms printed is defined by the number of students enrolled in each course section. Students who are unofficially auditing the course will not be included in calculating the number of forms required. Photocopied forms cannot be accepted, as they cannot be electronically scanned.

     

    The instructor teaches courses in more than one department

    It is the instructor's responsibility to ensure that any additional department heads receive copies of the report.

  • How are USAT results calculated?
     
    • The course evaluation report is generated from the scanned responses on the completed evaluation forms.
    • Evaluation of courses whose enrolment is less than 10 is discouraged for two reasons: (a) the results are not considered to be statistically valid and (b) students may be identified through their handwriting.
    • When the number of returned evaluations for a course is less than 10 (even though the enrolment might have been greater than 10), the results from that course are not included in the departmental means.
    • The Dean, department head, and student copies of the reports show only the results to the university-wide and department-wide evaluation items.
    • The instructor copy shows, in addition, the results to the instructor-chosen evaluation items.
    • For each individual course/instructor, the report shows, for the university- and department-wide items, the results for this specific course/instructor (the filled-in histogram bar) and, for comparison, the results for the department (the open histogram bar). Departmental comparisons are produced only when there are more than two courses being evaluated in the session being processed.
    • The numerical departmental results are calculated by summing the individual evaluation means and dividing by the number of course/section/instructors evaluated in the department - rather than summing all the responses to each item across the department (which would allow courses with a high enrolment to weigh the departmental mean).
    • The percentage departmental results are calculated by averaging the individual course percentages - again rather than creating a percentage of the individual responses across the whole department.
    • The standard deviation in the results for the individual course is given as a measure of the degree of distribution of the responses around the mean; the higher the standard deviation, the more spread-out are the responses; i.e. students show greater variation of opinion. The lower the standard deviation, the more in agreement are the students (standard deviation has been calculated using a population rather than a sample standard deviation function).
    • The "Number of returned evaluations" gives the total number of scanned forms.
    • The "Number of respondents" to each evaluation item gives the number of respondents providing a response to each item. Non-responders are not included in this number, so that this value may vary among the items and against the "Number of returned evaluations".
  • Who sees USAT results?
     

    The completed evaluation forms are returned to the USAT Office where they are scanned electronically, and the data are converted into an easy-to-read graphical report. The reports are then released in accordance with Article 29.3.5 of the Queen's University Faculty Association (QUFA) Collective Agreement, which states:

     

    "Quantitative responses to the questionnaires shall be sent to the Member, the Department Head (if applicable) and the appropriate Dean(s) after the data have been converted into a report and following the submission of final grades. The report shall contain aggregated responses to each quantitative question, with the mean, standard deviation, frequency and number of eligible respondents calculated. Student responses to the University's questionnaire shall be aggregated in such a way as to present a fair and accurate picture of the opinions of the respondents."

     

    Each report of quantitative results is also sent to the appropriate student society with the permission of the faculty member and after submission of final marks.

     

    The report is a graphical analysis of the scannable items on the front of the evaluation form. It allows the reader to compare the results of an instructor's evaluation to the average for all instructors in that department who taught in that term. In the case of fall/winter courses, instructors who taught in the fall term only are compared to their counterparts at the end of the winter term.

     

    The Dean and department head see the results of the university-wide and department-chosen items. They do not see the students' comments or the results of any additional items the instructor selected for the evaluation form. The completed evaluation forms are sent with the report to the instructor after final marks have been submitted. This practice is in keeping with the QUFA Collective Agreement, Article 29.3.2, which states:

     

    " The USAT form, with the exception of any written comments, shall not be signed by the student." and Article 29.3.7, which states, " Written comments shall be sent only to the Member, and the University will take the appropriate measures to ensure that only the Member receives such responses."

    Kingston, Ontario, Canada. K7L 3N6. 613.533.2000