Open Letter Regarding Course Assessments

After completing several course assessments, it seemed apparent from my own experience and the murmurings of my classmates that we were disappointed in how the assessments had been completed. To this end, I began to consult with coursemates and MLIS Student Council regarding the issue and drafted up the following letter. This letter has been submitted to Rosanne Greene, manager of Graduate Student Services at FIMS, and will also be hosted here so as to be available to MLIS students. Additionally, a transcript of the faculty-specific paperwork is openly available on our Google Drive so that the student body might have the opportunity to contemplate these forms at their leisure.

If you have any further concerns, I encourage you to raise them with MLIS Student Council or Rosanne Greene.

-------------------------------------------------------------------------------------------------

2014 December 15

Rosanne M. Greene Manager, Graduate Student Services Room 240, North Campus Bldg. Faculty of Information and Media Studies University of Western Ontario

Dear Rosanne Green;

I am contacting you to discuss some concerns regarding the course evaluations. After consulting with MLIS Student Council and fellow students, it is my understanding that many of us are concerned with the content of the evaluations as well as their implementation. This is particularly challenging for first termers, many of whom don’t have experience of these assessments as they are practiced at Western University. My concerns are, admittedly, amplified by the knowledge that the focus group of graduating students was cancelled this term; since it seems these course assessments are sometimes the only assessment conducted, it seems all the more critical that they be as refined as possible. To this end, in addition to outlining my concerns, I have included recommendations for your consideration at the end of this letter.

I recognize that these concerns are not necessarily easy to address, and that some of these concerns fall outside of the direct influence of the FIMS office, but I raise them with you anyhow to contribute to an ongoing conversation regarding how to improve the assessment process. My intention is not to lay blame or make demands, but rather to engage in open, critical discussion.

Scantron

Although I recognize that the scantron component falls outside of your direct influence, I still mention these concerns as the FIMS faculty is one of our primary conduits to the university administration.

We're asked our enthusiasm prior to taking the course, but not after taking it. This seems a missed opportunity.
It seems beneficial to include the option of Don’t Know or No Comment in addition to, or instead of, N/A.
Many of the questions include qualifiers such as “…considering class size” or “… considering the limitations of course material”. Although I appreciate the apparent intent of such addendums, their inclusion seems to necessarily skew responses upwards.

Preamble

These experiences will vary from person to person, but I include them here to highlight the discrepancy in how these assessments are administered.

The preamble was not read in all of my classes; I had filled out several assessments before I’d had the opportunity to hear it at all.
I had one class where the professor put the effort into explaining the use and impact of these assessments. This was after I had already filled out several, and I wonder if I would have filled those initial ones out more carefully if I had received this information at the beginning of the week.

Faculty-specific paperwork

I appreciate that the faculty has invested the time and effort to include their own assessment elements, but there remain issues to address.

The information is overwhelming. This issue is amplified by the fact that students are often in the midst of term-end projects, filling them out in in whatever time is available once the lecture is complete, and the previously mentioned issues regarding the inconsistency of whether or not the preamble is read aloud.
Even after reading over the explanation of the various forms several times, it does not seem to clearly explain the distinction between remarks on the scantron as opposed to the faculty-specific paperwork. Although the faculty-specific paperwork is described fairly well, it is not explicitly stated how these comments are distinct from comments left on the scantron.
Some important aspects of the courses do not seem to have space dedicated to discussing them, such as the impact of TAs. Considering the prevalence of TAs in first term courses, this seems a critical oversight. Notably, several of the FIMS student councils have raised this issue.
The five point scales lack a No Comment or Don’t Know option, and the middle option is often phrased as a positive rather than neutral response (e.g. “somewhat useful”).

Procedure

I was struck by how differently these assessments were distributed in each class. Of course, some variation is to be expected, but it suggests that a review of best practices may be in order.

One of my classes had the TA collect the assessments. I want to emphasize that I have no suspicions regarding this TA’s handling of the assessments, and the TA in question was even quite considerate when I approached them about the matter. Their involvement in the collection of the assessments still struck me as quite problematic, particularly as they were so involved in the execution of the course and the grading. After speaking with my peers, it seems this was not an isolated event.
One class had the assessments collected by having them spread out in distinct piles on a table instead of placed directly in the envelope. This compromised their confidentiality as classmates, and even a TA, were easily able to see any answers facing upwards. The TA that was present even remarked to one student when they handed their assessment in about the student’s decision to not write any comments. I suspect the TA was trying to motivate the student to contribute and likely meant no harm, but this experience does emphasize how critically confidentiality was abandoned in this particular case. Yet again, upon consulting with my peers, this does not seem to be an isolated event.

Recommendations

I believe strongly that people willing to discuss problems should strive to also discuss solutions. Here are some recommendations for your consideration.

Completing the assessments at the beginning of class might discourage students from rushing.
Having online assessments instead of, or in addition to, the in-class assessments might allow respondents to participate when they are less distracted by end of term projects.
Conducting the assessments earlier or later in the term may avoid conflict with the stressful workload often experienced at end of term.
Sending out an email prior to the assessments in order to inform and alert the students that the assessments are about to be administered might help to ensure information is available to students in advance of being expected to fill out the forms. I recognize there are many issues with email, but as it is the primary method of communication used to disseminate information to students, it seems the best place to distribute this information.
Ensure the assessment templates are available so that students may become more familiar with the content of the assessments. If these documents are already available, I have been unable to find them on the website and would like to know where they might be located.
Emphasizing the responsibility of professors, particularly in the mandatory courses primarily populated by first term students, to read the preamble before handing over the assessments to a student to administer.
Establishing guidelines precluding TA involvement collecting course evaluations in courses for which they worked.

As I understand it, there has been an ongoing conversation regarding how to assess the program and its courses. I hope that this letter can in some small way contribute to that conversation. If further information or clarification is desired, please contact me at kseanor@uwo.ca.

Thank you for your time;

Kirstyn Seanor Co-chair MLIS Student Council