By Dr Victoria Escaip (Spanish Subject co-ordinator) and Richard Davies (Flexible Learning Advisor)

When you have the problem of high student numbers, limited staff time, and a desire for a personalised assessment process, what do you do? This article explores one recently tried solution for language assessment that makes the most of the technology academic staff and  students have at their disposal, in conjunction with the Learning Management System at UC.

Around the middle of last year, there was a goal. An idea to find ways to make use of the technology LEARN has to offer to make the Spanish courses better for students and staff alike. Some aspects were relatively straightforward to implement such as improved course design, use of the Gradebook and electronic submissions. One of the trickier and more difficult problems was how to find a better way to assess the speaking skills of a large number of students. The existing face-to-face method was complex and exhausting for the staff, and anxiety ridden for students.

The solution proposed was to use the recording function in a LEARN quiz question to both record the questions and the answers. Initially the idea of using just audio was investigated but it quickly became apparent there was no way of verifying who had actually taken the test. As an alternative, better solution, recording both video and audio was looked into.

After some initial testing which proved promising, there was another concern. How could we ensure the students didn’t just tell each other what the questions were going to be so they could prep ahead of time? The solution to this was to create a bank of questions at three different difficulty levels in which the computer each time would randomly pick some out to quiz the student. This randomisation would simulate what the teacher did in person.

Having dealt with that problem there were two more issues, what devices would the quiz be suitable for and how do we make sure it is fool proof for the students. Some sample questions were tested on a variety of devices. Research found that the only 100% reliable approach was to use a laptop or desktop computer. Although there was some concern that some students may not have access to a laptop or desktop, use of one was offered to students if necessary. The instructions that students would read were then carefully gone through with images included to ensure there was no misunderstanding. Each question included the instructions as due to the random sorting we would not know which question would come first.

A practice quiz was set up to ensure students had a chance to check their device was working before they did the formal test. This practice test had two questions that were not going to be used in the formal test, and students could do this practice test as many times as they liked. This allowed students to make sure their equipment was working correctly in advance and, most importantly, to familiarize themselves with the procedure.

It was important to get the timing and the number of questions right. If you set too short a time the students will not be able to get through all of the questions. If you give them too long then they will have enough time to look up the answers before they respond. So two senior students were asked to do the quizzes a couple of times and their feedback allowed the adjustment of timing.

Finally, the best way to grade was considered. Rather than marking each question one at a time it was considered a more efficient approach to view all of the answers given as a collection, just as it is done in a face-to-face situation, and then give an overall mark by entering it directly into the Gradebook.

What were the results?

1. The time problem was solved. The oral assessment online maximized the resources available without affecting the quality of the assessment. There was not a single complaint and all 83 students went through this procedure smoothly.
2. A relaxed test environment. We were pleased to see that students were much more relaxed than in a face-to-face oral test situation. Some of them were even wearing their pyjamas and looked really comfortable in their skins. We believe this procedure enabled them to give their best and helped them to achieve better marks.
3. Fair marking. On one hand, due to the appropriate time given for the test, higher-level students were able to answer all the questions, or most of them, thus getting a high mark, while lower-level students would lose time when looking up for answers before responding, which prevented them from completing the test, achieving a lower mark, which in all cases corresponded to the lecturer’s criteria. On the other hand, the fact of having more time to mark the tests on the lecturer’s own time, without the rush of having to assess one student after the other, contributed to a fairer marking when assigning grades since each answer was listened more carefully, and a comparative grading scale could be established before assigning the final mark for each student.
4. Other positive outcomes. A pleasing and surprising outcome was that all the students’ answers to the question “What are you like?” (a traditional question to which students need to respond using vocabulary and the verb ‘to be’, a tricky verb in Spanish) were really positive. Answers such as “I’m intelligent”, “I’m hardworking”, “I’m good looking” were the norm, in opposition to the usual ones in a face-to-face situation: “I’m dumb”, “I’m ugly”, “I’m lazy, ha ha ha”. This says much about how a relaxing environment can provide them with more personal confidence and even boost their self-esteem!

All in all, the students were more spontaneous, they did not have a bad time, as it happens for many in a face-to-face oral test situation, and it actually seemed that they had a lot of fun video recording themselves and looking at themselves on the screen.

What did the end result look like? See below: