By Dr Victoria Escaip (Spanish Subject co-ordinator) and Richard Davies (Flexible Learning Advisor)

When you have the problem of high student numbers, limited staff time, and a desire for a personalised assessment process, what do you do? This article explores one recently tried solution for language assessment that makes the most of the technology academic staff and  students have at their disposal, in conjunction with the Learning Management System at UC.

Around the middle of last year, there was a goal. An idea to find ways to make use of the technology LEARN has to offer to make the Spanish courses better for students and staff alike. Some aspects were relatively straightforward to implement such as improved course design, use of the Gradebook and electronic submissions. One of the trickier and more difficult problems was how to find a better way to assess the speaking skills of a large number of students. The existing face-to-face method was complex and exhausting for the staff, and anxiety ridden for students.

The solution proposed was to use the recording function in a LEARN quiz question to both record the questions and the answers. Initially the idea of using just audio was investigated but it quickly became apparent there was no way of verifying who had actually taken the test. As an alternative, better solution, recording both video and audio was looked into.

After some initial testing which proved promising, there was another concern. How could we ensure the students didn’t just tell each other what the questions were going to be so they could prep ahead of time? The solution to this was to create a bank of questions at three different difficulty levels in which the computer each time would randomly pick some out to quiz the student. This randomisation would simulate what the teacher did in person.

Having dealt with that problem there were two more issues, what devices would the quiz be suitable for and how do we make sure it is fool proof for the students. Some sample questions were tested on a variety of devices. Research found that the only 100% reliable approach was to use a laptop or desktop computer. Although there was some concern that some students may not have access to a laptop or desktop, use of one was offered to students if necessary. The instructions that students would read were then carefully gone through with images included to ensure there was no misunderstanding. Each question included the instructions as due to the random sorting we would not know which question would come first.

A practice quiz was set up to ensure students had a chance to check their device was working before they did the formal test. This practice test had two questions that were not going to be used in the formal test, and students could do this practice test as many times as they liked. This allowed students to make sure their equipment was working correctly in advance and, most importantly, to familiarize themselves with the procedure.

It was important to get the timing and the number of questions right. If you set too short a time the students will not be able to get through all of the questions. If you give them too long then they will have enough time to look up the answers before they respond. So two senior students were asked to do the quizzes a couple of times and their feedback allowed the adjustment of timing.

Finally, the best way to grade was considered. Rather than marking each question one at a time it was considered a more efficient approach to view all of the answers given as a collection, just as it is done in a face-to-face situation, and then give an overall mark by entering it directly into the Gradebook.

What were the results?

1. The time problem was solved. The oral assessment online maximized the resources available without affecting the quality of the assessment. There was not a single complaint and all 83 students went through this procedure smoothly.
2. A relaxed test environment. We were pleased to see that students were much more relaxed than in a face-to-face oral test situation. Some of them were even wearing their pyjamas and looked really comfortable in their skins. We believe this procedure enabled them to give their best and helped them to achieve better marks.
3. Fair marking. On one hand, due to the appropriate time given for the test, higher-level students were able to answer all the questions, or most of them, thus getting a high mark, while lower-level students would lose time when looking up for answers before responding, which prevented them from completing the test, achieving a lower mark, which in all cases corresponded to the lecturer’s criteria. On the other hand, the fact of having more time to mark the tests on the lecturer’s own time, without the rush of having to assess one student after the other, contributed to a fairer marking when assigning grades since each answer was listened more carefully, and a comparative grading scale could be established before assigning the final mark for each student.
4. Other positive outcomes. A pleasing and surprising outcome was that all the students’ answers to the question “What are you like?” (a traditional question to which students need to respond using vocabulary and the verb ‘to be’, a tricky verb in Spanish) were really positive. Answers such as “I’m intelligent”, “I’m hardworking”, “I’m good looking” were the norm, in opposition to the usual ones in a face-to-face situation: “I’m dumb”, “I’m ugly”, “I’m lazy, ha ha ha”. This says much about how a relaxing environment can provide them with more personal confidence and even boost their self-esteem!

All in all, the students were more spontaneous, they did not have a bad time, as it happens for many in a face-to-face oral test situation, and it actually seemed that they had a lot of fun video recording themselves and looking at themselves on the screen.

What did the end result look like? See below:

The e-Learning support team is hosting a number of events for UC Teaching Month. This presents a great opportunity to learn about what is happening in e-Learning at UC and internationally. All are invited to attend.

Events include:

Relearning E-Learning
Wednesday, 17 July 2019: 1:30- 3PM, Rehua 103 (no registration required)

UC lecturers share their technology enabled teaching practices that support student learning and success. Come along if you are looking to try something new in your technology enabled teaching.

Speakers include:

• Tanja Mitrovic – Video based learning.
• Kate Pedley – Using Echo360 to engage students.
• Jerry Maroulis (Wageningen University) – To MOOC or not to MOOC.
• Viktoria Papp – Digital note taking vs handwritten notes and their effect on student achievement.\
• Victoria Escaip – Using online video for language assessment.

Teaching Month Online Seminars and Workshops

Unbundling University Project
Online: Wednesday, 10 July 2019 (free – no registration required).
The seminar will be live streamed via zoom and all are welcome to attend.

Professor Laura Czerniewicz (Director, Centre for Innovation in Learning and Teaching (CILT) at University of Cape Town) is presenting at Victoria University, Wellington from 3-4pm on her Unbundling University project. Laura is speaking about the Unbundled University project which examines the profound confluence which constitutes the unbundled university – the intersection of increasingly disaggregated curricula and services, the affordances of digital technologies, the growing marketization of the higher education sector itself and the deep inequalities which characterise both the sector and the contexts in which they are located.

Information about the seminar can be found at
https://hail.to/flexiblelearningNZ/article/5ChDA7r

The link to the seminars online meeting space is
https://vlnprimary.zoom.us/j/958518941

ACODE 79: Supporting Universities for Future Qualifications’ & Pedagogical Models
Online 15-26th July: free.

The Australasian Council on Open, Distant & e-Learning (ACODE) is conducting a fully online workshop over 15-26th July

Led by a team of ACODE facilitators working with international speakers, the workshops will explore the following questions and reflect on how we can work to shape future planning and support for learning and teaching.

1. How are universities structuring their activities and offerings and what challenges does this present for those enabling technology enhanced learning?
2. How are university-vendor relationships changing in the sector and how will this affect our aspirations and capabilities for the future of technology enhanced learning?
3. How are universities defining their places as higher education institutions in diverse and competing contexts? How can technology help maintain a community of learners and scholars while also supporting scale and access?

More information at https://www.acode.edu.au/course/view.php?id=3#section-1

Register at https://docs.google.com/forms/d/e/1FAIpQLSeEVg14kPWhf6-hLr4mzhFq_GR4zdi27rbV1okyB7DF_DVjEw/viewform

Information about all UC Teaching Month Events can be found at https://intranet.canterbury.ac.nz/academicservices/Teaching%20Month.shtml