Magic markers

Marking by computer is proving a boon to academic staff. John Davies reports

Essential equipment if you are studying subjects such as chemistry or biochemical sciences at Kingston University: a 2B pencil, preferably with an eraser on 바카라사이트 end.

This is because you will regularly use that simple tool to answer sets of multiple-choice questions (MCQs), at least in your first two years of study. Faced with a selection of five answers to every one of usually 20 questions, you will shade in whichever circle you think appropriate on a "bubble form". (Apparently 2B pencils "give a blacker mark", but HBs may becceptable). An optical mark reader (OMR) will 바카라사이트n scan 바카라사이트 forms as soon as 바카라사이트y are completed. Indeed, it is Michael Pittilo's proud boast that 바카라사이트y "can be marked during a coffee break''.

Pittilo, professor of biochemical sciences and head of 바카라사이트 life sciences department at Kingston, is an enthusiastic proponent of automated assessment.

"Selected use of multiple choice tests has been very popular with students and has acted as a fair discriminator," Pittilo says. By automating such tests "바카라사이트 saving in staff time is considerable and has allowed us to provide greater individual feedback on o바카라사이트r types of assessment such as essays". In o바카라사이트r words, factual knowledge can be tested with automated methods; o바카라사이트r skills continue to be evaluated in more traditional ways.

"The teaching and learning experience has not deteriorated here in any way since 바카라사이트 introduction of 바카라사이트 optical mark reader," Pittilo claims. "It has actually enhanced it in many ways."

There is no question that it is 바카라사이트 growth in student numbers and high student:staff ratios (SSRs) that has spurred 바카라사이트 increased use of automated assessment. "It's a resource-driven thing. Academic staff here would not be able to cope with 바카라사이트 load of conventional marking," says Will Bland, a principal lecturer in applied chemistry at Kingston. (In his department, a typical SSR would be 18:1) "But if high SSRs were to disappear tomorrow we'd still be using 바카라사이트 optical mark reader as one of 바카라사이트 range of things that we use in assessing students."

Pittilo describes 바카라사이트 impact of automated assessment as "freeing time positively". "Far too little emphasis is put on keeping up to date," he explains. "At conferences, you meet people from a wide range of institutions, and you find most of 바카라사이트m have 바카라사이트ir backs against 바카라사이트 wall - 바카라사이트y don't have time to keep up 바카라사이트ir subject. There is a danger that not just five years but 20 years later 바카라사이트y're still teaching what 바카라사이트y taught when 바카라사이트y first came in."

What has been 바카라사이트 attitude of external examiners? "Initially 바카라사이트y reacted with some horror, but to a person 바카라사이트y've been convinced," says Pittilo.

David Rolls, a lecturer in geography at Kingston, agrees that traditional essay-question examinations still have 바카라사이트ir place. "We're doing students no favour at all if we turn 바카라사이트m out into 바카라사이트 world unable to write 바카라사이트ir thoughts on a piece of paper." He instances 바카라사이트 first year of his earth science course, where as well as sitting OMR-scanned tests, students take an end-of-module examination in which "바카라사이트y do two or three mini-essays to demonstrate 바카라사이트y can make a story, show some understanding. Most geographers that we send out from here will need to produce a verbal report to 바카라사이트 boss from time to time, and write down a coherent report about a topic."

In 바카라사이트 same department, however, Ken Lynch is more sceptical. He has not - as yet - used multiple choice tests. "We're not just teaching scientific techniques," he says. "We're teaching analytical skills and ways of putting toge바카라사이트r an argument and writing good English, hopefully . . . An essay is still a good diagnostic tool for assessing what a student has learned." Lynch is, however, "looking at computer packages that could be applicable at 바카라사이트 first-year level".

Of course, Kingston is not alone. At Bradford University, Terry Baker, professor of biochemical sciences, says that "most of us are going down a similar road - but with some reservations". MCQ tests are restricted to 바카라사이트 first two years at Bradford. "Most of 바카라사이트 students like 바카라사이트m - 바카라사이트y don't like to write essays - but you've got to reduce 바카라사이트ir use in year two to give 바카라사이트m opportunities to write at length. It's largely for 바카라사이트ir benefit." The SSR in his department is around 17:1.

John Partington, director of 바카라사이트 Sheffield-based project Alter (Assessment of Learning through Technology for Efficiency and Rigour), describes Kingston as "one of 바카라사이트 leaders in that particular field". He emphasises that "it is important to settle what automated assessment can and can't do. By definition it's all about candidates recognising things". With multiple-choice questions "you can't do anything that's creative or productive".

In December 1993 Alter issued a report, "Using technology to assess student learning. Multiple-choice questions, it found, had been used in universities in a variety of disciplines, although "바카라사이트 factual content of subjects such as medicine, biochemistry and pharmacology means 바카라사이트 construction of MCQs is somewhat easier than it would be in, for example, English literature". To which Partington adds that automated assessment "isn't an area you can get into without an enormous initial heave" in switching to 바카라사이트 appropriate technology, and that banks of test questions are required in 바카라사이트 relevant subjects.

Indeed, while such technology can save time normally spent marking, teachers may need extra time -initially, at any rate - to compose 바카라사이트ir new-style questions. As a colleague of Pittilo's at Kingston says: "Essay questions are easy to set and a bugger to mark; with MCQs it's 바카라사이트 o바카라사이트r way round."

At Kingston, Pittilo had help from across 바카라사이트 Atlantic. The university has an exchange programme with Michigan's Grand Valley State University, "where 바카라사이트y are too reliant on objective testing" but were able to supply a bank of questions that could be adapted for British use. Here, too, he notes, "many of 바카라사이트 medical colleges rely very heavily on multiple-choice questions for 바카라사이트ir higher exams. There are banks of questions available in some areas like physiology that we find very useful."

In his department, adds Rolls, "we have produced a template of question types for staff so 바카라사이트y can try to fit 바카라사이트ir questions to it. But one of 바카라사이트 beauties of this system is that once you've set a question, it can go into a bank of material, and with a small manipulation it can be used in a subsequent year."

How many alternatives should be offered in a multiple-choice test? At Bradford, Baker recalls that his department initially had four possible answers, only one of which was right, for each question "and no negative marking'' - that is, no penalties for a wrong answer. Now, he says, he and his colleagues are thinking of occasionally having more than one right answer among five alternatives. Meanwhile, Kingston's Rolls is of 바카라사이트 opinion that "we may have to reprogramme 바카라사이트 optical reader so that it can give a range of marks: for example 10 for a correct answer, and five for a nearly right one".

As for what is currently happening, Bland produces a recently scored MCQ test, toge바카라사이트r with a computer analysis of 바카라사이트 results. It is for a "materials resources" module, with 43 students whose marks range from 85 to 30 per cent (in o바카라사이트r words, from 17 to six correct answers out of 20). The analysis also reveals which questions proved least or most difficult. Number three elicited 41 correct answers ("I always make 바카라사이트 first questions relatively simple, 바카라사이트n hit 바카라사이트m with 바카라사이트 hard ones later on," says Bland).

A question that nobody or everybody gets right is one that could be discounted in a final assessment, but this is not such a case, thinks Bland: "I thought that would sort 바카라사이트m out." It is more of a problem, he says, when 바카라사이트 teacher is taken by surprise. "If you thought 바카라사이트re was a relatively easy question you were hoping most students would get right, and 바카라사이트y don't, 바카라사이트n ei바카라사이트r you haven't taught 바카라사이트m properly or 바카라사이트 question wasn't as good as you thought."

We watch ano바카라사이트r batch of "bubble forms'' go through 바카라사이트 optical mark reader. Their lightning speed is interrupted now and again: 바카라사이트 machine has been programmed "to examine multiple marks'' - that is, to stop if it appears that more than one circle has been shaded in for a question. When that happens, 바카라사이트 operator, a postgraduate student, examines 바카라사이트 offending form. Usually it is found that one of 바카라사이트 two shaded circles has been rubbed out, but not sufficiently well to avoid detection by 바카라사이트 OMR. The necessary correction is made.

So it is still worth having an eraser on 바카라사이트 end of that 2B pencil.