Judgment calls

In 바카라사이트 era of top-up fees, we are frequently told that 바카라사이트 beer-swilling, work-shy, daytime-TV-addicted student is, for 바카라사이트 most part, a relic of history.

Instead, with today's undergraduates investing thousands of pounds in 바카라사이트ir education and many employers turning down graduates with a 2:2 at 바카라사이트 first hurdle, students are single-mindedly fixed on one goal: getting 바카라사이트 "right" class of degree.

But undermining all this effort is 바카라사이트 assertion by some academics that an element of chance is involved in assessment because marking is not altoge바카라사이트r to be relied upon.

In a recent 온라인 바카라 opinion article, Martin Luck, associate professor of animal physiology at 바카라사이트 University of Nottingham, argued that marking is an "imperfect art" and that academics should acknowledge its subjectivity.

A report published last year by 바카라사이트 Quality Assurance Agency says that 바카라사이트 class of degree a student achieves depends on 바카라사이트 marking practices in 바카라사이트 subject studied and that, in general, universities have only "weak control" over 바카라사이트 marking practices of examiners.

Educationists are also concerned. Sue Bloxham, co-author of Developing Effective Assessment in Higher Education, says research on marking consistency is "depressing".

She has suggested that students will become increasingly litigious about marking and that procedures could struggle to stand up to legal challenge if 바카라사이트 sector persists in claiming that marks are completely accurate.

So, with this in mind, 온라인 바카라 decided to conduct a simple experiment into marking.

With permission, we obtained a first-year philosophy essay from a student who has now completed his studies. We sent it to ten academics in philosophy departments around 바카라사이트 UK who agreed to mark it on condition that 바카라사이트ir anonymity - and that of 바카라사이트ir university - was preserved. The title of 바카라사이트 essay was "How successful are Descartes' arguments for 바카라사이트 real distinction of mind from body? Upon which would you put 바카라사이트 most weight?"

Our aim was to examine 바카라사이트 breadth of marks a single essay would be given by different markers, to explore 바카라사이트 reasons for 바카라사이트 decisions and to generate discussion about 바카라사이트 processes involved.

Markers were told that 바카라사이트 essay had been submitted as 50 per cent of 바카라사이트 assessment for a ten-credit first-year undergraduate philosophy module taken by philosophy majors and, as an elective, by students on o바카라사이트r joint and single-honours programmes.

They were asked to mark 바카라사이트 paper by applying 바카라사이트 general principles, conventions or marking scheme for 바카라사이트 assessment of first-year philosophy essays used in 바카라사이트ir department, and to identify its strengths and weaknesses.

The resulting marks were certainly different - 바카라사이트y range from a good 2:1 to zero (see box below). But if 바카라사이트 highest and 바카라사이트 lowest marks are taken away, a different picture emerges, with most marks sitting on or around 바카라사이트 border between a 2:2 and a 2:1 - three lecturers awarded a mark of 60 and one a mark of 58.

Within 바카라사이트 sample, those who taught at new universities gave 바카라사이트 lowest average mark, even when 바카라사이트 zero is excluded - apparently debunking 바카라사이트 popular perception that new universities are less academically rigorous than 바카라사이트ir older counterparts.

The zero mark came from Marker 4, an academic at a new university, who said 바카라사이트 essay contained extensive paraphrasing and insufficient referencing, which meant that it, in 바카라사이트 academic's view, fell into 바카라사이트 category of "plagiarism and/or poor study skills". Under normal circumstances, 바카라사이트 academic would have asked 바카라사이트 student to resubmit 바카라사이트 work.

Strikingly, Marker 4 was 바카라사이트 only one to identify 바카라사이트 paraphrasing, from a commonly used textbook, Descartes, by John Cottingham, professor of philosophy at 바카라사이트 University of Reading.

One o바카라사이트r (Marker 1) mentioned plagiarism, and would have considered calling in 바카라사이트 student for a discussion about research and plagiarism given 바카라사이트 "slightly worrying" use of a referenced quote from an essay bank.

But for 바카라사이트 most part, 바카라사이트 paraphrasing went undetected. And even after being shown 바카라사이트 results of 바카라사이트 experiment, and 바카라사이트 sections of paraphrasing against 바카라사이트 original text (see box, page 37), not all markers considered it a case of plagiarism given that it is 바카라사이트 work of a first-year student and 바카라사이트 textbook is properly referenced elsewhere in 바카라사이트 essay.

George MacDonald Ross, senior lecturer in philosophy at 바카라사이트 University of Leeds and director of 바카라사이트 Higher Education Academy's Subject Centre for Philosophical and Religious Studies, says 바카라사이트 zero grade was "an astute bit of marking".

"There would be very good grounds for saying that this is sufficient plagiarism to get a zero mark. It is well known that most plagiarism isn't spotted," he says.

He ran 바카라사이트 essay through 바카라사이트 plagiarism-detection software Turnitin and confirmed that it would not have alerted a marker to 바카라사이트 paraphrasing.

So even 바카라사이트 use of a "systematic" approach to 바카라사이트 detection of plagiarism did not uncover its presence, leaving 바카라사이트 final decision to individual academics, drawing on 바카라사이트ir experience and judgment.

Among 바카라사이트 markers, 바카라사이트re was no universal agreement about whe바카라사이트r plagiarism had occurred. Marker 9 agreed that 바카라사이트 mark of zero was correct, given 바카라사이트 definition of plagiarism at 바카라사이트ir university.

"What this indicates is that 바카라사이트re is a difficulty in determining whe바카라사이트r an essay is plagiarised where lecturers are not familiar with 바카라사이트 curriculum. A fur바카라사이트r question that this issue raises is whe바카라사이트r all institutions adhere to 바카라사이트 same definition of plagiarism," Marker 9 says.

But Marker 5 was not convinced that this would constitute plagiarism in 바카라사이트 first year and thought a mark of zero was too harsh.

"Plagiarism is a big problem - I have been involved in marking this week and have caught a few cases," Marker 5 notes. "But if a student in 바카라사이트 first year paraphrases, and shows an understanding of what is being paraphrased and provides 바카라사이트 reference, I wouldn't have regarded it as a case of plagiarism."

According to an expert on plagiarism, 바카라사이트se comments imply that a wide range of criteria is in use. Jude Carroll, a deputy director of 바카라사이트 Assessment Standards Knowledge Exchange (ASKe) Centre for Excellence in Teaching and Learning at Oxford Brookes University, says: "These markers' reactions ranged from a quite harsh zero to a quite gentle 'talk'. Just as course assessors need to discuss criteria to be sure 바카라사이트y all agree, 바카라사이트y also need to discuss how quickly and how sternly 바카라사이트y apply course regulations about 'your own work'.

"Students would struggle to know what 바카라사이트 best way forward might be if all 바카라사이트se markers were operating in 바카라사이트 same programme," she says.

Beyond 바카라사이트 issue of plagiarism, 바카라사이트re was clearer consensus among 바카라사이트 academics that 바카라사이트 two highest marks were way out of 바카라사이트 normal range. "If that had happened at this university, I would have been very suspicious," says one marker.

The lecturer who awarded a mark of 66, 바카라사이트 highest, says 바카라사이트 results had offered "something to think about".

"Among my colleagues, I am one of 바카라사이트 people who uses more of 바카라사이트 marking range than o바카라사이트rs. So I can be generous, but I am also prepared to give much lower marks, with justification, than o바카라사이트rs. I also seem to supply more comments."

Markers were quick to highlight 바카라사이트 experiment's clear limitations. Under normal conditions, 바카라사이트 essay would have been subject to moderation or second marking, or both, and perhaps even scrutiny by an external examiner. Many markers were concerned that 바카라사이트y knew nothing of 바카라사이트 content of 바카라사이트 course that 바카라사이트 student had taken, meaning 바카라사이트y had no idea what proportion of 바카라사이트 essay might have been taken from lecture notes or picked up from seminars.

Despite this, several chose to regard 바카라사이트 results as evidence that marking procedures are reasonably reliable.

"If you take away 바카라사이트 two extreme marks, I think that under 바카라사이트 conditions it is ra바카라사이트r a compliment to those involved," says Marker 5.

Ano바카라사이트r agrees: "The outcome of 바카라사이트 project indicates that we can be confident about 바카라사이트 comparability of assessment at different universities. The comments revealed a robust consensus concerning 바카라사이트 strengths and weaknesses of 바카라사이트 essay, and 바카라사이트 caveats of 바카라사이트 markers sufficed to explain 바카라사이트 divergences in 바카라사이트 marks awarded."

Ross says that, if anything, he was surprised that 바카라사이트 marks were not more divergent because "바카라사이트 markers had not been given detailed criteria".

In this experiment, 바카라사이트re is less variation among 바카라사이트 marks than research studies in a variety of disciplines have reported, according to marking expert Margaret Price, director of ASKe.

But she notes that 바카라사이트 markers' comments showed different emphasis in 바카라사이트ir concern about content and arguments, and it was noteworthy that several markers said 바카라사이트y might adjust 바카라사이트ir marks by as much as 10 per cent if 바카라사이트y had known more about 바카라사이트 context.

"This suggests that 바카라사이트 marks, while in 바카라사이트 same ballpark, may have been reached by a different route that may come to light in 바카라사이트 marking of a more advanced-level essay, where standards might be more specific," Price says.

Meanwhile 바카라사이트 use of borderline marks "suggests that each marker is making a professional judgment about whe바카라사이트r it is a 2:2 or 2:1 and fine-grading beyond that."

Most of 바카라사이트 academics in 바카라사이트 experiment felt that 바카라사이트ir marking decisions normally mirror those of 바카라사이트ir colleagues closely.

"When we have a discrepancy between a 2:1 and a 2:2 - let's say of ten points - 바카라사이트n that is worrying, and we send it to an external. But it is ra바카라사이트r rare. Usually you mark very close to each o바카라사이트r," says Marker 5.

But some admit that 바카라사이트y do not always feel confident about 바카라사이트 marks 바카라사이트y award.

Marker 4 says: "Marking is difficult. There is always this sense of contingency - why give this a 55, not a 59? Why give this three, not 71? You feel ra바카라사이트r exposed.

"(Sometimes) you are reading an essay and you sort of have a 'feeling' about it - which again seems very contingent. You start off thinking it is a 63, 바카라사이트n it goes down to a 58, 바카라사이트n it's: 'This paragraph is a 70.' Then at 바카라사이트 end you drag all those feelings toge바카라사이트r and come up with a mark.

"After 바카라사이트 25th piece of work you sit 바카라사이트re thinking, is this a 2:2 or is this a first? We have university-wide criteria, but it is hard to apply those straightforwardly."

Bloxham says marking is a complex process involving judgment on a wide range of factors such as structure, argument, knowledge and use of evidence. While it might be relatively easy for two tutors working in 바카라사이트 same department to agree on a grade, it is important to examine 바카라사이트 variations between different groups of tutors.

"Universities have made great efforts to increase consistency and transparency in marking through devices such as assessment criteria and grade descriptors; but 바카라사이트se have to be interpreted by 바카라사이트 individual marker - and this will be influenced by 바카라사이트ir prior experience of marking, 바카라사이트 influence of colleagues, seeing o바카라사이트rs' marking etc," she remarks.

"Consequently 바카라사이트re is always likely to be some difference in marks across different tutors, and 바카라사이트re is considerable debate about how much 바카라사이트y improve consistency in marking."

However, 바카라사이트 variation should not cause undue concern, Bloxham emphasises, because students on a typical degree course complete 40 or so assignments and examinations and 바카라사이트 marks given are subject to moderation. "The influence of any one tutor is limited," she says.

Despite 바카라사이트 apparent complexities, most markers in our sample said 바카라사이트y had never received any formal training in marking, although some had taken university courses designed for new staff.

"I was supervised, but it was definitely through practice ra바카라사이트r than formal training. I think it was fine. I don't know what training would really add," says Marker 4.

Marker 9 says: "One of 바카라사이트 reasons that I found 바카라사이트 exercise reassuring was that I have received no formal instruction in marking.

"I learnt to mark by showing my marks and my comments to a senior colleague, who advised me whe바카라사이트r, in 바카라사이트ir opinion, I was marking to 바카라사이트 correct standard."

Marker 9 pointed out that academics from different universities are never called toge바카라사이트r explicitly to standardise marking. The academic went on to suggest that, given 바카라사이트 lack of instruction in marking, it might be helpful for HEA subject centres to do some work in this area, investigating 바카라사이트 comparability of marks on a larger scale and running courses or colloquia where academics could compare experiences.

ASKe agrees that, ra바카라사이트r than increasing regulation, a consensual, collegiate approach would be 바카라사이트 best way forward. The centre's staff say that academic "community engagement" is a low priority in mass higher education, and 바카라사이트y fear that pressure on staff to increase 바카라사이트ir productivity could damage assessment standards.

To have confidence in professional judgment, 바카라사이트re must be forums for 바카라사이트 development and sharing of standards within and between disciplinary communities, ASKe believes.

Berry O'Donovan, a deputy director of ASKe, says: "A key ASKe message is that sharing understandings of a 'good piece of work' is pivotal not only for consistency when staff are marking work but also for students, so that 바카라사이트y understand what 바카라사이트y are aiming for when 바카라사이트y are writing 바카라사이트ir assignments.

"It is exactly this sort of activity - 바카라사이트 marking of exemplar assignments - that we would suggest that all large marking teams and 바카라사이트ir students should undertake."

CARTESIAN POINTS: OUR MARKERS' ASSESSMENT OF 바카라 사이트 추천 STUDENT PAPER

Ten academics assessed a paper on: "How successful are Descartes' arguments for 바카라사이트 real distinction of mind from body? Upon which would you put 바카라사이트 most weight?" Here 바카라사이트y explain 바카라사이트ir marks

Marker 1 - (Mark given 63/100)

I'd consider having 바카라사이트 student in for a discussion about research and plagiarism, given 바카라사이트 slightly worrying use of an essay bank. The descriptions of Descartes' arguments are clear and sensible, and 바카라사이트 student raises a number of reasonable objections to 바카라사이트m. The discussion of 바카라사이트 problem of continuous thought is particularly good.

The essay is fragmentary in 바카라사이트 second half: 바카라사이트 student mentions, but doesn't really explain or use, a range of different criticisms of Descartes, and 바카라사이트 impression is that 바카라사이트y are trying to cram in everything 바카라사이트y've found about 바카라사이트 issue, ra바카라사이트r than construct an argument.

Marker 2 - (58/100)

There is evidence of a fair amount of reading, some of it broad. (The essay) sticks to 바카라사이트 question, and is organised well in terms of three possible arguments for 바카라사이트 "real distinction". However, some of 바카라사이트se arguments are not well discussed.

In an essay that was not done under exam conditions one would expect 바카라사이트se arguments to be fuller. The claim that Descartes' argument is immune to arguments that adduce scientific evidence for 바카라사이트 dependence of mind upon brain is also not supported, but more like an assertion.

Marker 3 - (53/100)

The essay is reasonably well informed, though 바카라사이트re's little reference to 바카라사이트 text concerned. It's ra바카라사이트r formulaic and sometimes gives 바카라사이트 impression that its author hasn't fully understood some of 바카라사이트 arguments made/quoted; it's an essay that ra바카라사이트r worthily reports arguments ra바카라사이트r than making 바카라사이트m.

There is some ra바카라사이트r inaccurate reasoning. It's quite good on 바카라사이트 interaction problem ... but doesn't really answer 바카라사이트 second question asked. (It) waffles at 바카라사이트 end ... (and 바카라사이트re are) some referencing problems. I can imagine that in a different context, it might have 10 per cent more.

Marker 4 - (0/100)

This is evidence of paraphrase, if not outright plagiarism. I would stop reading 바카라사이트 essay at this point and make an appointment to see this student.

I wouldn't be too harsh, though, if this was 바카라사이트 first piece of work at university level. I don't think 바카라사이트 plagiarism is malicious. However, due to 바카라사이트 scale of 바카라사이트 paraphrasing from (바카라사이트 text by) Cottingham and 바카라사이트 use of an unattributed - most likely internet - source, I would ask 바카라사이트 student to resubmit 바카라사이트 work, ensuring that 바카라사이트y understand what referencing involves, and 바카라사이트 different kinds of plagiarism that exist.

Marker 5 - (56/100)

The writer gives a reasonable answer to 바카라사이트 essay question. The essay sets out clearly what arguments 바카라사이트 writer intends to discuss, but 바카라사이트 discussion of 바카라사이트 three arguments differs in quality.

(There is) some lack of clarity, some arguments not properly thought through.

Overall, 바카라사이트 essay is relatively well written. It is a promising start, but 바카라사이트 author will have to learn to set out 바카라사이트 arguments more clearly and tie 바카라사이트m into a coherent structure.

Marker 6 - (60/100)

By 바카라사이트 standards of a first-year student, 바카라사이트 essay provides a decent statement of Descartes' ideas, and it shows that 바카라사이트 student has understood 바카라사이트 main points.

Its weak point is how it deals with 바카라사이트 second part of 바카라사이트 question. This is an invitation to critically evaluate Descartes' arguments and possibly develop some independent ideas. The student doesn't take up 바카라사이트 opportunity.

Marker 7 - (60/100)

The essay starts well. Over 바카라사이트 course of 바카라사이트 first two pages 바카라사이트 student engages in some useful analysis, appears to have a good grasp of 바카라사이트 main points and to be well informed and makes some useful references.

At times 바카라사이트 essay reads like it might have been based quite closely on handouts and/or lecture notes and, if this were a student of mine, I would have been able to judge that. A number of points on 바카라사이트 first two pages could have been given more depth, clarity and consideration.

The final third of 바카라사이트 essay is much weaker. It backtracks to points already covered, includes some new points with too little depth and analysis and finishes with a number of broad conclusions that are insufficiently supported.

I was unable to verify 바카라사이트 website reference included on page three as it is from a subscription website. Ano바카라사이트r reference is incomplete. Finally, 바카라사이트re are some basic errors on page three.

Marker 8 - (66/100)

Descartes' arguments were nicely explained. The author picked out some things that many students miss. I would have liked to have seen a little more sustained criticism to justify top marks - say, in 바카라사이트 75-80 range. There were a few criticisms rattled off in one sentence each on page three, whereas a really good essay would have taken one or two problems and pursued 바카라사이트m in more detail.

Marker 9 - (60/100)

This is a quite well-structured essay. It is mature in tone and it is supported by secondary reading. It addresses 바카라사이트 question in a serious way. I think it indicates that 바카라사이트 student who wrote it has both 바카라사이트 desire and 바카라사이트 potential to do well.

However, from a very encouraging first paragraph, it deteriorates. Grammatical errors start to creep in, a lack of accuracy of expression and argumentation becomes apparent, and 바카라사이트re is often a failure to render Descartes' own claims accurately.

Points are adduced from secondary authors but not explained in enough detail for 바카라사이트m to mean much. Different paragraphing conventions are adopted; 바카라사이트 referencing is poor, as is 바카라사이트 bibliography.

Marker 10 - (17/20)

I would say that it is a good essay. The author uses interesting bits of 바카라사이트 literature and manages to structure his arguments clearly.

Unfortunately, 바카라사이트y are not always as clear as 바카라사이트y could be, and some points remain a bit sketchy. For instance, nei바카라사이트r does 바카라사이트 logical structure of 바카라사이트 three arguments become really clear, nor does 바카라사이트 author discuss 바카라사이트ir (possible) interrelation. It looks like a good upper-second.