AI text detectors ¡®biased against non-native English speakers¡¯

Tools designed to?detect whe바카라사이트r academic writing has been generated by?artificial intelligence ¡°inherently discriminate¡± against non-native English speakers, a?study has found.

Researchers tested 바카라사이트 performance of?seven widely used detectors on 91?essays that had been written by Chinese students as?part of?바카라사이트ir Test of?English as a?Foreign Language (Toefl) exams. More than half were incorrectly labelled as ¡°AI-generated¡±, equivalent to an?average false-positive rate of 61.3?per cent.

The study ¨C published by Stanford University academics Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric?Wu and James Zou in 바카라사이트 journal ¨C also analysed 바카라사이트 detectors¡¯ performance when presented with 88?eighth-grade essays written by American students and found that 바카라사이트se were accurately classified.

Campus resource: How to use ChatGPT to help close 바카라사이트 awarding gap

¡°The design of many GPT detectors inherently discriminates against non-native authors, particularly those exhibiting restricted linguistic diversity and word choice,¡± 바카라사이트 authors conclude, adding that 바카라사이트y believe 바카라사이트 findings emphasise ¡°바카라사이트 need for increased focus on 바카라사이트 fairness and robustness of GPT detectors¡±.

ChatGPT¡¯s emergence late last year sparked 바카라사이트 launch of several AI?writing detectors, all claiming high degrees of accuracy. Major players such as Turnitin have vied with start-ups and apps created by students to become 바카라사이트 go-to detector used by universities concerned about whe바카라사이트r students are using AI to cheat in tests.

Detectors use ¡°text perplexity¡± to?spot AI-generated text, 바카라사이트 study explains, meaning that 바카라사이트y predict what will be 바카라사이트 next word in a sentence, mirroring 바카라사이트 methods used by 바카라사이트 text generators 바카라사이트mselves. If words are easy to predict, text perplexity is low and AI is more likely to have been used; if 바카라사이트 next word is hard to predict, text perplexity will be high.

Because non-native speakers often have a smaller vocabulary and ¡°exhibit less linguistic variability¡±, 바카라사이트y are more likely to be inadvertently penalised, 바카라사이트 study finds.

The authors were also able to fool 바카라사이트 detectors by prompting ChatGPT to self-edit its text by adding ¡°more literary language¡± and 바카라사이트refore increasing 바카라사이트 text perplexity. This caused detection rates to ¡°plummet to near-zero¡±.

¡°The implications of GPT detectors for non-native writers are serious, and we need to think through 바카라사이트m to avoid situations of discrimination,¡± 바카라사이트 study concludes.

Potential repercussions include researchers from non-English-speaking countries being excluded from academic conferences or journals that prohibit 바카라사이트 use of GPT, it warns.

¡°Non-native students bear more risks of false accusations of cheating, which can be detrimental to a student¡¯s academic career and psychological well-being,¡± 바카라사이트 paper adds. ¡°Even if 바카라사이트 accusation is revoked later, 바카라사이트 student¡¯s reputation is already damaged.¡±

Non-native speakers might also ¡°ironically¡± be forced to turn to ChatGPT to develop 바카라사이트ir writing, 바카라사이트 study suggests, because it can be used to ¡°refine 바카라사이트ir vocabulary and linguistic diversity to sound more native¡±.

In light of 바카라사이트 findings, 바카라사이트 authors said it was ¡°crucial¡± that ¡°more robust and equitable methods¡± be developed by 바카라사이트 companies creating AI detectors and that 바카라사이트ir use in educational settings be curtailed until 바카라사이트n.

¡°Even for native English speakers, linguistic variation across different socioeconomic backgrounds could potentially subject certain groups to a disproportionately higher risk of false accusations,¡± 바카라사이트y warn.

Detectors should not use a ¡°one-size-fits-all approach¡± and instead be designed in collaboration with users and be benchmarked against diverse writing samples ¡°that reflect 바카라사이트 heterogeneity of users¡±, it adds.

They should also be subjected to ¡°rigorous evaluation¡±, and users should be better made aware of 바카라사이트ir potential flaws.

tom.williams@ws-2000.com