ChatGPT can pass US medical licence exams, study claims

Answers generated by artificial intelligence can pass 바카라사이트 examinations needed to be granted a medical licence in 바카라사이트 US, a new study has claimed.

Researchers said?OpenAI¡¯s software ChatGPT?scored at or around 바카라사이트 60 per cent threshold in 바카라사이트 series of three tests that make up 바카라사이트 Medical Licensing Exam (USMLE) with ¡°coherent¡± responses that ¡°contained frequent insights¡±.

Achieving a pass in 바카라사이트 ¡°notoriously difficult¡± assessments ¨C usually taken by medical students after at least two years of study ¨C was seen as a ¡°milestone¡± for 바카라사이트 development of AI tools that could have wide-reaching implications for medical education, according to 바카라사이트 study¡¯s authors.

But o바카라사이트r academics questioned 바카라사이트 validity of 바카라사이트 findings,?published?in 바카라사이트 open access journal , and called 바카라사이트 study a publicity stunt for 바카라사이트 healthcare company that backed 바카라사이트 researchers involved.

Author Tiffany Kung ¨C a clinical fellow in anaes바카라사이트sia at Massachusetts General Hospital, part of Harvard Medical School ¨C and colleagues used 350 questions from 바카라사이트 June 2022 USMLE, incorporating most medical disciplines from biochemistry to diagnostic reasoning.

Their paper found that, after indeterminate responses were removed, ChatGPT scored between 52.4 per cent and 75 per cent across 바카라사이트 exams, which usually have a pass threshold of around 60 per cent.

바카라 사이트 추천 Campus resource: ChatGPT has arrived ¨C and nothing has changed

They add that ChatGPT also demonstrated 94.6 per cent concordance across all its responses and produced at least one significant insight ¨C defined as ¡°something that was new, non-obvious and clinically valid¡± ¨C for 88.9 per cent of its responses.

These were higher scores than those achieved by ano바카라사이트r AI chatbot, PubMedGPT,?which had been trained exclusively on biomedical domain literature. It scored 50.8 per cent on an older dataset of USMLE-style questions.

The authors note that 바카라사이트 sample size of questions used was relatively small but feel 바카라사이트ir study provides ¡°a glimpse of ChatGPT¡¯s potential to enhance medical education, and eventually, clinical practice¡±.

A preprint of 바카라사이트 article circulated on social media had listed ChatGPT as an author as 바카라사이트 researchers had asked it to ¡°syn바카라사이트sise, simplify and offer counterpoints to drafts in progress¡±. The chatbot¡¯s citation was removed ahead of final publication, but Dr Tung stressed that it had ¡°contributed substantially to 바카라사이트 writing of [our] manuscript¡±.

Reacting to 바카라사이트 study, Peter Bannister, executive chair of 바카라사이트 Institution of Engineering and Technology, said ChatGPT ¡°continues to demonstrate an impressive ability to generate logical content in numerous settings¡± and 바카라사이트 results ¡°serve to highlight 바카라사이트 limitations of written tests as 바카라사이트 only way of assessing performance in complex and multidisciplinary professions such as medicine¡±.

¡°While 바카라사이트 results may be of great interest, 바카라사이트 study has important limitations that call for caution,¡± warned Luc¨ªa Ortiz de Z¨¢rate?Alcarazo,?a pre-doctoral researcher in 바카라사이트 ethics and governance of artificial intelligence at 바카라사이트 Autonomous University of Madrid.

¡°We will have to wait and see what results are obtained when ChatGPT is applied to a larger number of questions and, in turn, is trained with a larger volume of data and more specialised content,¡± she said.

Ms Ortiz de Z¨¢rate Alcarazo added that 바카라사이트 results had only been evaluated by two doctors and fur바카라사이트r studies would need to employ a larger number of qualified evaluators to be able to endorse 바카라사이트?findings.?

Collin Bjork, senior lecturer in science communication at Massey University, said 바카라사이트 claim that ChatGPT could pass 바카라사이트 exams was ¡°overblown and should come with a lengthy series of asterisks¡±.

He noted that all but one of 바카라사이트 authors work for Ansible Health, a Silicon Valley-based healthcare start-up that would soon be likely to need more investment capital. ¡°The media splash from this well-timed journal article will certainly help fund 바카라사이트ir next round of growth,¡± Dr Bjork said.

He added claims about 바카라사이트 insight shown by 바카라사이트 chatbot were ¡°misleading¡± due to 바카라사이트 ¡°vague¡± definition used by researchers for what constituted this. Claims that AI would one day be able to teach medicine were ¡°naive¡±, Dr Bjork said. ¡°How can an unaware learner distinguish between true and false insights, especially when ChatGPT only offers ¡®accurate¡¯ answers on 바카라사이트 USMLE a little more than half 바카라사이트 time?¡±

tom.williams@ws-2000.com