A recent survey showed that ChatGPTadvanced language model, can be successful in US Medical Licensing Exam (USMLE), although his estimate is not considered exceptional. A study by AI and medical experts examined ChatGPT’s ability to answer complex and difficult medical questions and concluded that the model performs adequately, but not outstandingly.

Researchers from startup Ansible Health published last Thursday (9) in an article published in the scientific journal PLOS Digital Health, the results of their study. The data shows that ChatGPT was able to achieve a score of “at a pass threshold of approximately 60 percent”, meaning that the model was able to answer a number of complex medical questions with a satisfactory level of accuracy.

The United States Medical Licensing Examination (USMLE) is a three-part exam that assesses the medical skills and knowledge of professionals seeking a license to practice medicine in the United States. In addition, the test also takes into account the values ​​and attitudes of the candidates.

After excluding image-based questions, ChatGPT scored 350 out of 376 USMLE questions for June 2022. The results showed that ChatGPT scored between 52.4% and 75% on each of the three stages of the exam. It is important to remember that in general the approval threshold is close to 60%. ChatGPT also outperformed a PubMedGPT model trained solely on biomedical literature, which scored 50.8%.

“Getting a passing score on this notoriously difficult specialized exam without any human support marks a remarkable milestone in the clinical maturation of AI.“, said the authors of the study.

Approved, but with reservations

Following the publication of a study on the effectiveness of ChatGPT’s extended language model in the US Medical Licensing Examination (USMLE), the Federation of State Medical Councils and the National Council of Medical Examiners, who co-founded USMLE, released a statement. compound.

In the statement, they emphasize that the study used exam preparation materials and practice questions, not actual USMLE exam questions. This raises questions about the validity and reliability of the Ansible Health study results. The statement suggests that using actual USMLE questions may yield different and more accurate results about ChatGPT’s ability to perform the test.

“It is important to note that the practice questions used by ChatGPT do not reflect the depth and breadth of the USMLE exam content that test takers will encounter. For example, studies did not include certain types of questions, such as questions using images, heart sounds, and computer simulations of clinical skills. This means that other critical test constructs are not fully represented in the studies.“, the federation said in a statement.

However, they do not rule out the possibility of ChatGPT or artificial intelligence: “While there isn’t enough evidence to support current claims that AI can pass the USMLE exams, we wouldn’t be surprised if AI models improve their performance significantly as technology advances.“

“When used correctly, these tools can have a positive impact on how assessments are built and how students learn.“, he concluded.

Mundo Conectado Deal Center: Selection of Discounts and Lowest Prices
Best deals on electronics, cell phones, TVs, soundbars, drones and more

Via: BGR Source: PLOS Digital Health

…..