We are excited to share some breaking news that was presented during the 2023 Annual Meeting of the American College of Allergy Asthma and Immunology (ACAAI). Brace yourselves because we're diving into the world of artificial intelligence and food allergy research!
Picture this: you're out at a restaurant, eagerly scanning the menu for something delicious to indulge in. But suddenly, that familiar pang of anxiety creeps in. What if the dish you choose contains an ingredient that could trigger your allergies? It's a scenario that many people with food allergies are all too familiar with.
Enter the latest research unveiled at the ACAAI meeting – a fusion of cutting-edge technology and allergy management. Of course, like any emerging technology, there are still hurdles to overcome. Ensuring the accuracy and reliability of AI algorithms, as well as addressing concerns around data privacy, are critical challenges that must be addressed.
Here is the summary of the research findings:
Artificial Intelligence Gives Artificial Information: Comparing ChatGPT Responses to the Food Allergy Clinical Guidelines
Background:
ChatGPT is an artificial intelligence (AI) chatbot and a popular tool for accessing information through open-ended queries. Little is known about the quality and reliability of medical information provided in the field of Allergy/Immunology.
Objective:
We aimed to evaluate the quality and reliability of medical information provided by ChatGPT on the topic of food allergies.
Methods:
Five domains of the AAAAI/ACAAI food allergy guidelines (prevalence, symptoms, diagnostic tests, management, prevention) were queried to ChatGPT. Medical information provided by ChatGPT was measured with the DISCERN instrument, a validated questionnaire designed to assess the quality of written health information. Agreement between the guidelines and the AI answers was measured by 4 authors independently to weigh the internal consistency of ChatGPT.
Results:
The information provided by ChatGPT across the five domains resulted in an average DISCERN score of 3.4/5, indicating fair-good quality information, with 78% agreement with guideline recommendations. Fleiss κ score was 0.71, which represented significant agreement between the graders of ChatGPT responses. Of significant concern, only 39% of the 41 total sources provided by the AI chatbot were accurate. The remaining 61% consisted of expired web pages (32%), fabricated journal articles (17%), and miscited articles (12%).
Conclusions:
Artificial intelligence ChatGPT information on food allergies appeared to reflect the currently accepted Allergy/Immunology guidelines but the chatbot was an unreliable source of information due to errors in over 60% of provided sources. The search results included completely made-up or fabricated journal articles (most concerning), expired web pages and inaccurate citations.
References:
Reliability Of Food Allergy Information Presented By An Artificial Intelligence Bot Versus Clinical Guidelines.
FOOD ALLERGY | A030| VOLUME 131, ISSUE 5, SUPPLEMENT 1, S7, NOVEMBER 2023
