OpenAI's ChatGPT is no closer to replacing your family physicians, as the increasingly advanced chatbot failed to accurately diagnose the vast majority of hypothetical pediatric cases.
The findings were part of a new study published in JAMA Pediatrics on Jan. 2, conducted by researchers from Cohen Children's Medical Center in New York. The researchers analyzed the bot's responses to requests for medical diagnosis of child illnesses and found that the bot had an 83 percent error rate across tests.
The study used what are known as pediatric case challenges, or medical cases originally posted to groups of physicians as learning opportunities (or diagnostic challenges) involving unusual or limited information. Researchers sampled 100 challenges published on JAMA Pediatrics and NEJM between the years 2013 and 2023.
ChatGPT provided incorrect diagnoses for 72 out of 100 of the experimental cases provided, and generated 11 answers that were deemed "clinically related" to the correct diagnosis but considered too broad to be correct.
The researchers attribute part of this failure to the generative AI's inability to recognize relationships between certain conditions and external or preexisting circumstances, often used to help diagnose patients in a clinical setting. For example, ChatGPT did not connect "neuropsychiatric conditions" (such as autism) to commonly seen cases of vitamin deficiency and other restrictive-diet-based conditions.
The study concludes that ChatGPT needs continued training and involvement of medical professionals that feeds the AI not with an internet-generated well of information, which can often cycle in misinformation, but on vetted medical literature and expertise.
AI-based chatbots relying on Large Language Models (LLMs) have been previously studied for their efficacy in diagnosing medical cases and in accomplishing the daily tasks of physicians. Last year, researchers tested generative AI's ability to pass the three-part United States Medical Licensing Exam It passed.
But while it's still highly criticized for its training limits and potential to exacerbate medical bias, many medical groups, including the American Medical Association, don't view the advancement of AI in the field just as a threat of replacement. Instead, better trained AI's are considered ripe for their administrative and communicative potential, like generating patient-side text, explaining diagnoses in common terms, or in generating instructions. Clinical uses, like diagnostics, remain a controversial, and hard to research, topic.
To that extent, the new report represents the first analysis of a chatbot's diagnostic potential in a purely pediatric setting acknowledging the specialized medical training undertaken by medical professionals. Its current limitations show that even the most advanced chatbot on the public market can't yet compete with the full range of human expertise.
Originally posted here:
ChatGPT is terrible at diagnosing child medical cases, according to new study - Mashable
- Sang Pediatrics: When to see the doctor for a fever - YourCentralValley.com - May 12th, 2024 [May 12th, 2024]
- Should You Put Sunscreen on Infants? Not Usually - FDA.gov - May 12th, 2024 [May 12th, 2024]
- Thousands of bike helmets to be distributed to Ohio children | 10tv.com - 10TV - May 12th, 2024 [May 12th, 2024]
- How to talk to your teenaged patient about pregnancy and contraception - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- Adult RSV Shots Given to More Than 30 Babies by Mistake - Medpage Today - May 12th, 2024 [May 12th, 2024]
- Pediatric Health Care Teams Can Help Prevent Youth Suicide - The Pew Charitable Trusts - May 12th, 2024 [May 12th, 2024]
- Small patients, big discoveries | Penn Today - Penn Today - May 12th, 2024 [May 12th, 2024]
- HCPs can play an important role in addressing the US infant formula shortage - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- East Wenatchee Doctor Awarded For Boosting Child Vaccinations - KPQ - May 12th, 2024 [May 12th, 2024]
- HPV vaccination coverage of US-children aged 9-17 years in 2022 - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- Weekly review: Profound genetic deafness gene therapy, measles increases, and more - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- Pediatric Telehealth Platform Market is expected to Expand at a Massive CAGR of 26.6% through 2030 - openPR - May 12th, 2024 [May 12th, 2024]
- DB-OTO improved hearing to normal in child with profound genetic deafness - Contemporary Pediatrics - May 12th, 2024 [May 12th, 2024]
- The IT strategy behind a groundbreaking new $2B pediatric hospital - Healthcare IT News - May 12th, 2024 [May 12th, 2024]
- Modest Changes in Kids' Developmental Scores Seen From Pandemic - Medpage Today - April 23rd, 2024 [April 23rd, 2024]
- This pediatrics subspecialty has the best work-life balance - MD Linx - April 23rd, 2024 [April 23rd, 2024]
- PM Pediatrics Cobble Hill closing Pediatric urgent care options in Brooklyn and lower Manhattan - Brooklyn Bridge Parents - April 23rd, 2024 [April 23rd, 2024]
- Bethesda Pediatrics awarded $18,000 grant by GIVE - Yahoo! Voices - April 23rd, 2024 [April 23rd, 2024]
- Nemours Children's Health, UCF Partner to Elevate the Quality of Healthcare for Florida's Children - UCF - April 23rd, 2024 [April 23rd, 2024]
- Pediatricians told to talk to patients and parents about climate. - Grist - April 23rd, 2024 [April 23rd, 2024]
- Getting a good night's rest is vital for neurodiverse children pediatric sleep experts explain why - The Conversation - April 23rd, 2024 [April 23rd, 2024]
- Tina Cheng Honored as Exceptional CMO - Research Horizons - Research Horizons - April 23rd, 2024 [April 23rd, 2024]
- Let's talk about childhood injury prevention | Hub - The Hub at Johns Hopkins - April 23rd, 2024 [April 23rd, 2024]
- More young people with depression, anxiety in Virginia | Headlines - InsideNoVa - April 23rd, 2024 [April 23rd, 2024]
- Pediatrics fundraising event at Altru helps children with critical needs - Grand Forks Herald - April 15th, 2024 [April 15th, 2024]
- Public perceptions of families affected by pediatric cancer and educational work in pediatric oncology | Pediatric ... - Nature.com - April 15th, 2024 [April 15th, 2024]
- The pediatrician, the resident, the strawberry monster, and the zebra - VUMC News - VUMC Reporter - April 15th, 2024 [April 15th, 2024]
- Receptor targeting to identify medications for pediatric mental health concerns - Contemporary Pediatrics - April 15th, 2024 [April 15th, 2024]
- Hypoglossal nerve stimulation offers hope for pediatric Down syndrome patients with OSA - News-Medical.Net - April 15th, 2024 [April 15th, 2024]
- Efforts to reduce wait times for developmental evaluations - Baylor College of Medicine | BCM - April 15th, 2024 [April 15th, 2024]
- Nasal oxidative stress mediating the effects of colder temperature exposure on pediatric asthma symptoms | Pediatric ... - Nature.com - April 15th, 2024 [April 15th, 2024]
- Week in review: A new dermatology case study, and a new treatment for severe asthma - Contemporary Pediatrics - April 15th, 2024 [April 15th, 2024]
- Chordae Tendinea Rupture and Pulmonary Valve Vegetation in Pediatric Endocarditis: A Rare Condition - Cureus - April 7th, 2024 [April 7th, 2024]
- Pruitt named director of academic pediatrics division Washington University School of Medicine in St. Louis - Washington University School of... - April 7th, 2024 [April 7th, 2024]
- Researchers Show Injury Prevention Program Reduces Injuries in Young Children | Newsroom - UNC Health and UNC School of Medicine - April 7th, 2024 [April 7th, 2024]
- Provident Healthcare Partners Advises SouthernMED Pediatrics and Sweetgrass Pediatrics in a Partnership with U.S. ... - PR Newswire - April 7th, 2024 [April 7th, 2024]
- Online child safety laws could help or hurt 2 pediatricians explain what's likely to work and what isn't - The Conversation - April 7th, 2024 [April 7th, 2024]
- Hundreds of blue pinwheels draw attention for the need of doctors who specialize in child abuse pediatrics - MyStateline.com - April 7th, 2024 [April 7th, 2024]
- Pediatricians can help teens quit vaping and using tobacco - Odessa American - April 7th, 2024 [April 7th, 2024]
- CUTTING-EDGE RESEARCH SHAPING CHILDREN'S HEALTH SHOWCASED AT THE PEDIATRIC ACADEMIC ... - PR Newswire - April 7th, 2024 [April 7th, 2024]
- New NIAID report on maternal COVID-19 vaccination offers direct evidence of protection for newborns - Contemporary Pediatrics - April 7th, 2024 [April 7th, 2024]
- Navigating Pediatric Diabetes: A Q&A With Leading Expert David Maahs, MD - Stanford Medicine Children's Health Blog - Stanford Children's Health - March 29th, 2024 [March 29th, 2024]
- Linaclotide Succeeds for Functional Constipation in Children | GI and Hepatology News - MDedge - March 29th, 2024 [March 29th, 2024]
- Testing and diagnoses capacity of 2009 influenza and COVID-19 pandemics - Contemporary Pediatrics - March 29th, 2024 [March 29th, 2024]
- Predicting Infection Risk in Childhood Cancer - News Center - Feinberg News Center - March 29th, 2024 [March 29th, 2024]
- Healing the divide: Advancing access to pediatric care - HealthLeaders Media - March 29th, 2024 [March 29th, 2024]
- McMaster Child Health Research Day charts the future of pediatric health - Brighter World - March 29th, 2024 [March 29th, 2024]
- This is serious: The results of a measles outbreak have harmful outcomes - Contemporary Pediatrics - March 29th, 2024 [March 29th, 2024]
- Unlocking the Power of Influenza Vaccines for Pediatric Population: A Narrative Review - Cureus - February 29th, 2024 [February 29th, 2024]
- Human Papillomavirus Vaccines - Immunizations - American Academy of Pediatrics - February 29th, 2024 [February 29th, 2024]
- The role of the pediatrician in school-based mental health services - Contemporary Pediatrics - February 29th, 2024 [February 29th, 2024]
- Reminder for IPRC Webinar Preventing Secondary Trauma and Practical Self Care for Pediatric Rehabilitation ... - RCPA - February 29th, 2024 [February 29th, 2024]
- PCORI Awards CYSHCNet Researchers $5.5 Million to Study Transitions for Children and Caregivers from Hospital-to ... - UNC Health and UNC School of... - February 29th, 2024 [February 29th, 2024]
- Viewpoint: Journal Pediatrics reneges on its commitment to print response to botched article claiming GMOs are ... - Genetic Literacy Project - February 29th, 2024 [February 29th, 2024]
- NYC medical school will go tuition-free thanks to $1 billion gift from former pediatrics professor - Anchorage Daily News - February 29th, 2024 [February 29th, 2024]
- Innovative New Pediatric Facility Helps Alleviate Patient Anxiety - Healthcare Construction and Operations News - February 29th, 2024 [February 29th, 2024]
- Most pediatricians aren't prepared to handle teen opioid use, study says - WCVB Boston - February 29th, 2024 [February 29th, 2024]
- As Medicaid Shrinks, Clinics for the Poor Are Trying to Survive - The New York Times - February 29th, 2024 [February 29th, 2024]
- Many children hospitalized with COVID continue to face challenges. - Baltimore Sun - February 29th, 2024 [February 29th, 2024]
- Exposure to neighborhood violence linked to unmet health needs and increased care utilization in children - News-Medical.Net - February 29th, 2024 [February 29th, 2024]
- OUtMATCH: Omalizumab Proves Utility in Multiple Food Allergies for Pediatric, Adult Patients - MD Magazine - February 29th, 2024 [February 29th, 2024]
- Neffy pharmacokinetic profiles compared to epinephrine injections - Contemporary Pediatrics - February 29th, 2024 [February 29th, 2024]
- Clinician quiz: AAP clinical practice guideline for treatment of children with obesity - Contemporary Pediatrics - February 29th, 2024 [February 29th, 2024]
- Contributed: Shortcomings and opportunities for health equity in pediatrics - Mobihealth News - January 27th, 2024 [January 27th, 2024]
- Committed to physician education to enhance children's health - UChicago Medicine - January 27th, 2024 [January 27th, 2024]
- What's leading to youth sports burnout? American Academy of Pediatrics report says 70% of kids are dropping out by 13 - WLS-TV - January 27th, 2024 [January 27th, 2024]
- Washington University School of Medicine joins collaboration to improve pediatric heart disease care - News-Medical.Net - January 27th, 2024 [January 27th, 2024]
- How to Treat Coughs in Kids - Health Matters - January 27th, 2024 [January 27th, 2024]
- COVID-19 vaccination during pregnancy and offspring neurodevelopment - Contemporary Pediatrics - January 27th, 2024 [January 27th, 2024]
- Discontinuing contact precautions for pediatric patients with methicillin-resistant Staphylococcus aureus - Contemporary Pediatrics - January 27th, 2024 [January 27th, 2024]
- Viewpoint: American Academy of Pediatrics fiasco International Association for the Research on Cancer is the ... - Genetic Literacy Project - January 27th, 2024 [January 27th, 2024]
- Recent reports of measles in multiple states - Contemporary Pediatrics - January 27th, 2024 [January 27th, 2024]
- Expert shares advice for parents navigating pediatric influenza, COVID-19, and RSV this winter - Newswise - January 27th, 2024 [January 27th, 2024]
- Global Group of Researchers Release New Criteria for Diagnosing Pediatric Sepsis and Septic Shock - AboutLawsuits.com - January 27th, 2024 [January 27th, 2024]
- Recurrent Syncope Unveiling Pulmonary Hypertension Secondary to Pulmonary Artery Thrombi in a Pediatric Patient - Cureus - January 8th, 2024 [January 8th, 2024]
- ChatGPT bombs test on diagnosing kids' medical cases with 83% error rate - Ars Technica - January 8th, 2024 [January 8th, 2024]
- The clinical takeaways of the RSV immunizations - Contemporary Pediatrics - January 8th, 2024 [January 8th, 2024]
- Missouri pediatricians provide eight parenting goals to start the new year strong - Kirksville Daily Express and Daily News - January 8th, 2024 [January 8th, 2024]
- Reviewing the 2023 RSV season and an outlook on 2024 - Contemporary Pediatrics - January 8th, 2024 [January 8th, 2024]
- Predictive Factors for Insulin Resistance in Pediatric Obesity: A Comprehensive Analysis - Physician's Weekly - December 31st, 2023 [December 31st, 2023]