Human Oversight and AI in Healthcare
Relevance
GS3: Science and Technology
Context
AI tools are rapidly transforming healthcare with applications ranging from foetal dating and high-risk pregnancy management to virtual autopsies and clinical chatbots. However, the promise of AI comes with critical concerns including automation bias, weak regulation, privacy issues, and the need for human oversight.
Key Developments in India
Garbhini-GA2: AI for Foetal Dating
- Developed by IIT-Madras and Translational Health Science and Technology Institute.
- Trained on ultrasound scans from ~3,500 pregnant women.
- Accuracy: Error margin of half a day, compared to up to 7 days error using Hadlock’s formula (which is based on Western population data).
- Ongoing validation across diverse Indian datasets.
AI in High-Risk Pregnancy (HRP) Management
- Nearly 50% of pregnancies in India are high-risk.
- Mumbai-based NGO ARMMAN, with UNICEF and State Governments, is using AI chatbots to assist auxiliary nurse-midwives (ANMs) in managing HRPs.
- AI chatbot provides text and voice responses and supports decision-making.
- Challenges:
- Speech recognition limitations in regional accents.
- Human-in-the-loop essential for complex queries.
Virtual Autopsy (Virtopsy)
- Spearheaded in India by Amar Jyoti Patowary at NEIGRIHMS.
- Uses CT and MRI scans to create 3D models for autopsies.
- Advantages:
- Faster: ~30 minutes vs. ~2.5 hours in conventional autopsy.
- Allows multiple virtual dissections.
- Limitations:
- Misses small soft tissue injuries, color changes, and odor cues.
- Can be enhanced by verbal autopsy and visual examination.
AI-Assisted Patient Data Handling
- Example: MediBuddy’s AI bot for preliminary diagnosis and clinical data collection.
- Concerns:
- Data Privacy: Personal data masking and role-based access control used to protect patient data.
- Current legal framework (IT Act 2000, Digital Personal Data Protection Act 2023) lacks clarity on AI-specific regulation.
Key Challenges
Automation Bias
- Over-reliance on AI recommendations by human professionals, potentially leading to misdiagnosis.
- Example:
- A study showed even experienced radiologists’ accuracy dropped when they trusted incorrect AI suggestions during mammogram assessments.
Data Privacy and Security
- India’s data protection laws do not explicitly regulate AI in healthcare.
- Need for robust data governance and AI accountability frameworks.
Technological Gaps
- Speech recognition models in AI struggle with regional languages and accents.
- Existing AI tools have limited adaptability to diverse linguistic and cultural contexts.
Way Forward
- Human-in-the-Loop Design:
- AI should assist, not replace, human decision-making in healthcare.
- Robust Regulation:
- Update privacy and healthcare laws to address AI-specific challenges.
- Training and Sensitisation:
- Medical professionals must be trained to understand AI’s capabilities and limitations.
- Localization of AI Models:
- Improve AI’s adaptability to regional languages and cultural contexts.
- Transparent Algorithms:
- Ensure explainability and traceability in AI decision-making.
Mains Questions
- Discuss the ethical and regulatory challenges of integrating Artificial Intelligence in India’s healthcare system.
- What is automation bias? Explain with examples how it can affect decision-making in the healthcare sector.
- Artificial Intelligence is revolutionizing healthcare but also raising concerns regarding privacy and accountability. Discuss.
