Artificial intelligence is increasingly gaining traction in pathology, offering tools for tumor detection, object identification, and tissue quantification. A recent study co-authored by Inotiv’s Dr. Kristen Hobbie, explored the intra-rater and inter-rater agreement among veterinary toxicologic pathologists in scoring progressive cardiomyopathy (PCM), a common background finding in laboratory rodents. Pathologists scored the severity of PCM and evaluated an AI-based algorithm (AIA) specifically trained to detect and score PCM in rodents. The AIA quantifies lesion severity by analyzing necrosis, inflammation, fibrosis, and mineralization, offering a continuous numeric score representing the affected heart area.
PCM poses challenges in pathology due to its complex progression and varying severity. Diagnosing and grading PCM consistently is crucial in distinguishing spontaneous lesions from those caused by toxicological agents. However, variability among pathologists often limits consistency. These recent advancements in artificial intelligence (AI) offer promising solutions to improve diagnostic accuracy and harmonization in toxicologic pathology.
The Challenge of Consistency
PCM lesions, ranging from cardiomyocyte degeneration to fibrosis and mineralization, evolve over time, complicating their diagnosis and grading. Inter-rater (between different pathologists) and intra-rater (by the same pathologist) agreements are critical metrics for ensuring reliability in pathology studies. However, these agreements can vary due to subjective interpretations of lesion severity, diagnostic criteria, and personal biases.
Inter-rater and intra-rater agreement have been extensively studied in human and veterinary pathology, but less so in toxicologic pathology. Factors influencing agreement include diagnostic confidence, scoring system complexity, and lesion difficulty. Lower resolution systems with fewer severity levels tend to produce higher agreement. But despite efforts to harmonize criteria, variability persists—an issue AI could potentially resolve.
Key Findings
Three hundred heart sections were evaluated from a panel of five pathologists, including 100 slides previously examined. Pathologists assigned diagnoses (no abnormality, PCM, other) and severity grades (0-5) based on modified criteria. The AIA assessed cardiomyopathy extent based on necrosis, inflammation, fibrosis, and mineralization, providing a continuous numeric value representing the percentage of the heart affected.
- Intra-Rater Agreement:
Pathologists demonstrated substantial consistency in scoring, with weighted Cohen’s kappa values ranging from 0.64 to 0.80. This indicated that individual pathologists could reliably replicate their own assessments over time. - Inter-Rater Agreement:
Agreement among pathologists was moderate, with weighted Cohen’s kappa values between 0.46 and 0.72. While this highlights variability, it also underscores the challenge of achieving consensus in multi-rater scenarios. - AI Repeatability:
The AIA proved highly repeatable, showing minimal differences across repeated analyses of the same slides. This consistency positions AI as a reliable tool for supplementing pathologist assessments. - AI vs. Manual Scoring:
While differences between AIA and manual scores were noted, the AI offered continuous, quantitative data, potentially enhancing sensitivity in detecting lesions.
The Future of AI in Pathology
The study revealed that AI could play a transformative role in toxicologic pathology by acting as:
- A Consistency Booster: AI reduces diagnostic discrepancies, enabling more reliable comparisons across studies.
- A Peer Reviewer: AI can flag significant differences in pathologist scores, prompting reevaluations and fostering a learning environment.
- A Training Tool: By comparing trainee scores with AI predictions, pathology education can become more data-driven and precise.
Conclusion
The integration of AI in veterinary toxicologic pathology is still in early stages, but its potential as a tool for pathologists to increase sensitivity and specificity of histopathologic assessments is undeniable. While challenges like regulatory acceptance and scalability remain, the collaboration between AI and toxicologic pathologists could herald a new era of pathology in nonclinical studies. AI algorithms are potential tools for pathologists to increase sensitivity and specificity of histopathologic assessments.
Read the Full Article: https://journals.sagepub.com/eprint/VWZTZW24RVCSRXEBG2JH/full
