By: Marie SOCKEEL (Primaa) Bertille POMMIER (Primaa) Stéphane SOCKEEL (Primaa) Rémy PEYRET (Primaa) Clara SIMMAT (Primaa) Nicolas POZIN (Primaa) Sophie PREVOT (Hôpital Bicêtre)
Aurélie MARAN-GONZALEZ (Institut régional du Cancer de Montpellier) Gaëtan MAC GROGAN (Institut Bergonié), Anne VINCENT SALOMON (Institut Curie).

Background & Objective

The use of AI based diagnostic tools in digital pathology is revolutionizing the field. However, the evaluation of devices is a crucial issue today. We conducted a study on Cleo Breast by Primaa in the context of use by 10 pathologists on 200 slides of Breast or Lymph Nodes samples in real routine conditions (both ‘standalone’ or fully integrated version).
The results show a statistical non-inferiority in performance (up to 10% gain with AI) and a significant reduction in pathologist’s time (up to 50% time savings).

Study design

The study was conducted between May and July 2023 across four French centers: Institut Curie, Institut Bergonié, ICM, and AP-HP. A total of 10 investigator pathologists and 4 expert pathologists.
Two different sites (Curie Institute Paris and Curie Institute Saint-Cloud provided 200 breast slides and associated lymph nodes. The WSI were digitized with the scanner S360 Hamamatsu at x40.
Ground Truth has been established with Labelson platform.

CLEOBREAST

Cleo Breast is a cutting-edge diagnostic support tool designed specifically for pathologists. It assists in the detection of biomarkers on breast pathology slides and associated lymph nodes by offering the following features:

• Detection of Invasive and In Situ Carcinoma Lesions
• Automated Mitotic Count over 2 mm²
• Lymph Node Metastasis Detection
• Calcification Detection

In this study, we used Cleo Breast both in standalone and fully integrated with Sectra amplifier IMS.

Study Results

Methods

• crossover superiority study
• internal eCRF (StudIA)
• independent expert biostatistician
• sensitivity and specificity etablished for all biomarkers against groundtruth
• McNemar test for paired data
  

Time Savings

The median time that pathologists took to interpret a slide is 114 seconds [62:211] without AI and 103 seconds [54:195] with AI, representing a time saving of 9.6%. To interpret the 40 lymph node slides, the median time is 68 seconds [25:123] without AI and 46 seconds [23:75] with AI, representing a time saving of 32.4%.

Increased Diagnostic precision

Cleo Breast demonstrates non-inferiority in diagnostic performance, showcasing high sensitivity in detecting invasive carcinoma (99.1% [98:99,7] without AI, 99.8% [99:100] with AI), carcinoma in situ (92.1% [89,7:94,1] without AI, 92,2% [89,8:94,2] with AI), mitotic scores (Kappa cohen’s 0.748 [0,717:0,78] without AI, 0.774 [0,744:0,805] with AI), calcifications (Kappa cohen’s 0.82 [0,795:0,846] without AI, 0.84 [0,816:0,864] with AI) and lymph node metastases (97.6% [93,2:99,5] without AI, 100% [97:100] with AI). (Fig 1).
The results between the physicians using Cleo Breast AI on the Cleo viewer and Sectra are significantly similar (Fig 2).

Discussion

• Accuracy and Efficiency: AI demonstrates comparable accuracy to traditional analysis, with high sensitivity and specificity in detecting critical biomarkers such as invasive carcinoma and lymph node metastases.
• Inter-rater Agreement: The Kappa coefficient shows that AI maintains a good level of agreement between evaluators, which is crucial for consistent diagnostics.
• Impact on Analysis Time: Using AI results in a slight increase in analysis time per slide, likely due to the integration and interpretation of AI suggestions. However, this extra time may be justified by potential improvements in diagnostic accuracy.
• User Acceptability: While AI slightly reduces user confidence and comfort, these metrics remain high, suggesting good acceptability of the tool among healthcare professionals.

Conclusion

This study marks a significant step in evaluating Cleo Breast’s clinical effectiveness in breast pathology diagnosis. It confirms its ability to enhance diagnostic accuracy, especially by streamlining mitotic scoring, thus improving pathologist’s efficiency.

Integration with a IMS signifies a notable advancement in diagnostic workflow.

Cleo Breast emerges as a transformative diagnostic tool, promising to expedite and refine diagnostic processes, leading to better patient care outcomes. This research paves the way for further Cleo Breast adoption in clinical settings.