Marie SOCKEEL, MD, Bertille POMMIER, PharmD, Rémy PEYRET, PhD.

Background

Breast cancer is the most prevalent cancer among women in France and represents the leading cause of cancer-related deaths in females. The recent reduction of the mortality is partly due to advances in therapies and early diagnostics, notably through organized breast cancer screening programs.

Primaa has developed Cleo Breast, an AI-powered tool for the analysis of 5 key features: cancer detection, mitosis counting, detection of emboli and microcalcifications, and nodal metastasis identification. As the first CE-marked tool for daily detection of breast cancer biomarkers in routine practice, Cleo Breast is at the vanguard of integrating artificial intelligence in pathology.

The objective of our multicentric study is to assess the time-saving benefits of utilizing our Cleo Breast tool across a representative sample of pathologists, ranging from experts to juniors, thereby solidifying our commitment to transforming breast cancer diagnosis through precision technology.

Algorithms developed for Cleo Breast

The Cleo Breast AI system, a state-of-the-art machine learning solution, has been trained on an extensive and varied dataset, comprising thousands of annotated breast tissue images. It excels in identifying a comprehensive range of breast cancer biomarkers, calibrated to the highest standards of diagnostic accuracy.
Our rigorous validation process across several pathology laboratories has demonstrated the AI’s exceptional performance, marked by significant improvements in diagnostic sensitivity and specificity. The AI’s impact is quantified by the following metrics:

Note : The validation of the AI’s capabilities was carried out independently of the training data, affirming the robustness and generalizability of the system).

Methods

• An independent methodologist structured the study, which was underpinned by a sample size that was statistically calculated to robustly assess the performance of each feature of the Cleo Breast analytical tool.
• The study’s setup was intended to minimize bias and maximize the reliability of the study’s outcomes.
• This multicentric performance clinical diagnostic study used a fully crossed multiples reader to assess Cleo Breast as an aid to the pathologists to detect the presence or absence of dedicated biomarkers of woman breast tissue lesions in WSI with tissus sample stained with HE/HES.
• The study’s ten pathologists were divided into two groups and analyzed all 200 WI in the study both AI and without AI. The distribution of investigators in group A and group B will be balanced according to their level of experience.
• The two groups reviewed every WSI in the opposite modality (with AI vs without AI). Then after one month of wash-out period, the pathologists reviewed the WSI for a second time using the opposite modality.
• The establishment of a ground truth for comparative analysis was conducted by a consensus of three expert physicians and a reconciler, ensuring an accurate benchmark for evaluating Cleo Breast’s diagnostic accuracy.
• The expert pathologists reviewed independently the 200 WSI blinded from the initial diagnosis.
• A total of 200 WSI are used in the study. The WSI are taken from breast biopsies, breast resection specimens and lymph nodes resection specimens.
• These, WSI are fully anonymized and come from two different laboratories.
  The calibration process involved meticulous review and annotation of 50 slides by the expert panel, which served to fine-tune the AI’s algorithms prior to the main study.
• This careful preparation ensured that the AI tool was optimized for both specificity and sensitivity in real-world clinical applications.

Results

The establishment of a ground truth for comparative analysis was conducted by a consensus of three expert physicians and a reconciler, ensuring an accurate benchmark for evaluating Cleo Breast’s diagnostic accuracy. The expert pathologists reviewed independently the 200 WSI blinded from the initial diagnosis.

A total of 200 WSI are used in the study. The WSI are taken from breast biopsies, breast resection specimens and lymph nodes resection specimens. These, WSI are fully anonymized and come from two different laboratories.

The calibration process involved meticulous review and annotation of 50 slides by the expert panel, which served to fine-tune the AI’s algorithms prior to the main study. This careful preparation ensured that the AI tool was optimized for both specificity and sensitivity in real-world clinical applications.

Conclusion

Our results attest to the tool’s robust performance in detecting Invasive and In Situ Carcinoma, mitotic figures, and metastases, with or without AI assistance. Notably, the significant time savings achieved in these areas underscore the efficiency that AI integration brings to cancer pathology.

With Cleo Breast, the promise of AI in medicine is realized through accelerated diagnostic processes, which can lead to faster patient management and potentially better outcomes. As we move forward, the next steps involve validating these performances in real-world clinical settings. Given the learning curve associated with any new technology and the full integration into diagnostic workflows, we anticipate that the already impressive benefits of Cleo Breast will become even more pronounced.