Here's a continuation of our enlightening blog series, in this section we are delving into the intriguing synergy between two technological frontiers - Whole Slide Imaging (WSI) and Artificial Intelligence (AI). In this exploration, we embark on a journey that unveils the transformative potential of their integration, transcending the boundaries of medical diagnostics, research, education and beyond.
Education
Equipped with whole-slide imaging, artificial intelligence (AI) tools can help further training of the next generation of pathologists by providing on demand, standardized, and interactive digital slides that can be shared with multiple users anywhere, at any time (92,93). Additionally, AI tools can provide automated annotations in the form of quizzes for trainees. With the help of these interactive tools trainees can view, pan, and zoom enhanced digital slides, which can provide tutoring in real-time and in a dynamic teaching environment. For the purpose of generating synthetic images, researchers extracted individual and clustered nuclei that were both positively and negatively stained from real whole-slide imaging images, and systematically placed the extracted nuclei clumps on an image canvas - cut-and-paste approach. These images were evaluated by trained pathologists in the task of estimating the ratio of positive to total number of nuclei. The resulting concordance correlation coefficients between the pathologist and the true ratio range from 0.86 to 0.95 (94). The main idea is to force the generator to learn the underlying distribution of the images from the training data. Generation of numerous synthetic histopathology images could be useful because it will give pathology trainees the opportunity to test their skills, besides being useful for quality control and understanding the perceptual and cognitive challenges that pathologists face (75,94).
Quality Assurance
The development of automated, high-speed, and high resolution WSI has a substantial effect on QA. Digitized slides that are readily available to pathologists in the LIS or on the intranet can be used for several QA tasks, including teleconsultation, gauging inter-observer and intra-observer variance, proficiency testing, and archiving of slides (8,39,95,96). For example, CAP optionally sends Whole slide (WS) images in addition to glass slides for certain proficiency testing cases. AI can have an important role in QA. By providing feedback manually or with intelligent deep learning and AI tools, a pathologist has the potential to keep improving on his or her performance. AI can be used as a supplement to these manual digital reviews in routine diagnostic workflow or as a complement to the more formal quality reviews that are part of a pathology laboratory’s quality management process. AI can also provide a quality check on the diagnosis rendered by a pathologist by applying automated diagnostic algorithms. These methods can continue to serve as patient safety mechanisms to improve the quality of diagnosis and to prevent error (75).
Pathological Diagnosis
WSI and AI have been increasingly used for routine pathological diagnosis. Several studies have shown a concordance of 89% to 99% when comparing diagnostic interpretation using digital slides to diagnoses rendered using glass slides and a conventional light microscope (19,32,43,70,92,97). The quality of images produced by WSI scanners has a direct influence on the readers’ performance and their reliability of diagnosis. Most modern scanners come equipped with autofocus optics system to select focal planes to accurately capture the three-dimensional tissue morphology similar to a two-dimensional digital image (43). To account for varying thickness of tissue sections, these systems determine a set of focus points at different focal planes from which scanners capture images to produce sharp tissue representation. However, still digital images with out-of-focus areas may be produced if the autofocus optics system erroneously selects focus points that lie in a different plane than the proper height of the tissue (19,43). To overcome this, AI automatically identifies out-of-focus regions allowing WSI scanners to add a few extra focal points to those regions by either feature engineering or via a representation learning approach. Another approach called Deep Focus, based on representation learning, automatically discovers features from the images to identify blurry regions. Because the Deep Focus program automatically learns features at different levels of abstraction, it can generalize to different types of tissues and even to color variations due to different types of staining, H&E and IHC (10). Standardization of the color displayed by digital slides is important for the accuracy of AI. Color variations in digital slides are often produced because of different lots or manufacturers of staining reagents, variations in thickness of tissue sections, difference in staining protocols, and disparity in scanning characteristics. For this reason, the absence of color normalization in an AI pipeline could negatively affect the performance of machine learning algorithms. For a long time, collecting color statistics to perform color matching across images has remained the main source of color normalization. However, progresses in the generative models have presented novel ways of color normalization (98).
Image Analysis
Image analysis tools can automate and quantify with greater consistency and accuracy than light microscopy (81, 99). Computer-aided diagnosis is widely used for Estrogen receptor (ER), Progesterone receptor (PR), and HER2/neu assessments in breast cancer (100, 101), Ki67 assessment in neuroendocrine neoplasms (102, 103), PD-L1 as immune checkpoint molecules in various solid organ malignancies, as well as multiple other clinical and research stains. The reliability of these methods requires the standardization of the image acquisition step. AI methods aid in enabling the regions of interest selection (104, 105). Nuclear segmentation in WSI enables extraction of high-quality features for nuclear morphometrics and other analysis in computational pathology (94). For this reason, automatic nuclei segmentation is among the most studied problems in AI. In general, these algorithms estimate a probability map of the nuclear and non-nuclear (two-class) regions on the basis of learned nuclear appearances and rely on complex methods after processing to obtain the final nuclear shapes and separation between touching nuclei (106). Kumar and colleagues (107) have created a well-annotated database consisting of 30 whole-slide images of digitized tissue samples from several organs. The slides were taken from the publically available database The Cancer Genome Atlas. The images were generated at 18 different hospitals, which adds to the diversity of this dataset in terms of variation in slide preparation protocols among laboratories. Over 21,000 nuclei were manually annotated to train a deep learning algorithm. Unlike former methods, a nuclei segmentation as a three-class problem was created. They considered the nuclei edges as a third class when generating the tertiary probability map. This map was subjected to region growing to segment the individual nuclei.
During most pathological analysis, pathologists are interested in identifying a subset of nuclei in a particular anatomical region. For example, in T1 bladder cancer (108), pathologists are interested in identifying the tumor nuclei within lamina propria. Similarly, in breast and neuroendocrine tumors, the pathologists are interested in the ratio of Ki67 tumor positive nuclei to total tumour nuclei within the hotspots. In follicular lymphoma, the analysis is limited to only the presence of centroblasts within the neoplastic follicles. For these reasons, there is an increasing interest in developing AI algorithms that can identify a subset of cells within a certain anatomical region. Also, whole slide is partitioned into superpixels on the basis of similarity at some magnification. Superpixels are grouped into anatomical regions (specifically epithelium) on the basis of graph clustering. Finally, each cluster is classified as ductal carcinoma in situ or benign or normal on the basis of features extracted by deep learning (104,109, 110).
In the dynamic convergence of Whole Slide Imaging (WSI) and Artificial Intelligence (AI), we witness the epitome of transformative innovation. This amalgamation not only underscores the potential to revolutionize medical diagnostics and research but also exemplifies the harmonious partnership between human expertise and computational prowess. As WSI continues to empower pathologists with unprecedented levels of data-rich insights, AI augments their capabilities by swiftly identifying patterns, anomalies, and predictive trends. The synergy between these realms holds the promise of accelerated diagnoses, enhanced treatment decisions, and novel scientific breakthroughs. As we conclude this blog series, we stand on the cusp of a new era in healthcare, one where WSI and AI harmonize to unravel the mysteries of the microcosmic world, inspiring optimism for a healthier and more informed future.
References
92. Parwani AV. Digital pathology enhances cancer diagnostics. MLO: Medical Laboratory Observer. 2017 Feb 1;49(2):25.
93. Somanchi S, Neill DB, Parwani AV. Discovering anomalous patterns in large digital pathology images. Statistics in medicine. 2018 Nov 10;37(25):3599-615.
94. Niazi MK, Parwani AV, Gurcan MN. Digital pathology and artificial intelligence. The lancet oncology. 2019 May 1;20(5):e253-61.
95. Hanna MG, Pantanowitz L, Evans AJ. Overview of contemporary guidelines in digital pathology: what is available in 2015 and what still needs to be addressed? Journal of Clinical Pathology. 2015 Jul 1;68(7):499-505.
96. Bauer TW, Schoenfield L, Slaw RJ, Yerian L, Sun Z, Henricks WH. Validation of whole slide imaging for primary diagnosis in surgical pathology. Arch Pathol Lab Med. 2013;137(4):518-524.
97. Yu H, Gao F, Jiang L, Ma S. Development of a whole slide imaging system on smartphones and evaluation with frozen section samples. JMIR mHealth and uHealth. 2017 Sep 15;5(9):e8242.
98. Senaras C, Niazi MK, Lozanski G, Gurcan MN. DeepFocus: detection of out-of-focus regions in whole slide digital images using deep learning. PloS one. 2018 Oct 25;13(10):e0205387.
99. Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A. Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nature reviews Clinical oncology. 2019 Nov;16(11):703-15.
100. Yamada M, Saito A, Yamamoto Y, Cosatto E, Kurata A, Nagao T, et al. Quantitative nucleic features are effective for discrimination of intraductal proliferative lesions of the breast. Journal of Pathology Informatics. 2016;7.
101. Li AC, Zhao J, Zhao C, Ma Z, Hartage R, Zhang Y, et al. Quantitative digital imaging analysis of HER2 immunohistochemistry predicts the response to anti-HER2 neoadjuvant chemotherapy in HER2-positive breast carcinoma. Breast cancer research and treatment. 2020 Apr;180(2):321-9.
102. Feng M, Deng Y, Yang L, Jing Q, Zhang Z, Xu L, et al. Automated quantitative analysis of Ki-67 staining and HE images recognition and registration based on whole tissue sections in breast carcinoma. Diagnostic Pathology. 2020 Dec;15(1):1-2.
103. Volynskaya Z, Mete O, Pakbaz S, Al-Ghamdi D, Asa SL. Ki67 quantitative interpretation: insights using image analysis. Journal of pathology informatics. 2019;10.
104. Komura D, Ishikawa S. Machine learning approaches for pathologic diagnosis. Virchows Archiv. 2019 Aug;475(2):131-8.
105. Chang HY, Jung CK, Woo JI, Lee S, Cho J, Kim SW, et al. Artificial intelligence in pathology. Journal of pathology and translational medicine. 2019 Jan;53(1):1.
106. Colling R, Pitman H, Oien K, Rajpoot N, Macklin P, CM‐Path AI in Histopathology Working Group, et al. Artificial intelligence in digital pathology: a roadmap to routine use in clinical practice. The Journal of pathology. 2019 Oct;249(2):143-50.
107. Kumar A, Prateek M. Localization of nuclei in breast cancer using whole slide imaging system supported by morphological features and shape formulas. Cancer Management and Research. 2020;12:4573.
108. Niazi MK, Yazgan E, Tavolara TE, Li W, Lee CT, Parwani A, et al. Semantic segmentation to identify bladder layers from H&E Images. Diagnostic Pathology. 2020 Dec;15(1):1-8.
109. Mahmood T, Arsalan M, Owais M, Lee MB, Park KR. Artificial intelligence-based mitosis detection in breast cancer histopathology images using faster R-CNN and deep CNNs. Journal of clinical medicine. 2020 Mar;9(3):749.
110. Komura D, Ishikawa S. Machine learning methods for histopathological image analysis. Computational and structural biotechnology journal. 2018 Jan 1;16:34-42
Blog Author
Sambit K Mohanty, MD