Virtual Immunohistochemistry for Breast Cancer Biomarker Prediction From H&E-Stained Images Using Generative Network

Shuying Wu; Shiwei Xu

doi:10.5566/ias.3613

Authors

Shuying Wu School of Artificial Intelligence, Wenzhou Polytechnic
Shiwei Xu School of Artificial Intelligence, Wenzhou Polytechnic, Wenzhou City

DOI:

https://doi.org/10.5566/ias.3613

Keywords:

Virtual staining, Generative adversarial network, H&E, IHC

Abstract

Immunohistochemistry (IHC) is essential in diagnostic pathology but is often constrained by cost, time, and limited tissue availability. Virtual IHC staining, which predicts IHC stains from standard hematoxylin and eosin (H&E) images, presents a promising alternative. This study introduces a novel Conditional Generative Adversarial Network (cGAN) architecture based on a U-Net with depthwise separable convolutions to enhance the accuracy and efficiency of virtual IHC staining. This architectural refinement improves computational efficiency while preserving high image quality. We trained and evaluated our model using the BCI and MIST datasets and compared its performance against established image-to-image translation techniques, including Pix2Pix, CycleGAN, and a U-Net variant with standard convolutions. Performance was assessed using quantitative metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Fréchet Inception Distance (FID). The results showed that our model outperformed these benchmarks, achieving higher PSNR and SSIM scores, lower MAE and RMSE values, and a significantly reduced FID, indicating superior image quality and closer resemblance to ground-truth IHC images. Furthermore, the integration of depthwise separable convolutions led to a notable decrease in inference time and model size, improving its feasibility for clinical applications. These findings highlight the potential of our method as a significant advancement in virtual IHC staining, offering improved accuracy, efficiency, and suitability for broader clinical use.

References

Allison KH, Hammond MEH, Dowsett M, McKernin SE, Carey LA, Fitzgibbons PL, Hayes DF, Lakhani SR, Chavez-MacGregor M, Perlmutter J, et al. (2020). Estrogen and progesterone receptor testing in breast cancer: Asco/cap guideline update. J Clin Oncol 38:1346–66.

Bancroft JD, Gamble M (2008). Theory and practice of histological techniques. Elsevier health sciences.

BenTaieb A, Hamarneh G (2017). Adversarial stain transfer for histopathology image analysis. IEEE Trans Med Imaging 37:792–802.

DoanNgan B, Angus D, Sung L, et al. (2022). Label-free virtual HER2 immunohistochemical staining of breast tissue using deep learning. BME Front.

Engel KB, Moore HM (2011). Effects of preanalytical variables on the detection of proteins by immunohistochemistry in formalin-fixed, paraffin-embedded tissue. Arch Pathol Lab Med 135:537–43.

Fischer AH, Jacobson KA, Rose J, Zeller R (2008). Hematoxylin and eosin staining of tissue and cell sections. Cold Spring Harb Protoc 2008:pdb–prot4986.

Goldstein NS, Ferkowicz M, Odish E, Mani A, Hastah F (2003). Minimum formalin fixation time for consistent estrogen receptor immunohistochemical staining of invasive breast carcinoma. Am J Clin Pathol 120:86–92.

Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2020). Generative adversarial networks. Commun ACM 63:139–44.

Group EBCTC, et al. (2015). Aromatase inhibitors versus tamoxifen in early breast cancer: patient-level meta-analysis of the randomised trials. Lancet 386:1341–52.

Gurcan MN, Boucheron LE, Can A, Madabhushi A, Rajpoot NM, Yener B (2009). Histopathological image analysis: A review. IEEE Rev Biomed Eng 2:147–71.

Harb R, Pock T, Muller H (2024). Diffusion-based generation of histopathological whole slide images at a gigapixel scale. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.

Henry J, Natalie T, Madsen D (2021). Pix2pix GAN for image-to-image translation. Research Gate Publication:1–5.

Holzinger A, Langs G, Denk H, Zatloukal K, Muller H (2019). Causability and explainability of artificial intelligence in medicine. Wiley Interdiscip Rev Data Min Knowl Discov 9:e1312.

Hou L, Agarwal A, Samaras D, Kurc TM, Gupta RR, Saltz JH (2019). Robust histopathology image analysis: To label or to synthesize? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Howat WJ, Lewis A, Jones P, Kampf C, Ponten F, van der Loos CM, Gray N, Womack C, Warford A (2014). Antibody validation of immunohistochemistry for biomarker discovery: recommendations of a consortium of academic and pharmaceutical based histopathology researchers. Methods 70:34–8.

Li F, Hu Z, Chen W, Kak A (2023). Adaptive supervised patchnce loss for learning H&E-to-IHC stain translation with inconsistent groundtruth image pairs. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer.

Li W, Li J, Polson J, Wang Z, Speier W, Arnold C (2022). High resolution histopathology image generation and segmentation through adversarial training. Med Image Anal 75:102251.

Liu S, Zhu C, Xu F, Jia X, Shi Z, Jin M (2022a). BCI: Breast cancer immunohistochemical image generation through pyramid Pix2Pix. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.

Liu S, Zhu C, Xu F, Jia X, Shi Z, Jin M (2022b). BCI: Breast cancer immunohistochemical image generation through pyramid Pix2Pix. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Liu Y, Li X, Zheng A, Zhu X, Liu S, Hu M, Luo Q, Liao H, Liu M, He Y, et al. (2020). Predict Ki–67 positive cells in H&E–stained images using deep learning independently from IHC–stained images. Front Mol Biosci 7:183.

Ma L, Rathgeb A, Mubarak H, Tran M, Fei B (2022). Unsupervised super-resolution reconstruction of hyperspectral histology images for whole-slide imaging. J Biomed Opt 27:056502.

Orlandini LF, Antonio MVdN, Espreafico Jr CR, Bosquesi PL, Poli-Neto OB, de Andrade JM, Dos Reis FJC, Tiezzi DG (2021). Epidemiological analyses reveal a high incidence of breast cancer in young women in Brazil. JCO Glob Oncol 7:81–8.

Peng Q, Lin W, Hu Y, Bao A, Lian C, Wei W, Yue M, Liu J, Yu L, Wang L (2024). Advancing H&E-to-IHC virtual staining with task-specific domain knowledge for HER2 scoring. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer.

Qu L, Zhang C, Li G, Zheng H, Peng C, He W (2024). Advancing H&E-to-IHC stain translation in breast cancer: A multi-magnification and attention-based approach. In: 2024 IEEE International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE International Conference on Robotics, Automation and Mechatronics (RAM). IEEE.

Rivenson Y, Wang H, Wei Z, de Haan K, Zhang Y, Wu Y, Gunaydin H, Zuckerman JE, Chong T, Sisk AE, et al. (2019). Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat Biomed Eng 3:466–77.

Rizvi S, Cicalese P, Seshan S, Sciascia S, Becke J, Nguyen H (2022). Histopathology DatasetGAN: Synthesizing large-resolution histopathology datasets. In: 2022 IEEE Signal Processing in Medicine and Biology Symposium (SPMB). IEEE.

Ronneberger O, Fischer P, Brox T (2015). U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18. Springer.

Slamon DJ, Leyland-Jones B, Shak S, Fuchs H, Paton V, Bajamonde A, Fleming T, Eiermann W, Wolter J, Pegram M, et al. (2001). Use of chemotherapy plus a monoclonal antibody against HER2 for metastatic breast cancer that overexpresses HER2. N Engl J Med 344:783–92.

Stepec D, Skocaj D (2020). Image synthesis as a pretext for unsupervised histopathological diagnosis. In: Simulation and Synthesis in Medical Imaging: 5th International Workshop, SASHIMI 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4, 2020, Proceedings 5. Springer.

Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F (2021). Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 71:209–49.

Taylor C, Levenson RM (2006). Quantification of immunohistochemistry—issues concerning methods, utility and semiquantitative assessment II. Histopathology 49:411–24.

Vasiljevic J, Nisar Z, Feuerhake F, Wemmert C, Lampert T (2022). CycleGAN for virtual stain transfer: Is seeing really believing? Artif Intell Med 133:102420.

Wilm F, Benz M, Bruns V, Baghdadlian S, Dexl J, Hartmann D, Kuritcyn P, Weidenfeller M, Wittenberg T, Merkel S, et al. (2022). Fast whole-slide cartography in colon cancer histology using superpixels and CNN classification. J Med Imaging 9:027501.

Wolff AC, Hammond MEH, Allison KH, Harvey BE, Mangu PB, Bartlett JM, Bilous M, Ellis IO, Fitzgibbons P, Hanna W, et al. (2018). Human epidermal growth factor receptor 2 testing in breast cancer: American Society of Clinical Oncology/College of American Pathologists clinical practice guideline focused update. Arch Pathol Lab Med 142:1364–82.