Zero-shot mineral segmentation in reflected-light microscopy using the segment anything model

Adriaan Campo; Eric Pirard

doi:10.5566/ias.4060

Authors

Adriaan Campo Université de Liège GeMMe – Georesources and Geoimaging Sart Tilman B52 Allée de la Découverte 9 4000 Liège, Belgium; Ghent University Faculty of Arts and Philosophy Institute for Psychoacoustics and Electronic Music (IPEM) Miriam Makebaplein 1 9000 Ghent, Belgium
Eric Pirard Université de Liège GeMMe – Georesources and Geoimaging Sart Tilman B52 Allée de la Découverte 9 4000 Liège, Belgium

DOI:

https://doi.org/10.5566/ias.4060

Keywords:

Automated mineralogy, Image analysis, Ore microscopy, Reflected light microscopy, Segment anything model, Zero-shot segmentation

Abstract

Automated mineral segmentation underpins quantitative mineralogy and stereological characterization, but supervised deep-learning methods require annotated data and may generalize poorly across mineralogical domains. This study evaluated the Segment Anything Model (SAM), a general-purpose foundation model, for reflected-light microscopy (RLM) mineral-image analysis in a purely zero-shot setting, without task-specific training, fine-tuning, or domain adaptation. Two complementary case studies were conducted. First, three SAM encoder variants (ViT-B, ViT-L, and ViT-H) were evaluated on publicly available iron-ore (FeM) and copper-ore (Cu) RLM datasets using SEM-derived reference masks. SAM achieved mean F1-scores of 0.77–0.88, with ViT-L and ViT-H reaching 93–94% of the previously reported supervised DeepLabv3+ F1-score benchmark for FeM and approximately 93% for Cu. Second, SAM was applied to polarized RLM image stacks of dolomite to assess its influence on downstream stereological characterization. No significant differences from the gradient-based method were detected for any RGB configuration or PCA ViT-B in major or minor intercept length or bireflectance. The largest deviations occurred for PCA-preprocessed ViT-H, with median major and minor intercept lengths approximately 21% and 17% higher, respectively, and bireflectance approximately 16% lower. No significant method-dependent difference was detected in principal anisotropy orientation. These results show that zero-shot SAM can support mineral-image segmentation and downstream stereological characterization without annotated training data, while highlighting the influence of preprocessing and encoder choice on the derived grain representation

References

Álvarez Iglesias JC, Gomes OFM, Paciornik S (2011). Automatic recognition of hematite grains under polarized reflected light microscopy through image analysis. Miner Eng 24(12):1264-70.

Álvarez Iglesias JC, Santos RBM, Paciornik S (2019). Deep learning discrimination of quartz and resin in optical microscopy images of minerals. Miner Eng 138:79-85.

Archit A, Freckmann L, Nair S, Khalid N, Hilt P, Rajashekar V, Freitag M, Teuber C, Spitzner M, Tapia Contreras C, Buckley G, von Haaren S, Gupta S, Grade M, Wirth M, Schneider G, Dengel A, Ahmed S, Pape C (2025). Segment Anything for Microscopy. Nat Methods 22(3):579-91.

Azizi S, Mustafa B, Ryan F, Beaver Z, Freyberg J, Deaton J, Loh A, Karthikesalingam A, Kornblith S, Chen T, Natarajan V, Norouzi M (2021). Big self-supervised models advance medical image classification. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), 3478-88.

Badrinarayanan V, Kendall A, Cipolla R (2017). SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481-95.

Beucher S (1992). The watershed transformation applied to image segmentation. Scanning Microsc Suppl 6:299-314.

Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, Bernstein MS, Bohg J, Bosselut A, Brunskill E, Brynjolfsson E, Buch S, Card D, Castellon R, Chatterji N, Chen A, Creel K, Davis JQ, Demszky D, Donahue C, Doumbouya M, et al. (2021). On the opportunities and risks of foundation models. arXiv:2108.07258.

Canny J (1986). A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679-698.

Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2018). DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell 40(4):834-48.

Cruz-Orive LM (2017). Stereology: a historical survey. Image Anal Stereol 36(3):153-77.

De Castro B, Benzaazoua M, Roychowdhury S, Chopard A, Quintal Lauzon F, Plante B (2024). Novel technique for the preparation and analysis of powder-based polished sections by automated optical mineralogy: Part 2 - use of deep learning approach for transparent mineral detection. Miner Eng 206:108508.

Delesse A (1848). Procédé mécanique pour déterminer la composition des roches. Ann Mines 13:379-88.

Dunn OJ (1961). Multiple comparisons among means. J Am Stat Assoc 56(293):52–64.

Fandrich R, Gu Y, Burrows D, Moeller K (2007). Modern SEM-based mineral liberation analysis. Int J Miner Process 84(1-4):310-20.

Felzenszwalb PF, Huttenlocher DP (2004). Efficient graph-based image segmentation. Int J Comput Vis 59(2):167-81.

Filippo MP, Gomes OFM, da Costa GAOP, Mota GLA (2021). Deep learning semantic segmentation of opaque and non-opaque minerals from epoxy resin in reflected light microscopy images. Miner Eng 170:107007.

Frayssinet L, Grinberg P, Grecco HE, Comerio MA, Domené EA (2026). Automated segmentation using computer vision applied to petrographic thin sections of unconventional tight gas sandstone reservoirs. Discov Geosci 4:57.

Friedman M (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675-701.

Gomes OFM, Paciornik S, Filippo MP, da Costa GAOP, Mota GLA (2021a). Cu dataset - A copper ore labeled images dataset for segmentation training and testing. Zenodo. doi:10.5281/zenodo.5020566.

Gomes OFM, Paciornik S, Filippo MP, da Costa GAOP, Mota GLA (2021b). FeM dataset - An iron ore labeled images dataset for segmentation training and testing. Zenodo. doi:10.5281/zenodo.5014700.

Gu Y (2003). Automated scanning electron microscope based mineral liberation analysis: an introduction to JKMRC/FEI mineral liberation analyser. J Miner Mater Charact Eng 2(1):33-41.

Hu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W (2021). LoRA: low-rank adaptation of large language models. arXiv:2106.09685.

Jaimes Contreras R, Pilawski D, Califice A, Pirard E (2010). Quantitative microtexture analysis of carbonate rocks using bireflectance imaging. In: Proceedings of IAMG 2010; Budapest, Hungary.

Kendall MG, Smith BB (1939). The problem of m rankings. Ann Math Stat 10(3):275-87.

Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, Xiao T, Whitehead S, Berg AC, Lo WY, Dollár P, Girshick R (2023). Segment Anything. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), 4015-26.

Koh EJY, Amini E, McLachlan GJ, Beaton N (2021). Utilising convolutional neural networks to perform fast automated modal mineralogy analysis for thin-section optical microscopy. Miner Eng 173:107230.

Kolesnikov A, Beyer L, Zhai X, Puigcerver J, Yung J, Gelly S, Houlsby N (2020). Big Transfer (BiT): general visual representation learning. In: Proceedings of the European Conference on Computer Vision (ECCV), 491-507.

Li C, Han X, Yao C, Guo Y, Li Z, Jiang L, Liu W, Huang H, Fu H, Ban X (2025). A novel training-free approach to efficiently extracting material microstructures via visual large model. Acta Mater 290:120962.

Otsu N (1979). A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62-6.

Pérez-Barnuevo L, Pirard E, Castroviejo R (2013). Automated characterisation of intergrowth textures in mineral particles: a case study. Miner Eng 52:136-42.

Pirard E (2004). Multispectral imaging of ore minerals in optical microscopy. Mineral Mag 68(2):323-33.

Pirard E, Lebichot S, Krier W (2007). Particle texture analysis using polarized light imaging and grey level intercepts. Int J Miner Process 84(1-4):299-309.

Quiñonero-Candela J, Sugiyama M, Schwaighofer A, Lawrence ND, eds. (2009). Dataset shift in machine learning. Cambridge, MA: MIT Press.

Reinke A, Tizabi MD, Baumgartner M, Eisenmann M, Heckmann-Nötzel D, Kavur AE, Rädsch T, Sudre CH, Acion L, Antonelli M, Arbel T, Bakas S, Benis A, Buettner F, Cardoso MJ, Cheplygina V, Chen J, Christodoulou E, Cimini BA, Farahani K, et al. (2024). Understanding metric-related pitfalls in image analysis validation. Nat Methods 21(2):182-94.

Ronneberger O, Fischer P, Brox T (2015). U-Net: convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 234-41.

Schulz B, Sandmann D, Gilbricht S (2020). SEM-based automated mineralogy and its application in geo- and material sciences. Minerals 10(11):1004.

Taha AA, Hanbury A (2015). Metrics for evaluating 3D medical image segmentation: analysis, selection, and tool. BMC Med Imaging 15:29.

Underwood EE (1970). Quantitative stereology. Reading, MA: Addison-Wesley.

Vellappally A, Hou S, Emmings J (2024). Automated grain segmentation and mineral classification in rock thin sections. In: Proceedings of the EAGE Annual Conference & Exhibition, 1-5