Skip to main content

A case study of pre-processing strategies for robust deep learning segmentation of peripheral nerve fibers

In brief

Rat nerve segmentation reaches Dice 0.717 on unseen staining conditions

In rat sciatic nerve images, models trained on raw images barely segmented data from unseen staining conditions, with Dice scores of 0.0 to 0.17. Preprocessing helped when training data covered only one domain, while diverse training data plus preprocessing and moderate image augmentation achieved a Dice score of 0.717 on held-out images. Whether these strategies transfer to other nerves and laboratories remains unknown.

Journal
Computers in biology and medicine (Q1)
Published
9 November 2026
Study design
Unclassified
Evidence level
Level 5, Expert Opinion (CEBM 5)
Authors
Krzysztof Spaliński, Magdalena Jasionowska-Skop, Michał Kopka, Karolina Kopka, Antonina Spalińska, Artur Przelaskowski
PMID
43002890
DOI
10.1016/j.compbiomed.2026.112038

Why clinicians should know about it

  • Picked for Histology (top studies of the week, 15 November 2026): Evaluates preprocessing robustness for peripheral nerve histology

Abstract

BACKGROUND AND OBJECTIVES: Automated approaches for peripheral nerve morphometry often suffer from poor generalizability due to variability in tissue preparation, staining protocols, and image acquisition conditions. In practical deployment, models must operate reliably on previously unseen data shaped by subtle, interacting effects of tissue properties and staining chemistry. This study systematically evaluates whether explicit pre-processing strategies can improve the robustness of automated myelin segmentation to intra-stain batch effects and cross-domain shifts, addressing an underrepresented problem in peripheral nerve histology. METHODS: Using the Mysh framework, we performed a comprehensive evaluation on rat sciatic nerve images, training models on single-domain and combined-domain data with seven pre-processing strategies and three augmentation regimes across Toluidine Blue (TB) and Immunohistochemistry (IHC) images. Robustness was assessed via intra-stain batch stability (Light-TB vs. Dark-TB) and cross-modality generalization between optical density-based TB and binary-like IHC signals. RESULTS: Baseline models trained on raw images failed to generalize to out-of-distribution data (Dice ≈0.0-0.17). Pre-processing substantially recovered performance in single-domain training scenarios, with optimal strategies depending on the direction of domain shift: contrast enhancement (CLAHE) for IHC-to-TB generalization and feature simplification (BeerLaNet) for TB-to-IHC. When diverse multi-domain training data were available, aggressive color augmentation achieved strong generalization (Dice 0.712 on held-out Dark-TB), though combining pre-processing with moderate augmentation yielded comparable or superior results (Dice 0.717). CONCLUSIONS: For laboratories with limited single-domain data, explicit pre-processing is essential for robust deployment across unseen imaging conditions. When diverse training data is accessible, augmentation-based strategies become highly effective, though pre-processing remains a valuable complementary tool. The observed stability of IHC highlights its potential as a reliable modality for automated morphometry in regenerative medicine.

Abstract as published, via PubMed.

View on PubMedFull text at the publisherOpen in the app

For healthcare professionals. The summary is generated by AI from the published abstract, and the evidence level is assigned automatically from the study design on the Oxford CEBM hierarchy. Neither is medical advice. Read the full paper before changing practice.