Increasing the Image Sharpness of Light Microscope Images Using Deep Learning
-
P. Krawczyk
Patrick Krawczyk received his bachelor’s degree in Electrical Engineering and his master’s degree in Computer Controlled Systems from Aalen University, Germany. He has been working as a research assistant at Materials Research Institute Aalen since 2020 and is currently pursuing his PhD in the field of image restoration. His research interests include deep neural n etworks, computer vision and materials microscopy.Andreas Jansche is currently pursuing his PhD in the area of applied machine learning for materials microscopy. He received his bachelor's degree in Computer Science and his research master's degree in Advanced Materials and Manufacturing from Aalen university, Germany. He has been working as a research assistant at the Materials Research Institute Aalen since 2015 and as a software engineer for automated microscopy since 2012. His research includes deep neural networks, computer vision and materials microscopy.
Abstract
Image-based qualitative and quantitative structural analyses using high-resolution light microscopy are integral parts of the materialographic work on materials and components. Vibrations or defocusing often result in blurred image areas, especially in large-scale micrographs and at high magnifications. As the robustness of the image-processing analysis methods is highly dependent on the image grade, the image quality directly affects the quantitative structural analysis. We present a deep learning model which, when using appropriate training data, is capable of increasing the image sharpness of light microscope images. We show that a sharpness correction for blurred images can successfully be performed using deep learning, taking the examples of steels with a bainitic microstructure, non-metallic inclusions in the context of steel purity degree analyses, aluminumsilicon cast alloys, sintered magnets, and lithium-ion batteries. We furthermore examine whether geometric accuracy is ensured in the artificially resharpened images.
Kurzfassung
In der Materialographie von Werkstoffen und Bauteilen ist die qualitative und quantitative Gefügeanalyse mittels Aufnahmen aus hochauflösenden Lichtmikroskopen fester Bestandteil. Besonders bei hohen Vergrößerungen und großflächigen Mikroskopieaufnahmen entstehen aufgrund von Schwingungen oder Defokussierung oft unscharfe Bildbereiche. Die Bildqualität der Aufnahmen hat einen direkten Einfluss auf die quantitative Gefügeanalyse, da die Robustheit der bildverarbeitenden Analysemethoden stark von der Güte der Aufnahmen abhängig ist. Wir stellen ein Deep Learning Modell vor, mit welchem unter Verwendung passender Trainingsdaten die Bildschärfe von Lichtmikroskopieaufnahmen erhöht werden kann. Am Beispiel von Stählen mit bainitischem Gefüge, nichtmetallischen Einschlüssen bei der Stahlreinheitsgradanalyse, Aluminium-Silizium-Gusslegierungen, Sintermagneten und Lithium-Ionen-Batterien zeigen wir, dass unscharfe Aufnahmen erfolgreich mittels Deep Learning nachgeschärft werden können. Des Weiteren wird untersucht, ob die geometrische Genauigkeit in den künstlich nachgeschärften Aufnahmen gewährleistet ist.
About the authors
Patrick Krawczyk received his bachelor’s degree in Electrical Engineering and his master’s degree in Computer Controlled Systems from Aalen University, Germany. He has been working as a research assistant at Materials Research Institute Aalen since 2020 and is currently pursuing his PhD in the field of image restoration. His research interests include deep neural n etworks, computer vision and materials microscopy.
Andreas Jansche is currently pursuing his PhD in the area of applied machine learning for materials microscopy. He received his bachelor's degree in Computer Science and his research master's degree in Advanced Materials and Manufacturing from Aalen university, Germany. He has been working as a research assistant at the Materials Research Institute Aalen since 2015 and as a software engineer for automated microscopy since 2012. His research includes deep neural networks, computer vision and materials microscopy.
4 Acknowledgements
We would like to thank Gaby Ketzer, Elvira Reiter, Alexander Banholzer, Dominic Hohs, and Tim Schubert for the valuable materialographic support and Amit Kumar Choudhary for his assistance with the segmentation models. We would also like to thank the Federal Ministry of Education and Research (BMBF) for financing the research project EBiMA (13FH176PX8) under the FHProfUnt funding program.
4 Danksagung
Wir danken Gaby Ketzer, Elvira Reiter, Alexander Banholzer, Dominic Hohs und Tim Schubert für die wertvolle materialographische Unterstützung, sowie Amit Kumar Choudhary für die Unterstützung bei den Segmentierungsmodellen. Dank gilt dem Bundesministerium für Bildung und Forschung (BMBF) für die Finanzierung des Forschungsprojekts EBiMA (13FH176PX8) im Rahmen des FHProfUnt-Förderprogramms.
References / Literatur
[1] S. Nah, T. H. Kim, and K. M. Lee: Deep multiscale convolutional neural network for dynamic scene deblurring, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, p. 257–265. DOI: 10.1109/CVPR.2017.3510.1109/CVPR.2017.35Search in Google Scholar
[2] S. Brehm, S. Scherer and R. Lienhart: High-Resolution Dual-Stage Multi-Level Feature Aggregation for Single Image and Video Deblurring, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2020, pp. 1872–1881. DOI: 10.1109/CVPRW50498.2020.0023710.1109/CVPRW50498.2020.00237Search in Google Scholar
[3] H. Zhang, Y. Dai, H. Li, and P. Koniusz: Deep stacked hierarchical multi-patch network for image deblurring, 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 5971–5979. DOI: 10.1109/CVPR.2019.0061310.1109/CVPR.2019.00613Search in Google Scholar
[4] P. Krawczyk, H. Baumgartl, A. Jansche, T. Bernthaler, R. Buettner, G. Schneider: Comparison of deep learning methods for imagedeblurring on light optical materials microscopy data, 2021 IEEE 17th International Conference on Automation Science and Engineering (CASE), 2021.10.1109/CASE49439.2021.9551404Search in Google Scholar
[5] U. Sara, M. Akter, and M. Uddin: Image quality assessment through FSIM, SSIM, MSE and PSNR-a comparative study, Journal of Computer and Communications, Vol. 7, pp. 8–18, 2019. DOI: 10.4236/jcc.2019.7300210.4236/jcc.2019.73002Search in Google Scholar
© 2021 Walter de Gruyter GmbH, Berlin/Boston, Germany
Articles in the same Issue
- Contents
- Editorial
- Editorial
- Increasing the Image Sharpness of Light Microscope Images Using Deep Learning
- Visualization of Prior Austenite Grain Boundaries in Low-Carbon Steels
- Failure analysis
- Metallurgical Failure Investigation of Fractured Dog Bone Seal Retainer Ring Fillet Welds in the Turbine Exhaust Casing of a Heavy-duty Gas Turbine Engine
- Report on the 55th Metallography Conference from 29.09. – 01.10.2021
- Picture of the month
- Presentation of the DGM Metallography Award 2021
- Presentation of 33th Buehler Best Paper Award 2020
- Presentation of Struers Best Contribution and Best Poster Award 2021
- Winners of the Photo Competition / Gewinner des Fotowettbewerbs
- Winners of the Photo Competition
- News
- Großer Bahnhof für Gerhard Schneider
- Meeting Diary
- Meeting Diary
Articles in the same Issue
- Contents
- Editorial
- Editorial
- Increasing the Image Sharpness of Light Microscope Images Using Deep Learning
- Visualization of Prior Austenite Grain Boundaries in Low-Carbon Steels
- Failure analysis
- Metallurgical Failure Investigation of Fractured Dog Bone Seal Retainer Ring Fillet Welds in the Turbine Exhaust Casing of a Heavy-duty Gas Turbine Engine
- Report on the 55th Metallography Conference from 29.09. – 01.10.2021
- Picture of the month
- Presentation of the DGM Metallography Award 2021
- Presentation of 33th Buehler Best Paper Award 2020
- Presentation of Struers Best Contribution and Best Poster Award 2021
- Winners of the Photo Competition / Gewinner des Fotowettbewerbs
- Winners of the Photo Competition
- News
- Großer Bahnhof für Gerhard Schneider
- Meeting Diary
- Meeting Diary
