Analysis of mango fruit surface temperature using thermal imaging and deep learning

Pathmanaban Pugazhendi; Gnanavel Balakrishnan Kannaiyan; Shanmuga Sundaram Anandan; Chermadurai Somasundaram

doi:10.1515/ijfe-2022-0302

Article

Analysis of mango fruit surface temperature using thermal imaging and deep learning

, , and

Published/Copyright: June 12, 2023

Published by

Become an author with De Gruyter Brill

Submit Manuscript Author Information

From the journal International Journal of Food Engineering Volume 19 Issue 6

Abstract

Thermal imaging has the potential to measure the object’s surface temperature. This study investigated the thermal behavior of mango fruit stored in a refrigerated environment. Thermal images of the fruit were collected with sufficient quality by supplying hot air to the acquisition environment. Grey-Level Co-occurrence Matrix (GLCM) features of mango images were determined to distinguish the subtle and noticeable changes. The thermal images were analyzed to find the temperature difference between the different regions of the fruit. The temperature of the bruise boundary (T_bd) was higher than the bruised center (T_C) throughout the storage period. In addition, an enhanced deep-learning model was used to predict the damaged mango. Over 10 days, 3500 thermal images were obtained from the 400 mangoes. In that, 80 % of the images were used for training, 10 % for testing, and 10 % for validation. The model achieved a classification accuracy of 99.6 %.

Keywords: deep learning; GLCM feature; mango fruit; surface temperature; thermal image

Corresponding author: Pathmanaban Pugazhendi, Velammal Engineering College, Chennai, Tamilnadu, India, E-mail: pathmanaban@velammal.edu.in

Acknowledgments

The authors would like to acknowledge the Department of Computer Science Engineering and Food Technology of Velammal Engineering College, Easwari Engineering College and Sree Sastha institute of Engineering and Technology (cold Storage) for providing the facilities for this work, and Mrs.K.Vaishnavi of the English Department of Velammal Engineering College for proofreading the manuscript.

Author contributions: Pathmanaban P: Conceptualization; Investigation; Writing – original draft; Validation; Methodology; Visualization. Gnanavel B.K: Supervision; Data curation; Validation; Methodology. Shanmuga Sundaram Anandan: Writing – review & editing; Visualization; Software; Formal analysis; Project administration; Resources; Methodology; Validation. S. Chermadurai: Writing – review & editing; Visualization; Software; Formal analysis; Project administration;
Research funding: No funding was received for this study.
Conflict of interest statement: P. Pathmanaban declares that he has no conflict of interest. Gnanavel B. K. declares that he has no conflict of interest. Shanmuga Sundaram Anandan declares that he has no conflict of interest. S. Chermadurai declares that he has no conflict of interest.
Data availability: The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.
Ethics approval: This article does not contain any studies with humans or animals.
Consent to participate: Not applicable.

References

1. Mango GIdata. Nov 2022 [online]. Available from: https://agriexchange.apeda.gov.in/ProductSearch/Product_Detail.aspx?hscode=080450.Search in Google Scholar

2. Lawson, T, Lycett, GW, Ali, A, Chin, CF. Characterization of southeast Asia mangoes (Mangifera indica L) according to their physicochemical attributes. Sci Hortic 2019;243:189–96. https://doi.org/10.1016/j.scienta.2018.08.014.Search in Google Scholar

3. Musale, SS, Patil, PM. Database development of defective and healthy Alphonso Mangoes. Int J Adv Agric Environ Eng 2014;1:155–60.10.15242/IJAAEE.C614532Search in Google Scholar

4. Vélez-Rivera, N, Blasco, J, Chanona-Pérez, J, Calderón-Domínguez, G, de Jesús Perea-Flores, M, Arzate-Vázquez, I, et al.. Computer vision system applied to classification of (Manila) Mangoes during ripening process. Food Bioprocess Technol 2014;7:1183–94. https://doi.org/10.1007/s11947-013-1142-4.Search in Google Scholar

5. Soltani Firouz, M, Sardari, H. Defect detection in fruit and vegetables by using machine vision systems and image processing. Food Eng Rev 2022;14:353–79. https://doi.org/10.1007/s12393-022-09307-1.Search in Google Scholar

6. Duc Thong, N, Truong Thinh, N, Thanh Cong, H. Mango sorting mechanical system uses machine vision and artificial intelligence. Int J Eng Technol 2019;11:321–7. https://doi.org/10.7763/ijet.2019.v11.1169.Search in Google Scholar

7. Shamili, M. The estimation of mango fruit total soluble solids using image processing technique. Sci Hortic 2019;249:383–9. https://doi.org/10.1016/j.scienta.2019.02.013.Search in Google Scholar

8. Rungpichayapichet, P, Nagle, M, Yuwanbun, P, Khuwijitjaru, P, Mahayothee, B, Mu, J. Prediction mapping of physicochemical properties in mango by hyperspectral imaging. Biosyst Eng 2017;159:109–20. https://doi.org/10.1016/j.biosystemseng.2017.04.006.Search in Google Scholar

9. Damage, M, Mango, IN, Indica, M. Visible spectral imaging technology for early detection of mechanical damage in Mango (Mangifera Indica L). In: The 16th TSAE national conference and the 8th TSAE international conference; 2015.Search in Google Scholar

10. Alberto, LR, Ardila, CEC, Ortiz, FAP. A computer vision system for early detection of anthracnose in sugar mango (Mangifera indica) based on UV-A illumination. Inf Process Agric 2022;10:204–15. https://doi.org/10.1016/j.inpa.2022.02.001.Search in Google Scholar

11. Zeng, X, Miao, Y, Ubaid, S, Gao, X, Zhuang, S. Detection and classification of bruises of pears based on thermal images. Postharvest Biol Technol 2020;161:111090. https://doi.org/10.1016/j.postharvbio.2019.111090.Search in Google Scholar

12. Yu, Y, Zhang, K, Yang, L, Zhang, D. Fruit detection for strawberry harvesting robot in non-structural environment based on Mask-RCNN. Comput Electron Agric 2019;163:104846. https://doi.org/10.1016/j.compag.2019.06.001.Search in Google Scholar

13. Pathmanaban, P, Gnanavel, BK, Anandan, SS. Recent application of imaging techniques for fruit quality assessment. Trends Food Sci Technol 2019;94:32–42. https://doi.org/10.1016/j.tifs.2019.10.004.Search in Google Scholar

14. Vadivambal, R, Jayas, DS. Applications of thermal imaging in agriculture and food industry-a review. Food Bioprocess Technol 2011;4:186–99. https://doi.org/10.1007/s11947-010-0333-5.Search in Google Scholar

15. Fito, PJ, Ortolá, MD, de los Reyes, R, Fito, P, de los Reyes, E. Control of citrus surface drying by image analysis of infrared thermography. J Food Eng 2004;61:287–90. https://doi.org/10.1016/s0260-8774(03)00120-1.Search in Google Scholar

16. Veraverbeke, EA, Verboven, P, Lammertyn, J, Cronje, P, de Baerdemaeker, J, Nicolaï, BM. Thermographic surface quality evaluation of apple. J Food Eng 2006;77:162–8. https://doi.org/10.1016/j.jfoodeng.2005.06.059.Search in Google Scholar

17. Baranowski, P, Mazurek, W, Wozniak, J, Majewska, U. Detection of early bruises in apples using hyperspectral data and thermal imaging. J Food Eng 2012;110:345–55. https://doi.org/10.1016/j.jfoodeng.2011.12.038.Search in Google Scholar

18. Baranowski, P, Lipecki, J, Mazurek, W, Walczak, RT. Detection of watercore in “Gloster” apples using thermography. Postharvest Biol Technol 2008;47:358–66. https://doi.org/10.1016/j.postharvbio.2007.07.014.Search in Google Scholar

19. Kim, G, Kim, GH, Park, J, Kim, DY, Cho, BK. Application of infrared lock-in thermography for the quantitative evaluation of bruises on pears. Infrared Phys Technol 2014;63:133–9. https://doi.org/10.1016/j.infrared.2013.12.015.Search in Google Scholar

20. Ershad, SF. Texture classification approach based on combination of edge & Co-occurrence and Local binary pattern. In: Proceedings of the international conference on image processing, computer vision, and pattern recognition. IPCV; 2011.Search in Google Scholar

21. Jawale, D, Deshmukh, M. Real time automatic bruise detection in (Apple) fruits using thermal camera. In: Proceedings of the IEEE international conference on communication and signal processing. ICCSP; 2017.10.1109/ICCSP.2017.8286542Search in Google Scholar

22. Meshalkin, Y, Koroteev, D, Popov, E, Chekhonin, E, Popov, Y. Robotized petrophysics: machine learning and thermal profiling for automated mapping of lithotypes in unconventionals. J Pet Sci Eng 2018;167:944–8. https://doi.org/10.1016/j.petrol.2018.03.110.Search in Google Scholar

23. Boulent, J, Foucher, S, Théau, J, St-Charles, PL. Convolutional neural networks for the automatic identification of plant diseases. Front Plant Sci 2019;10:941. https://doi.org/10.3389/fpls.2019.00941.Search in Google Scholar PubMed PubMed Central

24. Sun, X, Subedi, P, Walsh, KB. Achieving robustness to temperature change of a NIRS-PLSR model for intact mango fruit dry matter content. Postharvest Biol Technol 2020;162:111117. https://doi.org/10.1016/j.postharvbio.2019.111117.Search in Google Scholar

25. Ramyapriyanandhini, G, Bagyammal, T, Parameswaran, L, Vaiapury, K. Anomaly detection in thermal images of perishable items using deep learning. Lect Notes Netw Syst 2022;373:647–59.10.1007/978-981-16-8721-1_61Search in Google Scholar

26. Dong, YY, Huang, YS, Xu, BL, Li, BC, Guo, B. Bruise detection and classification in jujube using thermal imaging and DenseNet. J Food Process Eng 2022;45:1–10. https://doi.org/10.1111/jfpe.13981.Search in Google Scholar

27. Guo, B, Li, B, Huang, Y, Hao, F, Xu, B, Dong, Y. Bruise detection and classification of strawberries based on thermal images. Food Bioprocess Technol 2022;15:1133–41. https://doi.org/10.1007/s11947-022-02804-5.Search in Google Scholar

28. Ali, MM, Hashim, N, Aziz, SA, Lasekan, O. Characterisation of pineapple cultivars under different storage conditions using infrared thermal imaging coupled with machine learning algorithms. Agriculture 2022;12:1013. https://doi.org/10.3390/agriculture12071013.Search in Google Scholar

29. Sachan, R, Kundra, S, International, AD. An efficient algorithm for object detection in thermal images using convolutional neural networks and thermal signature of the objects. In: International conference on energy, power, and environment (ICEPE), Shillong, India; 2022.10.1109/ICEPE55035.2022.9798144Search in Google Scholar

30. Ellong, EN, Adenet, S, Rochefort, K. Physicochemical, nutritional, organoleptic characteristics and food applications of four mango (Mangifera indica) varieties. Food Nutr Sci 2015;06:242–53. https://doi.org/10.4236/fns.2015.62025.Search in Google Scholar

31. Russakovsky, O, Deng, J, Su, H, Krause, J, Satheesh, S, Ma, S, et al.. ImageNet large scale visual recognition challenge. Int J Comput Vis 2015;115:211–52. https://doi.org/10.1007/s11263-015-0816-y.Search in Google Scholar

32. Lecun, Y, Bengio, Y, Hinton, G. Deep learning. Nature 2015;521:436–44. https://doi.org/10.1038/nature14539.Search in Google Scholar PubMed

33. Yang, Q, Liu, Y, Chen, T, Tong, Y. Federated machine learning: concept and applications. ACM Trans Intell Syst Technol 2019;10:1–12. https://doi.org/10.1145/3298981.Search in Google Scholar

34. Smith, SL, Kindermans, P-J, Ying, C, le Google Brain, Qv. Don’t decay the learning rate, increase the batch size. In: International conference on learning representations (ICLR); 2018.Search in Google Scholar

35. Costianes, PJ, Plock, JB. Gray-level co-occurrence matrices as features in edge enhanced images. In: Proceedings – applied imagery pattern recognition workshop, Washington, DC, USA; 2010.10.1109/AIPR.2010.5759705Search in Google Scholar

36. Patel, KK, Kar, A, Khan, MA. Potential of reflected UV imaging technique for detection of defects on the surface area of mango. J Food Sci Technol 2019;56:1295–301. https://doi.org/10.1007/s13197-019-03597-w.Search in Google Scholar PubMed PubMed Central

37. Bhole, V, Kumar, A. Mango quality grading using deep learning technique: perspectives from agriculture and food industry. In: Proceedings of the 21st annual conference on information technology education. Association for Computing Machinery, Inc., Omaha, NE, USA; 2020.10.1145/3368308.3415370Search in Google Scholar

38. Ardila, CEC, Ramirez, LA, Ortiz, FAP. Spectral analysis for the early detection of anthracnose in fruits of Sugar Mango (Mangifera indica). Comput Electron Agric 2020;173(105357 Suppl):1–17. https://doi.org/10.1016/j.compag.2020.105357.Search in Google Scholar

39. Rizwan Iqbal, HM, Hakim, A. Classification and grading of harvested mangoes using convolutional neural network. Int J Fruit Sci 2022;22:95–109. https://doi.org/10.1080/15538362.2021.2023069.Search in Google Scholar

40. Naik, S, Desai, P. Mango (Mangifera indica L.) classification using convolutional neural network and linear classifiers. In: Proceedings of third international conference on sustainable computing. Advances in Intelligent Systems and Computing, Singapore; 2022.10.1007/978-981-16-4538-9_17Search in Google Scholar

41. Naik, S, Patel, B. Thermal imaging with fuzzy classifier for maturity and size based non-destructive mango (Mangifera Indica L.) grading. In: International conference on emerging trends and innovation in ICT, ICEI 2017. Institute of Electrical and Electronics Engineers Inc.; 2017.10.1109/ETIICT.2017.7977003Search in Google Scholar

42. Pathmanaban, P, Gnanavel, BK, Anandan, SS. Guava fruit (Psidium guajava) damage and disease detection using deep convolutional neural networks and thermal imaging. Imag Sci J 2023;70:102–16. https://doi.org/10.1080/13682199.2022.2163536.Search in Google Scholar

43. Holman, JP. Experimental methods for engineers eighth edition [Internet]. Available from: www.mhhe.com/holman.Search in Google Scholar

Received: 2022-11-18

Accepted: 2023-05-27

Published Online: 2023-06-12

You are currently not able to access this content.

Articles in the same Issue

https://doi.org/10.1515/ijfe-2022-0302

Keywords for this article

deep learning; GLCM feature; mango fruit; surface temperature; thermal image