Multi-objective optimization of temperature profile using reinforcement learning in batch crystallization process
-
Suneet Dhanasekaran
Abstract
The temperature profile is a critical parameter in industrial batch cooling crystallization processes which directly impacts the crystal size distribution (CSD) and subsequent down streaming operations. Conventional cooling approaches, including natural and linear cooling, often fail to achieve optimal results. Linear cooling may trigger excessive primary nucleation by crossing metastable limits, while natural cooling leads to inconsistent nucleation and growth, resulting in smaller and non-uniform crystals. Hence, optimized cooling profiles are used to achieve the desired target CSD. This study employs reinforcement learning (RL) to optimize the temperature profile for batch cooling crystallization for two systems: paracetamol in water and potassium dihydrogen phosphate (KDP) in aqueous solution. RL dynamically adjusts the cooling trajectory by iteratively interacting with a process model, aiming to maximize mean crystal size while minimizing the coefficient of variation. A comprehensive mathematical model incorporating population balance equations, mass balance, and energy balance is developed to simulate the crystallization process. For the paracetamol system, the RL-optimization strategy resulted in a 107.5 % increase in mean crystal size compared to the natural cooling profile. In the case of the KDP system, a 12 % increase in mean crystal length is achieved relative to a linear cooling profile, along with a significant reduction in the coefficient of variation, indicating improved crystal size uniformity. The optimization results obtained using RL are also compared with that from a genetic algorithm for both cases, and RL demonstrated superior performance. This work underscores the potential of RL in advancing broader applications in chemical process optimization.
Acknowledgments
Authors are acknowledged the Department of Chemical Engineering A.C. Tech Anna University for the support.
-
Research ethics: Not applicable.
-
Informed consent: Not applicable.
-
Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.
-
Use of Large Language Models, AI and Machine Learning Tools: None declared.
-
Conflict of interest: The authors state no conflict of interest.
-
Research funding: None declared.
-
Data availability: Not applicable.
References
1. Braatz, RD, &Hasebe, S. Particle size and shape control in crystallization processes. In: AIChE symposium series. New York: American Institute of Chemical Engineers; 1998:307–27 pp.Search in Google Scholar
2. Kim, DY, Paul, M, Rapke, JU, Wozny, G, Yang, DR. Modelling of crystallization process and optimization of the cooling strategy. Kor J Chem Eng 2009;26:1220–5.10.1007/s11814-009-0207-6Search in Google Scholar
3. Petsagkourakis, P, Sandoval, IO, Bradford, E, Zhang, D, del Rio-Chanona, EA. Reinforcement learning for batch bioprocess optimization. Comput Chem Eng 2020;133:106649. https://doi.org/10.1016/j.compchemeng.2019.106649.Search in Google Scholar
4. Zhou, Z, Li, X, Zare, RN. Optimizing chemical reactions with deep reinforcement learning. ACS Cent Sci 2017;3:1337–44. https://doi.org/10.1021/acscentsci.7b00492.Search in Google Scholar PubMed PubMed Central
5. Nikita, S, Tiwari, A, Sonawat, D, Kodamana, H, Rathore, AS. Reinforcement learning based optimization of process chromatography for continuous processing of biopharmaceuticals. Chem Eng Sci 2021;230:116171. https://doi.org/10.1016/j.ces.2020.116171.Search in Google Scholar
6. Paengjuntuek, W, Arpornwichanop, A, Kittisupakorn, P. Product quality improvement of batch crystallizers by a batch-to-batch optimization and nonlinear control approach. Chem Eng J 2008;139:344–50. https://doi.org/10.1016/j.cej.2007.08.010.Search in Google Scholar
7. Oh, DH, Adams, D, Vo, ND, Gbadago, DQ, Lee, CH, Oh, M. Actor-critic reinforcement learning to estimate the optimal operating conditions of the hydrocracking process. Comput Chem Eng 2021;149:107280. https://doi.org/10.1016/j.compchemeng.2021.107280.Search in Google Scholar
8. Powell, BKM, Machalek, D, Quah, T. Real-time optimization using reinforcement learning. Comput Chem Eng 2020;143:107077. https://doi.org/10.1016/j.compchemeng.2020.107077.Search in Google Scholar
9. Quah, T, Machalek, D, Powell, KM. Comparing reinforcement learning methods for real-time optimization of a chemical process. Processes 2020;8:1497. https://doi.org/10.3390/pr8111497.Search in Google Scholar
10. Nagy, ZK, Fujiwara, M, Braatz, RD, Woo, XY. Determination of kinetic parameters for batch pharmaceutical crystallization using metastable zone experiments. Ind Eng Chem Res 2008;47:1245–52. https://doi.org/10.1021/ie060637c.Search in Google Scholar
11. Ramakrishna, D. Population balances: theory and applications to particulate systems in engineering. USA: Academic Press; 2020.Search in Google Scholar
12. Hemalatha, K, Rani, KY. Multiobjective optimization of unseeded and seeded batch cooling crystallization processes. Ind Eng Chem Res 2017;56:6012–21. https://doi.org/10.1021/acs.iecr.7b00586.Search in Google Scholar
13. Benyahia, B, Anandan, PD, &Rielly, C. Robust model-based reinforcement learning control of a batch crystallization process. In: 9th international conference on systems and control (ICSC). France: IEEE; 2021:89–94 pp.10.1109/ICSC50472.2021.9666494Search in Google Scholar
14. Samad, NAFA, Singh, R, Sin, G, Gernaey, KV, &Gani, R. A generic multi-dimensional model-based system for batch cooling crystallization processes. Comput Chem Eng 2011;35:828–43. https://doi.org/10.1016/j.compchemeng.2011.01.029.Search in Google Scholar
15. Ma, DL, Tafti, DK, Braatz, RD. High-resolution simulation of multidimensional crystal growth. Ind Eng Chem Res 2002;41:6217–23. https://doi.org/10.1021/ie010680u.Search in Google Scholar
16. Gunawan, R, Ma, DL, Fujiwara, M, Braatz, RD. Identification of kinetic parameters in multidimensional crystallization processes. Int J Mod Phys B 2002;16:367–74. https://doi.org/10.1142/s0217979202009883.Search in Google Scholar
17. Schulman, J, Wolski, F, Dhariwal, P, Radford, A, & Klimov, O Proximal policy optimization algorithms. arXiv preprint arXiv, 2017, 1707.06347.Search in Google Scholar
© 2025 Walter de Gruyter GmbH, Berlin/Boston
