Active Disturbance Rejection Control of Output Voltage of Solid Oxide Fuel Cell Based on Reinforcement Learning

doi:10.12096/j.2096-4528.pgt.24017

Abstract

Abstract:

Objectives In order to improve the performance and lifetime of solid oxide fuel cell (SOFC) systems, the 100 kW SOFC system was taken as the research object. The continuous adjustment of the controller coefficients was explored through reinforcement learning to realize the best comprehensive performance, while ensuring the output voltage tracking performance. Methods A mechanism-based SOFC output voltage system model was established, an improved nonlinear active disturbance rejection controller (NLADRC) was used to make the output voltage track the reference value well by controlling the input gas flow. Conventional single-channel controllers can only satisfy one objective at a time, and dual-channel controllers will increase system complexity, cost and risk of failure. An improved NLADRC controller based on the twin delayed deep deterministic policy gradient (TD3) was proposed to optimize the coefficients of nonlinear error feedback control law. Results The designed controller can improve SOFC output voltage tracking performance without violating fuel utilization constraints. Conclusions The designed controller has the advantages of strong adaptability, high stability, and the ability to overcome uncertainty, providing theoretical reference for designing output voltage controllers in practical SOFC systems.

Key words: solid oxide fuel cell (SOFC), twin delayed deep deterministic policy gradient (TD3), nonlinear active disturbance rejection control (NLADRC), fuel utilization, nonlinear error feedback control law, output voltage tracking, uncertainty

CLC Number:

TM 911.4

Chaojun GUAN, Zhengling LEI, Haibo HUO, Fang WANG, Guoquan YAO, Tao LIU. Active Disturbance Rejection Control of Output Voltage of Solid Oxide Fuel Cell Based on Reinforcement Learning[J]. Power Generation Technology, 2024, 45(6): 1163-1172.

Figures/Tables 9

Fig. 1 Solid oxide fuel cell model structure

Tab. 1 Values for SOFC system parameters

参数	数值
绝对温度 $T 0$ / $K$	1 273
法拉第常数 $F 0$ /（ $C / m o l$ ）	96 485
通用气体常数 $R 0$ / $[J / (m o l ⋅ K)]$	8.314
理想标准电位 $E 0$ / $V$	1.18
电池数 $N$ ₀	384
常数 $K r$ / $[m o l / (s ⋅ A)]$	$0.996 × 10 - 6$
氢气的阀摩尔常数 $K H 2$ $/ [m o l / (s ⋅ P a$ $)]$	$8.32 × 10 - 6$
水蒸气的阀摩尔常数 $K H 2 O$ $/ [m o l / (s ⋅ P a)]$	$2.77 × 10 - 6$
氧气的阀摩尔常数 $K O 2$ $/ [m o l / (s ⋅ P a$ $)]$	$2.49 × 10 - 6$
氢气流响应时间 $τ H 2$ $/$ $s$	26.1
水的响应时间 $τ H 2 O$ $/$ $s$	78.3
氧气流响应时间 $τ O 2$ $/$ $s$	2.91
燃料重整时间常数 $τ f$ $/$ $s$	5
氢氧比 $τ H - O$	1.145

Tab. 1 Values for SOFC system parameters

参数	数值
绝对温度 $T 0$ / $K$	1 273
法拉第常数 $F 0$ /（ $C / m o l$ ）	96 485
通用气体常数 $R 0$ / $[J / (m o l ⋅ K)]$	8.314
理想标准电位 $E 0$ / $V$	1.18
电池数 $N$ ₀	384
常数 $K r$ / $[m o l / (s ⋅ A)]$	$0.996 × 10 - 6$
氢气的阀摩尔常数 $K H 2$ $/ [m o l / (s ⋅ P a$ $)]$	$8.32 × 10 - 6$
水蒸气的阀摩尔常数 $K H 2 O$ $/ [m o l / (s ⋅ P a)]$	$2.77 × 10 - 6$
氧气的阀摩尔常数 $K O 2$ $/ [m o l / (s ⋅ P a$ $)]$	$2.49 × 10 - 6$
氢气流响应时间 $τ H 2$ $/$ $s$	26.1
水的响应时间 $τ H 2 O$ $/$ $s$	78.3
氧气流响应时间 $τ O 2$ $/$ $s$	2.91
燃料重整时间常数 $τ f$ $/$ $s$	5
氢氧比 $τ H - O$	1.145

Fig. 2 Structure of second-order nonlinear active disturbance rejection control

Fig. 3 TD3 algorithm structure

Fig. 4 Structure framework of ADRC-TD3 overall control

Tab. 2 Main hyperparameters of ADRC-TD3 algorithm

参数	取值
折扣因子	0.99
经验池大小	2×10⁶
Actor网络学习率	0.001
Critic网络学习率	0.000 1
样本学习个数	128
训练步数	4 000
软更新系数	0.005
延迟更新参数	2

Fig. 5 ADRC-TD3 algorithm adjustment process reward

Fig. 6 Comparison of simulation results of given voltage signal tracking experiments

Fig. 7 Comparison of simulation results for load current disturbance experiments

References 25

1	YANG B， LI Y， LI J，et al ．Comprehensive summary of solid oxide fuel cell control：a state-of-the-art review[J]．Protection and Control of Modern Power Systems，2022，7(3)：1-31． doi:10.1186/s41601-022-00251-0
2	NECHACHE A， HODY S ．Alternative and innovative solid oxide electrolysis cell materials：a short review[J]．Renewable and Sustainable Energy Reviews，2021，149：111322． doi:10.1016/j.rser.2021.111322
3	郭心如，郭雨旻，罗方，等．磷酸燃料电池的能效、㶲及生态特性分析[J]．发电技术，2022，43(1)：73-82．
	GUO X R， GUO Y M， LUO F，et al ．Analysis of energy，exergy and ecology characteristics of phosphoric acid fuel cell[J]．Power Generation Technology，2022，43(1)：73-82．
4	和萍，祁盼，申润杰，等．计及风电和燃料电池的综合能源系统阻尼特性分析[J]．电力科学与技术学报，2020，35(1)：14-23．
	HE P， QI P， SHEN R J，et al ．Analysis of damping characteristics of an integrated energy system with hybrid wind-fuel cells integrated[J]．Journal of Electric Power Science and Technology，2020，35(1)：14-23．
5	LAN T， STRUNZ K ．Multiphysics transients modeling of solid oxide fuel cells：methodology of circuit equivalents and use in EMTP-type power system simulation[J]．IEEE Transactions on Energy Conversion，2017，32(4)：1309-1321． doi:10.1109/TEC.2017.2687886
6	BAO C， WANG Y， FENG D，et al ．Macroscopic modeling of solid oxide fuel cell (SOFC) and model-based control of SOFC and gas turbine hybrid system[J]．Progress in Energy and Combustion Science，2018，66：83-140． doi:10.1016/j.pecs.2017.12.002
7	张瑞宇，王雨晴，任佳伟．基于丙烷催化部分氧化的微管式固体氧化物燃料电池系统特性研究[J]．发电技术，2024，45(3)：486-493．
	ZHANG R Y， WANG Y Q， REN J W ．Characteristics research of a micro-tubular solid oxide fuel cell system based on catalytic partial oxidation of propane[J]．Power Generation Technology，2024，45(3)：486-493．
8	李林，刘彤宇，李爽，等．甲醇重整制氢燃料电池发电研究进展[J]．发电技术，2022，43(1)：44-53． doi:10.12096/j.2096-4528.pgt.21116
	LI L， LIU T Y， LI S，et al ．Research progress of hydrogen production by methanol reforming for fuel cell power generation[J]．Power Generation Technology，2022，43(1)：44-53． doi:10.12096/j.2096-4528.pgt.21116
9	KOMATSU Y， BRUS G， KIMIJIMA S，et al ．The effect of overpotentials on the transient response of the 300 W SOFC cell stack voltage[J]．Applied Energy，2014，115：352-359． doi:10.1016/j.apenergy.2013.11.017
10	SUN L， JIN Y， SHEN J，et al ．Sustainable residential micro-cogeneration system based on a fuel cell using dynamic programming-based economic day-ahead scheduling[J]．ACS Sustainable Chemistry & Engineering，2021，9(8)：3258-3266． doi:10.1021/acssuschemeng.0c08725
11	王倩如，王彩霞，顾吉鹏．固体氧化物燃料电池的神经模糊控制策略研究[J]．热能动力工程，2022，37(10)：198-206． doi:10.1016/j.apenergy.2021.118214
	WANG Q R， WANG C X， GU J P ．Study on neural fuzzy control strategy of solid oxide fuel cell[J]．Journal of Engineering for Thermal Energy and Power，2022，37(10)：198-206． doi:10.1016/j.apenergy.2021.118214
12	ZHANG T， LI H， TU X，et al ．Optimization of SOC fractional PID control parameters for solid oxide battery based on improved firefly algorithm[C]//2021 3rd International Conference on Industrial Artificial Intelligence (IAI)．Shenyang，China：IEEE，2021：1-5． doi:10.1109/iai53119.2021.9619450
13	WU X， GAO D ．Optimal robust control strategy of a solid oxide fuel cell system[J]．Journal of Power Sources，2018，374：225-236． doi:10.1016/j.jpowsour.2017.10.070
14	ABBAKER A O， WANG H， TIAN Y ．Voltage control of solid oxide fuel cell power plant based on intelligent proportional integral-adaptive sliding mode control with anti-windup compensator[J]．Transactions of the Institute of Measurement and Control，2020，42(1)：116-130． doi:10.1177/0142331219867779
15	LI J， YU T， YANG B ．A data-driven output voltage control of solid oxide fuel cell using multi-agent deep reinforcement learning[J]．Applied Energy，2021，304：117541． doi:10.1016/j.apenergy.2021.117541
16	LI J， YU T ．A novel data-driven controller for solid oxide fuel cell via deep reinforcement learning[J]．Journal of Cleaner Production，2021，321：128929． doi:10.1016/j.jclepro.2021.128929
17	HAN J ．From PID to active disturbance rejection control[J]．IEEE Transactions on Industrial Electronics，2009，56(3)：900-906． doi:10.1109/tie.2008.2011621
18	ZHOU J， XUE S， XUE Y，et al ．A novel energy management strategy of hybrid electric vehicle via an improved TD3 deep reinforcement learning[J]．Energy，2021，224：120118． doi:10.1016/j.energy.2021.120118
19	PADULLES J， AULT G W， MCDONALD J R ．An integrated SOFC plant dynamic model for power systems simulation[J]．Journal of Power sources，2000，86(1/2)：495-500． doi:10.1016/s0378-7753(99)00430-9
20	SUN L， HUA Q， SHEN J，et al ．A combined voltage control strategy for fuel cell[J]．Sustainability，2017，9(9)：1517． doi:10.3390/su9091517
21	LI Y， SHEN J， LU J ．Constrained model predictive control of a solid oxide fuel cell based on genetic optimization[J]．Journal of Power Sources，2011，196(14)：5873-5880． doi:10.1016/j.jpowsour.2011.03.010
22	ZHAO C， LI D ．Control design for the SISO system with the unknown order and the unknown relative degree[J]．ISA Transactions，2014，53(4)：858-872． doi:10.1016/j.isatra.2013.10.001
23	LADOSZ P， WENG L， KIM M，et al ．Exploration in deep reinforcement learning：a survey[J]．Information Fusion，2022，85：1-22． doi:10.1016/j.inffus.2022.03.003
24	FUJIMOTO S， VAN HOOF H， MEGER D ．Addressing function approximation error in actor-critic methods[J]．Proceedings of the 35th International Conference on Machine Learning，2018，80：1587-1596．
25	NGUYEN T T， NGUYEN N D， NAHAVANDI S ．Deep reinforcement learning for multiagent systems：a review of challenges，solutions，and applications[J]．IEEE Transactions on Cybernetics，2020，50(9)：3826-3839． doi:10.1109/tcyb.2020.2977374

[1]	Jianqiang YE, Dunhu SUN. Research on Power Planning Based on Robust Optimization Under Carbon Trading Condition [J]. Power Generation Technology, 2024, 45(3): 566-574.
[2]	Ruiyu ZHANG, Yuqing WANG, Jiawei REN. Characteristics Research of a Micro-Tubular Solid Oxide Fuel Cell System Based on Catalytic Partial Oxidation of Propane [J]. Power Generation Technology, 2024, 45(3): 486-493.
[3]	Siqi GONG, Zaipeng YUN, Ming XU, Le AO, Chufu LI, Kai HUANG, Chen SUN. Numerical Simulation of Solid Oxide Fuel Cell Tail Gas Catalytic Combustion Based on Three-Way Catalyst [J]. Power Generation Technology, 2024, 45(2): 331-340.
[4]	Xiaobiao FU, Jiaqi HOU, Baoju LI, Yakun WEN, Xiaowen LAI, Lei GUO, Zhiwei WANG, Yao WANG, Haifeng ZHANG, Dexin LI. A Two-Modal Weather Classification Method and Its Application in Photovoltaic Power Probability Prediction [J]. Power Generation Technology, 2024, 45(2): 299-311.
[5]	Xiaoqiang JIA, Yongbiao YANG, Jiao DU, Haiqing GAN, Nan YANG. Study on Uncertainty Operation Optimization of Virtual Power Plant Based on Intelligent Prediction Model Under Climate Change [J]. Power Generation Technology, 2023, 44(6): 790-799.
[6]	Yanping ZHANG, Yuchao ZHANG, Yifei LIU. Optimization Design of Trough Solar Power Plant Based on Probabilistic Reliability [J]. Power Generation Technology, 2020, 41(6): 590-598.
[7]	Yunhui SHI,Chuangxin GUO,Xiao DING. Integrated Energy System Economic Dispatch Based on Affine Adjustable Robust Optimization [J]. Power Generation Technology, 2020, 41(2): 118-125.
[8]	Yunhui SHI,Chuangxin GUO. Optimal Dispatch of Integrated Energy System Considering Operational Risks in the Presence of Energy Storage [J]. Power Generation Technology, 2020, 41(1): 56-63.
[9]	Bin ZHANG,Chao ZHANG,Xiaojuan HAN. Research on Load Frequency Control Strategy Based on Cloud PI Controller in Large-scale Wind Power Connected to Grid [J]. Power Generation Technology, 2019, 40(6): 516-520.
[10]	LIU Shan-bo, YU Ding-yi. Application on Uncertainty Analysis in Boiler Efficiency Test [J]. Power Generation Technology, 2017, 38(1): 44-47.