基于计算机视觉的三维重建技术在燃气轮机行业的应用及展望

doi:10.12096/j.2096-4528.pgt.21031

发电技术 ›› 2021, Vol. 42 ›› Issue (4): 454-463.DOI: 10.12096/j.2096-4528.pgt.21031

基于计算机视觉的三维重建技术在燃气轮机行业的应用及展望

孔祥玲¹(), 付经伦¹^,²^,³^,⁴^,⁵()

¹ 中国科学院工程热物理研究所南京未来能源系统研究院, 江苏省南京市 210000
² 中国科学院工程热物理研究所先进燃气轮机实验室, 北京市海淀区 100190
³ 中国科学院大学, 北京市海淀区 100049
⁴ 中国科学院先进能源动力重点实验室, 北京市海淀区 100190
⁵ 中国科学院轻型动力创新研究院, 北京市海淀区 100190

收稿日期:2021-04-25 出版日期:2021-08-31 发布日期:2021-07-22
作者简介:孔祥玲(1984), 女, 博士, 副研究员, 从事机器人运动控制、机器视觉、最优化理论及应用、路径优化控制等方面的研究, kongxiangling@njiet.cn
付经伦(1979), 女, 博士, 研究员, 研究方向为燃气轮机透平气动冷却设计及数字化, fujl@iet.cn
基金资助:
国家自然科学基金项目(51776201)

Computer-Vision Based on Three-dimensional Reconstruction Technology and Its Applications in Gas Turbine Industry

Xiangling KONG¹(), Jinglun FU¹^,²^,³^,⁴^,⁵()

¹ Nanjing Institute of Future Energy System, IET, CAS, Nanjing 210000, Jiangsu Province, China
² Advanced Gas Turbine Laboratory, IET, CAS, Haidian District, Beijing 100190, China
³ University of Chinese Academy of Sciences, Haidian District, Beijing 100049, China
⁴ Key Laboratory of Advanced Energy and Power, CAS, Haidian District, Beijing 100190, China
⁵ Innovation Academy for Light-duty Gas Turbine, CAS, Haidian District, Beijing 100190, China

Received:2021-04-25 Published:2021-08-31 Online:2021-07-22
Supported by:
National Natural Science Foundation of China(51776201)

摘要/Abstract

摘要：

燃气轮机的整机性能及运行安全与其各部件的性能和工作状态密切相关，部件性能优劣和健康状态直接反映在表面的形貌上。因此，根据部件的形貌特征对其性能及健康状态进行判读，是评估燃机设计和健康状态的最为高效的方法，而三维重建技术是实现这一方法的关键技术。首先围绕基于计算机视觉的三维重建，系统地阐述了基于单目视觉、基于双目视觉及基于深度学习的三维重建技术及其发展现状；之后讨论了三维重建技术在燃气轮机行业的发展现状及可能的发展方向；最后，以燃气轮机透平叶片为对象，比较了基于阴影信息及多视图学习网络的三维重建效果并讨论了其各自的优缺点。

关键词: 燃气轮机, 计算机视觉, 三维重建, 燃机设计, 健康诊断

Abstract:

The operational safety and performance of gas turbine is closely related to the health status of its components. Generally, the status of gas turbine components can be evaluated by the morphological characters, such as size, shape, color, etc. Therefore, the method of feature extraction, especially the three- dimensional (3D) feature extraction, has become a key technology for component health status assessment. The existing computer vision based on 3D reconstruction algorithms was reviewed. After that, the research status and development directions of the 3D reconstruction in the gas turbine industry were discussed. In the end, the reconstruction results of a turbine blade using the shape from shading (SFS) algorithm and the Mvsnet were presented and compared.

Key words: gas turbine, computer vision, three-dimensional reconstruction, gas turbine design, health diagnosis

中图分类号:

TK05

孔祥玲, 付经伦. 基于计算机视觉的三维重建技术在燃气轮机行业的应用及展望[J]. 发电技术, 2021, 42(4): 454-463.

Xiangling KONG, Jinglun FU. Computer-Vision Based on Three-dimensional Reconstruction Technology and Its Applications in Gas Turbine Industry[J]. Power Generation Technology, 2021, 42(4): 454-463.

图/表 11

图1 结构光成像原理

Fig.1 Functional principle of structured-light imaging

表1 基于结构光信息的三维重建算法比较

Tab. 1 Comparison of structured-light based on 3D reconstruction algorithms

项目	算法名称
项目	莫尔等高线法	相位法	傅里叶变换法
光栅类型	主光栅、基准光栅	正弦光栅	罗奇光栅或正弦光栅
优点	精度高、鲁棒性强	精度较高、速度快	精度较高、速度快
缺点	合成大面积莫尔条纹困难、速度慢	易受高频噪声和散斑影响而产生误差	易受高频噪声和散斑影响而产生误差
适用范围	小部件的精密测量	大部件的快速测量	大部件的快速测量

图2 激光三角测量原理

Fig.2 Functional principle of laser triangulation

图3 基于光度立体技术的三维重建原理示意图

Fig.3 Functional principle of photometric stereo

图4 双目立体视觉的基本原理

Fig.4 Functional principle of stereo vision

图5 基于双目立体视觉的三维重建处理流程

Fig.5 Processing steps of 3D reconstruction based on binocular stereo-vision

图6 PointOutNet网络结构及其应用

Fig.6 PointOutNet structure and its applications

图7 3D-R2N2网络结构

Fig.7 Overview of 3D-R2N2 network

图8 ResMeshNet网络结构及其应用

Fig.8 Diagram of ResMeshNet architecture and

图9 基于明暗信息的三维重建法举例

Fig.9 Examples of shape from shading

图10 基于Mvsnet深度网络的深度重建

Fig.10 Depth map reconstruction by using Mvsnet deep learning frame

参考文献 50

1	方继辉, 王荣. 重型F级燃气轮机IGV开度对压气机效率的影响[J]. 发电技术, 2020, 41 (3): 317- 319.
	FANG J H , WANG R . Influence of IGV opening degree on the compressor efficiency of MITSUBISHI F4 gas turbine[J]. Power Generation Technology, 2020, 41 (3): 317- 319.
2	BUTIME J, GUTIERREZ I, CORZO L G, et al. 3D reconstruction methods, a survey[C]//Proceedings of the First International Conference on Computer Vision Theory and Applications, 2015: 457-463.
3	CHEN X , LU C , MA M , et al. Color-coding and phase-shift method for absolute phase measurement[J]. Optics Communications, 2013, 298/299, 54- 58. DOI
4	TAKEDA M , MUTOH K . Fourier transform profilometry for the automatic measurement of 3-D object shapes[J]. Applied Optics, 1983, 22 (24): 3977- 3982. DOI
5	FARAJIKHAH S , MADANIPOUR K , SAHARKHIZ S , et al. Shadow moiré aided 3-D reconstruction of fabric drape[J]. Fibers and Polymers, 2012, 13 (7): 928- 935. DOI
6	IDESAWA M , YATAGAI T , SOMA T . Scanning moiré method and automatic measurement of 3D shapes[J]. Applied Optics, 1977, 16 (8): 2152- 2162. DOI
7	SRINIVASAN V , LIU H C , HALIOUA M . Automated phase-measuring profilometry of 3-D diffuse objects[J]. Applied Optics, 1984, 23 (18): 3105- 3108. DOI
8	TAVARES P J , VAZ M A . Linear calibration procedure for the phase-to-height relationship in phase measurement profilometry[J]. Optics Communications, 2007, 274 (2): 307- 314. DOI
9	沈洋, 陈文静. 抽样对复合傅里叶变换轮廓术的影响[J]. 激光技术, 2008, 32 (1): 80- 83.
	SHEN Y , CHEN E J . Influence of sampling on composite Fourier-transform propfilometry[J]. Laser Technology, 2008, 32 (1): 80- 83.
10	HORN B. Shape from shading: a method for obtaining the shape of a smooth opaque object from one view[D]. Cambridge: Massachusetts Institute of Technology, 1970.
11	HORN B . Obtaining shape from shading information[M]. New York: McGraw-Hill, 1989: 123- 171.
12	王学梅. 不同成像条件的从明暗恢复形状算法研究[D]. 长沙: 国防科学技术大学, 2009.
	WANG X M. Research on shape recovery algorithm from light and shade under different imaging conditions[D]. Changsha: National University of Defense Technology, 2009.
13	BROOKS M J, HORN B. Shape and source from shading[R]. Cambridge: Massachusetts Institute of Technology, 1985.
14	PARAGIOS N , CHEN Y , FAUGERAS O . Handbook of mathematical models in computer vision[M]. New Nork: Springer, 2006: 108- 112.
15	孙玉娟. 基于光学图像的三维重建理论与技术[M]. 北京: 清华大学出版社, 2017: 125- 126.
	SUN Y J . Theory and technology of 3D reconstruction based on optical image[M]. Beijing: Tsinghua University Press, 2017: 125- 126.
16	ROUY E , TOURIN A . A viscosity solutions approach to shape-from-shading[J]. SIAM Journal on Numerical Analysis, 1992, 867- 884.
17	KIMMEL R , BRUCKSTEIN A M . Tracking level sets by level sets: a method for solving the shape from shading problem[J]. Computer Vision and Image Understanding, 1995, 62 (1): 47- 58. DOI
18	ZHANG R , TSAI P S , CRYER J E , et al. Shape from shading: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 21 (8): 690- 706. DOI
19	PENTLAND A . Shape information from shading: a theory about human perception[J]. Spat Vis, 1989, 4 (2/3): 165- 182.
20	TSAI P , SHAH M . Shape from shading using linear approximation[J]. Image and Vision Computing, 1994, 12 (8): 487- 498. DOI
21	ZHANG S , NEGAHDARIPOUR S . 3-D shape recovery of planar and curved surfaces from shading cues in underwater images[J]. IEEE Journal of Oceanic Engineering, 2002, 27 (1): 100- 116. DOI
22	COOPER A P R . A simple shape-from-shading algorithm applied to images of ice-covered terrain[J]. IEEE Transactions on Geoscience and Remote Sensing, 1994, 32 (6): 1196- 1198. DOI
23	CHO S Y , CHOW T W S . Shape and surface measurement technology by an improved shape-from-shading neural algorithm[J]. IEEE Transactions on Industrial Electronics, 2000, 47 (1): 225- 230. DOI
24	KONG F H. Reconstruction of solder joint surface based on hybrid shape from shading[C]//International Conference on Natural Computation. IEEE Computer Society, 2008: 593-597.
25	CASTELAN M, HANCOCK E R. Acquiring height maps of faces from a single image[C]//3D Data Processing, Visualization, and Transmission, International Symposium. IEEE Computer Society. Thessaloniki: IEEE, 2004: 183-190.
26	MCGUNNIGLE G , DONG J . Augmenting photometric stereo with coaxial illumination[J]. IET Computer Vision, 2011, 5 (1): 33- 49. DOI
27	ZHENG Q, CHELLAPPA R. Estimation of illuminant direction, albedo, and shape from shading[C]//IEEE Computer Society Conference on Computer Vision & Pattern Recognition. Maui: IEEE, 2002: 540-545.
28	WOODHAM R J . Photometric method for determining surface orientation from multiple images[J]. Optical Engineering, 1992, 19 (1): 151- 171.
29	GEORGHIADES A S , BELHUMEUR P N , KRIEGMAN D J . From few to many: illumination cone models for face recognition under variable lighting and pose[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 23 (6): 643- 660.
30	HAYAKAWA H . Photometric stereo under a light source with arbitrary motion[J]. Journal of the Optical Society of America A, 1994, 11 (11): 3079- 3089. DOI
31	PAPADHIMITRI T, FAVARO P. A new perspective on uncalibrated photometric stereo[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE, 2013: 1474-1481.
32	WOODHAM R J . Gradient and curvature from the photometric-stereo method, including local confidence estimation[J]. Journal of the Optical Society of America A, 1994, 11 (11): 3050- 3068. DOI
33	COLEMAN E N , JAIN R . Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry[J]. Computer Graphics and Image Processing, 1982, 18 (4): 309- 328. DOI
34	HERTZMANN A , SEITZ S M . Example-based photometric stereo: shape reconstruction with general, varying BRDFs[J]. IEEE Computer Society, 2005, 27 (8): 1254- 1264.
35	ZHANG Z . A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22 (11): 1330- 1334. DOI
36	CAI J . Integration of optical flow and dynamic programming for stereo matching[J]. IET Image Processing, 2012, 6 (3): 205- 212. DOI
37	ANSARI M E, MAZOUL A, BENSRHAIR A, et al. A real-time spatio-temporal stereo matching for road applications[C]//International IEEE Conference on Intelligent Transportation Systems. Washington: IEEE, 2011: 1483-1488.
38	STOJAKOVIC V , TEPAVCEVIC B . Image-based modeling approach in creating 3D morphogenetic reconstruction of Liberty Square in Novi Sad[J]. Journal of Cultural Heritage, 2011, 12 (1): 105- 110. DOI
39	CALAKLI F , TAUBIN G . SSD: smooth signed distance surface reconstruction[J]. Computer Graphics Forum, 2011, 30 (7): 1993- 2002. DOI
40	MAURICIO K , YUSHI G , TUKI T , et al. Robust 3D image reconstruction of pancreatic cancer tumors from histopathological images with different stains and its quantitative performance evaluation[J]. International Journal of Computer Assisted Radiology and Surgery, 2019, 14 (12): 2047- 2055. DOI
41	SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA, 2015.
42	ZHONG Z, JIN L, XIE Z, et al. High performance offline handwritten Chinese character recognition using googlenet and directional feature maps[C]//Proceeding of the 201513th International Conference on Document Analysis and Recognition (ICDAR). Tunis, Tunisia, 2015: 846-850.
43	HE K M, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA, 2016.
44	FAVALLI M , FORNACIAI A , ISOLA I , et al. Multiview 3D reconstruction in geosciences[J]. Computers & Geosciences, 2012, 44, 168- 176.
45	FAN H, HAO S, GUIBAS L. A point set generation network for 3D object reconstruction from a single image[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 605-613.
46	CHOY C B, XU D, GWAK J, et al. 3D-R₂N₂: a unified approach for single and multi-view 3D object reconstruction[C]//Proceeding of the European Conference on Computer Vision. 2016.
47	PAN J, LI J, HAN X, et al. Residual meshnet: learning to deform meshes for single-view 3D reconstruction[C]//Proceeding of the 2018 International Conference on 3D Vision, 2018.
48	RUAN C , YU T , CHEN F , et al. Experimental characterization of the spatiotemporal dynamics of a turbulent flame in a gas turbine model combustor using computed tomography of chemiluminescence[J]. Energy, 2019, 170, 744- 751. DOI
49	WANG Q , ZHANG Y . Spark characteristics investigation of a gas turbine igniter[J]. Combustion Science and Technology, 2012, 184 (10/12): 1526- 1540.
50	HUANG J , LIU H , WANG Q , et al. Limited-projection volumetric tomography for time-resolved turbulent combustion diagnostics via deep learning[J]. Aerospace Science and Technology, 2020, 106, 106123. DOI

[1]	崔则阳, 孔祥玲, 付经伦, 施佳君. 一种基于图像的燃气轮机叶型参数测量方法[J]. 发电技术, 2024, 45(1): 106-112.
[2]	杨旸, 李耀强, 张金琦. 基于数值方法的燃气轮机贫预混旋流燃烧室单头部结构设计[J]. 发电技术, 2023, 44(5): 712-721.
[3]	杨旸, 郭德三, 李耀强, 张金琦. 燃气轮机贫预混多旋流组合燃烧室头部结构设计[J]. 发电技术, 2023, 44(2): 183-192.
[4]	张弘毅, 曲立涛. 9F级燃机选择性催化还原脱硝数值模拟研究与应用[J]. 发电技术, 2023, 44(1): 78-84.
[5]	金云峰, 刘超, 邓高峰, 关运龙, 郝建刚, 黄海舟, 蒋东翔. 燃气轮机进气过滤系统维修周期经济性分析[J]. 发电技术, 2022, 43(1): 119-125.
[6]	白明亮, 张冬雪, 刘金福, 刘娇, 于达仁. 基于深度自编码器和支持向量数据描述的燃气轮机高温部件异常检测[J]. 发电技术, 2021, 42(4): 422-430.
[7]	邱彬, 付经伦. 燃气轮机排气扩压器研究现状[J]. 发电技术, 2021, 42(4): 437-446.
[8]	魏凯, 罗忠, 孙永航, 王宇. 燃气轮机带阀片引射混合器内流特性分析[J]. 发电技术, 2021, 42(4): 431-436.
[9]	朱华, 严彪, 刘雨松, 李亮. 湿空气透平冷却技术研究[J]. 发电技术, 2021, 42(4): 412-421.
[10]	管金, 何宗泽, 吕小静, 翁一武. 30kW微型燃气轮机发电机组启动实验研究[J]. 发电技术, 2021, 42(4): 404-411.
[11]	金云峰, 刘超, 邓高峰, 关运龙, 田鑫, 黄海舟, 蒋东翔. 燃气轮机进气压差建模方法研究[J]. 发电技术, 2021, 42(4): 395-403.
[12]	黄友华, 马善为, 刘吉, 武正华. 燃气轮机烟气SCR脱硝系统优化设计与工程应用[J]. 发电技术, 2021, 42(3): 350-356.
[13]	方继辉,王荣. 重型F级燃气轮机IGV开度对压气机效率的影响[J]. 发电技术, 2020, 41(3): 317-319.

基于计算机视觉的三维重建技术在燃气轮机行业的应用及展望

Computer-Vision Based on Three-dimensional Reconstruction Technology and Its Applications in Gas Turbine Industry

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 50

相关文章 13

编辑推荐

Metrics

本文评价