Abstract: [Objectives] The multi-mode fault signals such as vibration anomalies with nonlinear and multi-scale characteristics may be generated during the operation of turbine units. It is difficult for traditional time series models to capture long range dependencies, and the single modal analysis method cannot integrate heterogeneous data features effectively. In response to the above issues, this paper proposes a multimodal heterogeneous graph hybrid feature extraction model consisting of Transformer Gram Angular Summation Fields (GASF) - Recurrent Plot (RP) - Two dimensional (2D) - Gate Recurrent Unit (GRU), aiming to improve the reliability and generalization ability of fault diagnosis. [Methods]Firstly, the time series data of hydroturbine units are converted into 2D images, GASF and RP methods are used to extract spatial features of time series data, and Transformer model is constructed. Meanwhile, GRU is used to capture dynamic time series features, and multi-modal feature fusion is used to combine temporal features, image spatial features and heterogeneous image features. Thus, the accuracy and robustness of fault identification are significantly improved. [Results]The proposed method shows higher accuracy and stronger generalization ability in the fault diagnosis task of hydraulic turbine, and the diagnosis accuracy reaches 100% on multiple test sets. [Conclusions]The method can effectively fuse time series data and image features, significantly enhance the model's ability to recognize nonlinear fault modes, and accurately capture the abnormal state of the device. 

Key words: water turbine, fault diagnosis, Transformer, optimization algorithm, gate recurrent unit, feature extraction, gram angular summation fields, recurrent plot, 2D image