author: Chen Yutong, Wang Ya, Huang Gang, et al.
作者：陈昱同，汪亚，黄刚，等.
keywords: physics-constrained deep learning, graph neural network, precipitation forecast
关键词：物理约束深度学习，图神经网络，降水预报
Abstract
Precipitation exerts far-reaching impacts on both socio-economic fabric and individual well-being, necessitating concerted efforts in accurate forecasting. Deep learning (DL) models have increasingly demonstrated their prowess in forecasting meteorological elements. However, traditional DL prediction models often grapple with heavy rainfall forecasting. In this study, we propose physics-informed localized graph neural network (GNN) models called ω-GNN and ω-EGNN, constrained by the coupling of physical variables and climatological background to predict precipitation in China. These models exhibit notable and robust improvements in identifying heavy rainfall while maintaining excellent performance in forecasting light rain by comparing to numerical weather prediction (NWP) and other DL models with multiple perturbation experiments in different data sets. Surprisingly, within a certain range, even when a DL model utilizes more input variables, GNN can still maintain its advantage. The methods to fuse physics into DL model demonstrated in this study may be promising and call for future studies.
降水深刻影响着社会经济和个人福祉，需要学界共同努力来让预报更加精准。虽然近年的深度学习（DL）模型预报气象要素的能力越来越强，但传统的深度学习算法仍然难以预测出强降水事件。本研究提出了一组两套图神经网络降水预报模型ω-GNN和ω-EGNN，这组模型被物理量的耦合规律和大尺度气候背景所约束。在预报中国全境降水的任务中，相比于数值天气预报（NWP）和其他深度学习模型，本研究提出的物理约束图神经网络模型不仅显著且稳定地改善了强降水的预报结果，而且小降水的预报水平也没有下降。即使给其他深度学习模型在一定范围内输入更多信息，它们的预报效果也没能超过这组物理约束的图神经网络模型。可见，这种通过图神经网络融合物理与深度学习的方法有进一步研究的前景。
https://doi.org/10.1029/2023GL106676
Point 1. Why physics-constrainted DL?
I am going to write a post specially for this issue. Please look forward to it.
我打算另开一帖来谈谈我对这个问题的理解，敬请期待。
Point 2. A graph of precipitation process
A graph is a data structure composed of vertices and edges. Convolution can be operated on a graph, similar to convolutional neural network (CNN), in which grid data can also be seen as a kind of graph. As shown in the figure, the value on a center node (node of interest, NoI) will be aggregated with that of its neighbour nodes. Such aggregation is something like the "coupling" of each NoI and its neighbour nodes. Many layers of graph convolution form a graph neural network (GNN), which can realize the coupling between high-order neighbour nodes.
GNN offers a potential tool to represent the coupling of physical variables in a specific process by putting physical variables on the vertices and connecting the related couple. Such a prior structure may reduce the degree of freedom of the deep learning (DL) model. In this study, a graph was abstracted from the precipitation process. The precipitation process was simply attributed to 2 conditions (physical variables): relative humidity (RH) and ascending motion (ω). ω was replaced by the coupling of other primary physical variables according to the ω-equation, as the data of ω is rare and low quality.
$$ (\partial_{pp} + \frac{\sigma}{f^2} \nabla^2) \omega \approx -\frac{1}{f} \partial_p {Adv}_\zeta - \frac{R}{pf^2} \nabla^2 {Adv}_T \tag{2.1}\label{2.1} $$The GNN based on this abstracted graph is the core of the proposed physics-constrained DL model.
图（Graph）是一种由顶点（Vertex）和边（Edge）组成的数据结构。在图上也可以进行类似卷积神经网络（CNN）中的卷积运算（格点数据也可以视为一种图）。如图所示，图卷积（Graph Convolution）把中心节点（NoI）和它周围的邻居节点上的值混合在一起运算，得到的结果再赋给中心节点。这种运算可以形象地称为“聚合函数”（Aggregation Functi-on），具体算法按需拟定。聚合函数在形式上可以看作中心节点与邻居节点的某种“耦合”。多个图卷积层相连就能使中心节点与更高阶的邻居节点耦合，这就是图神经网络（GNN）。
对于一个物理过程，如果把涉及的物理量放在顶点，再把两两间的相互作用连起来，就能用图来表征这个过程。在耦合的过程中具体发生了什么，由GNN自己学习。如此相当于提前告知了深度学习模型一些模糊的物理知识，模型只需要在此基础上深化认知，任务难度也随之下降了，否则还要从头学起——这就是给模型施加了一种物理约束，降低了模型的自由度。本研究的核心就是把降水过程抽象成图，以此来约束深度学习模型。
降水的发生要满足两个因素：1.充足的水汽（相对湿度RH）；2.要有抬升运动（垂直速度ω）（暂不考虑凝结核的因素）。ω是个很难观测和计算的量，因此，本研究根据ω-方程$ \eqref{2.1} $所反应的耦合关系，用其他更基本的物理量替换了ω。由此构造出表征降水的物理过程的图，并建立GNN。
Point 3. Embedding climate background
In this study, only regional data was used -- specifically, grid data around China. However, what about the information outside China, for example, the climate background of larger scales like ENSO (of which the signal is on the Pacific Ocean far from China), which can also strongly influence China's weather and climate? In fact, action at a distance doesn't exist, so the large-scale climate background and its signals far away will first propagate to and act on the local conditions. That means, the large-scale information is essentially contained in the local data.
There's more to the story. Again, local data potentially contains information on the climate background, which doesn't necessarily mean DL models can really capture this subtle information. Prior information will help DL models extract specific features more effectively, but how to do this?
It's not worthwhile to input global data into the model, or we could have build a global model rather than a local model. Climate background can be conveniently described by climate indices like Nino 3.4 and etc. More generally speaking, the ENSO effect on China has only 3 types: positive phase, normal phase, and negative phase. They are sparse. Some other factors similarly take sparse values, such as season (spring, summer, autumn, winter). The problem is to make the DL model understand these sparse factors, or vice versa, make the sparse factors a form that can be taken in by the DL model.
Entity embedding is such a technique (Guo and Berkhahn, 2016). It can map the sparse factors to dense, high-dimensional vectors according to the training data. Through entity embedding, this study turned ENSO phases, seasons, lead times, and valid times into understandable and learnable prompts for the DL model.
本研究只用到了中国附近的格点数据。那中国以外更大尺度的气候背景对中国的影响又怎么办？比如ENSO，其信号定义在南太平洋深处，并不在本研究划定的范围内。这一点其实不必担心，因为物理上并不存在“超距作用”，大尺度气候背景和遥远处的气候信号也必须先传播到研究区域内，对研究区域的大气状况产生作用，最终才能影响到当地的天气。换句话说，研究区域内的数据原来已经包含了更大尺度气候背景的信息。但这还不够，因为深度学习模型不能保证捕捉到这些微妙的气候背景信息。
还是之前的想法，要用先验知识提示模型去学习特定的特征。没必要把全球格点数据都输入模型，那样的话做区域模型就失去意义了。即便是大尺度的气候背景也可以只用一个“气候指数”值来表示，比如Nino3.4指数，用起来十分方便。按这种思路进一步简化，ENSO对中国大气状况的影响也不过三种情况：正相位、中性相位和负相位。这种分法得到的结果具有稀疏性，类似的还有季节（分为春夏秋冬四季）等。现在的问题在于，如何让深度学习模型“理解”这些取值稀疏的因子，或者反过来说，怎么把这些稀疏的值转换为深度学习模型可以“理解”的形式。
实体嵌入（Entity Embedding）技术能把稀疏的对象映射到稠密的高维空间里，变成可以输入深度学习模型中的高维向量（Guo and Berkhahn, 2016）。本研究用这种方法把ENSO相位、季节、提前时间和预报时间都作为可习得的先验“提示”输入给深度学习模型。
Point 4. The architecture of the model
The figure below shows the architecture of the study's proposed DL model. Generally speaking, it is structured as an "Encoder-Coupler-Decoder". The CNN Encoder extracts low-dimensional features (namely, "encoding") from the six input variables respectively, and then the six features will be coupled in the Coupler. The Coupler is a GNN based on the graph presented in Point 2. After coupling, the coupled feature of precipitation will be up-sampled to a prognostic field. Between the Encoder and the Decoder are the skip connections, like in the U-net.
Notice that the embedding vector of the climate background (see in Point 3) was concatenated to the input variable fields. This study proposed two GNN models: the one with the GNN Coupler (see in Point 2) but without the climate background embedding called ω-GNN, and the one with both the GNN Coupler and the climate background embedding called ω-EGNN.
如下图所示，本研究提出的深度学习模型大体上具有“编码器-耦合器-解码器”的结构。此处的编码器是一个CNN，负责提取输入数据的低维特征，此谓编码。把六个输入变量分别编码，再用Point 2中所述的GNN作为耦合器，对这六个变量的特征向量进行耦合运算。经过耦合后，把其中降水的特征向量挑出来，由上采样产生出预报场，即是解码。编码器和解码器之间还有类似U-net中的跳跃连接（Skip Connection）直接相连。注意：Point 3中提到的由气候背景因子转化成的向量直接和输入的变量场拼接在一起，以这种形式接受模型的后续处理。研究测试了两个GNN耦合模型，一个只有GNN耦合器，没有气候背景嵌入，称为ω-GNN；一个既有GNN耦合器，又有气候背景嵌入，称为ω-EGNN。
Results
Compared to numerical weather prediction (NWP) and other DL models, both the ω-GNN (with the GNN Coupler only) model and the ω-EGNN (with both the GNN Coupler and the climate background embedding) significantly improved the precipitation prediction, especially that of heavy precipitation. They also gained better stability than other DL models. ω-EGNN is slightly superior to ω-GNN. The two GNN models, as well as other DL models, were trained on two datasets of different sources, to find that the superiority of the two GNN models was maintained across different datasets.
As to the physical interpretability, the figure in Point 3 shows that the ω-EGNN model successfully distinguished between the four seasons and between the different ENSO phases, which demonstrates that the model can indeed learn the knowledge about these climate backgrounds.
ω-GNN模型和ω-EGNN模型均能显著地改善降水、特别是强降水的预报水平；相比其他深度学习模型，结果也更加稳定。二者之中，ω-EGNN模型略胜一筹。把实验用到的所有深度学习模型都在两个来源不同的数据集上分别训练，则发现两个GNN耦合模型的优势能够泛化到不同的数据集上。
除了模型本身架构（GNN耦合器）所蕴含的物理意义，还可以从气候背景嵌入的角度观察ω-EGNN模型的物理可解释性。从Point 3的图中可以看到，不同的季节和ENSO相位已经被区分开来，说明模型成功地学到了相关的知识。
