Online-Seminarserie: Künstliche Intelligenz in der Hydrometeorologie

Chaopeng Shen, Pennsylvania State University:

Differentiable parameter learning and differentiable modeling

Abstract
Big data and artificial intelligence (AI) methods are revolutionizing how knowledge is gained and predictions are made for sustainability sciences and the global environment. However, purely data-driven AI models often suffer from performance penalties when applied to data-scarce regions or extreme regimes, besides lacking interpretability. Here we show that differentiable models (a genre of physics-informed machine learning where gradients can be rapidly computed via a range of methods, allowing process-based equations to be seamlessly trained together with neural networks, https://t.co/qyuAzYPA6Y) are well-suited to capture unseen extremes because they utilize physical principles like mass balances and first-order exchanges to restrict the role of neural networks. They offer highly competitive performance and simultaneously provide physical process clarity. We demonstrate how the next-generation differentiable national-scale hydrologic model, multivariate water quality and ecosystem models achieve state-of-the-art results and open new avenues of learning. To improve computational efficiency, we further consider training highly efficient surrogate models for these models (mostly parameterized ordinary or partial differential equations) with the correct sensitivity. Recent advances in surrogate modeling, particularly deep learning frameworks like the Fourier Neural Operator (FNO), have demonstrated significant efficiency in approximating solution paths. However, these approximations result in inaccurate solutions to inverse problems due to inaccurate sensitivities and are highly sensitive to concept drift. We propose Sensitivity-Constrained Fourier Neural Operators (SC-FNO, presented in AI conference ICLR 2025, main session). SC-FNO ensures accuracy in the solution paths, inverse problems, and sensitivity calculations, even under sparse training data or concept drift scenarios. Differentiable modeling together with sensitivity-constrained neural operators are posed to drastically improve our simulation and learning capabilities for a wide range of engineering and geoscientific problems.

Biographical Sketch
Chaopeng Shen is Professor in Civil Engineering at The Pennsylvania State University. He received the Ph.D. degree in environmental engineering from Michigan State University, East Lansing, MI, USA, in 2009. His PhD research focused on computational hydrology and he developed the hydrologic model Process-based Adaptive Watershed Simulator (PAWS), which was later coupled to the community land model to study the interactions between hydrology and ecosystem. His recent efforts focused on harnessing the big data and machine learning (ML) and physics-informed ML opportunities in advancing hydrologic predictions and understanding. As an early advocate for ML in geosciences, he has written technical, editorial, review and collective opinion papers on hydrologic deep learning to call to attention the emerging opportunities for scientific advances. He currently promotes differentiable modeling which seamlessly integrates neural networks and physics for knowledge discovery.