In this study, the authors propose a framework for developing wall models for large-eddy simulation (LES) that can effectively capture pressure-gradient effects using multi-agent reinforcement learning. The framework utilizes distributed reinforcement learning agents that receive off-wall environmental states, including pressure gradient and turbulence strain rate, allowing adaptability to a wide range of flows characterized by pressure-gradient effects and separations. The agents in the framework determine an action to adjust the wall eddy viscosity based on these states, thereby influencing the wall-shear stress. Importantly, the model training is conducted in-situ with wall-modeled LES grid resolutions and does not rely on instantaneous velocity fields from high-fidelity simulations. Throughout the training process, the agents compute rewards from the relative error in estimating the wall-shear stress. This allows them to refine an optimal control policy that minimizes prediction errors. To validate their approach, the authors train wall models for two distinct subgrid-scale models using low-Reynolds-number flow over periodic hills. These trained models are then tested through simulations of flows over periodic hills at higher Reynolds numbers and flow over the Boeing Gaussian bump. The results demonstrate that the developed wall models successfully capture acceleration and deceleration of wall-bounded turbulent flows under pressure gradients and outperform equilibrium wall model in predicting skin friction. In addition to their proposed framework, this study provides further context by discussing existing experimental data available for validating computational fluid dynamics (CFD) codes and various computational approaches used for simulating flow around bumps. Previous studies have evaluated and compared methods such as Reynolds-averaged Navier–Stokes (RANS), direct numerical simulation (DNS), hybrid LES–DNS, WMLES (wall–modeled LES), and detached–eddy simulations (DES). Specifically, previous WMLES studies have shown that classical equilibrium wall models (EQWM) still have limitations in accurately predicting separation bubbles and peak values of skin friction. To address these limitations, a detailed comparison between simulations using EQWM and their developed reinforcement learning wall model (RLWM) is conducted which includes evaluating mean skin friction, mean pressure on bump surface as well as prediction of velocity field. The results highlight improved performance of RLWM compared to EQWM. Overall, this study presents a novel framework for developing wall models in large–eddy simulation that effectively captures pressure–gradient effects using multi–agent reinforcement learning. The validation results demonstrate its superiority over traditional equilibrium wall models in predicting skin friction and accurately simulating flows under pressure gradients.
- - Authors propose a framework for developing wall models for large-eddy simulation (LES) using multi-agent reinforcement learning
- - Framework utilizes distributed reinforcement learning agents to adjust wall eddy viscosity based on off-wall environmental states
- - Model training is conducted in-situ with wall-modeled LES grid resolutions, not relying on high-fidelity simulations
- - Agents compute rewards from relative error in estimating wall-shear stress to refine optimal control policy
- - Trained models successfully capture acceleration and deceleration of wall-bounded turbulent flows under pressure gradients
- - Outperform equilibrium wall model in predicting skin friction
- - Study provides context by discussing existing experimental data and computational approaches for simulating flow around bumps
- - Comparison between simulations using equilibrium wall model (EQWM) and reinforcement learning wall model (RLWM) highlights improved performance of RLWM
The authors of a study have come up with a way to make computer models that can predict how air or water flows around objects. They use a special kind of learning called reinforcement learning, where the computer learns by trying different things and getting rewards for doing well. The models are trained using simulations on a computer, not real-life experiments. The models are good at predicting how fast the flow is and how it changes when there is more or less pressure. They work better than other models that scientists have used before."
Definitions- Framework: A plan or structure for doing something.
- Reinforcement learning: A type of learning where a computer tries different things and gets rewards for doing well.
- Eddy viscosity: How thick or sticky the air or water is near a surface.
- Simulation: Using a computer program to create a model or representation of something happening in real life.
- Turbulent flow: When air or water moves in an unsteady and chaotic way.
Developing Wall Models for Large-Eddy Simulation with Multi-Agent Reinforcement Learning
Large-eddy simulation (LES) has become an increasingly popular tool for simulating turbulent flows. However, the accuracy of LES is dependent on the wall model used to capture pressure gradient effects. In this study, researchers propose a novel framework that utilizes multi-agent reinforcement learning (RL) to develop wall models that can effectively capture these effects. The proposed RLWM outperforms traditional equilibrium wall models (EQWM) in predicting skin friction and accurately simulating flows under pressure gradients.
Background
Previous studies have evaluated various computational approaches for simulating flow around bumps, including Reynolds-averaged Navier–Stokes (RANS), direct numerical simulation (DNS), hybrid LES–DNS, WMLES (wall–modeled LES), and detached–eddy simulations (DES). Specifically, previous WMLES studies have shown that classical EQWMs still have limitations in accurately predicting separation bubbles and peak values of skin friction. To address these limitations, this study proposes a new approach using multi-agent reinforcement learning agents to develop more accurate wall models.
Framework Overview
The proposed framework utilizes distributed RL agents that receive off-wall environmental states such as pressure gradient and turbulence strain rate from the flow field. These agents determine an action to adjust the eddy viscosity based on these states which influences the wall shear stress. During training, rewards are computed from relative errors in estimating the wall shear stress allowing them to refine an optimal control policy that minimizes prediction errors.
Validation Results
To validate their approach, two distinct subgrid scale models were trained using low Reynolds number flow over periodic hills then tested through simulations of higher Reynolds numbers and flow over a Boeing Gaussian bump. The results demonstrate improved performance of RLWM compared to EQWM when evaluating mean skin friction, mean pressure on bump surface as well as velocity fields predictions around bumps with acceleration or deceleration due to pressure gradients present in the flow field.
Conclusion
This research presents a novel framework for developing effective wall models in large–eddy simulation using multi–agent reinforcement learning which outperforms traditional equilibrium wall models in predicting skin friction and accurately simulating flows under pressure gradients. This study provides further context by discussing existing experimental data available for validating CFD codes as well as various computational approaches used for simulating flow around bumps making it an important contribution towards improving our understanding of turbulent flows and their modeling capabilities with LES techniques