Hybrid numerical methods for multiscale simulations of ... - CiteSeerX

SciDAC 2008 Journal of Physics: Conference Series 125 (2008) 012054

IOP Publishing doi:10.1088/1742-6596/125/1/012054

Hybrid numerical methods for multiscale simulations of subsurface biogeochemical processes T D Scheibe1, A M Tartakovsky1, D M Tartakovsky2, G D Redden3, P Meakin3,4, B J Palmer1 and K L Schuchardt1 1

Pacific Northwest National Laboratory, Richland, WA, 99352, USA University of California, San Diego, Department of Mechanical and Aerospace Engineering, La Jolla, CA, 92093, USA 3 Idaho National Laboratory, Idaho Falls, ID, 83415, USA 4 Physics of Geological Processes, University of Oslo, USA 2

E-mail: [email protected] Abstract. Many subsurface flow and transport problems of importance today involve coupled non-linear processes that occur in media exhibiting complex heterogeneity. Problems involving biological mediation of reactions fall into this class of problems. Recent experimental research has revealed important details about the physical, chemical, and biological mechanisms that control these processes from the molecular to laboratory scales. We are developing a hybrid multiscale modeling framework that combines discrete pore-scale models (which explicitly represent the pore space geometry at a local scale) with continuum field-scale models (which conceptualize flow and transport in a porous medium without a detailed representation of the pore space geometry). At the pore scale, we have implemented a parallel three-dimensional Lagrangian model of flow and transport using the smoothed particle hydrodynamics method and performed test simulations using millions of computational particles on the supercomputer at the Environmental Molecular Sciences Laboratory. We have also developed methods for gridding arbitrarily complex pore geometries and simulating pore-scale flow and transport using parallel implementations of grid-based computational fluid dynamics methods. Within the multiscale hybrid framework, we have coupled pore- and continuum-scale models to simulate coupled diffusive mixing, reaction, and mineral precipitation, and compared the results with conventional continuum-only simulations. The hybrid multiscale modeling framework is being developed using a number of SciDAC enabling technologies including the Common Component Architecture, advanced solvers, Grid technologies, scientific workflow tools, and visualization technologies.

1. Introduction Hybrid multiscale numerical modeling methods combine multiple models defined at fundamentally different length and time scales within the same overall spatial and temporal domain. In many cases, the models also have fundamentally different ways of representing the physical, chemical, and biological processes. For example, several models in the materials science and physics literature couple molecular dynamics (MD) simulations at the molecular scale to continuum mechanics (typically finite element) simulations at larger scales [15, 2, 3]. c 2008 IOP Publishing Ltd

1



Such simulations are motivated by classes of problems in which large-scale phenomena of interest (e.g., overall material failure and deformation) are strongly influenced by processes occurring at much smaller scales (e.g., microfracture propagation) that are not well represented by effective or averaged processes and/or properties. Executing an exhaustive simulation of processes at the smallest scales for a domain of engineering significance is currently impractical, and likely to remain so for a very long time. However, the hybrid multiscale approach, in which a small-scale model with high resolution is used in a small fraction of the overall domain and is linked to a large-scale model with coarse resolution over the remainder of the overall domain, can provide necessary efficiency of characterization and computation that may render solution of these problems practical. This SciDAC science spplication is developing and applying hybrid numerical techniques to simulate complex subsurface flow and biogeochemical reactions of relevance to subsurface contaminants at U.S. Department of Energy sites. Two related science application partnerships (SAPs) support this science application through the implementation of the Common Component Architecture for parallel model integration and development of a scientific workflow and data management environment to facilitate execution of complex model processes and management of simulation input and output data. This paper introduces a subsurface problem that motivates the hybrid multiscale approach and describes progress to date. 2. Motivation Important subsurface biogeochemical processes (e.g., microbial respiration or complexation of solutes with mineral surfaces) are currently best defined by events that take place at very small scales, typically ranging from molecular to cellular to small batch and column experimental scales (with time scales of minutes to days). However, the scales at which we are interested in predicting phenomena for engineering applications (e.g., fate and transport of contaminants in aquifers or geologic carbon sequestration) are very large, typically ranging from meters to many kilometers (with time scales of months to years or even a million years in the case of nuclear waster repositories). This disparity in scales creates computational challenges that motivate a hybrid modeling approach in which pore- and continuum-scale models execute concurrently within different portions of a model domain. In this paper, we focus specifically on porous media systems that exhibit strong coupling, with feedback, between flow, transport, and reaction processes, such as reactions involving biofilm development [12] and/or mineral precipitation [10]. In these systems, the biogeochemical reactions (controlled by flow and transport processes) lead to changes in pore structure, which in turn modify local flow and transport behavior. This coupling/feedback leads to strong non-linearity in apparent continuum-scale behavior, and this behavior is not well represented by averaged properties and state variables. An example is provided by the laboratory experiment in which two solutes mix and react to precipitate calcium carbonate minerals within a very narrow zone in the center of the experimental cell. We have performed pore- and continuum-scale simulations of these mixing and reaction processes. The pore-scale simulation results indicate that strong mixing of the two solutes is limited to a very local region (only a few grain diameters wide) and the mixing is inhibited by mineral precipitation leading to a strong negative feedback or coupling between the reaction and transport across the narrow mixing zone in which precipitation occurs. The conditions strictly required for validity of a continuum model are violated in the mixing region. Nevertheless, we found that a continuum model was able to qualitatively reproduce the observed phenomena if adaptive grid resolution was used in the mixing zone at a scale comparable to that of the grains of the porous media. However, the continuum model did not quantitatively predict the amount of mineral precipitation because of discrepancies between the pore- and continuum-scale effective reaction rates. Details of the experiment and models are reported in [11]. The hybrid multiscale modeling approach is well suited to this problem, as it allows for explicit pore-scale simulation in the mixing region while using the less computationally demanding continuum methods in the majority of the model domain.

2



3. Hybrid multiscale modeling Published reviews of hybrid multiscale modeling concepts are provided by [5, 6]. Hybrid multiscale modeling methods have been most widely applied in the fields of materials science and chemical engineering, in which atomic-scale models of molecular dynamics (MD) have been linked to continuum-scale models of material deposition, strength, deformation, and failure. A recent review of multiscale methods applied to materials science is given by [2]. Hybrid multiscale methods have also been applied in chemical engineering, catalysis and reactor processes (e.g., [14]), life sciences (e.g., [13]), and fluid mechanics (e.g., [4]). Previous applications in subsurface simulation are extremely limited [1]. We have implemented a two-scale hybrid model of diffusive mixing, reaction, and precipitation described in [11]. In this work, favorable comparisons were drawn between analytical solutions, a single-resolution model using high resolution throughout the model domain, and a two-scale hybrid model. Current efforts are focused on incorporating advection at both scales and on implementing a coupled SPH-finite element hybrid model. 4. Component-based pore-scale model We have developed a parallel three-dimensional SPH code utilizing the Common Component Architecture (CCA) programming framework. SPH is a particle-based numerical method that has been adapted for simulation of pore-scale flow and reactive transport [8, 9]. In SPH simulations continuous fields are represented by a superposition of smooth bell-shaped functions centered on a set of Lagrangian particles, and this provides a meshless discretization of the continuum hydrodynamic fields (i.e., Navier-Stokes equations). The SPH particles carry attributes such as the solvent and solute masses that allow transport and reactions to be simulated. The value of the continuum fields (e.g., velocity, concentration, density) is computed by convolving the values associated with neighboring particles using a smooth weighting function. SPH simulations are relatively easy to implement, and they can handle complex internal boundaries, free surface and moving boundaries with relative ease. A typical SPH simulation consists of a loop to integrate the equations of motion coupled with a force evaluation at each time step. The force evaluations involve interactions between each particle and nearby particles within a finite neighbourhood. For parallel simulation, particles are distributed among processors in spatial blocks. This approach requires communication of information among processors for particles that are near subdomain boundaries and for particles that move across boundaries within a time step. SPH code components include a particle integration component, force evaluation component, communication layer component, and setup and I/O components. The parallel SPH code has been developed using the CCAFFEINE implementation of CCA and runs on the supercomputer “MPP2” at the Environmental Molecular Sciences Laboratory (EMSL). Test simulations have been executed using the pore geometry of a sand pack measured by x-ray microtomography (courtesy of Brian Wood, Oregon State University) and the results were visualized by members of the SciDAC Institute for Ultrascale Visualization at UC Davis (see figure 1). We are currently working on incorporating reactions into the parallel code and testing the code on scientifically relevant and challenging pore-scale simulation problems. 5. Scientific workflow The application of high-performance computing to subsurface simulations creates the challenge of ever-increasing model complexity. Hybrid multiscale models as described above are one example of simulations of complex systems that generate large amounts of output data. Simulating uncertain processes often involves sensitivity analyses, Monte Carlo simulations, or other approaches that require the execution of multiple simulations with different input sets. Managing the resulting data (including metadata about the relationship among different simulations and recording of the process for a given simulation study) is a challenging task in this environment. To address this issue, we have applied workflow concepts and an integrated data and provenance system to automate these processes while providing verifiable and repeatable records. Our application 3



Figure 1. Visualization of water flow through a porous medium, calculated using the parallel SPH code developed by this project. Colored particles are SPH computational particles, where the color represents the local flow velocity (red is large, green is small). White regions represent solid grains of the porous medium. In the foreground portion of the image, the solid grains have been removed to clearly show the fluid particles and local streamlines. Visualization courtesy of Kwan-Liu Ma and Chad Jones, Institute for Ultrascale Visualization, University of California at Davis. is based on the Kepler workflow tool, developed by the SciDAC Scientific Data Management (SDM) Center. Our application of workflow is unique in that it strives to seamlessly combine automated workflow with manual research steps to create a unified record of the overall research process. We have developed a prototype workflow environment that utilizes a graph-based representation of model relationships as described in [7]. We have applied this prototype to support experimental design of ongoing experiments on mixing-controlled reactions in porous media similar to those in [11]. 6. Concluding remarks Our research addresses the development of multiscale hybrid numerical methods for simulation of subsurface flow and biogeochemical reactions. We have developed and published results from an initial hybrid two-scale model of diffusion and reaction in a model porous medium. We have developed a 3D pore-scale parallel SPH code using CCA technology and are currently testing and applying that code. We have applied pore- and continuum-scale codes to a mixing-controlled precipitation reaction problem and published results demonstrating the need for hybrid methods. We 4



have developed a prototype scientific workflow system and applied that system to parameter studies for design of the mixing-controlled reaction experiments. Acknowledgments This research is supported by the U. S. Department of Energy’s Office of Science under the Scientific Discovery through Advanced Computing (SciDAC) program. References [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15]

Balhoff M T, Thompson K E and Hjortso M 2007 Coupling pore-scale networks to continuumscale models of porous media Computers & Geosciences 33:393-410 Csanyi G, Albaret T, Moras G, Payne M C and De Vita D 2005 Multiscale hybrid simulation methods for material systems J. Physics-Condensed Matter 17:R691-R703 Curtin W A and Miller R E 2003 Atomistic/continuum coupling in computational materials science Modelling and Simulation in Materials Science and Engineering 11:R33-R68 Nie X B, Chen S Y, Robbins W N E and Robbins M O 2004 A continuum and molecular dynamics hybrid method for micro- and nano-fluid flow J. Fluid Mechanics 500:55-64 Oden J T, Prudhomme S, Romkes A and Bauman P T 2006 Multiscale modeling of physical phenomena: Adaptive control of models SIAM J. Sci. Computing 28:2359-2389 Rabczuk T, Xiao S P and SauerA 2006 Coupling of mesh-free methods with finite elements: basic concepts and test results Communications in Numerical Methods in Engineering 22:1031-1065 Schuchardt K, Chin G J, Chase J, Daily J and Scheibe T D 2008 Experiences in the design of scientific workflows for subsurface sciences IEEE Trans. Automation Science and Engineering (in review) Tartakovsky A M and Meakin P 2005 A smoothed particle hydrodynamics model for miscible flow in three-dimensional fractures and the two-dimensional Rayleigh-Taylor instability J. Computational Physics 207:610-624 Tartakovsky A M, Meakin P, Scheibe T D and Eichler West R M 2007 Simulations of reactive transport and precipitation with smoothed particle hydrodynamics J. Computational Physics 222:654-672 Tartakovsky A M, Meakin P, Scheibe T D and Wood B D 2007 A smoothed particle hydrodynamics model for reactive transport and mineral precipitation in porous and fractured porous media, Water Resources Research, 43 (2007). A. M. Tartakovsky, G. Redden, P. C. Lichtner, T. D. Scheibe and P. Meakin, Mixing-induced precipitation: Experimental study and multiscale numerical analysis Water Resources Res. 44 Tartakovsky A M, Scheibe T D and Meakin P 2008 Pore-scale model for reactive transport and biomass growth J. Porous Media (in press) Villa E, Balaeff A, Mahadevan L and Schulten K 2004 Multiscale method for simulating protein-DNA complexes Multiscale Modeling & Simulation 2:527-553 Vlachos D G, Mhadeshwar A B and Kaisare N S 2006 Hierarchical multiscale model-based design of experiments, catalysts, and reactors for fuel processing Computers & Chemical Engineering 30:1712-1724 Wang C Y and Zhang X 2006 Multiscale modeling and related hybrid approaches Current Opinion in Solid State & Materials Science 10:2-14

5