(PhD06) Dendro-GR: Fully Adaptive Octrees for Computational Relativity.
TimeMonday, June 17th1pm - 6pm
DescriptionMy research is primarily focused on developing computational algorithms for the numerical solution of large-scale partial differential equations. New discoveries in Science and Engineering are primarily driven by computer simulations(in lieu of physical experiments). In many cases, such as Gravitational Wave (GW) astronomy physical experiments are impossible. In the modern computational era, while computing resources have grown exponentially, they have also become increasingly complex with ever-increasing heterogeneity and fine-grain parallelism, making their use by domain-scientists has become increasingly difficult. For my current research, the main driving application has been computational relativity and GW astronomy, but the contributions of my research are fundamental and have also had a significant impact on other areas such as Computational Fluid Dynamics (CFD).
The very first GWs detected by LIGO, originated from two black holes (BH) spiraling around each other and merging together to form a single BH, an event that happened 1.3B years ago. GWs carry information about the early universe, and the newest member to join multi-messenger astronomy. Current GW observations have been limited to observations of binary mergers of roughly equal mass. To a large extent, this has been due to the inability of existing numerical codes (such as Einstein Toolkit) to perform large mass ratio simulations. State-of-the-art numerical relativity codes demonstrate poor scalability on modern heterogeneous architectures. They also use inefficient discretizations leading to an extremely large number of unknowns at higher mass ratios. In order to overcome these challenges, my research work on past 4 years has been focused on developing scalable parallel algorithms to perform fast numerical relativity simulations, where the massively parallel scalability enables to perform Intermediate Mass Ratio Inspirals (IMRIs) (mass ratios of 1: 100), which is not possible with existing numerical relativity codes. We present a highly-scalable framework (Dendro-GR) that targets problems of interest to the numerical relativity and broader astrophysics communities. The goal of this work is to perform advanced, massively parallel numerical simulations of Intermediate Mass Ratio Inspirals (IMRIs) of binary black holes with mass ratios on the order of 100:1. These studies will be used to generate waveforms as used in LIGO data analysis and to calibrate semi-analytical approximate methods. Our framework consists of a distributed memory octree-based adaptive meshing framework in conjunction with a node-local code generator. The code generator makes our code portable across different architectures. The equations corresponding to the target application are written in symbolic notation and generators for different architectures can be added independently of the application. Our adaptive meshing algorithms and data-structures have been optimized for modern architectures with deep memory hierarchies. This enables our framework to have achieved excellent performance and scalability on modern leadership architectures. Dendro-GR demonstrates excellent weak scalability up to 131K cores on ORNL's Titan for binary mergers for mass ratios up to 100.
For the next steps of my graduate thesis work, I am currently working on extraction of the GWs and incorporating relativistic fluid equations which enable us to perform Neutron Star(NS) merges.