Simulating Black Hole Accretion with Unprecedented Computational Resources
As part of the Black Hole PIRE project for the Event Horizon Telescope (EHT), for which I was the PI, we made record-breaking use of national computational resources to simulate accretion onto black holes. This effort included one of the largest allocations on TACC’s Frontera supercomputer, utilizing over 80 million CPU hours over five years, as well as a significant fraction of the total core hours available through the nationwide Open Science Pool. These simulations have provided critical insights into black hole dynamics, enabling transformative advancements in high-resolution black hole imaging and astrophysical modeling.
Pioneering GPU-Based Computing for Black Hole Research
Our group was among the first to adopt GPUs for high-performance computing in astrophysics and, in particular, in ray tracing simulations in general relativistic spacetimes. We developed GRay, a massively parallel GPU-based integrator capable of tracing billions of photon trajectories in curved spacetimes. GRay achieves a peak performance exceeding 300 GFLOP on a single GPU and operates two orders of magnitude faster than traditional CPU-based ray-tracing codes, enabling efficient exploration of theoretical predictions for black hole images, spectra, and light curves. Building on this, we developed GRay2, which improves geodesic integration in Kerr spacetimes by using special coordinates to eliminate singularities. By leveraging the OpenCL framework, GRay2 runs on diverse hardware accelerators, offering high accuracy and versatility in modeling particle and photon geodesics around black holes.
Algorithms our group has Developed or Co-Developed
- Markov Chains for Horizons (MARCH): a Bayesian MCMC algorithm for fitting interferometric data
- GRay2: A General Purpose Geodesic Integrator for Kerr Spacetimes
- HEROIC: A 3D general relativistic radiative post-processor with Comptonization
- HERO – A 3D general relativistic radiative post-processor
- GRay: A Massively Parallel GPU-based Code for Ray Tracing in Relativistic Spacetimes
- iRayNS: A Ray-tracing Algorithm for Spinning Neutron Stars
- iRay: A Ray-tracing Algorithm for Spinning Compact Object Spacetimes with Arbitrary Quadrupoles
- CODE-M: A COllisional Disk Evolution Model
- ABAX3d: Spectral Method Algorithm for 3D Self-Gravitating, Magnetohydrodynamic Disks
- ABAXGravity: Spectral Method Algorithm for 2D Hydrodynamic Disks with Self-Gravity
- ABAX2d: Spectral Method Algorithm for 2D Viscous, Hydrodynamic Disks
- Radiative Transfer Algorithm for Obliquely Illuminated Accretion Disks
- Radiative Transfer Algorithm for Compton Scattering in Static and Moving Media