GitHub

Achieving Petaflop Performance with Celeste.jl

Update time

As part of a multi-institution, multi-disciplinary team, including the National Energy Research Scientific Computing Center (NERSC), Princeton, Harvard, Intel, MIT, and JuliaHub, researchers develop a state-of-the-art parallel computing method to analyze massize astronomy datasets at petascale.

Package: jeff-regier/Celeste.jl

image

The Celeste team set a number of new milestones, including:

  • Loading an aggregate of ~178 terabytes of Sloan Digital Sky Survey (SDSS) image data
  • Producing parameter estimates for 188 million stars and galaxies from this dataset in 14.6 minutes
  • Achieving a peak performance of 1.54 petaflops using 1.3 million threads on 9,300 Knights Landing (KNL) nodes of the Cori supercomputer at NERSC
  • Achieving a performance improvement of 1,000x in single-threaded execution
  • Demonstrating conclusively that massive and complex datasets can be analyzed with Bayesian inference by applying variational inference (VI)
  • Providing not only point estimates for light sources but, for the first time, a principled measure of the quality of inference for each light source (posterior uncertainty for source type, brightness, and colors)

See the team's series of papers below and JuliaHub article for more:

  1. Jeffrey Regier, Andrew C. Miller, David Schlegel, Ryan P. Adams, Jon D. McAuliffe, & Prabhat (2019). Approximate inference for constructing astronomical catalogs from images. The Annals of Applied Statistics, 13(3), 1884 – 1926. [ADS]
  2. Regier, J., Pamnany, K., Fischer, K., Noack, A., Lam, M., Revels, J., Howard, S., Giordano, R., Schlegel, D., McAuliffe, J., Thomas, R., & Prabhat, .. (2018). Cataloging the Visible Universe Through Bayesian Inference at Petascale. In 2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS) (pp. 44-53). [ADS]
  3. Regier, J., Miller, A., McAuliffe, J., Adams, R., Hoffman, M., Lang, D., Schlegel, D., & Prabhat, M. (2015). Celeste: Variational inference for a generative model of astronomical images. In Proceedings of the 32nd International Conference on Machine Learning (pp. 2095–2103). PMLR. [ADS]

This page was generated using DemoCards.jl.