Article: Wang, JJ; Perrin, MD; Savransky, D; Arriaga, P; Chilcote, JK; De Rosa, RJ; Millar-Blanchaer, MA; Marois, C; Rameau, J; Wolff, SG; Shapiro, J; Ruffio, JB; Graham, JR; Macintosh, B; “The Automated Data Processing Architecture for the GPI Exoplanet Survey”, TECHNIQUES AND INSTRUMENTATION FOR DETECTION OF EXOPLANETS VIII, Proceedings of SPIE
Abstract: The Gemini Planet Imager Exoplanet Survey (GPIES) is a multi-year direct imaging survey of 600 stars to discover and characterize young Jovian exoplanets and their environments. We have developed an automated data architecture to process and index all data related to the survey uniformly. An automated and flexible data processing framework, which we term the GPIES Data Cruncher, combines multiple data reduction pipelines together to intelligently process all spectroscopic, polarimetric, and calibration data taken with GPIES. With no human intervention, fully reduced and calibrated data products are available less than an hour after the data are taken to expedite follow-up on potential objects of interest. The Data Cruncher can run on a supercomputer to reprocess all GPIES data in a single day as improvements are made to our data reduction pipelines. A backend MySQL database indexes all files, which are synced to the cloud, and a front-end web server allows for easy browsing of all files associated with GPIES. To help observers, quicklook displays show reduced data as they are processed in real-time, and chatbots on Slack post observing information as well as reduced data products. Together, the GPIES automated data processing architecture reduces our workload, provides real-time data reduction, optimizes our observing strategy, and maintains a homogeneously reduced dataset to study planet occurrence and instrument performance.
Funding Acknowledgement: NASA’s NEXSS program [NNX15AD95G]; Office of Science of the U.S. Department of Energy [DE-AC02-05CH11231]; National Science Foundation [ACI-1548562]; NASA through Hubble Fellowship – Space Telescope Science Institute [51378.01-A]; NASA [NAS5-26555]
Funding Text: The Gemini Observatory is operated by the Association of Universities for Research in Astronomy, Inc., under a cooperative agreement with the NSF on behalf of the Gemini partnership: the National Science Foundation (United States), the National Research Council (Canada), CONICYT (Chile), the Australian Research Council (Australia), Ministerio da Ciencia, Tecnologia e Inovacao (Brazil), and Ministerio de Ciencia, Tecnologia e Innovacion Productiva (Argentina). This work was supported in part by NASA’s NEXSS program, grant number NNX15AD95G. This research used resources of the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231. This work used the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1548562. Support for MMB’s work was provided by NASA through Hubble Fellowship grant #51378.01-A awarded by the Space Telescope Science Institute, which is operated by the Association of Universities for Research in Astronomy, Inc., for NASA, under contract NAS5-26555. We also thank the SDSC staff for their helpful support in providing resources and technical help. This research made use of Astropy, a community-developed core Python package for Astronomy.