Please visit Jefferson Lab Event Policies and Guidance before planning your next event:
May 8 – 12, 2023
Norfolk Waterside Marriott
US/Eastern timezone

Overview of the distributed image processing infrastructure to produce the Legacy Survey of Space and Time (LSST)

May 11, 2023, 11:45 AM
Norfolk Ballroom III-V (Norfolk Waterside Marriott)

Norfolk Ballroom III-V

Norfolk Waterside Marriott

235 East Main Street Norfolk, VA 23510
Oral Track 1 - Data and Metadata Organization, Management and Access Track 1 - Data and Metadata Organization, Management and Access


Hernandez, Fabio (IN2P3 / CNRS computing center)


The Vera C. Rubin observatory is preparing for execution of the most ambitious astronomical survey ever attempted, the Legacy Survey of Space and Time (LSST). Currently in its final phase of construction in the Andes mountains in Chile and due to start operations late 2024 for 10 years, its 8.4-meter telescope will nightly scan the southern sky and collect images of the entire visible sky every 4 nights using a 3.2 Gigapixel camera, the largest imaging device ever built for astronomy. Automated detection and classification of celestial objects will be performed by sophisticated algorithms on high-resolution images to progressively produce an astronomical catalog eventually composed of 20 billion galaxies and 17 billion stars and their associated physical properties.

In this contribution we will present an overall view of the system currently in construction to perform data distribution and the annual reprocessing campaigns of the entire image dataset collected since the beginning of the survey, using computing and storage resources provided by 3 Rubin data facilities (one in the US and two in Europe). Each year a data release will be produced and delivered to science collaborations for their studies in 4 science areas: probing dark energy and dark matter, taking an inventory of the solar system, exploring the transient optical sky and mapping the Milky Way.

We will present how we leverage some of the practices and tools used for large-scale distributed data processing by other projects in the high energy physics and astronomy communities and how we integrate them with tools developed by the Rubin project for meeting the specific challenges it faces.

Consider for long presentation Yes

Primary authors

Hernandez, Fabio (IN2P3 / CNRS computing center) Doidge, Matt (Lancaster University (GB)) Love, Peter (Lancaster University (GB)) Karavakis, Edward (Brookhaven National Laboratory) Jenness, Tim (AURA/Rubin Observatory) Yang, Wei (SLAC National Accelerator Laboratory) White, Brandon (Fermilab) Beckett, George Le Boulc'h, Quentin Mainetti, Gabriele Clark, Peter Noble, Timothy

Presentation materials

Peer reviewing