INDRA-ASTRA

Jan 13, 2021, 10:00 AM → Dec 31, 2021, 12:00 PM US/Eastern

In the INDRA-ASTRA project, we explore the possibility for automated calibrations using AI / ML which would allow a rapid turnaround from data taking to physics results.

We use BlueJeans for the remote meeting: https://bluejeans.com/785458355 

Wed, January 13

10:00 AM → 12:00 PM INDRA-ASTRA Meeting: Planning

Convener: Markus Diefenthaler (Jefferson Lab)

10:00 AM INDRA-ASTRA: Introduction

Streaming readout gives opportunity to streamline workflows and to take advantage of other emerging technologies, e.g., artificial intelligence (AI) or machine learning (ML). In the INDRA-ASTRA project, we have explored the possibility for automated calibrations using AI / ML which would allow a rapid turnaround from data taking to physics results.

Speaker: Markus Diefenthaler (Jefferson Lab)

Diefenthaler-INDRA-ASTRA-20210113.pdf

10:15 AM Automated Data Quality Monitoring with ADWIN2

We introduce the ADWIN algorithm for detecting changes in streaming data and present an example implementation of ADWIN to detect sudden and gradual changes in a sample of ZEUS MC data. We compare the algorithm between one-dimensional and two-dimensional cases and give an example of integrating a simple, near-real time calibration method with ADWIN.

Speaker: Abdullah Farhat (ODU)

Farhat-ADWIN-20210113.pdf

• Optimal confidence values for controlling the rate of detecting anomalies
• Adapting ADWIN to use higher moments instead of the mean
• Limiting the maximum window size to improve computational efficiency of ADWIN

10:45 AM Change Detection Based on Multiscale Basis

We have used the Multiscale basis to detect the sudden change. For the sudden change, the mean information of data is enough to detect the change. In the next step, we plan to detect the gradual change, this requires us to explore the higher moment information of the data. The corresponding multiscale basis also has a higher vanishing moment property.

Speaker: Ronglong Fang (ODU)

Fang-Multiscale-20210113.pdf

11:15 AM Discussion: Next steps (Further Evaluation, Tests with detector data)

Work towards detector tests Access and understand available data streams (files)

• Ed will share an example file and his decoder from the binary format to Python readable ASCII files
• Discuss with EPSCI on how to integrate the prototype implementation into ERSAP (JANA2 plugin in C++? Micro service in CLARA?)

New algorithm based on multiscale basis functions

• Abdullah will work with Ronglong on improving and evaluating the new algorithm based on multiscale basis functions

Fri, May 14

3:00 PM → 4:00 PM INDRA-ASTRA Meeting: Multiscale Algorithm

3:00 PM Update on Multiscale Algorithm

Speaker: Ronglong Fang (ODU)

Ronglong Fang - Change Detect.pdf

3:30 PM Multiscale Algorithm for Streamed Data

Speaker: Abdullah Farhat (ODU)

3:45 PM Test Data from the INDRA Lab

Speaker: Holly Szumila-Vance (JLab)

Fri, June 4

4:00 PM → 5:00 PM INDRA-ASTRA Meeting: GEM Prototype

4:00 PM Working with the Detector Data

Speakers: Holly Szumila-Vance (JLab), Ronglong Fang (ODU)

read and detect data.pdf

save Data.pdf

Wed, June 9

2:00 PM → 3:00 PM INDRA-ASTRA Meeting: Note on Sudden Change Detect: Knowledge Exchange with AIEC

Convener: David Lawrence (Jefferson Lab)

multiscale_shift_modify.ipynb

multiscale_shift_modify.pdf

2:00 PM AIEC

Speaker: Torri Jeske (Jefferson Lab)

AIECintro.pdf

2:15 PM INDRA-ASTRA

Speaker: Markus Diefenthaler (Jefferson Lab)

INDRA-ASTRA-Overview.pdf

INDRA-ASTRA-Overview.pptx

2:30 PM Discussion

Fri, June 11

4:00 PM → 4:45 PM INDRA-ASTRA Meeting: Online Mode

4:00 PM Report from AIEC and INDRA-ASTRA meeting

Speaker: Markus Diefenthaler (Jefferson Lab)

4:15 PM Working with the Detector Data

Speakers: Abdullah Farhat (ODU), Holly Szumila-Vance (JLab), Ronglong Fang (ODU)

Fri, June 18

4:00 PM → 5:00 PM INDRA-ASTRA Meeting: Online Mode

4:00 PM Performance of Online Mode

Speakers: Abdullah Farhat (ODU), Markus Diefenthaler (Jefferson Lab), Ronglong Fang (ODU), Yuesheng Xu (ODU)

Thu, June 24

4:00 PM → 4:50 PM INDRA-ASTRA Meeting: Introduction, GEM Data, Online Mode

4:00 PM Introduction Srijith Rajamohan

Speaker: Markus Diefenthaler (Jefferson Lab)

4:05 PM Online performance of ADWIN2 and Multiscale Methods

Speaker: Ronglong Fang (ODU)

multiscale_shift.pdf

4:35 PM Discussion: Next Steps

Fri, July 2

4:00 PM → 5:00 PM INDRA-ASTRA Meeting: Continuation from June 24

Fri, July 16

4:00 PM → 5:00 PM INDRA-ASTRA Meeting: Note on Sudden Change Detect

multiscale_shift_modify.ipynb

multiscale_shift_modify.pdf

Fri, July 23

4:00 PM → 5:00 PM INDRA-ASTRA Meeting: GEM Data, Multiscale Algorithm, Online Mode

4:00 PM Priority I: Detector tests

Status of test runs
Status of test run analysis

4:10 PM Priority II: Developing the Multiscale Method

Singlescale Shift Online Method: Is shifting by +1 sufficient?

How can we show that the method gives the correct results beyond detector tests?

4:30 PM Priority III: Online Mode

ERSAP Update
Status of compiling the code

ersap-in-nutshell.pdf

ersap-in-nutshell.pptx

4:40 PM Preparation for AI Town Hall

Fri, August 13

4:00 PM → 4:30 PM INDRA-ASTRA Meeting: Improved Algorithms

4:00 PM Updates

Speaker: Ronglong Fang (ODU)

mean_variance_change_detect.jpg

merge_shift_mean_variance.jpg

The figure 'mean_variance_change_detect.jpg' shows the detect results for the dataset 'hit_bbgem_1034.root'. We use the mean and standard deviation methods to detect the outlier.  The results show that we can detect all the peaks that can be observed by humans.

The figure 'merge_shift_mean_variance.jpg' shows the results for the gradual change. We create simple gradual change and use the test function with order 1 vanish moment. The results show that gradual change can be detected in a large scale. At this moment, the algorithm can not verify the type of change (sudden change or gradual change). 

Fri, August 27

4:00 PM → 5:00 PM INDRA-ASTRA Meeting: Preparations for GAP analysis

Fri, September 17

4:00 PM → 5:00 PM INDRA-ASTRA Meeting: Finding changes in the pedestals, pedestal correction

4:00 PM Update

Speaker: Ronglong Fang (ODU)

hit_304.pdf

4:30 PM Discussion

Fri, September 24

4:00 PM → 5:00 PM INDRA-ASTRA Meeting: Motivation, Publication

4:00 PM Motivation

Speaker: Markus Diefenthaler (Jefferson Lab)

Slides

4:15 PM Discussion

Fri, October 15

4:00 PM → 5:00 PM INDRA-ASTRA Meeting: Detector Calibration