INDRA-ASTRA

Jan 13, 2021, 10:00 AM → Dec 31, 2021, 12:00 PM US/Eastern

In the INDRA-ASTRA project, we explore the possibility for automated calibrations using AI / ML which would allow a rapid turnaround from data taking to physics results.

We use BlueJeans for the remote meeting: https://bluejeans.com/785458355 

Wednesday, January 13

INDRA-ASTRA Meeting: Planning

Convener: Markus Diefenthaler (Jefferson Lab)

1 INDRA-ASTRA: Introduction

Streaming readout gives opportunity to streamline workflows and to take advantage of other emerging technologies, e.g., artificial intelligence (AI) or machine learning (ML). In the INDRA-ASTRA project, we have explored the possibility for automated calibrations using AI / ML which would allow a rapid turnaround from data taking to physics results.

Speaker: Markus Diefenthaler (Jefferson Lab)

Diefenthaler-INDRA-ASTRA-20210113.pdf

2 Automated Data Quality Monitoring with ADWIN2

We introduce the ADWIN algorithm for detecting changes in streaming data and present an example implementation of ADWIN to detect sudden and gradual changes in a sample of ZEUS MC data. We compare the algorithm between one-dimensional and two-dimensional cases and give an example of integrating a simple, near-real time calibration method with ADWIN.

Speaker: Abdullah Farhat (ODU)

Farhat-ADWIN-20210113.pdf

• Optimal confidence values for controlling the rate of detecting anomalies
• Adapting ADWIN to use higher moments instead of the mean
• Limiting the maximum window size to improve computational efficiency of ADWIN

3 Change Detection Based on Multiscale Basis

We have used the Multiscale basis to detect the sudden change. For the sudden change, the mean information of data is enough to detect the change. In the next step, we plan to detect the gradual change, this requires us to explore the higher moment information of the data. The corresponding multiscale basis also has a higher vanishing moment property.

Speaker: Ronglong Fang (ODU)

Fang-Multiscale-20210113.pdf

4 Discussion: Next steps (Further Evaluation, Tests with detector data)

Work towards detector tests Access and understand available data streams (files)

• Ed will share an example file and his decoder from the binary format to Python readable ASCII files
• Discuss with EPSCI on how to integrate the prototype implementation into ERSAP (JANA2 plugin in C++? Micro service in CLARA?)

New algorithm based on multiscale basis functions

• Abdullah will work with Ronglong on improving and evaluating the new algorithm based on multiscale basis functions

Friday, May 14

INDRA-ASTRA Meeting: Multiscale Algorithm

5 Update on Multiscale Algorithm

Speaker: Ronglong Fang (ODU)

Ronglong Fang - Change Detect.pdf

6 Multiscale Algorithm for Streamed Data

Speaker: Abdullah Farhat (ODU)

7 Test Data from the INDRA Lab

Speaker: Holly Szumila-Vance (JLab)

Friday, June 4

INDRA-ASTRA Meeting: GEM Prototype

8 Working with the Detector Data

Speakers: Holly Szumila-Vance (JLab), Ronglong Fang (ODU)

read and detect data.pdf

save Data.pdf

Wednesday, June 9

INDRA-ASTRA Meeting: Note on Sudden Change Detect: Knowledge Exchange with AIEC

Convener: David Lawrence (Jefferson Lab)

multiscale_shift_modify.ipynb

multiscale_shift_modify.pdf

9 AIEC

Speaker: Torri Jeske (Jefferson Lab)

AIECintro.pdf

10 INDRA-ASTRA

Speaker: Markus Diefenthaler (Jefferson Lab)

INDRA-ASTRA-Overview.pdf

INDRA-ASTRA-Overview.pptx

11 Discussion

Friday, June 11

INDRA-ASTRA Meeting: Online Mode

12 Report from AIEC and INDRA-ASTRA meeting

Speaker: Markus Diefenthaler (Jefferson Lab)

13 Working with the Detector Data

Speakers: Abdullah Farhat (ODU), Holly Szumila-Vance (JLab), Ronglong Fang (ODU)

Friday, June 18

INDRA-ASTRA Meeting: Online Mode

14 Performance of Online Mode

Speakers: Abdullah Farhat (ODU), Markus Diefenthaler (Jefferson Lab), Ronglong Fang (ODU), Yuesheng Xu (ODU)

Thursday, June 24

INDRA-ASTRA Meeting: Introduction, GEM Data, Online Mode

15 Introduction Srijith Rajamohan

Speaker: Markus Diefenthaler (Jefferson Lab)

16 Online performance of ADWIN2 and Multiscale Methods

Speaker: Ronglong Fang (ODU)

multiscale_shift.pdf

17 Discussion: Next Steps

Friday, July 2

INDRA-ASTRA Meeting: Continuation from June 24

Friday, July 16

INDRA-ASTRA Meeting: Note on Sudden Change Detect

multiscale_shift_modify.ipynb

multiscale_shift_modify.pdf

Friday, July 23

INDRA-ASTRA Meeting: GEM Data, Multiscale Algorithm, Online Mode

18 Priority I: Detector tests

Status of test runs
Status of test run analysis

19 Priority II: Developing the Multiscale Method

Singlescale Shift Online Method: Is shifting by +1 sufficient?

How can we show that the method gives the correct results beyond detector tests?

20 Priority III: Online Mode

ERSAP Update
Status of compiling the code

ersap-in-nutshell.pdf

ersap-in-nutshell.pptx

21 Preparation for AI Town Hall

Friday, August 13

INDRA-ASTRA Meeting: Improved Algorithms

22 Updates

Speaker: Ronglong Fang (ODU)

mean_variance_change_detect.jpg

merge_shift_mean_variance.jpg

The figure 'mean_variance_change_detect.jpg' shows the detect results for the dataset 'hit_bbgem_1034.root'. We use the mean and standard deviation methods to detect the outlier.  The results show that we can detect all the peaks that can be observed by humans.

The figure 'merge_shift_mean_variance.jpg' shows the results for the gradual change. We create simple gradual change and use the test function with order 1 vanish moment. The results show that gradual change can be detected in a large scale. At this moment, the algorithm can not verify the type of change (sudden change or gradual change). 

Friday, August 27

INDRA-ASTRA Meeting: Preparations for GAP analysis

Friday, September 17

INDRA-ASTRA Meeting: Finding changes in the pedestals, pedestal correction

23 Update

Speaker: Ronglong Fang (ODU)

hit_304.pdf

24 Discussion

Friday, September 24

INDRA-ASTRA Meeting: Motivation, Publication

25 Motivation

Speaker: Markus Diefenthaler (Jefferson Lab)

Slides

26 Discussion

Friday, October 15

INDRA-ASTRA Meeting: Detector Calibration