Metadata-Version: 2.1
Name: module_qc_analysis_tools
Version: 1.0
Summary: Module qc analysis tools
Project-URL: Homepage, https://gitlab.cern.ch/atlas-itk/pixel/module/module-qc-analysis-tools
Project-URL: Bug Tracker, https://gitlab.cern.ch/atlas-itk/pixel/module/module-qc-analysis-tools/-/issues
Project-URL: Source, https://gitlab.cern.ch/atlas-itk/pixel/module/module-qc-analysis-tools
Author-email: Jay Chan <jay.chan@cern.ch>
Maintainer-email: Giordon Stark <gstark@cern.ch>
License: Copyright (c) 2018 The Python Packaging Authority
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.7
Requires-Dist: jsonschema
Requires-Dist: matplotlib
Requires-Dist: module-qc-data-tools>=0.3.0
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: tabulate
Description-Content-Type: text/markdown

# module-qc-analysis-tools

A general python tool for running ITkPixV1.1 module QC test analysis. An
overview of the steps in the module QC procedure is documented in the
[Electrical specification and QC procedures for ITkPixV1.1 modules](https://gitlab.cern.ch/atlas-itk/pixel/module/itkpix-electrical-qc/)
document and in
[this spreadsheet](https://docs.google.com/spreadsheets/d/1qGzrCl4iD9362RwKlstZASbhphV_qTXPeBC-VSttfgE/edit#gid=989740987).
The analysis scripts in this repository require input files with measurement
data. The measurement data should be collected using the
[module-qc-measurement-tools](https://gitlab.cern.ch/atlas-itk/pixel/module/module-qc-tools)
package.

## Requirements

This tool requires users to have >python3.6 with the following packages
installed:

- `numpy`
- `scipy`
- `tabulate`
- `matplotlib`
- `jsonschema`

## Installation

First clone the project:

```
git clone https://gitlab.cern.ch/atlas-itk/pixel/module/module-qc-analysis-tools.git
```

Upon a successful checkout, `cd` to the new `module-qc-analysis-tools` directory
and run the following to install the necessary software:

```verbatim
$ python3 -m venv env
$ source env/bin/activate
$ python -m pip install --upgrade pip
$ python -m pip install -e .
```

Note that users should use the latest python version (check python version via
`python3 -V`). Python3.7 is the minimum requirement for developers. See
[For Developer](#for-developer) section.

## Scripts

### `Analyze ADC Calibration`

This analysis script performs the ADC calibration. It produces several
diagnostic plots and an output file with the ADC calibration slope and offset.

<details> <summary> analysis-ADC-CALIBRATION --help </summary>

```
analysis-ADC-CALIBRATION --help
usage: analysis-ADC-CALIBRATION [-h] -i INPUT_MEAS [-o OUTPUT_DIR] [-q QC_CRITERIA] [-l LAYER] [--permodule]
                                [-f {root,numpy}] [-v VERBOSITY]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT_MEAS, --input-meas INPUT_MEAS
                        path to the input measurement file or directory containing input measurement files.
  -o OUTPUT_DIR, --output-dir OUTPUT_DIR
                        output directory
  -q QC_CRITERIA, --qc-criteria QC_CRITERIA
                        path to json file with QC selection criteria (default: $(module-qc-analysis-tools --prefix)/analysis_cuts.json)
  -l LAYER, --layer LAYER
                        Layer of module, used for applying correct QC criteria settings. Options: L0, L1, L2
                        (default)
  --permodule           Store results in one file per module (default: one file per chip)
  -f {root,numpy}, --fit-method {root,numpy}
                        fitting method
  -v VERBOSITY, --verbosity VERBOSITY
                        Log level [options: DEBUG, INFO (default), WARNING, ERROR]
```

</details>

### `Analyze Analog Readback`

This analysis script performs the Analog Readback. It produces an output file
with the calculated internal biases, temperature from the internal and external
temperature sensor, and VDDA/VDDD vs Trim, including diagnostic plots with slope
and offset.

<details> <summary> analysis-ANALOG-READBACK --help </summary>

```
$ analysis-ANALOG-READBACK --help
usage: analysis-ANALOG-READBACK [-h] -i INPUT_MEAS [-o OUTPUT_DIR] [-q QC_CRITERIA] [-l LAYER] [--permodule]
                                [-f {root,numpy}] [-v VERBOSITY] [--verbose]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT_MEAS, --input-meas INPUT_MEAS
                        path to the input measurement file or directory containing input measurement files.
  -o OUTPUT_DIR, --output-dir OUTPUT_DIR
                        output directory
  -q QC_CRITERIA, --qc-criteria QC_CRITERIA
                        path to json file with QC selection criteria (default: $(module-qc-analysis-tools --prefix)/analysis_cuts.json)
  -l LAYER, --layer LAYER
                        Layer of module, used for applying correct QC criteria settings. Options: L0, L1, L2
                        (default)
  --permodule           Store results in one file per module (default: one file per chip)
  -f {root,numpy}, --fit-method {root,numpy}
                        fitting method
  -v VERBOSITY, --verbosity VERBOSITY
                        Log level [options: DEBUG, INFO (default), WARNING, ERROR]
  --verbose             verbose mode

```

</details>

### `Analyze SLDO`

This script analyses the SLDO curve. It produces several diagnostic plots and an
output file with several parameters extracted from the SLDO curves.

<details> <summary> analysis-SLDO --help </summary>

```
$ analysis-SLDO --help
usage: analysis-SLDO [-h] -i INPUT_MEAS [-o OUTPUT_DIR] [-q QC_CRITERIA] [-l LAYER] [--permodule] [-n NCHIPS]
                     [-f {root,numpy}] [-v VERBOSITY] [--lp-enable]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT_MEAS, --input-meas INPUT_MEAS
                        path to the input measurement file or directory containing input measurement files.
  -o OUTPUT_DIR, --output-dir OUTPUT_DIR
                        output directory
  -q QC_CRITERIA, --qc-criteria QC_CRITERIA
                        path to json file with QC selection criteria (default: $(module-qc-analysis-tools --prefix)/analysis_cuts.json)
  -l LAYER, --layer LAYER
                        Layer of module, used for applying correct QC criteria settings. Options: L0, L1, L2
                        (default)
  --permodule           Store results in one file per module (default: one file per chip)
  -n NCHIPS, --nChips NCHIPS
                        Number of chips powered in parallel (e.g. 4 for a quad module, 3 for a triplet, 1 for an
                        SCC.)
  -f {root,numpy}, --fit-method {root,numpy}
                        fitting method
  -v VERBOSITY, --verbosity VERBOSITY
                        Log level [options: DEBUG, INFO (default), WARNING, ERROR]
  --lp-enable           low power mode

```

</details>

### `Analyze VCal Calibration`

This analysis script performs the VCal calibration. It produces several
diagnostic plots and an output file with the VCal calibration slope and offset.

<details> <summary> analysis-VCAL-CALIBRATION --help </summary>

```
$ analysis-VCAL-CALIBRATION --help
usage: analysis-VCAL-CALIBRATION [-h] -i INPUT_MEAS [-o OUTPUT_DIR] [-q QC_CRITERIA] [-l LAYER] [--permodule]
                                 [-f {root,numpy}] [-v VERBOSITY]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT_MEAS, --input-meas INPUT_MEAS
                        path to the input measurement file or directory containing input measurement files.
  -o OUTPUT_DIR, --output-dir OUTPUT_DIR
                        output directory
  -q QC_CRITERIA, --qc-criteria QC_CRITERIA
                        path to json file with QC selection criteria (default: $(module-qc-analysis-tools --prefix)/analysis_cuts.json)
  -l LAYER, --layer LAYER
                        Layer of module, used for applying correct QC criteria settings. Options: L0, L1, L2
                        (default)
  --permodule           Store results in one file per module (default: one file per chip)
  -f {root,numpy}, --fit-method {root,numpy}
                        fitting method
  -v VERBOSITY, --verbosity VERBOSITY
                        Log level [options: DEBUG, INFO (default), WARNING, ERROR]
```

</details>

### `Analyze Injection capacitance`

This analysis script performs the injection capacitance. It produces several
diagnostic plots and an output file with the measured pixel injection
capacitance.

<details> <summary> analysis-INJECTION-CAPACITANCE --help </summary>

```
$ analysis-INJECTION-CAPACITANCE
usage: analysis-INJECTION-CAPACITANCE [-h] -i INPUT_MEAS [-o OUTPUT_DIR] [-q QC_CRITERIA] [-l LAYER] [--permodule]
                                      [-v VERBOSITY]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT_MEAS, --input-meas INPUT_MEAS
                        path to the input measurement file or directory containing input measurement files.
  -o OUTPUT_DIR, --output-dir OUTPUT_DIR
                        output directory
  -q QC_CRITERIA, --qc-criteria QC_CRITERIA
                        path to json file with QC selection criteria (default: $(module-qc-analysis-tools --prefix)/analysis_cuts.json)
  -l LAYER, --layer LAYER
                        Layer of module, used for applying correct QC criteria settings. Options: L0, L1, L2
                        (default)
  --permodule           Store results in one file per module (default: one file per chip)
  -v VERBOSITY, --verbosity VERBOSITY
                        Log level [options: DEBUG, INFO (default), WARNING, ERROR]
```

</details>

## Notes

### `Example commands for a chip in a quad module (L2):`

```
analysis-ADC-CALIBRATION -i ../module-qc-tools/emulator/outputs/SLDO_reference/1674670016/ --layer L2
analysis-SLDO -i ../module-qc-tools/emulator/outputs/SLDO_reference/1674670016/ --layer L2
analysis-ANALOG-READBACK -i ../module-qc-tools/emulator/outputs/SLDO_reference/1674670016/ --layer L2
analysis-VCAL-CALIBRATION -i ../module-qc-tools/emulator/outputs/SLDO_reference/1674670016/ --layer L2
analysis-INJECTION-CAPACITANCE -i ../module-qc-tools/emulator/outputs/SLDO_reference/1674670016/ --layer L2
```

### `Update Trim, ADC Calibration Parameters, VCal calibration parameters`

After each analysis, update the setting in the chip config by running the
following scripts:

```
analysis-overwrite-ADCcalPar -i <path to ADC CALIBRATION analysis output directory> -c <path to YARR config directory> -t <config type>
analysis-overwrite-trim -i <path to ANALOG READBACK analysis output directory> -c <path to YARR config directory> -t <config type>
analysis-overwrite-VcalPar -i <path to VCAL CALIBRATION analysis output directory> -c <path to YARR config directory> -t <config type>
```

### `JsonChecker and DataExtractor`

Two classes are designed for general purposes of the module qc analysis tool.

1. `JsonChecker` a. Check whether the test type is implemented b. For a specific
   task, check if required keywords exist c. Check if lengths of measurements
   are identical d. Check if there are negative numbers of measurements

2. `DataExtractor` This class finds measurements by Vmux value and convert them
   to quantites.

### `Params overwrite`

The script `overwrite_config.py` writes fitted parameters to a chip config file.
Recursion is used to search for the `Parameter` part of the original config file
and values corresponding to the key in the analysis output json file will
replace the original parameters.

Usage:

<details> <summary> analysis-overwrite-config --help </summary>
```

$ analysis-overwrite-config -h usage: analysis-overwrite-config [-h] -i
INPUT_FILE -c CONFIG_FILE

optional arguments: -h, --help show this help message and exit -i INPUT_FILE,
--input-file INPUT_FILE analysis output file -c CONFIG_FILE, --config-file
CONFIG_FILE the config file to be modified

```
</details>

<details> <summary> Example </summary>
```

$ analysis-overwrite-config -i
overwrite/0x16a4c_vcal_calib_InjVcalHigh_LargeRange_1662583926_anad.json -c
overwrite/metadata.json

```
</details>

## For Developer

### python version

A python version higher than 3.7 is needed for this repository. Check the local
python version with `python -V`.

If the local python version is lower, set up a virtual python environment
following the instructions
[here](https://itk.docs.cern.ch/general/Virtual_Environments/).

### pre-commit

Install pre-commit to avoid CI failure. Once pre-commit is installed, a git hook
script will be run to identify simple issues before submission to code review.

Instruction for installing pre-commit in a python environment:

1. Activate python environment, e.g. `source venv/bin/activate`.
2. Run `python3 -m pip install pre-commit`.
3. Run `pre-commit install` to install the hooks in `.pre-commit-config.yaml`.

After installing pre-commit, `.pre-commit-config.yaml` will be run every time
`git commit` is done. Redo `git add` and `git commit`, if the pre-commit script
changes any files.
```
