Metadata-Version: 2.1
Name: ConfMatrixCalc
Version: 0.6.0
Summary: Statistical Analysis of Phoneme Confusion Matrices
Home-page: UNKNOWN
Author: Arne Leijon
Author-email: leijon@kth.se
License: MIT
Description: Package **ConfMatrixCalc** implements probabilistic Bayesian analysis
        of phoneme identification test results.
        The analysis approach was presented and validated in (Leijon et al., 2016).
        
        Phoneme identification tests are used, for example,
        to evaluate the detailed ("microscopic") speech-recognition ability of
        listeners using two or more different hearing aids
        or other sound-transmission instruments or algorithms.
        Phoneme identification performance is often tested using nonsense "words" with
        a fixed structure, e.g., CVC, VCV, or CVCVC, where
        C is a consonant and V is a vowel.
        This makes the test material more difficult than real words or sentences,
        because the listener can not make use of prior lexical and semantic knowledge.
        However, this may actually be an advantage, because interesting test results can be
        obtained at realistic speech-to-noise ratios, where listeners might
        otherwise get nearly perfect identification results with an easier test material.
        
        Early speech research showed that the phoneme identification ability is
        correlated with general sentence understanding (Fletcher and Steinberg, 1929, Fig. 11).
        
        ## Phoneme Confusion Matrices
        The test results are usually recorded as two-dimensional arrays of *confusion counts*.
        A matrix element with index (s, r) shows how many times
        the listener responded by the *r*th category, when the *s*th stimulus was presented.
        
        The statistical analysis of confusion-matrix data is non-trivial,
        because the matrix is usually quite sparse for each listener.
        For example, in a consonant-identification test with 16 consonants,
        each stimulus type might be presented, say, five times, i.e., 80 presentations in total.
        Then each matrix row will have at least 16 - 5 = 11 elements with a zero count.
        This makes it difficult to estimate underlying response probabilities and to
        quantify the statistical reliability of observed test results.
        The Bayesian analysis method handles these problems in a coherent manner.
        
        ## Analysis Results
        1. **Overall performance** is indicated by two measures,
            each with a *credible range* to indicate the uncertainty of the estimate:
        
            1. **Probability of Correct** identification (PC), across all presented phonemes.
        
            1. The **Mutual Information** (MI) between stimulus and response (Miller and Nicely, 1955),
                sometimes called "transmitted information".
                This measure indicates the average amount of information about the stimulus category,
                received by the listener by hearing each presented phoneme.
        
        1. **Detailed performance** is shown by *credible confusion pattern*, i.e., a set of
            stimulus-response pairs where listeners' response probabilities are
            jointly credibly different between test conditions.
        
        The Bayesian model is hierarchical.
        The package estimates predictive distributions of results for
        * a random individual in the population from which participants were recruited,
        * each individual in the group of test participants.
        
        ## Phoneme Identification Experiments
        The package can analyse data from simple or rather complex experimental designs,
        including the following features:
        
        1. Phoneme identification data may be collected in one or more **Test Conditions**.
            Each test condition may be a combination of categories from one or more *Test Factors*.
            For example, the main test factor may be *Hearing Aid*,
            with categories *A*, *B*, or *Unaided*.
            Another test factor may be, e.g.,
            *Background*, with categories *Quiet*, or *Noisy*.
            A third factor may be *Position*, with categories *C1* or *C2*, indicating
            the consonant position in CVC nonsense words.
            The analysis shows credible differences between categories within the first (main) test factor,
            for each combination of categories in other (secondary) test factors.
        
        1. One or more **Listener Groups** may be included.
            The analysis shows systematic differences between groups.
        
        1. The analysis model does not require anything about the number of
            test presentations for each phoneme category.
            The validation (Leijon et al., 2016) showed that reliable results
            could be derived with as few as five presentations per phoneme.
            The analysis estimates the **statistical credibility**
            of all observed results, given the amount of collected data.
        
        ## Package Documentation
        General information is given in the package doc-string that may be accessed by command
        `help(ConfMatrixCalc)`.
        
        Specific information about the organisation and accepted formats of input data files
        is presented in the doc-string of module cm_data, accessible via `help(ConfMatrixCalc.cm_data)`.
        
        After running an analysis, the logging output briefly explains
        the analysis results presented in figures and tables.
        
        ## Usage
        1. Install the most recent package version:
            `python3 -m pip install --upgrade ConfMatrixCalc`
        
        1. Copy the template script `run_cm.py` to your work directory, rename it,
            and edit the copy as guided by comments in the template, to specify
            - your experimental layout,
            - the top input data directory,
            - a directory where all output result files will be stored.
        
        1. Run your edited script: `python3 run_my_cm.py`.
        
        ## Requirements
        This package requires Python 3.6 with Numpy, Scipy, and Matplotlib,
        as well as a support package samppy,
        and the Openpyxl package for reading data from Excel workbook documents.
        The pip installer will check and install the required packages if needed.
        
        ## References
        A. Leijon, G. E. Henter, and M. Dahlquist (2016).
        Bayesian analysis of phoneme confusion matrices.
        *IEEE Trans Audio, Speech, and Language Proc* 24(3):469–482.
        doi: 10.1109/TASLP.2015.2512039.
        
        G. A. Miller and P. E. Nicely (1955).
        An analysis of perceptual confusions among some English consonants.
        *J Acoust Soc Amer* 27(2):338–352, 1955.
        doi: 10.1121/1.1907526.
        
        H. Fletcher and J. Steinberg (1929). Articulation testing methods.
        *Bell System Technical Journal* 8:806–854.
        doi: 10.1002/j.1538-7305.1929.tb01246.x.
        
        This Python package is a re-implementation and generalization of a similar MatLab package,
        developed by Arne Leijon for *ORCA Europe, Widex A/S, Stockholm, Sweden*.
        The MatLab development was financially supported by *Widex A/S, Denmark*.
        
        
Keywords: phoneme-identification confusion-matrix Bayesian speech
Platform: UNKNOWN
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3.6
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Multimedia :: Sound/Audio
Requires-Python: >=3.6
Description-Content-Type: text/markdown
