Metadata-Version: 2.1
Name: soundscape-generation
Version: 0.1.0
Summary: Generate soundscapes based on images.
Home-page: https://github.com/hslu-abiz/soundscape-generation
Author: ABIZ Lab
Author-email: abiz@hslu.ch
License: UNKNOWN
Description: # Soundscape Generation
        
        ## Table of Contents
        
        1. [Installation](#installation)
        2. [Usage](#usage)
        3. [References](#references)
        
        ## Installation
        
        ### Scaper Installation
        
        The sound generation module was developed using Scaper. Given a collection of isolated sound events, Scaper acts as a
        high-level sequencer that can generate multiple soundscapes from a single probabilistically defined specification.
        
        Follow the instructions give in the following link:
        
        * [Scaper installation](https://scaper.readthedocs.io/en/latest/installation.html)
        
        ### Download Dependencies
        
        ```bash
        pip install -r requirements.txt
        ```
        
        ### Download Cityscapes Dataset
        
        To download the dataset, a cityscapes account is required for the authentification. Such an account can be created
        on [www.cityscapes-dataset.com](https://www.cityscapes-dataset.com/). After the registration, run the `download_data.sh`
        script. During the download, it will ask you to provide your email and password for authentification.
        
        ```bash
        ./download_data.sh
        ```
        
        ## Usage
        
        For the object detection module a
        pre-trained [ERFNet](http://www.robesafe.es/personal/eduardo.romera/pdfs/Romera17tits.pdf) is used, which is then
        finetuned on the Cityscapes dataset.
        
        ### Train Object Segmentation Network
        
        To train the network, run the follwing command.
        
        ```bash
        python train.py --num_epochs 70 --batch_size 8 --evaluate_every 1 --save_weights_every 1
        ```
        
        By default, training resumes from the latest saved checkpoint. If the `checkpoints/` directory is missing, the training
        starts from scratch.
        
        ### Test the Segmentation Network
        
        Run the following command to predict the semantic segmentation of every image in the `test_images/` directory (note:
        results are saved in the `test_segmentations/` directory)
        
        ```bash
        python predict.py
        ```
        
        Ensure that you specify the image's file type in the image path variable in `predict.py`.
        
        ### Generate soundscapes
        
        Run the file soundGeneration.py to generate soundscapes of every image in the `test_images/` directory (note: results
        are saved in the `soundscapes/` directory). Ensure that you specify the image type of the image in the image path
        variable of `predict.py`.
        
        ## Results
        
        ### Object Detection
        
        ![](assets/test1.png)
        ![](assets/test2.png)
        ![](assets/test3.png)
        
        The above predictions are produced by a network trained for 67 epochs that achieves a mean class IoU score of 0.7084 on
        the validation set. The inference time on a Tesla P100 GPU is around 0.2 seconds per image. The model was trained for 70 epochs on a single Tesla P100. After the training, the checkpoint
        that yielded to highest validation IoU score was selected. The progression of the IoU metric is shown below.
        
        ![](assets/iou_plot.png)
        
        ## References
        
        * [J. Salamon, D. MacConnell, M. Cartwright, P. Li and J. P. Bello, "Scaper: A library for soundscape synthesis and augmentation," 2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2017, pp. 344-348, DOI: 10.1109/WASPAA.2017.8170052.](http://www.justinsalamon.com/uploads/4/3/9/4/4394963/salamon_scaper_waspaa_2017.pdf)
        * [E. Romera et al., "ERFNet: Efficient Residual Factorized ConvNet for Real-time Semantic Segmentation", 2017](http://www.robesafe.es/personal/eduardo.romera/pdfs/Romera17tits.pdf)
        * [Official PyTorch implementation of ERFNet](https://github.com/Eromera/erfnet_pytorch)
        
Keywords: soundscapes,generation,tensorflow
Platform: UNKNOWN
Classifier: Development Status :: 1 - Planning
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Description-Content-Type: text/markdown
