Metadata-Version: 2.1
Name: microt-preprocessing
Version: 0.0.18
Summary: A package that transforms raw sensor data collected from Time study app into intermediate CSV file for analysis of various purposes
Home-page: https://bitbucket.org/mhealthresearchgroup/microt_preprocessing/src/master/
Author: Jixin Li, Aditya Ponnada
Author-email: li.jix@northeastern.edu
License: UNKNOWN
Project-URL: Bug Tracker, https://bitbucket.org/mhealthresearchgroup/microt_preprocessing/issues
Description: # Introduction
        
        ---
        
        ### What is this repository for? ###
        * This repository include Python scripts that transform **raw sensor data** collected from Android mobile devices into **intermediate CSV file** for analysis of various purposes.
        * The preprocessing scripts here intend to be used to provide **reliable, consistent, and structured** intermediate data file for data analysis related to MicroT project.  
        
        
        # Summary of Preprocessed Intermediate File
        
        ---
        The discussion on data source to include can bee found in the [#86 Issue of DataProcessing repository ](https://bitbucket.org/mhealthresearchgroup/dataprocessing/issues/86/analytical-data-structure-and-requirements)
        
        ### 1. Android Smartwatch ###
        
        | Information                           | Raw Data File                                                                                                     | Included Data                                                                                                      | Omitted Data  | Intermediate file                                                                                                                    |
        | --------------------------------------|-------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|---------------|--------------------------------------------------------------------------------------------------------------------------------------|
        | Prompt response                       | [participant_folder] / logs-watch / DATE / HOUR / PromptResponses.log.csv                                         | ID, EMA_type, Date, Prompt Timestamp, Time Zone, Completion Status, Reprompt, Response Timestamp, Q-Key, Response  | None          | [save_path]/[participant_id]/DATE/watch_prompt_response_PARTICIPANT_DATE.csv                                                         |
        | Battery level                         | [participant_folder] / data-watch / DATE / HOUR / Battery.##.event.csv                                            | timestamp, battery_level, battery_charging                                                                         | None          | [save_path]/[participant_id]/DATE/watch_battery_PARTICIPANT_DATE.csv                                                                 |
        | Accelerometer data                    | [participant_folder] / data-watch / DATE / HOUR / AndroidWearWatch-AccelerationCalibrated-NA.*.sensor.baf         | header_timestamp, accelation_meters_per_second_squared(X,Y,Z axis), MIMS-unit                                      | None          | [participant_folder] / data-watch / DATE / HOUR / 020000000000-AccelerationCalibrated.*.sensor.csv, mims_DATE_HOUR.csv; a copy of former two in [save_path]/[participant_id]/DATE/          |
        | App usages                            | [participant_folder] / data-watch / DATE / HOUR / AppEventCounts.csv                                              | log_time, last_hour_timestamp, current_hour_time_stamp, app_package_name, event_time_stamp, app_event              | None          | [save_path]/[participant_id]/DATE/phone_app_usage_PARTICIPANT_DATE.csv                                                               |
        
        ### 2. Android Smartphone ###
        
        | Information                           | Raw Data File                                                                   | Included Data                                                                                                      | Omitted Data  | Intermediate file                                                                                    |
        | --------------------------------------|---------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------|---------------|------------------------------------------------------------------------------------------------------|
        | Prompt response                       | [participant_folder] / logs / DATE / HOUR / PromptResponses.log.csv             | ID, EMA_type, Date, Prompt Timestamp, Time Zone, Completion Status, Reprompt, Response Timestamp, Q-Key, Response  | None          | [save_path]/[participant_id]/DATE/phone_prompt_response_PARTICIPANT_DATE.csv                         |
        | GPS data                              | [participant_folder] / data / DATE / HOUR / GPS.csv                             | log_time, location_time, lat, long, horizontal_accuracy, provider, speed, altitude, bearing                        | None          | [save_path]/[participant_id]/DATE/phone_GPS_PARTICIPANT_DATE.csv                                     |
        | step count                            | [participant_folder] / data / DATE / HOUR / StepCounterService.csv              | log_time_stamp, steps_last_hour, accumulated_steps                                                                 | None          | [save_path]/[participant_id]/DATE/phone_stepCount_PARTICIPANT_DATE.csv                               |
        | Phone state and detected activities   | [participant_folder] / data / DATE / HOUR / ActivityDetected.csv                | log_time, in_vehicle, on_bike, on_foot, running, still, tilting, walking, unknown                                  | None          | [save_path]/[participant_id]/DATE/phone_detected_activity_PARTICIPANT_DATE.csv                       |
        | Phone usage events and broadcasts     |                                                                                 |                                                                                                                    |               | [save_path]/[participant_id]/DATE/phone_usage_broadcasts_PARTICIPANT_DATE.csv                        |
        | Environmental sensors                 | LightSensorStats.csv, ProximitySensorManagerService.csv, AmbientPressManagerService.csv, AmbientTempManagerService.csv, AmbientHumidManagerService.csv |    log time, sensor value, sensor max       | None          |                                                                                                      |
        
        # Code Book
        
        ---  
        
        Detailed explanation of columns in intermediate files can be found in the [code book](https://docs.google.com/document/d/1RsxueU1tCGNSl8-ClNjRPpkv-y9AHej1IfkIU53Lo6E/edit?usp=sharing).
        
        
        ### Python Version ###
        Python 3.6+ (Other versions haven't been tested but should be fine)
        
        ### Dependencies ###
        * For user who wants to include accelerometer and MIMS-unit, the below are extra set-up:
        	1. The MIMS-unit depends on particular [R package](https://www.r-pkg.org/pkg/MIMSunit), so install R on your system.
        	2. Add R to your environment variables. For windowsOS users, put path similar to "C:\Program Files\R\R-4.0.2\bin\x64" to your path in system variables, and reboot your computer.
        
        # Usage Option1: Install and use as a python package
        
        ---
        ### Install package ###
        The pypi link can be accessed [here](https://pypi.org/project/microt-preprocessing/).
        ```
        > pip install microt-preprocessing
        ```  
        
        ### Import package in python ###
        ```
        from microt_preprocessing import time_study_preprocessing_main
        
        microT_root_path = <microT_root_path>                                        # path to the data source folder
        intermediate_file_save_path = <intermediate_file_save_path>                  # path to the destination folder
        decrypt_password = <decrypt_password>                                        # decryption password for GPS file
        delete_raw = "0" or "1"				                                         # "0" denotes not deleting data source, "1" denotes deleting data source
        date = "2020-06-11"			                                                 # specific dates to be preprocessed
        
        time_study_preprocessing_main.preprocessing_all_ema.run_ema_main(microT_root_path, intermediate_file_save_path, decrypt_password, delete_raw, date)
        ```
        
        
        # Usage Option2: Clone this project and run scripts
        
        ---
        
        	
        * **Run script**
        
        	* **For smartphone** 
        
        	```
        	python preprocessing_all_ema.py <microT_root_path> <intermediate_file_save_path> [date_start]
        	```
        
        	* **For smartwatch**
        
        	```
        	python preprocessing_all_uema.py <microT_root_path> <intermediate_file_save_path> <participants_included_text_file_path> [date_start] [date_end]
        	```  
        	
        
        
        # Who do I talk to?
        
        ---
        
        ### Maintained by Aditya and Jixin ###
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.7
Description-Content-Type: text/markdown
