Checking for non-preferred file/folder path names (may take a long time depending on the number of files/folders) ...

Supporting data and tools for "An Open-source, Semi-supervised Water End Use Disaggregation and Classification Tool"
Authors:
Owners: This resource does not have an owner who is an active HydroShare user. Contact CUAHSI (help@cuahsi.org) for information on this resource.
Type: Resource
Storage: The size of this resource is 142.3 MB
Created: May 24, 2021 at 6:44 p.m.
Last updated: Jan 30, 2023 at 2:13 p.m. (Metadata update)
Published date: Jan 30, 2023 at 2:13 p.m.
DOI: 10.4211/hs.3143b3b1bdff48e0aaebcb4aedf02feb
Citation: See how to cite this resource
Sharing Status: Published
Views: 1563
Downloads: 182
+1 Votes: 1 other +1 this
Comments: No comments (yet)

Abstract

The files provided here are the supporting data and code files for the analyses presented in "An Open-source, Semi-supervised Water End Use Disaggregation and Classification
Tool" a manuscript submitted to the Journal of Water Resource Planning and Management. The data included in this resource were collected using the CIWS-Logger (https://github.com/UCHIC/CIWS-WM-Logger) data logging device. Cyberinfrastructure for Intelligent Water Supply (CIWS) is an open-source, modular, generalized architecture designed to automate the process from data collection to analysis and presentation of high temporal residential water use data. The CIWS-Logger is a low cost device capable of collecting this type of data on existing, magnetically driven water meters. The code included in this resource (CIWS-Disaggregator) demonstrates a new water end use disaggregation and classification tool that builds on existing end use disaggregation studies and addresses the unavailability of code and data used by prior studies. The tool was developed in Python and can be accessed via any current Python programming environment. It was tested on anonymized, high temporal resolution datasets for five homes selected from a larger dataset for 31 homes located in the Cities of Logan and Providence Utah, USA. Results from different meter types and sizes are presented to demonstrate the accuracy of the tool in disaggregating and classifying high temporal resolution data into individual end use events. The results of this paper are reproducible using openly available code and data, representing an accessible platform for advancing end use disaggregation tools. The tool can be adapted to specific research needs.

Subject Keywords

Coverage

Spatial

Coordinate System/Geographic Projection:
WGS 84 EPSG:4326
Coordinate Units:
Decimal degrees
Place/Area Name:
Logan and Providence, Utah, USA
North Latitude
41.7666°
East Longitude
-111.7709°
South Latitude
41.6857°
West Longitude
-111.8650°

Temporal

Start Date:
End Date:

Content

readme.md

Folders in this resource are organized as follows:

1. CIWS_Disaggregator: contains the semi-supervised water end use disaggregation and classification machine learning algorithm used to generate the results in the paper. The folder has three sub-folders, and three files:

  • RawData Folder contains the 4 second temporal resolution water use data collected at each of the 5 houses included in the study for a period of at least two weeks during the summer when outdoor water use was active and two weeks during the winter when no outdoor water use was expected.

  • Results Folder contains the disaggregated and classified water end use events extracted from the high resolution water use data for the 5 residential homes included in the study.

  • Events Folder is an empty folder. Once a user runs the CIWS-Disaggregator code, all of its outputs will be saved automatically in the Events folder.

  • Sites_ResidentialStudy_Info.csv is a .csv file that contains the metadata of the 5 houses included in the study, including the meter size, meter resoultion, number of residents, and irrigation type (sprinkler/hose).

  • TrainingDataset_SS.csv is a .csv file that contains manually labeled water end use events collected by one participating household.

  • CIWS_Disaggregator_SS.py is the python script module used for water end use disaggregation. All water end use disaggregation functions and classes are defined in the module including the filtering, disaggregation, classification, and label assigning functions.

2. Plots: contains the Python and R scripts used to generate figures presented in the Results section of the article. The folder also contains a file All_Events_L.csv that has classified events from all 5 households.

Instructions for reproducing results

Complete the following steps to reproduce the classified events results presented in the article:

  1. Download the complete CIWS_Disaggregator folder.

  2. Leave the files together in the folders to ensure the paths to the files remain correct.

  3. Open the CIWS_Disaggregator_SS_FV.ipynb Jupyter Notebook and execute it.

  4. The Outputs of the tool are .csv files named LabelledEvents_seasonHHID that contain the water end use events extracted from a seasonHHID input file. The results should match the results in the Results folder.

Complete the following steps to reproduce the figures:

  1. Download the complete Plots folder

  2. Leave the files together in the folders to ensure the paths to the files remain correct.

  3. Open the Plots.ipynb Jupyter Notebook and execute it. The Python Juypter Notebook can be used to reproduce Plots 8, 9 and all plots in the Appendix.

Required code libraries

The Python code provided in this resource was developed using Python 3.7.3. The following Python packages are required for running the provided scripts:

  • pandas - Version 1.2.3.
  • matplotlib - Version 3.0.3.
  • seaborn - Version 0.9.0.
  • numpy - Version 1.20.3.
  • sklearn - Version 0.20.3.
  • scipy - Version 1.2-1.

Related Resources

This resource belongs to the following collections:
Title Owners Sharing Status My Permission
Collection of resources that illustrate data processing methods and computational and modeling libraries in HydroShare and linked JupyterHub computing platforms David Tarboton  Public &  Shareable Open Access

Credits

Funding Agencies

This resource was created using funding from the following sources:
Agency Name Award Title Award Number
National Science Foundation CAREER: Cyberinfrastructure for Intelligent Water Supply (CIWS): Shrinking Big Data for Sustainable Urban Water CBET 1552444

How to Cite

Attallah, N., C. J. Bastidas Pacheco (2023). Supporting data and tools for "An Open-source, Semi-supervised Water End Use Disaggregation and Classification Tool", HydroShare, https://doi.org/10.4211/hs.3143b3b1bdff48e0aaebcb4aedf02feb

This resource is shared under the Creative Commons Attribution CC BY.

 http://creativecommons.org/licenses/by/4.0/
CC-BY

Comments

There are currently no comments

New Comment

required