Checking for missing content type metadata ...
This resource contains content types with missing metadata required to make it public or discoverable. Show missing content type metadata.
Click on the edit button ( ) below to edit this resource.
Checking for non-preferred file/folder path names (may take a long time depending on the number of files/folders) ...
This resource contains some files/folders that have non-preferred characters in their name. Show non-conforming files/folders.
This resource contains content types with files that need to be updated to match with metadata changes. Show content type files that need updating.
| Authors: |
|
|
|---|---|---|
| Owners: |
|
This resource does not have an owner who is an active HydroShare user. Contact CUAHSI (help@cuahsi.org) for information on this resource. |
| Type: | Resource | |
| Storage: | The size of this resource is 8.6 GB | |
| Created: | Oct 30, 2024 at 2:27 a.m. (UTC) | |
| Last updated: | Oct 04, 2026 at 3:37 p.m. (UTC) | |
| Citation: | See how to cite this resource | |
| Content types: | CSV Content |
| Sharing Status: | Public |
|---|---|
| Views: | 2382 |
| Downloads: | 298 |
| +1 Votes: | Be the first one to this. |
| Comments: | No comments (yet) |
Abstract
This resource provides the complete reproducibility package for the study Developing Testing Datasets for Benchmarking Automated Quality Control Algorithm Performance with Aquatic Sensor Data. It contains a labeled benchmark dataset derived from 2020–2024 observations at five Logan River Observatory monitoring sites in northern Utah, USA. The resource includes raw aquatic and climate sensor data, manually quality-controlled data, field notes, configuration files, Python code, and the resulting labeled time-series and anomaly-event datasets.
The workflow identifies technician-QC-derived anomaly events by comparing raw and manually corrected observations, then assigns point-level and event-level labels, anomaly patterns, inferred causes, and event features. The benchmark covers water temperature, pH, dissolved oxygen, water stage, and specific conductance. It is intended to support transparent, reproducible evaluation and comparison of automated quality-control and anomaly-detection methods for aquatic sensor time series, and to provide a reusable workflow that can be extended to other monitoring networks and hydrologic settings.
Subject Keywords
Coverage
Spatial
Temporal
| Start Date: | |
|---|---|
| End Date: |
Content
README.md
Summary
- The main Python script automates all key steps for anomaly detection, categorization, and labeling.
- A YAML configuration file (
Initial_Settings.yaml) allows users to customize the analysis without modifying the script. - The pipeline generates labeled datasets, visual plots, and categorized anomaly event reports for quality-controlled water sensor data.
- It generates standardised benchmark testing datasets to evaluate the existing QC methods/tools in water-related data.
Setup Instructions
1. Download the Zip input file (InputDatasets.zip)
2. Install Required Python Packages
bash
pip install -r requirements.txt
3. Configuration
You can modify the Initial_Settings.yaml file to change:
- site: The monitoring site name
- variable: The variable of interest (e.g., SpCond, Temperature)
- start_year, end_year: Time range for analysis
- Any other parameters or plot options Once updated, rerun the script.
4. Run the Script 'Script.py'
Repository Structure
AutomatedAnomalyDetectionLabeling/
│
├── scripts/
│ ├── Script.py
│ └── Initial_Settings.yaml
│
├── InputDatasets/
│ ├── FieldNote_data/
│ ├── Processed_data/
│ └── Raw_data/
│
├── Results/
│
├── Plots/
│
├── requirements.txt
│
└── README.md
- scripts: provides script needed to run the pipeline.
- InputDatasets: retrieve and store input files and essential outputs (e.g., field notes, corrected and raw data).
* After running the script, this directory will automatically be created in your local repository.
- results: provides .CSV files generated from the analysis, which contain the labeled datasets, list of the anomaly events, and complementary files.
* After running the script, this directory will automatically be created in your local repository.
- Plots: provides plots for visualizing the results.
* After running the script, this directory will automatically be created in your local repository.
- Note: The .idea/, .vscode/, and docs/ folders are not required to run the project. They contain editor settings and documentation for the GitHub Pages site.
HydroShare Repository
Explore the full workflow and products (e.g., benchmark dataset, plots) within the Logan River Observatory sites.
🔗 Logan River Observatory – HydroShare Repository
References
Related Geospatial Features
This HydroShare resource is linked to the following geospatial features
Learn more about Related Geospatial Features
We highly recommend that you add Spatial Coverage to this resource before searching for related geospatial features. Otherwise query times can be excessive.
| ${value.text} | ${value.text} |
Click a point to search for features that overlap with that location.
Select a feature for more information.
Credits
Funding Agencies
This resource was created using funding from the following sources:
| Agency Name | Award Title | Award Number |
|---|---|---|
| Cooperative Institute for Research to Operations in Hydrology (CIROH) | None | NA22NWS4320003 |
| Utah Water Research Laboratory, Utah State University | None | None |
How to Cite
This resource is shared under the Creative Commons Attribution CC BY.
http://creativecommons.org/licenses/by/4.0/
Comments
There are currently no comments
New Comment