README  Model Instance

This folder contains the scripts, input files, and supporting files required to run and review the analysis reported in the forthcoming manuscript by Humphreys and Mays (2023). All codes run on R v4.1.2.

CONTENTS

1. Instructions
2. Subfolders
3. Files

1. INSTRUCTIONS

The purpose of the wrapper script _Wrapper.R is to perform a complete analysis of the South Fork Clearwater River stream gage and water quality data for the Johns Creek, Red River, and South Fork Red River basins during the period 1986-2011. This analysis is reported in Humphreys and Mays (2023). All codes developed and run on R v4.1.2.

The analysis consists of:

1st loading in synthetic hydrographs (created using the USGS MOVE.1 regression technique via the SREF application).

2nd creating modeled sediment concentration and loading trends using the WRTDS statistical algorithm via the EGRET R package.

3rd loading in land management, precipitation (GridMET), and snow water equivalent (SWE) (snotel) data.

4th performing regressions (multi-variable linear and random forest) to test relationships between modeled sediment trends and disturbance data to determine if we can predict changes in sediment trends. 

Each step of the analysis is completed using a unique script called by this wrapper, allowing for a "one-click" execution of the complete analysis. 

File structure must be as follows:
	/base_dir
		/Input_Data
			/Regressions
		/Plots
		/Results
		/Scripts

WARNING: The wrapper script _Wrapper.R requires the user to update the variable base_dir to match the path and name of their base directory.

This script will take several minutes to run completely, and warnings are printed to console.

2. SUBFOLDERS

/Model_Instance
	/Input_Data
		/QC_and_Supporting
		/Regressions
		/SREF
	/Plots
	/Results
	/Scripts

2.1. /Input_Data (input data required for model analysis) 

Stream data provided by United States Forest Service, Nez Perce-Clearwater National Forest. Full citations available in forthcoming manuscript, Humphreys and Mays (2023).

2.1.1. /Input_Data/QC_and_Supporting (sensitivity analysis results, calibration results, hypothesis testing adjustments, and QC files) 

These files are not directly called by any script but provide supporting information and documentation and supplementary procedures for quality assurance/quality control (QA/QC).

2.1.2. /Input_Data/Regressions (inputs and supporting files for regression analysis)

Land management data provided by United States Forest Service, Region 1. Precipitation data from GridMET (Abatzoglou, 2011). SWE data from SNOTEL, Natural Resources Conservation Service. Full citations available in forthcoming manuscript, Humphreys and Mays (2023)

2.1.3. /Input_Data/SREF (supporting files, inputs, and outputs from the MOVE.1 analysis in SREF) 

SREF v1.0 by USGS, Granato (2009)

2.2. Plots (figures output from scripts) 

2.3. Results (figures and datasets output from scripts)

2.4. Scripts (scripts to run full analysis, developed and run in R v4.1.2)

R package dependencies:	EGRET v3.0.7, tidyverse v1.3.2, lubridate v1.8.0, ggpubr v0.4.0, randomForest v4.7-1.1, see v0.7.4, writexl v1.4.1, and dataRetrieval v.2.7.11.

3. FILES

Metadata format: Filename; Description

/Input_Data/

/QC_and_Supporting; Subdirectory
/Regressions; Subdirectory
/SREF; Subdirectory

JohnsDaily.csv; Synthetic discharge record, Johns Creek
JohnsEstimatedQ.csv; Estimated discharge, Johns Creek
JohnsInfoSSC.csv; Site information (SSC), Johns Creek
JohnsInfoTurb.csv; Site information (turbidity), Johns Creek[MD1]
JohnsQ.csv; Observed discharge, Johns Creek
JohnsSSC.csv; Observed SSC, Johns Creek
JohnsTurb.csv; Observed turbidity, Johns Creek

RedDaily.csv; Synthetic discharge record, Red River
RedEstimatedQ.csv; Estimated discharge, Red River
RedInfoSSC.csv; Site information (SSC), Red River
RedInfoTurb.csv; Site information (turbidity), Red River[MD2]
RedQ.csv; Observed discharge, Red River
RedSSC.csv; Observed SSC, Red River
RedTurb.csv; Observed turbidity, Red River

SFRedDaily.csv; Synthetic discharge record, South Fork Red River
SFRedEstimatedQ.csv; Estimated discharge, South Fork Red River
SFRedInfoSSC.csv; Site information (SSC), South Fork Red River
SFRedInfoTurb.csv; Site information (turbidity), South Fork Red River[MD3]
SFRedQ.csv; Observed discharge, South Fork Red River
SFRedSSC.csv; Observed SSC, South Fork Red River
SFRedTurb.csv; Observed turbidity, South Fork Red River

/Input_Data/QC_and_Supporting/
/Bulletin17B; Subdirectory


Adjusted_Pvalues_Bonferroni.csv; P-value adjustments by the Bonferonni method, output from p.adjust in R
Adjusted_Pvalues_FDR.csv; P-value adjustments by the False Discovery Rate method, output from p.adjust in R
JohnsSSC.xlsx; Observed SSC QAQC, Johns Creek
PowerFunction_Fits_WithEquations.txt; Power function fitting to observed data, all three tributary basins
RedSSC.xlsx; Observed SSC QAQC, Red River
Sensitivity_Johns.xlsx; Time lag sensitivity analysis, Johns Creek
SFRedSSC.xlsx; Observed SSC QAQC, South Fork Red River
WRTDS_Calibration.xlsx; Calibration of WRTDS parameters for all three tributary basins

/Input_Data/QC_and_Supporting//Bulletin17B
All files (including input, HEC-SSC, and output) related to Bulletin 17B analysis to determine Q1.5 effective discharge in each subbasin 

/Input_Data/Regressions/

GridMET_SFCWR_1986-2011.csv; GridMET precipitation for a polygon approximating the SFCWR basin, accessed via USGS Geo Data Portal
Johns_Anthro_Disturbances.csv; Filtered land management records, Johns Creek
Johns_ChannelBuildingFlows.csv; time series of annual channel building flow events, Johns Creek
Red_Anthro_Disturbances.csv; Filtered land management records, Red River
Red_ChannelBuildingFlows.csv; time series of annual channel building flow events, Red River
RegressionData.xlsx; Regression model data summary, all three tributary basins
Regressions_Compare.xlsx; Regression model results comparison, all three tributary basins
SFRed_Anthro_Disturbances.csv; Filtered land management records, South Fork Red River
SFRed_ChannelBuildingFlows.csv; time series of annual channel building flow events, South Fork Red River
Snotel_MountainMeadows.csv; SWE from NRCS Mountain Meadows #650
tuneRF_Johns.txt; Random forest regression tuning (via tuneRF in randomforest R package), Johns Creek
tuneRF_Red.txt; Random forest regression tuning (via tuneRF in randomforest R package), Red River
tuneRF_SFRed.txt; Random forest regression tuning (via tuneRF in randomforest R package), South Fork Red River

/Input_Data/SREF/

11111111-PeriodOfRecordDates.txt[MD4]; Period of record date range, Johns Creek
11111111by13338500MV1Q.txt; MOVE.1 estimated discharge, Johns Creek by USGS Stites
11111111DAT.txt; Input data to MOVE.1 regression, Johns Creek and USGS sites
11111111MOV.txt; MOVE.1 results and model statistics, Johns Creek by USGS Stites
13336500Q.txt; Observed Discharge, USGS at Selway
13337000Q.txt; Observed Discharge, USGS at Lochsa
13338500Q.txt; Observed Discharge, USGS at Stites
22222222-PeriodOfRecordDates.txt; Period of record date range, Red River
22222222by13338500MV1Q.txt; MOVE.1 estimated discharge, Red River by USGS Stites
22222222DAT.txt; Input data to MOVE.1 regression, Red River and USGS sites
22222222MOV.txt; MOVE.1 results and model statistics, Red River by USGS Stites
33333333-PeriodOfRecordDates.txt; Period of record date range, South Fork Red River
33333333by13338500MV1Q.txt; MOVE.1 estimated discharge, South Fork Red River by USGS Stites
33333333DAT.txt; Input data to MOVE.1 regression, South Fork Red River and USGS sites
33333333MOV.txt; MOVE.1 results and model statistics, South Fork Red River by USGS Stites
IndexStations.txt; List of index sites (Stites, Lochsa, Selway)
JohnsPR.txt; Partial record observed discharge, Johns Creek
LochsaDailyDischarge.xlsx; Index observed discharge, USGS Lochsa
RedPR.txt; Partial record observed discharge, Red River
SelwayDailyDischarge.xlsx; Index observed discharge, USGS Selway
SFRedPR.txt; Partial record observed discharge, South Fork Red River
StitesDailyDischarge.xlsx; Index observed discharge, USGS Stites

/Plots/

Comparison_Johns.png; Plot comparing observed SSC, WRTDS estimated SSC, and WRTDS flow normalized SSC. Includes smoothed line of annual estimated means at[MD5] Johns Creek
Comparison_Red.png; Plot comparing observed SSC, WRTDS estimated SSC, and WRTDS flow normalized SSC. Includes smoothed line of annual estimated means at Red River
Comparison_SFRed.png; Plot comparing observed SSC, WRTDS estimated SSC, and WRTDS flow normalized SSC. Includes smoothed line of annual estimated means at South Fork Red River
EP_Q_Johns.png; Exceedance curve, observed and synthetic discharge, Johns Creek
EP_Q_Red.png; Exceedance curve, observed and synthetic discharge, Red River
EP_Q_SFRed.png; Exceedance curve, observed and synthetic discharge, South Fork Red River
EP_SSC_Johns.png; Exceedance curve, observed SSC, Johns Creek
EP_SSC_Red.png; Exceedance curve, observed SSC, Red River
EP_SSC_SFRed.png; Exceedance curve, observed SSC, South Fork Red River
EP_Turb_Johns.png; Exceedance curve, observed turbidity, Johns Creek
EP_Turb_Red.png; Exceedance curve, observed turbidity, Red River
EP_Turb_SFRed.png; Exceedance curve, observed turbidity, South Fork Red River
Fit_Johns_SSCbyQ.png; Plot of linear function fit to observed data, SSC by discharge, Johns Creek
Fit_Johns_SSCbyQ_Power.png; Plot of power function fit to observed data, SSC by discharge, Johns Creek
Fit_Johns_SSCbyTurb.png; Plot of linear function fit to observed data, SSC by turbidity, Johns Creek
Fit_Red_SSCbyQ.png; Plot of linear function fit to observed data, SSC by discharge, Red River
Fit_Red_SSCbyQ_Power.png; Plot of power function fit to observed data, SSC by discharge, Red River
Fit_Red_SSCbyTurb.png; Plot of linear function fit to observed data, SSC by turbidity, Red River
Fit_SFRed_SSCbyQ.png; Plot of linear function fit to observed data, SSC by discharge, South Fork Red River
Fit_SFRed_SSCbyQ_Power.png; Plot of power function fit to observed data, SSC by discharge, South Fork Red River
Fit_SFRed_SSCbyTurb.png; Plot of linear function fit to observed data, SSC by turbidity, South Fork Red River
HalfViolin_Discharge.png; Half violin plot of distribution of observed discharge, all three tributary basins
HalfViolin_SSC.png; Half violin plot of distribution of observed SSC, all three tributary basins
HalfViolin_Turb.png; Half violin plot of distribution of observed turbidity, all three tributary basins
QSynth_Johns.png; Synthetic hydrograph (observed augmented by MOVE.1 estimates), Johns Creek
QSynth_Red.png; Synthetic hydrograph (observed augmented by MOVE.1 estimates), Red River
QSynth_SFRed.png; Synthetic hydrograph (observed augmented by MOVE.1 estimates), South Fork Red River
RF_Johns_VarImpPlot.png; Plot of random forest regression variable importance, Johns Creek full period of record
RF_Johns_VarImpPlot_AT.png; Plot of random forest regression variable importance, Johns Creek post-TMDL
RF_Johns_VarImpPlot_AT_FN.png; Plot of random forest regression variable importance, Johns Creek post-TMDL flow normalized
RF_Johns_VarImpPlot_BT.png; Plot of random forest regression variable importance, Johns Creek pre-TMDL
RF_Johns_VarImpPlot_BT_FN.png; Plot of random forest regression variable importance, Johns Creek pre-TMDL flow normalized
RF_Johns_VarImpPlot_D.png; Plot of random forest regression variable importance, Johns Creek dry years
RF_Johns_VarImpPlot_D_FN.png; Plot of random forest regression variable importance, Johns Creek dry years flow normalized
RF_Johns_VarImpPlot_FN.png; Plot of random forest regression variable importance, Johns Creek full period of record flow normalized
RF_Johns_VarImpPlot_W.png; Plot of random forest regression variable importance, Johns Creek wet years
RF_Johns_VarImpPlot_W_FN.png; Plot of random forest regression variable importance, Johns Creek wet years flow normalized
RF_Red_VarImpPlot.png; Plot of random forest regression variable importance, Red River full period of record
RF_Red_VarImpPlot_AT.png; Plot of random forest regression variable importance, Red River post-TMDL
RF_Red_VarImpPlot_AT_FN.png; Plot of random forest regression variable importance, Red River post-TMDL flow normalized
RF_Red_VarImpPlot_BT.png; Plot of random forest regression variable importance, Red River pre-TMDL
RF_Red_VarImpPlot_BT_FN.png; Plot of random forest regression variable importance, Red River pre-TMDL flow normalized
RF_Red_VarImpPlot_D.png; Plot of random forest regression variable importance, Red River dry years
RF_Red_VarImpPlot_D_FN.png; Plot of random forest regression variable importance, Red River dry years flow normalized
RF_Red_VarImpPlot_FN.png; Plot of random forest regression variable importance, Red River full period of record flow normalized
RF_Red_VarImpPlot_W.png; Plot of random forest regression variable importance, Red River wet years
RF_Red_VarImpPlot_W_FN.png; Plot of random forest regression variable importance, Red River wet years flow normalized
RF_SFRed_VarImpPlot.png; Plot of random forest regression variable importance, South Fork Red River full period of record
RF_SFRed_VarImpPlot_AT.png; Plot of random forest regression variable importance, South Fork Red River post-TMDL
RF_SFRed_VarImpPlot_AT_FN.png; Plot of random forest regression variable importance, South Fork Red River post-TMDL flow normalized
RF_SFRed_VarImpPlot_BT.png; Plot of random forest regression variable importance, South Fork Red River pre-TMDL
RF_SFRed_VarImpPlot_BT_FN.png; Plot of random forest regression variable importance, South Fork Red River pre-TMDL flow normalized
RF_SFRed_VarImpPlot_D.png; Plot of random forest regression variable importance, South Fork Red River dry years
RF_SFRed_VarImpPlot_D_FN.png; Plot of random forest regression variable importance, South Fork Red River dry years flow normalized
RF_SFRed_VarImpPlot_FN.png; Plot of random forest regression variable importance, South Fork Red River full period of record flow normalized
RF_SFRed_VarImpPlot_W.png; Plot of random forest regression variable importance, South Fork Red River wet years
RF_SFRed_VarImpPlot_W_FN.png; Plot of random forest regression variable importance, South Fork Red River wet years flow normalized
WRTDS_Johns_Flux.png; Plot of WRTDS modeled suspended sediment loading, Johns Creek
WRTDS_Johns_SSC.png; Plot of WRTDS modeled suspended sediment concentration, Johns Creek
WRTDS_Red_Flux.png; Plot of WRTDS modeled suspended sediment loading, Red River
WRTDS_Red_SSC.png; Plot of WRTDS modeled suspended sediment concentration, Red River
WRTDS_SFRed_Flux.png; Plot of WRTDS modeled suspended sediment loading, South Fork Red River
WRTDS_SFRed_SSC.png; Plot of WRTDS modeled suspended sediment concentration, South Fork Red River

/Results/

Hydro.csv; Summary of hydrologic predictor variables for input to regression models
Johns_WRTDS_Results.csv; WRTDS model results (SSC), Johns Creek
MVLR_Johns.fit.txt; Multivariable linear regression results and model statistics, Johns Creek
MVLR_Red.fit.txt; Multivariable linear regression results and model statistics, Red River
MVLR_SFRed.fit.txt; Multivariable linear regression results and model statistics, South Fork Red River
PowerFunction_Fits.txt; Power function fits to observed data, all tributary basins
Red_WRTDS_Results.csv; WRTDS model results (SSC), Red River
RegressionData_Johns.csv; Sample regression model input, Johns Creek
RegressionData_Red.csv; Sample regression model input, Red River
RegressionData_SFRed.csv; Sample regression model input, South Fork Red River
Regressions_Compare.csv; Output from Regression_Comparison_Tables.R, regression model statistics summary
RF_Johns.fit.txt; Random forest regression model statistics, Johns Creek
RF_Red.fit.txt; Random forest regression model statistics, Red River
RF_SFRed.fit.txt; Random forest regression model statistics, South Fork Red River
SFRed_WRTDS_Results.csv; WRTDS model results (SSC), South Fork Red River
WRTDS_Johns.txt; WRTDS model fit statistics, Johns Creek
WRTDS_Red.txt; WRTDS model fit statistics, Red River
WRTDS_SFRed.txt; WRTDS model fit statistics, South Fork Red River

/Scripts/

_Wrapper.R; Wrapper script to establish file structure, load dependencies, and execute all other scripts in analysis (requires user to update the variable base_dir to match their base directory)
ComparisonPlot_Johns.R; Generate comparison plot of observed and WRTDS estimated SSC, Johns Creek
ComparisonPlot_Red.R; Generate comparison plot of observed and WRTDS estimated SSC, Red River
ComparisonPlot_SFRed.R; Generate comparison plot of observed and WRTDS estimated SSC, South Fork Red River
ExceedenceCurves.R; Generate discharge, SSC, and turbidity exceedance curves for all three tributary basins
Load_Inputs.R; Load required input data into R for water quality modeling
Load_Regression_Data.R; Load required input data into R for regression analyses
Observed_Fits.R; Fitting of linear functions to observed datasets and plotting, all three tributary basins
Observed_Fits_Power.R; Fitting of power functions to observed datasets and plotting, all three tributary basins
Observed_ViolinPlots.R; Generate half violin distribution/box plots of observed data, all three tributary basins
QSyn_Plot_Johns.R; Generate and plot synthetic hydrograph, Johns Creek
QSyn_Plot_Red.R; Generate and plot synthetic hydrograph, Red River
QSyn_Plot_SFRed.R; Generate and plot synthetic hydrograph, South Fork Red River
Regressions_Comparison_Tables.R; Generate regression model statistic datasets for comparison
Regressions_Johns.R; Generate regression models, plot and export results, Johns Creek
Regressions_Red.R; Generate regression models, plot and export results, Red River
Regressions_SFRed.R; Generate regression models, plot and export results, South Fork Red River
tuneRF.R; Tune random forest regression models and print results to txt file
USGS_DataRetrieval.R; Retrieve USGS stream data for Stites, Lochsa, Selway
WriteHydro.R; Compile hydrologic predictor variable datasets
WRTDS_Johns_SSC.R; Generate WRTDS model for SSC and plot results, Johns Creek
WRTDS_Red_SSC.R; Generate WRTDS model for SSC and plot results, Red River
WRTDS_SFRed_SSC.R; Generate WRTDS model for SSC and plot results, South Fork Red River

[MD1]Each of these two files is only two lines long. Is that correct?
[MD2]Ditto.
[MD3]Ditto.
[MD4]Apparently Windows 10 sorts files differently from your system. It put this file 4th, after the other three starting with 1111111. No changes needed. Just an observation.
[MD5]Changed to avoid using the semicolon for something other than a record delimiter between the file name and the file description.