Checking for non-preferred file/folder path names (may take a long time depending on the number of files/folders) ...
This resource contains some files/folders that have non-preferred characters in their name. Show non-conforming files/folders.
This resource contains content types with files that need to be updated to match with metadata changes. Show content type files that need updating.
| Authors: |
|
|
|---|---|---|
| Owners: |
|
This resource does not have an owner who is an active HydroShare user. Contact CUAHSI (help@cuahsi.org) for information on this resource. |
| Type: | Resource | |
| Storage: | The size of this resource is 31.0 MB | |
| Created: | May 26, 2023 at 8:56 p.m. | |
| Last updated: | Sep 28, 2023 at 5:38 p.m. | |
| Citation: | See how to cite this resource |
| Sharing Status: | Public |
|---|---|
| Views: | 926 |
| Downloads: | 414 |
| +1 Votes: | 2 others +1 this |
| Comments: | No comments (yet) |
Abstract
Scientific and management challenges in the water domain require synthesis of diverse data. Many data analysis tasks are difficult because datasets are large and complex; standard data formats are not always agreed upon or mapped to efficient structures for analysis; scientists may lack training for tackling large and complex datasets; and it can be difficult to share, collaborate around, and reproduce scientific work. Overcoming barriers to accessing, organizing, and preparing datasets for analyses can transform the way water scientists work. Building on the HydroShare repository’s cyberinfrastructure, we have advanced two Python packages that make data loading, organization, and curation for analysis easier, reducing time spent in choosing appropriate data structures and writing code to ingest data. These packages enable automated retrieval of data from HydroShare and the USGS’s National Water Information System (NWIS) (i.e., a Python equivalent of USGS’ R dataRetrieval package), loading data into performant structures that integrate with existing visualization, analysis, and data science capabilities available in Python, and writing analysis results back to HydroShare for sharing and publication. While these Python packages can be installed for use within any Python environment, we will demonstrate how the technical burden for scientists associated with creating a computational environment for executing analyses can be reduced and how sharing and reproducibility of analyses can be enhanced through the use of these packages within CUAHSI’s HydroShare-linked JupyterHub server.
This HydroShare resource includes all of the materials presented in a workshop at the 2023 CUAHSI Biennial Colloquium.
Subject Keywords
Content
readme.md
Workshop Materials
This HydroShare resource contains the materials for a workshop delivered at the 2023 CUAHSI Biennial Colloquium in Lake Tahoe, California June 12-14, 2023.
Workshop Title: Using Python Packages and HydroShare to Advance Open Data Science and Analytics for Water
Files included in this resource:
- Workshop_Slides_6-6-2023.pptx: These are the PowerPoint slides presented during the workshop session.
- USGS_dataretrieval_Example_1.ipynb: A Jupyter notebook with example code for retrieving daily streamflow data for a USGS streamflow gage.
- USGS_dataretrieval_Example_2.ipynb: A Jupyter notebook with example code for retrieving 15-minute realtime discharge data for a USGS streamflow gage. Additional code examples show how to retrieve multiple sites at once and how to deal with timestamps for the data.
- USGS_Dataretrieval_Example_3.ipynb: A Jupyter notebook with example code for using additional functions from the USGS dataretrieval package.
- HydroShare_SnowtoFlow_Example.ipynb: A Jupyter notebook that demonstates an analysis that retrieves streamflow data from USGS NWIS, snow water equivalent data from the NRCS SNOTEL system and then develops a regression model to predict peak streamflow from peak snow water equivalent. This notebook was developed for sharing in HydroShare.
- HydroShare_hsclient_Example_1.ipynb: A Jupyter notebook that demonstrates how to create a new HydroShare resource, create and edit the metadata for the resource, and then add content files.
- HydroShare_hsclient_Example_2.ipynb: A jupyter notebook that demonstrates how to retrieve data content from HydroShare using hsclient and then automatically load data into a performant data structure for visualization and analysis.
Running the Notebooks in this Resource
If you want to run the Jupyter notebooks in this resource, you will first need to have a HydroShare user account and be a member of the CUAHSI Cloud Computing Group in HydroShare. Once you have a HydroShare account, click here and then request to join the group. Once you are a member of the group, you will be able to access the CUAHSI JupyterHub server.
Once you are a member of the CUAHSI Cloud Computing Group and are signed into HydroShare, do the following to run the Jupyter Notebooks in this resource:
- Click on the "Open With" button at the top of this resource
- Select "CUAHSI Jupyter Hub".
- Agree to the terms of use and click the "Sign in with HydroShare" button.
- You will then authorize the CUAHSI JupyterHub server to use your HydroShare login.
- When presented with the Server Options page, select the "Python - v3.8" option and then click the "Start" button. It will take a few moments for your JupyterHub server to start.
- Once your JupyterHub server is running, you will see the Jupyter notebooks in the left panel of the window. Double click on one of the Jupyter notebooks to execute it.
Additional Example Notebooks
See the Related Resources metadata below for links to HydroShare resources containing additional example Jupyter notebooks demonstrating how to use the USGS dataretrieval Python package and the hsclient Python client package.
Related Resources
| The content of this resource references | Horsburgh, J. S., S. S. Black (2021). HydroShare Python Client Library (hsclient) Usage Examples, HydroShare, http://www.hydroshare.org/resource/7561aa12fd824ebb8edbee05af19b910 |
| The content of this resource references | Horsburgh, J. S., A. S. Jones, S. S. Black, T. O. Hodson (2022). USGS dataretrieval Python Package Usage Examples, HydroShare, http://www.hydroshare.org/resource/c97c32ecf59b4dff90ef013030c54264 |
Credits
Funding Agencies
This resource was created using funding from the following sources:
| Agency Name | Award Title | Award Number |
|---|---|---|
| National Science Foundation | Collaborative Research: Elements: Advancing Data Science and Analytics for Water (DSAW) | 1931297 |
How to Cite
This resource is shared under the Creative Commons Attribution CC BY.
http://creativecommons.org/licenses/by/4.0/
Comments
There are currently no comments
New Comment