Geospatial data

US Geological Survey

• USGS

A. Office of Science Quality and Integrity

• Office of Science Quality and Integrity (OSQI)

A1. Trusted Digital Repositories

• USGS Trusted Digital Repositories (TDR) - Distinction Between a USGS TDR and an Acceptable Digital Repository

B. Core Science Systems mission area

• Core Science Systems - one of USGS mission areas

B1. CSS/SSAR

• Science Synthesis Analysis, and Research (SSAR) Program

B1a. Science Data Management

• Science Analytics and Synthesis (SAS) Science Data Management Branch (SDM)
• Data Management guidance, best practices, and tools
• ScienceBase
  • ScienceBase large file uploads
  • ScienceBase-AGOL integration
• Asset Identifier Service (AIS) - Swagger system. Persistent identifiers to make our research more Findable, Accessible, Interoperable, and Reusable (FAIR)
• Metadata creation - tools, best practices, validation, etc.

B2. National Geospatial Program

• • • National Geospatial Program (NGP)

B2a. National Geospatial Program

• [https://www.usgs.gov/3d-elevation-program
• [https://www.usgs.gov/3d-hydrography-program
• The National Map

C. Other Programs

• Center of Excellence for Geospatial Information Science (CEGIS)
  • 3DEP Feature Extraction and Conflation Pull mountain tops, valles, etc out of maps
  • Combine data from The National Map with selected datasets
• USGS ArcGIS Online (AGOL)

Organizations

• Radiant Earth
• Open Geospatial Consortium
• The Open Source Geospatial Foundation
  • Puts on the FOSS4G ("Free and Open Source Software for Geospatial") conference
• Open Topography out of UCSD
• Planetary Computer - The Microsoft Planetary Computer is a platform that lets users leverage the power of the cloud to accelerate environmental sustainability and Earth science.
  • The Data Catalog, which includes petabytes of data about Earth systems, hosted on Azure and made available to users for free.
  • APIs that allow users to search for the data they need across space and time.
  • The Hub, a fully managed computing environment that allows scientists to process massive geospatial datasets.
  • Applications, built by our network of partners, that put the Planetary Computer platform to work for environmental sustainability.
• Federal Geographic Data Committee (FGDC)
  • FGDC org chart - Chaired by Secretary of the Interior, vice chaired by OMB deputy director
• The Pangeo community - develop software and infrastructure to enable Big Data geoscience research

Industry/trade Newspapers/Press releases

• Geospatial World
• Jan 24, 2024: Inflation Reduction Act accelerates USGS effort to collect high-resolution data of landscapes across 25 states
• NOAA Digital Coast GeoZone
  • [https://geozoneblog.wordpress.com/2021/09/29/new-ways-to-access-lidar/

Open Geospatial Consortium

• OGS list of standards

STAC standard

Terminology

• "catalog" = library
• "collections" = section of the library, type of book, author, etc.
• items = books
• asset = the dataset

General

• 2023 | Earth-Search: A STAC API of Open datasets on AWS - Matthew Hanson
• Chris Holmes - Product Architect @ Planet, Board Member @ Open Geospatial Consortium, Technical Fellow @ Radiant.Earth
  • CNG Part 6: Metadata in a Cloud Native Geospatial World

tools

• pystac - Python library for working with any SpatioTemporal Asset Catalog (STAC)
  • docs
• stac-fastapi - STAC API implementation with FastAPI.
  • docs
• pgstac - Schema, functions and a python library for storing and accessing STAC collections and items in PostgreSQL
  • docs

examples

• TiTiler: Working With STAC - At Scale

Specification

• STAC-SPEC - SpatioTemporal Asset Catalog specification - making geospatial assets openly searchable and crawlable
  • docs
• STAC spec proposal submission to OGC
• STAC API proposal submission to OGC
• STAC Extensions
• STAC best practices

Other metadata specifications

• FGDC-CSDGM - Content Standard for Digital Geospatial Metadata
• ISO 19115: Geographic information - Metadata - finalized in 2003 and endorsed by the FGDC in 2010. A series of additional ISO 191 standards have been developed to update, extend, and supplement the 19115 standard.

OpenTopography

• based out of UCSD
• Example Jupyter notebooks

OT dev resources

• myOpenTopo Workbench
• developer info page
• OpenTopographyAPI which consists of
  • Global datasets API
  • USGS 3DEP Raster API
  • OpenTopography Data Catalog - Bounding box-based catalog search of topography datasets hosted by OpenTopography
• Catalogue Service for the Web (CSW)
• Estr geoportal server - competitor to STAC catalog
  • [https://github.com/Esri/geoportal-server-harvester
  • Geoportal example implementation
  • Geoportal server catalog
  • Elasticsearch
  • opensearch vs. elasticsearch

Links

• Spatial Data on the Web Best Practices
• Common Metadata Repository (CMR)
• PANGEO project
• Cloud-Optimized Geospatial Formats guide
• USGS 3DEP FTP User Guide

• RockyWeb explainer
• DOGAMI
• NRCS USDA data gateway
• AWS openData bucket
• Earth Science Information Partners
• Development Seed
• FirstStreet
• U.S. Interagency Elevation Inventory
• HEC-RAS
• Esri Geoportal Server - Geoportal Server is a standards-based, open source product that enables discovery and use of geospatial resources including data and services.
• Nebraska GeoCloud
  • The Nebraska GeoCloud (NGC) is a web-based digital platform for geophysical, geological, and groundwater data and models.
  • The purpose of the NGC is to archive Nebraska’s vast volume of data and make it accessible to both model builders and model users.
  • The NGC consists of databases, web servers, and web interfaces designed for data storage, sharing, and distribution.
  • It contains one interface for Projects and another interface for Data.
  • Projects may include software files, reports, and other information related to a project.
  • It can be used to store and share project files, or it can be used as the final repository for completed projects.
  • The Data interface is built upon structured databases that support the upload and download of data and models used in typical hydrogeological studies.
  • Users can access the data contained in this part of the NGC via the GeoScene3D data portal.
  • These data can also be viewed on an interactive web map and they are accessible via a web map service (WMS) in GIS programs (e.g. ArcGIS, QGIS).
• Virtual Point Clouds - multiple tile stitching explainer
• [1]
• [2]
• [3]
• [4]
• [5]

• PDAL - pronounced poodle
• github hobu USGS lidar - AWS Entwine Point Tiles USGS LiDAR Public Dataset GitHub repo
• How's MY waterway EPA.gov
• Esri Developer Events - Supplemental source code 💻, example projects, slides 📚, and resources supporting Esri Developer Events.
• VEDA Dashboard - VEDA (Visualization, Exploration, and Data Analysis) is NASA's open-source Earth Science platform in the cloud.
• The Earth Observation API
• developmentseed/eoAPI
• NGAC Assessment of the 3d elevation program
• copd

Data

Data Sources - S3 Buckets

• USGS Lidar
• USGS Lidar STAC
• NOAA Coastel lidar
• noaa-nos-coastal-lidar-pds
• The National Map Staged Products Directory (AWS S3)

Data Portals

• U.S. Interagency Elevation Inventory
• [6]
• [7] - Rocky web

Prioritize

• A Review of Options for Storage and Access of Point Cloud Data in the Cloud
• New Ways to Access Lidar - Sept 2021