Marine Data Literacy 2.0

Providing instruction for managing, converting, analyzing and displaying oceanographic station data, marine meteorological data, GIS-compatible marine and coastal data or model simulations, and mapped remote sensing imagery

 

 

 

 

Editors:  Lica Krug PhD, Joao Gusman PhD           Publisher/Founder:   Murray Brown PhD

EXERCISES are unchanged, but in slightly DIFFERENT LOCATIONS from any lists or courses you may have.  This new version is infinitely easier to edit or add exercises, which can be familiar HTML docs with figure files, or PDFs -- anything that an internet browser can visualize.  Please let the editors know if you have problems or find errors.

HOW TO FIND EXERCISES BY TITLES OR TOPICS IN MDL 2.0:

1.  Manually scan the Tables of Contents below. or

2.  Use your browser's SEARCH function.  In Google use CTRL-F to open it.

 

MDL Table of Contents (top level)

 

1  Marine Data Basics

1A PC Tips

1B Data Tips

1C  MDL Secrets of Happiness

2 Marine GIS Operations and Tools

2A Software

2B Basic Mapping

2C Adding Shapes to Maps

2D Managing Shapes

2E Adding Grids/Rasters to Maps

3 Principal Global Archives of In Situ Ocean Data

4 Managing and Analyzing Marine Data in Ocean Data View

4A Data Collection & Quality Control

4B Special Data Additions

4C Standard and Special Analyses

4D Exporting Selected Data Images

5 Gridding Marine Data and Handling Grid Products

6 Plotting Motion Vectors

7 Handling Relief Data (Bathymetry and Topography)

8 Network Access Tools and Data Services

9 Our Oceans Today: Operational and Synoptic Data Products

9A General

9B Currents

9C In Situ/At Sea

9D Models & Forecasts

9E Optics/Pigments/Nutrients

9F Salinity (Models & Measurements)

9G Sea Surface Height/Tides

9H Sea Ice

9I Sea Temperature

9J Waves

9K Winds & Weather

9L  General and Ancillary

MDL Extras | 1PLs Co

COURSES - Under revision/construction

 

MDL Table of Contents (detailed)

 

1 Marine Data Basics

 1A PC Tips:

·        1.1 General Purpose Folder Structure - Install and use this folder structure to organize your files

·        1.2 Online Marine Data Calculators - Collection of sites; some are lists of sites

·        1.3 Running 32-Bit Saga on a Mac with WINE - A beta test, but it is reported to work well

·        1.4 Managing Your Identity and Digital Communications in Marine Science - It's time to clean up your ID act

·        1.9 Online Map Sketching - Deprecated

·        1.13 General File-Naming Convention for Earth Science Datasets - Expanded OceanTeacher article

1B Data Tips:

·        1.5 Data Sources for MDL Exercises - Links back to original sources

·        1.6 Common Data Formats Used by Marine Scientists - Most-frequently encountered formats, with doc links

·        1.7 Essential Global Data Layers for Marine Atlases - Used in GIS training, where local/regional data are not yet found

·        1.8 Selected Marine Data Software - Software in MDL, plus a few related favorites; not comprehensive

·        1.10 Marine Data Product Atlases - Can be used for quick comparison/QC purposes

·        1.11 Raster Scales and Vector Resolution - Rough methods to estimate both

·        1.12 Randomize and/or Subsample Table Data in Excel - Especially useful for converting data products to pseudo-datasets

1C MDL Secrets of Happiness:

·        FILENAMES:  Use detail-rich, long filenames to identify your data files (see article above)

o   The complete folder path plus long filename cannot exceed 256 characters on most PCs.

·        AVOID SEASON NAMES:  Never use spring, summer, fall (or autumn) and winter, because they will cause confusion.  Your work will be more understandable with season labels like JFM or JAS or NDJ, or however you group the months.    Even if your work is obviously in the far north or south, avoid problems by avoiding words that can be misinterpreted. 

o   POSSIBLE EXCEPTION:  If you're referring to clearly understood seasonal phenomena, you can use terms like "local autumn overturn", etc.

·        ASCII EDITORS

o   Install and use and easy, free editor for all ASCII editing. 

o   Never use Word, WordPad or NotePad because they sometimes make undesirable changes

·        ZIPPING or UNZIPPING:  If you don't have WinZip, install and use 7zip

·       FTP:  For new or revised MDL items; use FileZilla; ask management for a script file for quick uploads to our server

·        FILE/FOLDER MANAGEMENT:  

o   View and manage all files and folders in Windows Explorer

o   Never use My Computer or its child windows.

o   Always view DETAILS for the folder contents, and never view icons.

o   Find the control for hiding extensions of known file types and turn it off (somewhere in every browser); you must SEE those extensions!

·        FILE SAVING:  Whenever you use SAVE AS, or SAVE TARGET AS

o   Check the file to make sure that Windows has not added an informational header, such as "Downloaded from...etc."

o   Check for TXT added incorrectly to the end of a filename.

 

2 Marine GIS Operations & Tools - Making maps of earth features and metocean data analyses. This series of exercises includes all the basic tools for making maps of marine and coastal areas, using existing datafiles (vector or grid) or new ones made by you.  It begins with the assumption that an Area of Interest (AOI) is already known.  If you are not certain of your exact AOI, then begin these exercises with a very large area, say 3 times as wide (in longitude) and 3 times as tall (in latitude) as your first general guess for the AOI.  After a bit of experience, you'll decide on a final (smaller) AOI.  If you are working in a group project, then carefully make a joint decision.    

2A Software:  The MDL Geographic Information System (GIS) exercises are based on the Free and Open Source Software (FOSS) program Saga.  You could use just about any GIS program, because they have many common concepts and features.  Saga, used here, has so many easily accessible features, and it has a very dedicated working group keeping it fixed and updated.

2B Basic Mapping

·        2.1 Saga Preliminary Setup

o   MAP and DATA settings for best performance

·        2.2 MDL Exercises Project Area

o   Introduces area polygon (Liberia) for class exercises

o   Uses -180º to +180º mapping schema

·        2.3 Creating a Project Map in Saga

o   Load "World Borders" low resolution polygon shape for basic map

o   Showing and saving map "projects"

o   Exporting shapefiles to KML vector format

·        2.4 Creating Map Frames in Saga

o   Create line shape and polygon shape of map area

o   Graticules removed here; see new exercise 2.40

o   2.4.1 Accuracy Check for AOI Frames - Quick check of student progress

·        2.5 Creating Grid Templates in Saga

o   Create series of grids for the project area, at various "standard" cell-sizes

o   2.5.1 Accuracy Check for Grid Templates

·        2.13 Creating a 0-360˚ Map System in Saga for Pacific-Centered Analyses & Products*

o   Create a project map with 0º to 360º mapping schema

·        2.39 Setting Properties for Multiple Grids/Shapes in Saga

o   Simultaneous setting of display properties

2C Adding Shapes to Maps

·        2.6 Adding GEBCO Depth Contours to a Project Map in Saga

o   Display and save line shapes of depth contours from GEBCO DVD or from GEBCO-Lite

o   Load and display (color coded) contours in project map

o   2.6.1 Accuracy Check for GEBCO Depth Contours

·        2.8 Adding Administrative Boundaries and Coastline to a Project Map in Saga

o   Browse, select and download useful shapefiles for map enhancements

·        2.9 Global Collection of Boundaries and Named Zones Data for Marine GIS:  VLIZ MarineRegions

o   Browse, select and download useful shapefiles of marine "boundaries" for map enhancements

o   Named places and features are now covered by Exercise 2.37, below

·        2.10 Adding a Text-Dominated Spreadsheet to a Project Map in Saga

·        2.11 Adding a Number-Dominated Spreadsheet to a Project Map in Saga

·        2.29 Adding Navigation Charts to Maps in Saga: KAP, BSB, S-57

o   Chart files are located from global vendors or free sources

o   Grid versions are reprojected with special free software and loaded as georeferenced images

o   Vector versions are directly loaded into shape assemblies

·        2.33 Managing Global Coastline, River and Lake Vectors: GSHHG

o   Install the GSHHG global files; then install the GEODAS-NG software

o   Extract shores and rivers (as shapes) for Liberia

o   Note the alternate method for extracting special XY0 text files for use in gridding relief data

·        2.35 Managing & Visualizing MGD77 and XYZ Geologic Survey Data in Saga: NGDC

o   Browse and obtain survey data from a new web interface

o   Download simple ASCII tables for direct management and display in Saga

o   Relations with GEBCO are discussed

·        2.37 Global Gazetteer of Named Marine Places and Features for Marine GIS:  GeoNET Names Server

·        2.40 Adding Graticules to Saga Maps

o   Uses the new top-level Graticules function instead of the older Tool

·        2.41 Adding Global Imagery to Maps in Saga

2D Managing Shapes

·        2.12 Cutting a Shape to the Project Map Extent in Saga

o   Use appropriate frame shape to cut point shape, line shape or polygon shape down to desired size

o   2.12 Accuracy Check for Shape Cutting

·        2.15 Creating a New Shape in Saga from Published Coordinates

o   Load table of coordinates into a module that creates a shapefile

o Includes link to related exercise for body without any published coordinates

·        2.20 Removing "Land Stations" from a Marine Data Collection

o   Merge world borders polygons with point shape of stations to reveal country codes for points, i.e. points on land

·        2.26 Digitizing High-Resolution Borders and Coastlines in Google Earth

o   Browse Google Earth (GE) images down to smallest local image

o   Digitize salient points along land-water margins

o   Save KML files for use with GE or Saga

·        2.27 Reprojecting Vector Data in Saga:  UTM to WGS84

o   If necessary, convert data table to a point shape

o   Specify the current projection of the shape, and save it for security

o   Specify the new projection to apply

o   Save the new shape, appropriately named to indicate re-projection

·        2.28 Reprojecting Grid Data in Saga:  WGS84 to UTM

o   Specify the current projection of the grid, and save it for security

o   Specify the new projection to apply

o   Save the new grid, appropriately re-named to indicate re-projection

2E Adding Grids/Rasters to Maps

·        2.7 Adding a Sediment Thickness Grid to a Project Map in Saga

o   Locate and download global grid

o   Convert grid to -180º to +180º system (with 2.13)

o   Load and display grid on project map

·        2.18 Discontinued US Navy Marine Atlas: Global Wave Grids

o   Examine and find ASCII headers leading raw binary grids

o   Use header information to load grids

o   Use basic examination to find and use blanking and Z-factor information

o   Compare with 2.32, below

·        2.19 Visualizing Climatological Grid Analyses in IDV: THREDDS

o   General methods to find and use typical grids (format irrelevant)

·        2.21 Importing NetCDF Satellite T, S, and Chlorophyll Climatological or Operational Grids into Marine GIS: OceanColor Web

o   Uses data from Exercise 2.36

o   Uses new NetCDF and HDF5 global rasters from OCW

o   All methodology now combined into single metholodology, including descaling

·        2.22 Importing "Scaled" HDF Satellite Surface Temperature Climatologies into Marine GIS: OceanColor Web

o   Deprecated; please refer to 2.21, above

·        2.24 Visualizing High-Density/Small Coastal Area Environmental Survey Data in Saga

o   Load and grid a typical spreadsheet file of station data

o   Use high-resolution shoreline file (made with Exercise 2.26, above) to create a "water polygon"

o   Use water polygon to clip data grid to show only at-sea values

·        2.25 Visualizing a NetCDF Grid with Not-a-Number (NaN) Values in Saga

o   NaN values sometimes occur in NetCDF grids; until Saga is revised to handle them, use this method to recognize and omit them from visualizations

·        2.29 Adding Navigation Charts to Maps in Saga: KAP, BSB, S-57

o   Chart files are located from global vendors or free sources

o   Grid versions are reprojected with special free software and loaded as georeferenced images

o   Vector versions are directly loaded into shape assemblies

·        2.30 Adding UCSD/Scripps Mixed Layer Depths Climatology to Maps in Saga:  Argo

o   Find and load desired products from a wide range of surface layer products

o   Make corrections to the coordinate system

o   Subset and display MDL products for Liberia

·        2.31 Importing Satellite SST and Sea Ice Climatologies into Marine GIS: NCEI LAS

o   Obtain regionally subsetted data for any year-day from a standard Live Access Server

o   NetCDF downloads require careful examination in Saga to select the best display tool

·        2.32 Global Wave Grids from the EU Climatology

o   Explore the data product website and download appropriate data for your purposes

o   Perform necessary edits and conversions on the data for use in Saga

o   Visualize the data in Saga, appropriately displayed, and produce statistical tables

o   Compare to 2.18, above

·        2.34 Visualizing Altimeter and SAR Global Wave Data Grids (L4) in Saga:  GlobWave

o   Obtain and display climatological wave-height analyses

·        2.36 Obtaining Operational & Climatological Satellite Temperature, Salinity, and Chlorophyll Ocean Data: OceanColor Web

o   Identify and download high-quality monthly climatologies for T, S and Chlorophyll, in preparation for other exercises that use these data in GIS

o   Use these data in Exercise 2.21

·        2.38 Obtaining Satellite Chlorophyll-a Imagery from Giovanni

o   Deprecated in favor of 2.21 Importing NetCDF Satellite T, S, and Chlorophyll Climatological or Operational Grids into Marine GIS: OceanColor Web

·        2.41 Adding Global Imagery to Maps in Saga

3 Principal Global Archives of In Situ Ocean Data

·        3.2 Managing/Repatriating Area-Specific Marine Data from the World Ocean Database (WOD)

o   Use the graphic interface to select data by type, time and location

o   Download zipped data for direct use in Ocean Data View

·        3.2 Managing/Repatriating Area-Specific Marine Metadata from the World Ocean Database (WOD)

o   Visit the same website, but use options that give you access to huge archives of cruise information behind the actual data value downloaded above

·        3.3 Managing Synthetic Ocean Station Data from the World Ocean Atlas (WOA)

o   Download "stations" from a special Ocean Data View collection based on average values at static grid points

·        3.4 Managing Standard-Depth Analyzed Parameters from the World Ocean Atlas (WOA)

o   Find and download gridded WOA data

·        3.5 Managing/Repatriating Secchi Disk Depth Data from the Worldwide Ocean Optics Database (WOOD)

o   Specify and download Secchi data from a global compendium

o   Edit the text data for Saga use, then convert them to a point shape

o   Grid the point shape in Saga and display them

·        3.6 Managing Global Biological Records for One Taxon in OBIS

o   Explore OBIS to see what data and products it provides, focusing on one species

o   Download tabular data for a major taxon and learn to grid abundance-type data in Saga

·        3.7 Managing/Repatriating Local Biological Records of Multiple Taxa in OBIS

o   Explore OBIS to see its data and products it provides, focusing on one limited area

o   Download tabular data for one or more taxa found in a limited area

o   Learn how to find various OBIS statistics on taxonomic levels

·        3.8  Visualizing the Global Transmissometer Database Collection in Ocean Data View

·        3.9  Managing PANGAEA Research Data, with Visualization Methods for Ocean Data View

4 Managing & Analyzing Marine Data in Ocean Data View - The ODV software is introduced and its basic methods demonstrated.  Ocean Data View (ODV) is arguably the most widely used ocean station data analysis-and-display system in use today.  The following exercises take the user through the complete process of creating a collection, making various "standard" graphic analyses, some product exports, and some quality control-related work. 

4A Data Collection & Quality Control

·        3.1 Obtaining Area-Specific Marine Data from the World Ocean Database - Necessary to get data for the ODV exercises

·        4.1 Creating ODV Collections from World Ocean Database Data

o   4.1.1 Accuracy Check for ODV Collection

·        4.12 Quality Control of Marine Data Collections with Value Range Checking in ODV

·        4.13 Quality Control of Marine Data Collections with the GLODAP Benchmark Dataset

·        4.19 Editing Quality Flags for Sample, Station and Cruise Data in ODV

4B Special Data Additions

·        4.9 Adding Seabird CTD Data to an ODV Collection

·        4.14 Obtaining NetCDF Operational Marine Data (Argo Profilers/GTSPP/SOOP) from Coriolis

·        4.15 Importing NetCDF Operational Marine Data (Argo Profilers/GTSPP/SOOP) into ODV

·        4.16 Importing ODP Spreadsheet with ISO Date & Time into ODV

4C Standard & Special Analyses

·        4.2 Creating Marine Data Station Plots in ODV

·        4.3 Creating Marine Data Scatter Plots in ODV

·        4.4 Creating Marine Data Section Plots in ODV

·        4.5 Creating Marine Data Surface Plots in ODV

·        4.10 Creating Isotherm-Depth Ocean Front Climatologies in ODV

·        4.18 Visualizing ADCP-Type Multi-Depth Currents in ODV

·        9.2 Visualizing Operational Subsurface Ocean Data in IDV: CORIOLIS

4D Exporting Selected Data/Images

·        4.6 Exporting Marine Data Plot Images from ODV

·        Deprecated

·        4.8 Exporting Marine Data from ODV:  Data Subsets

5. Gridding Marine Data & Handling Grid Products - Gridding marine data from various sources, managing/analyzing grids and creating "standard products" for Marine GIS.  The MDL resources do not offer a rigorous study of gridding, so serious students who want to know more about the subject should consult the literature and the various OceanTeacher articles cited in the exercises below.  The intent here is to provide the big picture of what grids are, how they are made and managed, and how they are used with other marine data.  These basic "mechanics" are often much more valuable to new data managers than the complexities of various gridding algorithms and their applications.

·        5.1 Converting a Data Table to a Point Shape in Saga

o   Load TSV-type table into Saga

o   Export point shape

·        5.2 Gridding Point Shapes in Saga

o   Basic gridding exercise

o   Uses INVERSE DISTANCE WEIGHTED method

·        5.3 Resampling Gridded Data to the Project Map Extent with a "Dummy" Grid in Saga

o   Frequently used method to reduce/resize any grid to a different grid system

o   5.3.1 Accuracy Check for Grid Resampling

·        5.4 Masking Methods for Depth- or Height-Limited Grids in Saga

o   General method to leave blank all grid cells above or below a certain value

·        5.5 Creating "Standard" Grid Products with Saga

o   Making PNG + PGW (or other common graphic formats)

o   Making PNG + KML (or KMZ format if compressed)

o   Making GeoTIFF data raster format

·        5.6 Creating Color Palettes for Standard Product Rasters in Saga

o   General methods to create or work with color palettes for grids

·        5.7 Handling Classified Grids with Saga: GLCF AVHRR Classifications

o   Subsetting classified grids to preserve original information

o   Working with color palettes for classified grids

·        5.8 Creating Legends for Grids in Google Earth

o   Create a legend during the image export process

o   Write KML content to add the legend to KML displays

o   Make white margins around Saga maps invisible by tagging them as transparent

·        5.9 Correcting Faulty FillValue (i.e. Blank) Declarations in NetCDF Grids

o   Identify cases of faulty blank declarations

o   Obtain and install a necessary NetCDF editor

o   Write and run a bat file that runs the NetCDF editor to correct the declaratio

·        5.10 Basic Grid Operations, Calculations and Subsetting in IDV

o   Access methods to create various statistical grids in IDV (e.g. minima, maxima, average)

o   Calculate difference grids that compare any two grids

·        5.11 Converting Between 0-to-360º and -180-to-180º Grids in IDV

o   Move easily between Atlantic-centered and Pacific-centered views, so long as the grids are exactly 360º wide

·        5.12 Deprecated

·        5.13 Using Grid Statistics and Calculations for Ecological Assessments in Saga

o   Assemble a group of grids for a specific area, representing different seasons or conditions

o   Calculate new grids representing, for example, the AVERAGE, MINIMUM or MAXIMUM values (among others)

o   Use these new grids to calculate other products, such a value ranges, etc.

·        5.14 Managing NetCDF, HDF4/5 and GRIB Rasters with Principal Display Programs

o   Use a suite of grids to test loading, viewing and extracting subsets

o   Tests Saga, ncBrowse, Integrated Data Viewer

·        5.16 Converting Between 0-to-360º and -180-to-180º Grids in Saga

·        5.17 Creating a Histogram in Excel for IDV Grids

·        5.18 Convert GRIB Format to NetCDF with ToolsUI from UNIDATA

·        5.19 Convert NetCDF or GRIB grids to Excel with IDV - Under construction

·        5.20 Navigating an Image (to Geolocation) in IDV - Under construction

6. Plotting Motion Vectors - Visualizing winds or currents as vector "arrows" or "streamlines" on maps

·        6.2 Plotting Vector Arrows from U and V Component Grids with Saga - Demonstrates how to create motions vectors (i.e. arrows) from pairs of U and V component grids, the most commonly encountered data representation for wind or currents.  The software can also handle speed and direction grids, if encountered.

·        6.3  Surface Current Vectors from the Global Drifter Program: NetCDF Option - This laborious method (not currently recommended) converts the originally published NetCDF U and V data grids to ASCII, thence to gridding, management and display in Saga.  See 6.5 below.

·        6.4 Using a Speed Grid for Visual Scaling of Vector Arrows - It is sometimes necessary to emphasize the speed of the vector arrows by displaying a background grid color-mapped to speed.  This method requires the Speed (or "Slope") grid created separately in Saga.

·        6.5 Surface Current Vectors from the Global Drifter Program: ASCII Spreadsheets Option - Replaces 6.3 with a greatly streamlined process.

7. Handling Relief Data (Bathymetry & Topography) - Managing and plotting bathymetric and topographic data (grids or vectors).  These exercises are concerned with data describing the height of the land (topography) or the depth of the seafloor (bathymetry), collectively called global relief.  These data can be grids, containing relief values at regular or irregular intervals; they can also be expressed in the form of contour lines (or contour vectors) which trace the locus of points of equal relief value over the surface of the map area.  Mapping relief is extremely problematic, due to the intense public need for very high resolution products, as opposed to the often very poor available datasets.

·        7.1 GEBCO 08 Global Relief Grid: Local Processing of Products

o   Deprecated, in favor of 7.4 GEBCO 08 Global Relief Grid:  Online Processing of Products which is infinitely easier and refers to better data

·        7.2 Creating Bathymetric Contours from Grids: GEBCO 08 Shelf Estimations

o   Use Saga module to contour bathymetry grid for areas not covered by GEBCO

·        7.3 Viewing Relief and Estimating Topographic Contours in Saga: DIVA 30-sec SRTM

o   Download 30-sec grid for Liberia area

o   Use Saga module to contour topography

·        7.4 GEBCO 08 Global Relief Grid:  Online Processing of Products

o   Use online GEBCO function to download Area-of-Interest grid

·        7.5 Merging/ViewingTiled High-Resolution Relief Grids (2-D and 3-D): CGIAR 3-sec SRTM

o   Download 3-sec grids (as tiles) for Liberia area

o   Use Saga module for combining and cutting the grids to the Area-of-Interest

o   Display final grid in 2-D and 3-D

·        7.6 Displaying 3-D Relief Data ("Topography") and Vector Overlays ("Maps") in IDV

o   Use grids as "topography" for display of vectors "maps"

o   Currently topographies must be from online THREDDS catalogs to guarantee success

·        7.7 Displaying Point Cloud, LAS-Format LIDAR Data: Saga & FugroViewer

o   Compares Saga and FugroViewer visualizations to a class GRASS image of the famous Serpent Mounds

8. Network Access Tools and Data Services - Systems and software developed to allow users to access data and data products directly over networks, using new protocols.  During the past 3 years many state-of-the-art systems have been developed to bridge gaps between users and data resources, completely replacing the old download-reformat-display paradigm.  A leader in the development of innovative approaches has been the Open Geospatial Consortium (OGC) which focuses on melding environmental datasets into mapping systems.  Because self-describing formats (HDF, GRIB, NetCDF) are such important formats in these recent developments, general methods for their use are also included here.  And more recently the author has begun adding methods here that access statistical or graphical products from large data assemblies, without any need for further management work.

·        8.5 Data Browsing/Mining in THREDDS Servers

o   Thematic Realtime Environmental Distributed Data Services

·        8.6 Data Browsing/Mining in OPeNDAP & LAS

o   Open-source Project for a Network Data Access Protocol/ Live Access Server

·        8.7 Data Browsing/Mining in SOS Servers

o   Sensor Observations Service

·        8.8 Viewing and Managing Well-Formed NetCDF Grids in Saga and IDV

o   NetCDF

o   GRIB in IDV; possibly also in Saga

·        8.9 Viewing Web Mapping Service (WMS) Operational Product Images in IDV & Google Earth

o   GIF images

·        8.10 Image Browsing/Harvesting in ncWMS Servers: GODIVA2

o   Learn to find, download and visualize the available layers on a pilot ncWMS website

·        8.11 Managing SST Time-Series from Long-Term or Classic Climatologies:  NODC LAS

o   Find and explore a Live Access Server holding enormous archives of SST, winds and ice data from remote sensing missions

o   Learn how to construct time-series graphs from these sources

o   Learn how to capture and use special LAS links that can summon these graphs in other applications, such as maps

·        8.12 Accessing Data Products from NOAA's GEO-IDE Framework

o   Learn how to use NOAA's new master catalog of data sources, with access routes through IDV, LAS and OPENDAP (among others)

9. Our Oceans Today: Operational & Synoptic Data Products - Methods to obtain and visualize operational marine data and products, mainly with Integrated Data Viewer

 

9A General:  Most of the exercises in MDL are concerned with climatological products and/or long-term data archives containing historical records.  Sections 8 and 9, however, concern data delivery systems (8) and operational products (9), including satellite analyses, model simulations, and warning-type products for ice and storms.  Specifically operational formats (NetCDF, GRIB, BUFR, etc.) are emphasized because they are designed for such uses.  The most recent products are preferred, often from "today," but a few systems require multiple days to assemble a useful composite (e.g. salinity, pigments).  Some products are available as both nowcasts and forecasts, so a wide range of "times" is presented.

             

9B Currents

·        9.8 Visualizing Satellite-Measured Sea Surface Height & Geostrophic Current Estimates in IDV: AVISO GDET

o   1/3-degree grids of motion vector components and scalar height

o   Download regionally subsetted NetCDF grids for h, u and v

o   Requires AVISO pre-registration

·        9.9 Deprecated; use 9.10 instead

·        9.10 Visualizing Ocean Model Simulations (Rasters or Vectors) in IDV: HYCOM

o   1/12-degree 2-D and 3-D grids of modeled results (motion vector components and scalars)

o   Differs from the above download method, in that the HYCOM product catalog is loaded into IDV for direct grid selection, download and analysis of products; no files are stored locally unless desired

·        9.13 Visualizing Modeled Meteorological Products in IDV: GFS

o   Direct access via a THREDDS server to 1-degree grids (2-D and 2-D)

o   The Global Forecast System (GFS) products; widely used meteorological simulations, due to known accuracy, ease of access (through a THREDDS catalog and complete OPeNDAP compatibility) and the fact that no fees, licenses or registrations are required.

·        9.23 Visualizing Operational Drifting Buoy Trajectories in IDV: GDP

o   ASCII spreadsheets

·        9.28 Accessing Global Marine Meteorological Data Products

o   Various internationally derived data products (multiple data types)

o   Not always available in IDV-compatible formats

·        9.31 Visualizing Modeled Ocean Scalars & Vectors in IDV:  RTOFS

o   1/12-degree 2-D and 3-D grids of modeled results (motion vector components and scalars [S, T])

·        9.36 Visualization of Remotely Sensed Currents (from Winds and Sea Height) in IDV: OSCAR

o   Level 4 gridded global product based on altimeter sea height and scatterometer surface winds

·        9.38 Visualizing "From" or "Toward" Current and Wind Particle Trajectories in IDV

9C In Situ/At Sea - includes items from other categories

·        9.20 Visualizing Operational Subsurface Ocean Data in ODV & IDV: CORIOLIS

o   Download NetCDF profiles (Argo drifters, Ships-of-Opportunity, Voluntary Observing Ships, etc)

o   Use Ocean Data View as a bridge application to convert NetCDF to ASCII spreadsheets

o   Load spreadsheets into IDV

·        9.23 Visualizing Operational Drifting Buoy Trajectories in IDV: GDP

o   ASCII spreadsheets

·        9.37 Managing/Repackaging Recent Analyses from the Global Network of Fixed Stations: US NDBC

9D Models & Forecasts - includes items from other categories

·        9.3 Visualizing Modeled Wave Analysis Grids in IDV: Wave Watch III

o   Access 1.25-degree grids through THREDDS server

·        9.9 Visualizing Ocean Model Scalars & Vectors in IDV: HYCOM Download - Deprecated; use 9.10 instead

·        9.10 Visualizing Ocean Model Scalars & Vectors in IDV: HYCOM

o   1/12-degree 2-D and 3-D grids of modeled results (motion vector components and scalars)

o   Differs from the above download method, in that the HYCOM product catalog is loaded into IDV for direct grid selection, download and analysis of products; no files are stored locally unless desired

·        9.13 Visualizing Modeled Meteorological Products in IDV: GFS

o   Direct access via a THREDDS server to 1-degree grids (2-D and 2-D)

o   The Global Forecast System (GFS) products; widely used meteorological simulations, due to known accuracy, ease of access (through a THREDDS catalog and complete OPeNDAP compatibility) and the fact that no fees, licenses or registrations are required.

·        9.14 Plotting Atlantic and Eastern Pacific Tropical  Cyclone Paths in IDV: National Hurricane Center (NHC)

o   Download point, line and polygon shapefile (SHP) formats

o   Only useful if an actual hurricane is present in the Atlantic or Pacific

o   The US National Hurricane Center (NHC) publishes current location, expected trajectory, 3-day and 5-day probability cones for tropical cyclones in the western Atlantic and the eastern Pacific

·        9.30 Visualizing Gridded Tropical Cyclone Products in IDV: National Hurricane Center (NHC)

o   Download 10-km GRIB2 files of wind and wave forecasts

o   Requires a new "projection" in IDV for best viewing

·        9.31 Visualizing Ocean Model Scalars & Vectors in IDV:  RTOFS/HYCOM

o   1/12-degree 3-D grids of modeled results (motion vector components and scalars [S, T]

o   Access data through a THREDDS server

·        9.32 Plotting Western Pacific/Indian Ocean Typhoon Paths in Google Earth & IDV: Joint Typhoon Warning Center (JTWC)

o   Shapefiles of past and projected trajectories

o   Satellite images, some of which can be used for map overlays

·        9.33 Image Gallery for Operational Meteorological and Oceanographic Products: NMOC

o   Multiple products at various resolutions, from many different models or observing systems

o   KML-georeferenced images and animations only; no actual data files

·        9.35 Obtaining Tide Forecasts for 7000 Global Locations: UKMO EasyTide

o   Find and use 7-forecasts from the UKMO model website

o   Capture the product URLs into active links on a regional map as an easy interface to the forecasts

·        9.38 Visualizing "From" or "Toward" Current and Wind Particle Trajectories in IDV

o   Use U and V vector grids to map particle trajectories, either from a selected point or area, or toward a target point or area

o   Works with any IDV-compatible time-series of 2D or 3D UV grids

·        9.39 Visualizing Mercator/PISCES Ocean Biogeochemistry Model Products in IDV

o   Browse MyOcean data catalog to find/obtain PISCES grids (NC format)

o   Explore display options in IDV

·        9.40 Extracting Time-Series Plots for Points in 3-D Grids with IDV: Models or Climatologies

o   Use the "data probe" function in IDV to isolate data at a specific point and time in the multi-dimensional grid

o   Display a time-series for the data at that point, using available display options

9E Optics/Pigments/Nutrients

·        9.4 Visualizing Satellite-Measured Optical Properties and Pigments in IDV: ColorWeb

o   Download global PNG images from extensive image catalog

o   Write XIMG auxiliary files for IDV georeferencing

o   Demonstrates general method to load geographic images into IDV

·        9.18 Visualizing Satellite-Measured Optical Properties and Pigments in IDV: Giovanni

o   Download 4-km NetCDF grids (mislabeled as NCD)

o   Giovanni system of "portals" allows easy access to many datasets, but only a portion are marine

o   Subsetting (areal) and additional data product formats (HDF, ASCII, KML) are supported

·        9.24 Visualizing Satellite-Measured Optical Properties and Pigments in IDV: GlobColour

o   FTP download of 4-km (0.042-degree) global grid in NetCDF format

o   Subset/analyze locally with IDV

·        9.21 Visualizing Satellite Images in IDV, Google Earth & SAGA:  MODIS Subsets

o   Part of the US NASA Rapid Response system for hazards and disasters

o   Georeferenced JPG and KMZ images

·        9.25 Visualizing Satellite-Measured Primary Production in IDV: OceanWatch - Deprecated; OceanWatch/CoastWatch no longer provides a global primary production product

·        9.27 Visualizing Satellite-Measured Optical Properties and Pigments in IDV: MODIS-A/THREDDS

o   THREDDS catalog access to 4-km (0.042-degree) global grid

·        9.39 Visualizing Mercator/PISCES Ocean Biogeochemistry Model Products in IDV

o   Browse MyOcean data catalog to find/obtain PISCES grids (NC format)

o   Explore display options in IDV

9F Salinity (Models and Measurements)

·        9.10 Visualizing Ocean Model Simulations (Rasters or Vectors) in IDV: HYCOM

o   1/12-degree 2-D and 3-D grids of modeled results (motion vector components and scalars)

o   Differs from the above download method, in that the HYCOM product catalog is loaded into IDV for direct grid selection, download and analysis of products; no files are stored locally unless desired

·        9.20 Visualizing Operational Subsurface Ocean Data in ODV & IDV: CORIOLIS

o   Download NetCDF profiles (Argo drifters, Ships-of-Opportunity, Voluntary Observing Ships, etc)

o   Use Ocean Data View as a bridge application to convert NetCDF to ASCII spreadsheets

o   Load spreadsheets into IDV

·        9.22 Visualizing Satellite-Measured Sea Surface Salinity in IDV: AQUARIUS/THREDDS

o   Live Access Server access and downloads (NetCDF) of Ver. 3 products

o   THREDDS catalog access to 1-degree global gridf, via IDV

o   Avoids the new NASA line of HDF products for the same data

·        9.26 Visualizing Satellite-Measured Sea Surface Salinity in IDV: MIRAS via CoastWatch

o   Browse and obtain MIRAS surface salinity maps, for management and display in Integrated Data Viewer

o   Subsetting and grid smoothing are described

·        9.31 Visualizing Ocean Model Scalars & Vectors in IDV:  RTOFS/HYCOM

o   1/12-degree 3-D grids of modeled results (motion vector components and scalars [S, T])

o   Access data through a THREDDS server

·        9.43 Visualizing Satellite-Measured Salinity Data in xxx: ESA's SMOS - Incomplete; needs work.

9G Sea Surface Height/Tides

·        9.8 Visualizing Satellite-Measured Sea Surface Height & Geostrophic Current Estimates in IDV: AVISO GDET

o   1/3-degree grids of motion vector components and scalar height

o   Download regionally subsetted NetCDF grids for h, u and v

o   Requires AVISO pre-registration

·        9.35 Obtaining Tide Forecasts for 7000 Global Locations: UKMO EasyTide

o   Find and use 7-day forecasts from the UKMO model website

o   Capture the product URLs into active links on a regional map as an easy interface to the forecasts

9H Sea Ice

·        9.7 Visualizing Satellite-Measured Sea Ice in IDV: Norwegian Meteorological Institute

o   THREDDS access to NetCDF grids

o   Ice concentration, ice edge and ice type analyses

o   Very rough resolution estimate would be 1/10 degree

9I Sea Temperature

·        9.2 Visualizing Historical and NRT Satellite-Measured Surface Temperatures in IDV: GHRSST FTP

o   Completely revised; now covers only the FTP downloads part of GHRSST

o   Refined IDV methods to identify data grids and products

o   9.34 covers MUR products from GHRSST

o   9.42 covers G1SST products from GHRSST

·        9.13 Visualizing Modeled Meteorological Products in IDV: GFS

o   Direct access to SST and winds data via an OPENDAP server to 1-degree grids (2-D and 2-D)

o   The Global Forecast System (GFS) products; widely used meteorological simulations, due to known accuracy, ease of access and the fact that no fees, licenses or registrations are required.

·        9.20 Visualizing Operational Subsurface Ocean Data in ODV & IDV: CORIOLIS

o   Download NetCDF profiles (Argo drifters, Ships-of-Opportunity, Voluntary Observing Ships, etc)

o   Use Ocean Data View as a bridge application to convert NetCDF to ASCII spreadsheets

o   Load spreadsheets into IDV

·        9.31 Visualizing Ocean Model Scalars & Vectors in IDV:  RTOFS/HYCOM

o   1/12-degree 3-D grids of modeled results (motion vector components and scalars [S, T])

o   Access data through a THREDDS server

·        9.33 Image Gallery for Operational Meteorological and Oceanographic Products: NMOC

o   Multiple products at various resolutions, from many different models or observing systems

o   KML-georeferenced images and animations only; no actual data files

·        9.34 Visualizing Satellite-Measured Surface Temperature Fields in IDV:  JPL/MUR

o   Direct OPENDAP connection to archive

o   Daily products, 1 day old at very high resolution

o   Probably the best operational SST product available today

·        9.37 Managing/Repackaging Recent Analyses from the Global Network of Fixed Stations: US NDBC

·        9.42 Visualizing Operational Surface Temperatures "Blended" from Multiple Sources, in IDV: JPL/G1SST

o   Direct OPENDAP connection, easy subsetting

o   Daily products, 1 day old at very high resolution

9J Waves

·        9.3 Visualizing Modeled Wave Analysis Grids in IDV: Wave Watch III

o   Access 1.25-degree grids through THREDDS server

·        9.5 Visualizing Operational Satellite-Measured Wave Analyses (Mapped L2) in IDV: GlobWave

o   Along-track point data in NetCDF Format, downloaded from FTP site

o   Requires registration with the scalar grids source site, but this is easy and expeditious

o   Download multiple passes to get a good overview of global conditions

·        9.16 Visualizing Satellite-Measured Wind Speed & Waves in IDV:  AVISO GDET

o   FTP download of NetCDF grids at 1-degree resolution

o   Typical visualizations in IDV

o   Requires AVISO pre-registration

·        9.33 Image Gallery for Operational Meteorological and Oceanographic Products: NMOC

o   Multiple products at various resolutions, from many different models or observing systems

o   KML-georeferenced images and animations only; no actual data files

9K Winds/Weather

·        9.13 Visualizing Modeled Meteorological Products in IDV: GFS

o   Direct access to SST and winds data via an OPENDAP server to 1-degree grids (2-D and 2-D)

o   The Global Forecast System (GFS) products; widely used meteorological simulations, due to known accuracy, ease of access and the fact that no fees, licenses or registrations are required.

·        9.14 Plotting Atlantic and Eastern Pacific Tropical  Cyclone Paths in IDV: National Hurricane Center (NHC)

o   Download point, line and polygon shapefile (SHP) formats

o   Only useful if an actual hurricane is present in the Atlantic or Pacific

o   The US National Hurricane Center (NHC) publishes current location, expected trajectory, 3-day and 5-day probability cones for tropical cyclones in the western Atlantic and the eastern Pacific

·        9.16 Visualizing Satellite-Measured Wind Speed & Waves in IDV:  AVISO GDET

o   FTP download of NetCDF grids at 1-degree resolution

o   Typical visualizations in IDV

o   Requires AVISO pre-registration

·        9.17 Visualizing Satellite-Measured Winds in IDV:  ASCAT

o   Direct access to 1/4-degree grids via THREDDS server

o   Display near-real-time wind vectors from U, V components

·        9.19 Visualizing Winds and Precipitation from NOGAPS Model in IDV: USGODAE

o   Access 1/2-degree gobal grids

o   Visualize or download (NetCDF) via Live Access Server (LAS)

o   Direct access via THREDDS

·        9.28 Accessing Global Marine Meteorological Data Products

o   Various internationally derived data products (multiple data types)

o   Not always available in IDV-compatible formats

·        9.29 Visualizing Cumulative Recent TRMM Precipitation in IDV: Giovanni

o   Subset and download NetCDF grids (mislabeled NCD) from publisher's GUI

o   Resolution is 1/4-degree, but this is difficult to find

·        9.30 Visualizing Gridded Tropical Cyclone Products in IDV: National Hurricane Center (NHC)

o   Download 10-km GRIB2 files of wind and wave forecasts

o   Requires a new "projection" in IDV for best viewing

·        9.32 Plotting Western Pacific/Indian Ocean Typhoon Paths in Google Earth & IDV: Joint Typhoon Warning Center (JTWC)

o   Shapefiles of past and projected trajectories

o   Satellite images, some of which can be used for map overlays

·        9.33 Image Gallery for Operational Meteorological and Oceanographic Products: NMOC

o   Multiple products at various resolutions, from many different models or observing systems

o   KML-georeferenced images and animations only; no actual data files

·        9.37 Managing/Repackaging Recent Analyses from the Global Network of Fixed Stations: US NDBC

·        9.41 Visualizing Satellite-Measured Winds in IDV:  RapidScat

o   NetCDF swaths (L2) of global wind speeds (10-m)

o   Requires use of 3-D globe to handle 0º-to-360º "problem"

9L General & Ancillary

·        9.1 Integrated Data Viewer (IDV) Preliminary Setup

·        9.12 Converting Rasters and Vectors to Google Earth Images, using IDV

o   Any format that can be displayed in IDV

o   Manipulate the data in IDV, then show your selection and display in Google Earth

·        9.21 Visualizing Satellite Images in IDV, Google Earth & SAGA:  MODIS Subsets

o   Part of the US NASA Rapid Response system for hazards and disasters

o   Georeferenced JPG and KMZ images

·        9.40 Extracting Time-Series Plots for Points in 3-D Grids with IDV: Models or Climatologies

o   Use the "data probe" function in IDV to isolate data at a specific point and time in the multi-dimensional grid

o   Display a time-series for the data at that point, using available display option

MDL EXTRAS

·        Docents - Former students teaching MDL; check here to find potential instructors in your own language

·        Reference - How to cite MDL in other publications

·        Page Template - For authors

The exercises, notes and graphics throughout this website are copyrighted, and may not be copied or abstracted in any way, without explicit permission (in writing).  Making one copy for your personal use is allowed.   Please report any copyright infringement to the publishers (above).