 |
 |
 |
 |
 |
 |
The UAF approach for Data Management
In brief following the UAF approach requires:
- that data files contributed into UAF are generated using UAF-compatible conventions and formats -- most commonly this means using netCDF files that follow the CF conventions or using WMO-standard GRIB files
- that these data files are served over the Internet using the OPeNDAP protocol and THREDDS catalogs.
Instructions on contributing data into UAF are found here. The UAF solutions are mature for .gridded. data, but are still evolving with respect to many classes of observed data. (See discussion of "gridded" data under the Guidance section below.) Following this standards-based strategy promotes interoperability among data providers, data users, and data services in the following ways:
Data DocumentationTo allow users to easily understand and use data it is important to provide comprehensive documentation for the data. The global attribute section of a NetCDF file contains metadata that describes the contents of the file. UAF encourages including rich metadata in NetCDF global attributes. Click here to learn about what documentation to include in UAF datasets.
Data DiscoveryThe process for users to find data of interest to them is made simple by UAF. UAF derives descriptions of the data files (metadata) from their self-describing content. It then automatically places this information into data discovery portals. This allows users to search for and locate data using simple, friendly Web pages. Click here to try UAF data discovery tools.
Data AccessData contributed into UAF become readily available to users through tools of their choosing. The public may visualize the data (make maps and plots); select custom subsets of the data; and download these subsets in various data file formats through several UAF Web sites. UAF-served data are also directly accessible (without the need to download!) through a number of software applications that are popular with research scientists. Click here to learn more about UAF data access tools.
Data Integration and AnalysisData from various sources may be quantitatively intercompared on-line through UAF tools such as Live Access Server and ERDDAP.
Guidance for the creation of NOAA data management plans
As of 2012 the process of submitting a proposal for NOAA funding includes the formulation of a data management plan. If the data to be created by a proposal activity is "gridded", then NOAA's UAF approach provides a data management strategy that will be regarded as a best practice. Obvious examples of gridded data include model outputs, level 3 and 4 satellite products, digital elevation models, and gridded data syntheses, such as climatological averaged fields. One-dimensional sampling structures that are commonly produced by observational programs, such as point time series, vertical profiles, and track lines, may also be regarded as gridded, though the technology to handle such data is still a work in progress. For a definition of "gridded" and UAF supported data types see the GEO-IDE wiki.s Structural Data Types page.
Ensuring archival of important datasets
We need input from Ken and Glenn on what to say about archival, generally? What is the strategy that determines which datasets in UAF deserve archive? Does UAF have an active role in this? Here we discuss only model outputs -- good as far as it goes, but should we say more?
NOAA Model Archives
Model outputs datasets that are of sufficient maturity to deserve long term archival should be considered for submission to the NOAA data centers. Typically such datasets would include those that are cited in per-reviewed science papers or that contribute significantly to the production of products delivered to stake-holders. Archive policies for models are still evolving, but a point of contact for discussing individual datasets is the NOAA NOMADS project. NOMADS can also assist with modelers with participation in UAF under some circumstances.
How to Contribute to UAF
To add your datasets to those already available via the Unified Access Framework please follow these instructions.
The only requirements for adding datasets is that they be available via a THREDDS catalog in Climate and Forecast (CF) convention-compliant NetCDF format.
Instructions for how to set up a THREDDS catalog can be found here.
If you have any questions or difficulty in adding your datasets please contact
Kevin O'Brien, Pacific Marine Environmental Laboratory
Kevin.m.obrien@noaa.gov