The services offered by DOKIPY differs between the nodes. All nodes (IMR, NPI, METNO) handles metadata and makes these searchable. IMR and METNO do also handle the actual datasets. Currently only METNO offers online access to the datasets. Access to the datasets is controlled by the associated metadata. Some datasets are freely available, while others are protected by username and password for a period. Metadata are synchronised between the DOKIPY nodes facilitating searching through all Norwegian IPY datasets at any of the nodes.

Metadata can be inserted using forms at NPI and METNO nodes. Handling of the datasets differ between the nodes at IMR and METNO. METNO has automated the upload of datasets and monitors the upload process. This require the datasets to be uploaded using standardised file formats (NetCDF/CF). Conformance of files can be tested interactively at the web portal (http://dokipy.met.no/submission.html). Direct upload of files can be done at http://dokipy.met.no/r2/upl/index.php. Contact IMR using the contact details of the node list to receive information on how to upload files to the IMR node.

Back to the top

What is metadata?

Metadata are data about the data. Metadata describes who measured what, where and when. Many Many different standards for metadata exist, most data centres use one of the international standards. IPY has chosen Global Change Master Directory (GCMD) Directory Interchange Format (DIF). This is compatible with ISO19115 which INSPIRE require. All metadata for Norwegian IPY datasets are also available in Dublin Core.

More information on metadata are available at:

Back to the top

Why do we need metadata?

Funding agencies, projects and scientists all need an overview of who measured what, where and when. This information increase the benefit of observation campaigns, promotes interdisciplinary science and avoids duplication of efforts. It is also required for data management purposes, especially when concerned with archiving datasets. The intention of IPY data management procedures is to ensure that the data collected during this IPY are available to the scientists of the next IPY. Metadata ensures that IPY will be more than individual projects, it connects projects and scientists across projects, disciplines and nations.

Back to the top

What is an archive?

A data archive is a data centre with a long term mandata for data management. A data archive ensures that datasets can be tracked and used by future scientists ensuring proper credit is attributed to the scientists actually generating the dataset. Data archives have procedures and systems that ensures secure management of datasets independent of technological and personnel changes.

Back to the top

What is DOKIPY offering me?

DOKIPY ensures that datasets entered into the DOKIPY system are handled in accordance with the IPY data policy and its requirements.

Back to the top

How can I deliver a dataset?

Data delivery can be done in two different ways. If you only have a small dataset, you can upload it interactively. If you have a dataset that is regularly updated (e.g. every month, day, week, ...) a dedicated delivery interface can be set up for you. In both situations, only registered users can upload data to the data management system. Furthermore, this system is heavily automated and require preparation of datasets before upload.

Datasets delivered has to be formatted following the NetCDF/CF convention and to contain the global attributes described in the submission page. Files can be uploaded as:

NetCDF: NetCDF binary format containing the file suffix .nc.
CDL: ASCII representation of NetCDF files (network Common Data form Language) containing the file suffix .cdl.
Tape archive: Several CDL or NetCDF files collected in a tape archive file which can be gzipped or not. All files are considered to be part of the same dataset. To be handled as a single dataset, the title and abstract has to be common over all files in the dataset. Use product_name etc to separate products within a dataset.

Prior to uploading a dataset we recommend that a test file is generated and tested using the test functionality found in the upload interface. This use the same conformance testing backend that the upload interface is using, but data is not stored permanently. This conformance test will send you an email containing the link to the error report. Some of the errors are major and influence the visibility of the dataset within the search interface, while others are minor. Please ensure that no errors are found in global attributes.

Back to the top

How can I download a full dataset?

While waiting for the updated version of the METAMOD portal solution which will include a shopping basket feature, wget may be used directly towards the THREDDS Data Server.

A statement like the following: wget -nc -r -l2 -A.nc -I /thredds/fileServer/,/thredds/catalog/ 'URL'
will fetch all files ending with .nc (-A.nc) recursively (-r) up to to links away from that page (-l2). The -nc (no-clobber) option makes sure, existing files are not fetched twice in case of an abort and restart of wget.

The URL to specify is the address of the file-listing in thredds (catalog.html is optional). This will download all the .nc files, plus the catalog-files (which are not needed, but are small). The URL can be copied from the search results found.

E.g. to download all ITP2 products from this DOKIPY node use: wget -nc -r -l2 -A.nc -I /thredds/fileServer/,/thredds/catalog/ 'http://thredds.met.no/thredds/catalog/data/met.no/itp02/catalog.html'

Back to the top