MARISCO
The IAEA Marine Radioactivity Information System (MARIS) provides open access to radioactivity measurements in marine environments. Developed by the IAEA Environmental Laboratories in Monaco, MARIS offers data on seawater, biota, sediment, and suspended matter.
This Python package includes command-line tools to convert MARIS datasets into NetCDF
or .csv
formats, enhancing compatibility with various scientific and data analysis software.
Core Concept: Handlers
marisco
is built around the concept of handlers
- specialized modules designed to convert MARIS datasets into NetCDF format. Each handler is tailored to a specific data provider and implemented as a dedicated Jupyter notebook.
Literate Programming Approach
We’ve adopted a Literate Programming approach, which means:
- Documentation: Each handler serves as comprehensive documentation.
- Code Reference: The notebooks contain the actual implementation code.
- Communication Tool: They facilitate discussions with data providers about discrepancies or inconsistencies.
Powered by nbdev
To achieve this, we leverage nbdev, a powerful tool that allows us to:
- Write code within Jupyter notebooks
- Automatically export relevant parts as dedicated Python modules
This approach bridges the gap between documentation and implementation, ensuring they remain in sync.
See It in Action
For a concrete example of this approach, check out our HELCOM dataset handler implementation.
Please note that this project is still under development.
We have implemented the MARIS Legacy handler to convert all existing datasets from the MARIS master database into NetCDF format. For datasets that are frequently updated, such as HELCOM, OSPAR, and TEPCO/Fukushima-related datasets, individual handlers are currently being developed and will be available soon.
Install
Now, to install marisco
simply run
pip install marisco
Once successfully installed, run the following command:
maris_init
This command:
- creates a
.marisco/
directory containing various configuration/configurable files ((below)) in your/home
directory - creates a
configs.toml
file containing default but configurable settings (default paths, …) - creates a configurable
cdl.toml
file used to generate a MARIS NetCDF4 CDL (Common Data Language) template; - downloads several MARIS DB nomenclature/lookup table into
.marisco/lut/
directory - generate
maris-template.nc
, the MARIS NetCDF4 template generated fromcdl.toml
and use to encode MARIS datasets
For inexperienced Python users, please refers to How to setup Marisco
with Anaconda or How to setup Marisco
with Windows Subsystem for Linux (WSL) and Visual Studio Code editor.
Zotero API key
Upon conversion, marisco
will automatically retrieve the bibliographic metadata of each MARIS dataset from Zotero. To do so, you need to define the following environment variable ZOTERO_API_KEY
containing the MARIS Zotero API key. Please contact the MARIS team to get your API key.
Getting started
Command line utilities
All commands accept a -h
argument to get access to its documentation.
maris_init
Create configuration files, MARIS NetCDF CDL (Common Data Language) and donwload required lookup tables (nomenclatures).
maris_create_nc_template
Generate MARIS NetCDF template to be used when encoding datasets
maris_netcdfy
Encode MARIS dataset as NetCDF
Positional arguments:
handler_name
: Handler’s name (e.g helcom, …)str
: Path to dataset to encodedest
: Path to converted NetCDF4
Example:
maris_netcdfy helcom _data/accdb/mors/csv _data/output/helcom.nc
Development
FAQ
How is cdl.toml
created & what it is used for?
A Python dictionary named CONFIGS_CDL
specifying MARIS NetCDF attributes, variables, dimensions, … is defined in nbs/api/configs.ipynb
in the first instance. Running the command maris_init
will generate a toml
version of it named .marisco/cdl.toml
further used to create a MARIS NetCDF template named in .marisco/maris-template.nc
.
Once marisco
installed, further customization of the MARIS NetCDF template can be done directly through .marisco/cdl.toml
file then running the command maris_create_nc_template
.