import pandas as pd
from fastcore.test import test_eq
from netCDF4 import Dataset, EnumType
from marisco.configs import NC_CSV, nc_tpl_pathField Definitions
MARIS distributes data as NetCDF4 files, its primary format, and as CSV files for import into the MARIS database. A marisco column is the uppercase name a handler gives a field during curation, such as NUCLIDE. marisco writes each marisco column to its NetCDF variable and its CSV variable.
The summary table is generated from NC_CSV in marisco.configs and from the NetCDF template, which ncgen builds from nbs/files/cdl/maris.cdl. The field descriptions after it give each field’s meaning, lookup table and CSV type. A MARIS CSV field that marisco doesn’t write has no marisco column, so its description gives its CSV variable instead.
Summary table
The table has one row per marisco column. netcdf and csv give its variable name in each format, and — means marisco doesn’t write it in that format. type, long_name, units and groups come from the NetCDF template. An enum type stands for a MARIS lookup table.
tpl maps each template variable to its type, long_name, units and the groups that declare it:
def nc_type(v): return 'string' if v.dtype == str else f'enum `{v.datatype.name}`' if isinstance(v.datatype, EnumType) else str(v.dtype)
tpl = {}
with Dataset(nc_tpl_path()) as nc:
for g in nc.groups.values():
for name, v in g.variables.items():
info = tpl.setdefault(name, dict(type=nc_type(v), long_name=getattr(v, 'long_name', ''), units=getattr(v, 'units', ''), groups=[]))
info['groups'].append(g.name)
for info in tpl.values(): info['groups'] = 'all' if len(info['groups']) == len(nc.groups) else ', '.join(info['groups'])The encoder writes only variables that the template declares. A marisco column whose NetCDF variable is missing from the template would be left out of every file, so every NetCDF name in NC_CSV must be in the template:
test_eq({n for n,_ in NC_CSV.values() if n} - set(tpl), set())pd.DataFrame([dict(column=k, netcdf=n, csv=c, **tpl.get(n, {})) for k,(n,c) in NC_CSV.items()]).set_index('column').fillna('—')| netcdf | csv | type | long_name | units | groups | |
|---|---|---|---|---|---|---|
| column | ||||||
| AREA | area | area | enum `area_t` | Marine area/region id | all | |
| BIO_GROUP | bio_group | — | enum `bio_group_t` | Biota group | biota | |
| BODY_PART | body_part | bodypar_id | enum `body_part_t` | Body part | biota | |
| BOTTOM | bottom | slicedown | float32 | Bottom depth of sediment layer | sediment | |
| COUNT_MET | count_met | counmet_id | enum `count_met_t` | Counting method | all | |
| DL | dl | detection | enum `dl_t` | Detection limit threshold | all | |
| DLV | dlv | detection_lim | float32 | Detection limit value | all | |
| DRYWT | drywt | drywt | float32 | Dry weight of biota sample, expressed in grams. | biota, sediment | |
| FILT | filt | filtered | enum `filt_t` | Filtering method | all | |
| LAB | lab | lab_id | enum `lab_t` | laboratory | all | |
| LAT | lat | latitude | float32 | Measurement latitude | degrees_north | all |
| LON | lon | longitude | float32 | Measurement longitude | degrees_east | all |
| NUCLIDE | nuclide | nuclide_id | enum `nuclide_t` | Nuclide | all | |
| PERCENTWT | percentwt | percentwt | float32 | Dry weight as a percentage of fresh weight. | biota, sediment | |
| PH | ph | — | float32 | pH | all | |
| PREP_MET | prep_met | prepmet_id | enum `prep_met_t` | Preparation method | all | |
| PROFILE_ID | profile_id | profile_id | string | Data provider profile ID | all | |
| REF_ID | — | ref_id | — | — | — | — |
| SAL | sal | salinity | float32 | Salinity | all | |
| SAMP_MET | samp_met | sampmet_id | enum `samp_met_t` | Sampling method | all | |
| SAMPLE_TYPE | — | samptype_id | — | — | — | — |
| SED_TYPE | sed_type | sedtype_id | enum `sed_type_t` | Sediment type | sediment | |
| SMP_DEPTH | smp_depth | sampdepth | float32 | Sample depth below sea level | m | all |
| SMP_ID | id | — | uint64 | Measurement ID | all | |
| SMP_ID_PROVIDER | id_provider | samplabcode | string | Data provider sample ID | all | |
| SPECIES | species | species_id | enum `species_t` | Species | biota | |
| STATION | station | station | string | Station | all | |
| TAXONDB | — | taxondb | — | — | — | — |
| TAXONDBID | — | taxondb_id | — | — | — | — |
| TAXONDBURL | — | taxondb_url | — | — | — | — |
| TAXONNAME | — | taxonname | — | — | — | — |
| TAXONRANK | — | taxonrank | — | — | — | — |
| TAXONREPNAME | — | taxonrepname | — | — | — | — |
| TEMP | temp | temperatur | float32 | Temperature | all | |
| TIME | time | begperiod | uint64 | Time of measurement | seconds since 1970-01-01 00:00:00.0 | all |
| TOP | top | sliceup | float32 | Top depth of sediment layer | sediment | |
| TOT_DEPTH | tot_depth | totdepth | float32 | Total depth below sea level | m | all |
| UNC | unc | uncertaint | float32 | Uncertainty | all | |
| UNIT | unit | unit_id | enum `unit_t` | Unit | all | |
| VALUE | value | activity | float32 | Activity | all | |
| VOL | vol | volume | float32 | Volume | all | |
| WETWT | wetwt | wetwt | float32 | Wet weight of biota sample, expressed in grams. | biota, sediment |
Field descriptions
Sample Quality
Indicates the quality assessment of the sample. Common values: G (Good), C (Caution), F (Fail).
| Lookup table | No |
| CSV variable | sampquality |
| CSV type | string |
Sample Type
Classifies the sample into one of four MARIS groups. In NetCDF4, sample type is encoded as the group name rather than a column variable. In CSV format it is an integer: 1 = SEAWATER, 2 = BIOTA, 3 = SEDIMENT, 4 = SUSPENDED.
| marisco column | SAMPLE_TYPE |
| Lookup table | No |
| CSV type | integer |
Laboratory
Identifies the laboratory that processed and measured the sample.
| marisco column | LAB |
| Lookup table | dbo_lab.xlsx |
| CSV type | integer |
Latitude
Decimal degrees, −90 to 90.
| marisco column | LAT |
| Lookup table | No |
| CSV type | float |
Longitude
Decimal degrees, −180 to 180.
| marisco column | LON |
| Lookup table | No |
| CSV type | float |
Station
Name of the station where the sample was collected, as reported by the provider.
| marisco column | STATION |
| Lookup table | No |
| CSV type | string |
Sample ID
Identifies a row within a sample-type group of a file. marisco generates it as a sequence starting at 1, except for the MARIS legacy dump, which keeps its database sample_id. It isn’t exported to CSV.
| marisco column | SMP_ID |
| Lookup table | No |
| CSV type | — |
Provider Sample ID
The data provider’s sample identifier, stored verbatim. Several measurements of the same sample can share it. It is an empty string when the provider gives none.
| marisco column | SMP_ID_PROVIDER |
| Lookup table | No |
| CSV type | string |
Profile ID
An identifier for linking samples that form part of a vertical sequence (profile). Provided as-is by the data provider.
| marisco column | PROFILE_ID |
| Lookup table | No |
| CSV type | string |
Transect ID
An identifier for linking samples that form part of a horizontal sequence (transect). Provided as-is by the data provider.
| Lookup table | No |
| CSV variable | transect_id |
| CSV type | string |
Sampling Depth
Depth below the water surface at which the sample was taken, in metres. 0 = surface sample; −1 = depth not available.
| marisco column | SMP_DEPTH |
| Lookup table | No |
| CSV type | float |
Total Depth
Total water column depth at the sampling location, in metres.
| marisco column | TOT_DEPTH |
| Lookup table | No |
| CSV type | float |
Begin Period
Date when sample collection began. If only a year is known, the date is set to 1 January of that year; if year and month are known, it is set to the 1st of that month.
| marisco column | TIME |
| Lookup table | No |
| CSV type | datetime string (yyyy-mm-dd hh:mm:ss) |
End Period
Date when sample collection ended. Same date convention as Begin Period.
| Lookup table | No |
| CSV variable | endperiod |
| CSV type | datetime string (yyyy-mm-dd hh:mm:ss) |
Nuclide
Identifies the measured radionuclide via a MARIS standard ID.
| marisco column | NUCLIDE |
| Lookup table | dbo_nuclide.xlsx |
| CSV type | integer |
Detection Limit
Describes the status of the associated Activity value:
1(=) — measured value; an uncertainty should accompany it2(<) — below detection limit; Activity holds the MDA or ISO 11929 detection limit3(ND) — not detected; no activity value and no MDA reported4(DE) — derived; Activity is an aggregation (mean, sum, etc.) of multiple samples
| marisco column | DL |
| Lookup table | dbo_detectlimit.xlsx |
| CSV type | integer |
Activity
The measured activity or MDA for the nuclide. See Detection Limit for interpretation of this value.
| marisco column | VALUE |
| Lookup table | No |
| CSV type | float |
Uncertainty
1σ (k=1) measurement uncertainty, expressed in the same unit as Activity.
| marisco column | UNC |
| Lookup table | No |
| CSV type | float |
Unit
Unit of measurement for Activity and Uncertainty. MARIS stores seawater activities in Bq m⁻³ — values reported by providers in Bq L⁻¹ are converted (×1000).
| marisco column | UNIT |
| Lookup table | dbo_unit.xlsx |
| CSV type | integer |
Variable Type
Type of aggregation applied when Activity is not a single measurement. Common codes: AM (arithmetic mean), GM (geometric mean), MED (median), MAX, MIN.
| Lookup table | No |
| CSV variable | vartype |
| CSV type | string |
Frequency
Regularity of sampling or measurement (e.g. daily, weekly, monthly).
| Lookup table | No |
| CSV variable | freq |
| CSV type | string |
Range Low Detection
Detection limit status of the Range Low value (same codes as Detection Limit). Used when Activity is an aggregated value.
| Lookup table | dbo_detectlimit.xlsx |
| CSV variable | rl_detection |
| CSV type | integer |
Range Low
Minimum activity within an aggregated dataset, expressed in the unit given by Unit.
| Lookup table | No |
| CSV variable | rangelow |
| CSV type | float |
Range Upper
Maximum activity within an aggregated dataset, expressed in the unit given by Unit.
| Lookup table | No |
| CSV variable | rangeupp |
| CSV type | float |
Species (BIOTA)
Identifies the sampled species via a MARIS standard ID. When only a biological group is known (e.g. ‘Fish’), the coarsest matching species entry in the LUT is used.
| marisco column | SPECIES |
| Lookup table | dbo_species.xlsx |
| CSV type | integer |
Biological Group (BIOTA)
Grouping of related species (e.g. crustaceans, molluscs, fish). Always derived from the biogroup_id column in dbo_species.xlsx — never assigned independently.
| marisco column | BIO_GROUP |
| Lookup table | Derived from dbo_species.xlsx |
| CSV type | — |
Taxon Name (BIOTA)
Scientific name of the taxon, as defined in dbo_species.xlsx.
| marisco column | TAXONNAME |
| Lookup table | See dbo_species.xlsx |
| CSV type | string |
Taxon Reported Name (BIOTA)
Taxon name as reported by the data provider (may differ from the accepted scientific name).
| marisco column | TAXONREPNAME |
| Lookup table | No |
| CSV type | string |
Common Name (BIOTA)
Common (vernacular) name of the species or organism.
| Lookup table | No |
| CSV variable | Commonname |
| CSV type | string |
Taxon Rank (BIOTA)
Rank of the taxon in the biological classification system (e.g. species, genus, family), as defined in dbo_species.xlsx.
| marisco column | TAXONRANK |
| Lookup table | See dbo_species.xlsx |
| CSV type | string |
Taxon Database (BIOTA)
Name of the external taxonomy database where this taxon is registered (e.g. WoRMS, ITIS).
| marisco column | TAXONDB |
| Lookup table | See dbo_species.xlsx |
| CSV type | string |
Taxon Database ID (BIOTA)
Identifier for this taxon in the external taxonomy database.
| marisco column | TAXONDBID |
| Lookup table | See dbo_species.xlsx |
| CSV type | string |
Taxon Database URL (BIOTA)
URL of the taxon record in the external taxonomy database.
| marisco column | TAXONDBURL |
| Lookup table | See dbo_species.xlsx |
| CSV type | string |
Body Part (BIOTA)
Identifies the specific body part from which the sample was taken (e.g. flesh, whole animal, liver).
| marisco column | BODY_PART |
| Lookup table | dbo_bodypar.xlsx |
| CSV type | integer |
Slice Up / Top (SEDIMENT)
Top of the sediment core interval relative to the water–sediment interface, in cm.
| marisco column | TOP |
| Lookup table | No |
| CSV type | float |
Slice Down / Bottom (SEDIMENT)
Bottom of the sediment core interval relative to the water–sediment interface, in cm.
| marisco column | BOTTOM |
| Lookup table | No |
| CSV type | float |
Sediment Type (SEDIMENT)
Classification of sediment according to the Udden–Wentworth scale.
| marisco column | SED_TYPE |
| Lookup table | dbo_sedtype.xlsx |
| CSV type | integer |
Sediment Reported Name (SEDIMENT)
Sediment name as reported by the data provider; stored verbatim.
| Lookup table | No |
| CSV variable | SedRepName |
| CSV type | string |
Volume
Volume of the sample, in litres.
| marisco column | VOL |
| Lookup table | No |
| CSV type | float |
Salinity (SEAWATER)
Salinity in practical salinity units (PSU). If the provider reports absolute salinity (g/kg), convert to PSU using TEOS-10 guidelines.
| marisco column | SAL |
| Lookup table | No |
| CSV type | float |
Temperature (SEAWATER)
Water temperature at the time of sampling, in °C.
| marisco column | TEMP |
| Lookup table | No |
| CSV type | float |
Filtered (SEAWATER)
Indicates whether the sample was filtered before measurement. In NetCDF4, stored as an integer: 1 = filtered (dissolved fraction), 2 = unfiltered (total).
| marisco column | FILT |
| Lookup table | No |
| CSV type | string (Y = filtered, N = unfiltered, NA = not applicable) |
Filter Pore Size (SEAWATER)
Pore size of the filter used, in µm.
| Lookup table | No |
| CSV variable | filtpore |
| CSV type | float |
Acidified
Indicates whether the sample was acidified prior to analysis.
| Lookup table | No |
| CSV variable | acid |
| CSV type | string (A = acidified, NA = not acidified) |
Oxygen (SEAWATER)
Dissolved oxygen concentration of the sample.
| Lookup table | No |
| CSV variable | oxygen |
| CSV type | float |
Sample Area (SEDIMENT)
Surface area of the sediment sample, in cm².
| Lookup table | No |
| CSV variable | samparea |
| CSV type | float |
Dry Weight (BIOTA)
Dry weight of the sample, in grams.
| marisco column | DRYWT |
| Lookup table | No |
| CSV type | float |
Wet Weight (BIOTA)
Wet weight of the sample, in grams.
| marisco column | WETWT |
| Lookup table | No |
| CSV type | float |
Percent Weight (BIOTA)
Dry-to-wet weight ratio, expressed as a percentage: (dry weight / wet weight) × 100. Value must be between 0 and 100.
| marisco column | PERCENTWT |
| Lookup table | No |
| CSV type | float |
Sampling Method
Method used to collect the sample.
| marisco column | SAMP_MET |
| Lookup table | dbo_sampmet.xlsx |
| CSV type | integer |
Drying Method
Method used to dry the sample prior to analysis.
| Lookup table | dbo_sampmet.xlsx |
| CSV variable | drymet_id |
| CSV type | integer |
Preparation Method
Method used to prepare the sample for radioactivity measurement.
| marisco column | PREP_MET |
| Lookup table | dbo_prepmet.xlsx |
| CSV type | integer |
Counting Method
Analytical method used to measure radioactivity (e.g. gamma spectrometry, alpha spectrometry, LSC).
| marisco column | COUNT_MET |
| Lookup table | dbo_counmet.xlsx |
| CSV type | integer |
Reference
Identifies the source dataset or publication from which this record originates.
| marisco column | REF_ID |
| Lookup table | dbo_ref.xlsx (internal; not publicly distributed) |
| CSV type | integer |
Reference Note
Additional notes about the source reference.
| Lookup table | No |
| CSV variable | refnote |
| CSV type | string |
Sample Note
Additional notes about the sample.
| Lookup table | No |
| CSV variable | sampnote |
| CSV type | string |
Measurement Note
Additional notes about the measurement process.
| Lookup table | No |
| CSV variable | measurenote |
| CSV type | string |
Good for Export
Flag indicating whether the record is suitable for export from the MARIS database.
| Lookup table | No |
| CSV variable | gfe |
| CSV type | string |
Area
Named geographic region where the sample was collected.
| marisco column | AREA |
| Lookup table | dbo_area.xlsx |
| CSV type | integer |