load_data
def load_data()->dict:Fetch Fram Strait CSV records and return one combined SEAWATER DataFrame
Fram Strait 2022-2025
This handler converts five Fram Strait (FS) I-129 seawater CSV records from Zenodo into MARIS-standard NetCDF4 data. The records cover four cruises from 2022 to 2025, including the spring 2022 CIRFA expedition.
The source files contain station, position, collection date, hydrographic metadata, provider sample IDs, and I-129 values with absolute uncertainties. The spring 2022 record also reports I-129 in atoms per kilogram. The handler normalizes the minor differences between the files, reshapes the value and uncertainty columns into one row per measurement, and combines all records into the SEAWATER group.
The handler is structured to accept further compatible Fram Strait CSV records through RECORDS. Each input retains its cruise identifier, while the output uses MARIS identifiers for nuclide, unit, detection status, laboratory, time, position, and sample ID.
Exported source
RECORDS = {
"FS2022_i129": {
"url": "https://zenodo.org/records/20425174/files/FS2022_i129.csv?download=1",
},
"FS2022S_i129": {
"url": "https://zenodo.org/records/20628610/files/FS2022S_data_i129.csv?download=1",
},
"FS2023_i129": {
"url": "https://zenodo.org/records/20448047/files/FS2023_i129.csv?download=1",
},
"FS2024_i129": {
"url": "https://zenodo.org/records/20761705/files/FS2024_i129.csv?download=1",
},
"FS2025_i129": {
"url": "https://zenodo.org/records/20761832/files/FS2025_i129.csv?download=1",
},
}
status = 'Active'dfs = load_data()dfs['SEAWATER'].columnsIndex(['Cruise', 'Station', 'Latitude_degN', 'Longitude_degE', 'Date',
'Niskin', 'Pressure_dbar', 'PracticalSalinity', 'Temperature_degC',
'Sample_ID', 'I129_at_l', 'unc_I129_at_l', 'I129_at_kg',
'unc_I129_at_kg'],
dtype='str')print(dfs['SEAWATER'].describe(include='number').T[['count', 'mean', 'min', 'max']]) count mean min max
Station 639.0 1.976041e+02 1.000000e+00 4.150000e+02
Latitude_degN 639.0 7.887156e+01 7.867350e+01 8.040850e+01
Longitude_degE 639.0 -3.993580e+00 -1.409233e+01 8.005167e+00
Niskin 639.0 9.790297e+00 1.000000e+00 2.400000e+01
Pressure_dbar 639.0 2.019122e+02 1.992000e+00 2.703528e+03
PracticalSalinity 639.0 3.358688e+01 2.850500e+01 3.510550e+01
Temperature_degC 639.0 7.247288e-01 -1.815000e+00 9.699700e+00
Sample_ID 639.0 6.736933e+01 1.000000e+00 1.800000e+02
I129_at_l 639.0 3.170581e+09 1.424944e+08 7.422415e+09
unc_I129_at_l 639.0 7.311502e+07 2.905559e+06 1.887087e+08
I129_at_kg 112.0 3.029187e+09 1.390414e+08 5.795588e+09
unc_I129_at_kg 112.0 5.143359e+07 2.835150e+06 9.653079e+07Column renaming, time parsing, and depth conversion
FS is seawater-only. RenameColsCB maps provider metadata columns to MARIS working names, ParseDateTimeCB converts the collection date to UTC TIME, and AddDepthCB computes sampling depth from the reported Pressure_dbar using the TEOS-10 equation of state via the gsw library.
RenameColsCB
def RenameColsCB(
grps:list=None, # Groups to process; None = all groups in `tfm.dfs`
):Map Fram Strait provider columns to MARIS standard names
# Verify RenameColsCB maps provider columns to MARIS names
dfs_mock = {
"SEAWATER": pd.DataFrame({
"Station": [341],
"Latitude_degN": [78.832167],
"Longitude_degE": [-2.004667],
"PracticalSalinity": [34.9],
"Temperature_degC": [0.0538],
"Sample_ID": [1],
})
}
tfm = Transformer(dfs_mock, cbs=[RenameColsCB()])
tfm()
for col in ["STATION", "LAT", "LON", "SAL", "TEMP", "SMP_ID_PROVIDER"]:
test_eq(col in tfm.dfs["SEAWATER"].columns, True)
print("RenameColsCB: FS2025 columns mapped correctly. ✓")RenameColsCB: FS2025 columns mapped correctly. ✓tfm = Transformer(dfs, cbs=[RenameColsCB()])
tfm()
print(
tfm.dfs["SEAWATER"][
["LAT", "LON", "STATION", "SAL", "TEMP", "SMP_ID_PROVIDER"]
].head(2).to_string()
) LAT LON STATION SAL TEMP SMP_ID_PROVIDER
0 78.921501 0.025167 1 34.9743 2.4978 1
1 78.921501 0.025167 1 34.9631 2.9199 2ParseDateTimeCB
def ParseDateTimeCB(
grps:list=None, # Groups to process; None = all groups in `tfm.dfs`
):Parse FS collection date into a UTC TIME value
# Verify ParseDateTimeCB parses Date into UTC TIME
dfs_mock = {"SEAWATER": pd.DataFrame({"Date": ["2025-07-30"]})}
tfm = Transformer(dfs_mock, cbs=[ParseDateTimeCB()])
tfm()
test_eq("TIME" in tfm.dfs["SEAWATER"].columns, True)
test_eq("Date" not in tfm.dfs["SEAWATER"].columns, True)
print(f"ParseDateTimeCB: TIME = {tfm.dfs['SEAWATER']['TIME'].iloc[0]}. ✓")ParseDateTimeCB: TIME = 2025-07-30 00:00:00+00:00. ✓tfm = Transformer(dfs, cbs=[
RenameColsCB(),
ParseDateTimeCB()
])
tfm()
print(tfm.dfs["SEAWATER"][["TIME"]].head(3).to_string()) TIME
0 2022-09-10 00:00:00+00:00
1 2022-09-10 00:00:00+00:00
2 2022-09-10 00:00:00+00:00tfm.dfs["SEAWATER"].head()| Cruise | STATION | LAT | LON | Niskin | Pressure_dbar | SAL | TEMP | SMP_ID_PROVIDER | I129_at_l | unc_I129_at_l | I129_at_kg | unc_I129_at_kg | TIME | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | FS2022 | 1 | 78.921501 | 0.025167 | 3 | 400.574005 | 34.974300 | 2.4978 | 1 | 1.807664e+09 | 4.503699e+07 | NaN | NaN | 2022-09-10 00:00:00+00:00 |
| 1 | FS2022 | 1 | 78.921501 | 0.025167 | 4 | 250.466995 | 34.963100 | 2.9199 | 2 | 2.280645e+09 | 5.610007e+07 | NaN | NaN | 2022-09-10 00:00:00+00:00 |
| 2 | FS2022 | 1 | 78.921501 | 0.025167 | 5 | 200.367996 | 34.998501 | 3.4750 | 3 | 1.477992e+09 | 3.681163e+07 | NaN | NaN | 2022-09-10 00:00:00+00:00 |
| 3 | FS2022 | 1 | 78.921501 | 0.025167 | 6 | 150.195999 | 35.014400 | 3.9118 | 4 | 1.938010e+09 | 4.743911e+07 | NaN | NaN | 2022-09-10 00:00:00+00:00 |
| 4 | FS2022 | 1 | 78.921501 | 0.025167 | 7 | 100.371002 | 35.028000 | 4.6130 | 5 | 2.091373e+09 | 5.140067e+07 | NaN | NaN | 2022-09-10 00:00:00+00:00 |
ImportantFEEDBACK TO DATA PROVIDER
We use the Thermodynamic Equation Of Seawater - 2010 (TEOS-10) via the gsw Python package (gsw.z_from_p) to convert your reported Pressure_dbar to sampling depth in metres. The conversion uses the reported latitude for the gravitational acceleration correction. Do you confirm this is the correct approach and conversion for these samples? We have rounded depths to one decimal place — please let us know if your convention uses a different precision.
AddDepthCB
def AddDepthCB(
grps:list=None, # Groups to process; None = all groups in `tfm.dfs`
):Compute sampling depth using Thermodynamic Equation of SeaWater 2010 (TEOS-10)
tfm = Transformer(dfs, cbs=[
RenameColsCB(),
ParseDateTimeCB(),
AddDepthCB()
])
tfm()
print(tfm.dfs["SEAWATER"].SMP_DEPTH)0 396.0
1 247.7
2 198.2
3 148.6
4 99.3
...
634 198.3
635 148.6
636 99.1
637 49.6
638 4.9
Name: SMP_DEPTH, Length: 639, dtype: float64Reshaping I-129 results from wide to long format
The Fram Strait files use a wide layout. Each sample occupies one row, while the I-129 results for different units are stored in separate columns, such as I129_at_l and I129_at_kg. Each value column has a corresponding uncertainty column, such as unc_I129_at_l or unc_I129_at_kg.
This layout is common in scientific datasets because it keeps a sample and its associated measurements together. It is convenient for reporting, but it requires an extra transformation before the data can enter the MARIS measurement model. MARIS stores one row per measurement, with the nuclide, unit, value, and uncertainty in separate columns.
MeltI129CB performs this wide-to-long conversion. It keeps the sample and hydrographic metadata as identifier columns, melts the I-129 value columns into VALUE, and melts the matching uncertainty columns into UNC. It then derives the unit from each source column name, assigns the MARIS identifiers for I-129 and the unit, and removes rows where no value was reported.
For example, a source row with both I129_at_l and I129_at_kg becomes two measurement rows. A source row with only one reported value becomes one measurement row. This extra reshape is small, but it makes the ingestion code more involved than a long-format delivery would be.
ImportantFEEDBACK TO DATA PROVIDER
The Fram Strait files report each sample and its associated I-129 results on a single row. This is a common and understandable reporting format. During ingestion, we reshape the value and uncertainty columns into one row per measurement so that the data matches the MARIS structure.
For future deliveries, would it be possible to provide the data in long format, with separate columns for the nuclide, unit, value, and uncertainty? This is not required, since MARIS can process the current format, but it would reduce provider-specific parsing and make the ingestion process more direct.
MeltI129CB
def MeltI129CB(
grps:list=None, # Groups to process; None = all groups in `tfm.dfs`
):Reshape wide I-129 value and uncertainty columns into one row per measurement
dfs_mock = {
"SEAWATER": pd.DataFrame({
"STATION": [101, 102],
"LAT": [78.5, 79.0],
"LON": [1.0, 2.0],
"TIME": pd.to_datetime(["2025-07-30", "2025-07-31"], utc=True),
"I129_at_l": [1.2e9, 2.4e9],
"unc_I129_at_l": [1.0e8, 2.0e8],
"I129_at_kg": [3.4e9, np.nan],
"unc_I129_at_kg": [3.0e8, np.nan],
})
}
tfm = Transformer(dfs_mock, cbs=[MeltI129CB()])
tfm()
out = tfm.dfs["SEAWATER"].sort_values(["STATION", "UNIT"]).reset_index(drop=True)
print(out[["STATION", "UNIT", "VALUE", "UNC", "NUCLIDE"]].to_string(index=False))
test_eq(len(out), 3)
test_eq(out["UNIT"].tolist(), [9, 12, 12])
test_eq(out["NUCLIDE"].tolist(), [28, 28, 28])
test_eq(out["VALUE"].tolist(), [3.4e9, 1.2e9, 2.4e9])
test_eq(out["UNC"].tolist(), [3.0e8, 1.0e8, 2.0e8])
test_eq("I129_at_l" not in out.columns, True)
test_eq("I129_at_kg" not in out.columns, True)
print("MeltI129CB: wide I-129 results reshaped and missing values removed. ✓") STATION UNIT VALUE UNC NUCLIDE
101 9 3400000000.0 300000000.0 28
101 12 1200000000.0 100000000.0 28
102 12 2400000000.0 200000000.0 28
MeltI129CB: wide I-129 results reshaped and missing values removed. ✓The following example shows how MeltI129CB converts two wide I-129 result columns into one row per reported measurement, assigns MARIS unit and nuclide identifiers, and drops missing values.
tfm = Transformer(dfs, cbs=[
RenameColsCB(),
ParseDateTimeCB(),
AddDepthCB(),
MeltI129CB(),
])
tfm()
out = tfm.dfs['SEAWATER']
print(f"Rows: {len(out)}")
print(out.groupby(['Cruise', 'UNIT']).size())
print(out[['Cruise', 'UNIT', 'VALUE', 'UNC']].head(6).to_string())Rows: 751
Cruise UNIT
FS2022 12 104
FS2022S 9 112
12 112
FS2023 12 127
FS2024 12 119
FS2025 12 177
dtype: int64
Cruise UNIT VALUE UNC
0 FS2022 12 1.807664e+09 4.503699e+07
1 FS2022 12 2.280645e+09 5.610007e+07
2 FS2022 12 1.477992e+09 3.681163e+07
3 FS2022 12 1.938010e+09 4.743911e+07
4 FS2022 12 2.091373e+09 5.140067e+07
5 FS2022 12 3.640577e+09 8.820744e+07Detection Limit
The provider does not report a detection level, but MARIS requires this field. The available MARIS categories are:
get_lut('DL'){'Not applicable': -1,
'Not available': 0,
'Detected value': 1,
'Detection limit': 2,
'Not detected': 3,
'Derived': 4}AddDetectionLimitCB
def AddDetectionLimitCB(
grps:list=None, # Groups to process; None = all groups in `tfm.dfs`
):Assign missing Detection Limit column to MARIS ‘Detected value: 1’ category
Analysis laboratory
The MARIS LAB LUT maps the analysing laboratory to ID 345:
labs = get_lut('LAB')
labs['Laboratory of Ion Beam Physics _LIP_, ETZ Zürich, Switzerland']345AddLabCB
def AddLabCB(
grps:list=None, # Groups to process; None = all groups in `tfm.dfs`
):Assign LAB MARIS ID for LIP, ETH Zürich
Cumulative pipeline through the new column callbacks:
tfm = Transformer(dfs, cbs=[
RenameColsCB(),
ParseDateTimeCB(),
AddDepthCB(),
AddNuclideCB(),
AddValueCB(),
AddUnitCB(),
AddUncertCB(),
AddDetectionLimitCB(),
AddLabCB(),
])
tfm()
tfm.dfs["SEAWATER"][["NUCLIDE", "UNIT", "VALUE", "UNC", "LAB", "DL"]].head()--------------------------------------------------------------------------- NameError Traceback (most recent call last) Cell In[94], line 6 2 tfm = Transformer(dfs, cbs=[ 3 RenameColsCB(), 4 ParseDateTimeCB(), 5 AddDepthCB(), ----> 6 AddNuclideCB(), 7 AddValueCB(), 8 AddUnitCB(), 9 AddUncertCB(), NameError: name 'AddNuclideCB' is not defined
Standardise final columns
SanitizeLonLatCB: validates lat/lon ranges and corrects sign conventionEncodeTimeCB: encodesTIMEinto the NetCDF numeric representationAddSampleIDCB: assigns sequentialSMP_ID, preservesSMP_ID_PROVIDER
All three are imported from marisco.callbacks and need no FS-specific configuration.
Cast STATION to string before encoding
FS’s Station column is pure numeric, so pandas infers int64. But STATION maps to a string-typed NetCDF variable, so FormatStationCB casts it to str as the last step before encoding. This callback is defined locally in this notebook only.
FormatStationCB
def FormatStationCB(
grps:list=None, # Groups to process; None = all groups in `tfm.dfs`
):Cast STATION to str for the NetCDF4 string-typed station variable
# Verify FormatStationCB casts STATION to str
dfs_mock = {'SEAWATER': pd.DataFrame({'STATION': [341, 415]})}
tfm = Transformer(dfs_mock, cbs=[FormatStationCB()])
tfm()
out = tfm.dfs['SEAWATER']
test_eq(out['STATION'].tolist(), ['341', '415'])
test_eq(all(isinstance(v, str) for v in out['STATION']), True) # what the encoder's per-element NetCDF write actually needs
print("FormatStationCB: STATION cast to str. ✓")FormatStationCB: STATION cast to str. ✓tfm = Transformer(dfs, cbs=[
RenameColsCB(),
ParseDateTimeCB(),
AddDepthCB(),
MeltI129CB(),
AddDetectionLimitCB(),
AddLabCB(),
SanitizeLonLatCB(),
EncodeTimeCB(),
AddSampleIDCB(col_provider="SMP_ID_PROVIDER"),
FormatStationCB(),
])
tfm()
out = tfm.dfs['SEAWATER']
print(f"Final shape: {out.shape}")
print("Columns:", out.columns.tolist())
print(out[['SMP_ID', 'SMP_ID_PROVIDER', 'NUCLIDE', 'UNIT', 'LAB']].head(4).to_string())Final shape: (751, 18)
Columns: ['Cruise', 'STATION', 'LAT', 'LON', 'Niskin', 'Pressure_dbar', 'SAL', 'TEMP', 'SMP_ID_PROVIDER', 'TIME', 'SMP_DEPTH', 'VALUE', 'UNC', 'UNIT', 'NUCLIDE', 'DL', 'LAB', 'SMP_ID']
SMP_ID SMP_ID_PROVIDER NUCLIDE UNIT LAB
0 1 1 28 12 345
1 2 2 28 12 345
2 3 3 28 12 345
3 4 4 28 12 345print("Final data summary (uppercase columns only):")
upper_cols = [c for c in out.columns if c.isupper()]
print(out[upper_cols].describe().to_string())Final data summary (uppercase columns only):
LAT LON SAL TEMP TIME SMP_DEPTH VALUE UNC UNIT NUCLIDE DL LAB SMP_ID
count 751.000000 751.000000 751.000000 751.000000 7.510000e+02 751.000000 7.510000e+02 7.510000e+02 751.000000 751.0 751.0 751.0 751.000000
mean 78.863648 -4.091302 33.590974 0.610922 1.695834e+09 194.384820 3.149494e+09 6.988157e+07 11.552597 28.0 1.0 345.0 376.000000
std 0.209903 5.363101 1.667206 2.236245 4.102686e+07 323.691563 1.190056e+09 3.027445e+07 1.069375 0.0 0.0 0.0 216.939316
min 78.673500 -14.092334 28.504999 -1.815000 1.650672e+09 2.000000 1.390414e+08 2.835150e+06 9.000000 28.0 1.0 345.0 1.000000
25% 78.832497 -8.006667 31.994850 -1.430950 1.651018e+09 25.300000 2.206530e+09 4.907503e+07 12.000000 28.0 1.0 345.0 188.500000
50% 78.833333 -4.005833 34.415298 0.032500 1.693613e+09 99.600000 3.016349e+09 6.420043e+07 12.000000 28.0 1.0 345.0 376.000000
75% 78.836667 -1.000000 34.908400 2.075250 1.724544e+09 198.600000 4.002510e+09 8.692992e+07 12.000000 28.0 1.0 345.0 563.500000
max 80.408500 8.005167 35.105500 9.699700 1.755043e+09 2658.000000 7.422415e+09 1.887087e+08 12.000000 28.0 1.0 345.0 751.000000NetCDF encoder
The encoder wraps the full pipeline and writes the standardised data to a NetCDF4 file. Global attributes are assembled via GlobAttrsFeeder with BboxCB, DepthRangeCB, TimeRangeCB, plus keywords and processing logs.
The resulting file contains spatial, depth, and time coverage derived from the transformed seawater data, together with FS keywords and the recorded processing steps.
Exported source
FS_KEYWORDS = [
"Fram Strait","Greenland Sea","I-129","radionuclides","seawater","Arctic Ocean",
]
def get_attrs(tfm):
"Retrieve global attributes for Fram Strait"
return GlobAttrsFeeder(tfm.dfs, cbs=[
BboxCB(),
DepthRangeCB(),
TimeRangeCB(),
KeyValuePairCB("keywords", ", ".join(FS_KEYWORDS)),
KeyValuePairCB("publisher_postprocess_logs", ", ".join(tfm.logs)),
])()encode
def encode(
dest:NoneType=None, # Output NetCDF file path
src:NoneType=None, # Unused; Fram Strait 2025 fetches its data from RECORDS
**kwargs
):Fram Strait 2002-2025 Iodine-129 seawater radionuclide data
Exported source
def encode(
dest=None, # Output NetCDF file path
src=None, # Unused; Fram Strait 2025 fetches its data from RECORDS
**kwargs # Additional arguments
):
"Fram Strait 2002-2025 Iodine-129 seawater radionuclide data"
dfs = load_data()
tfm = Transformer(dfs, cbs=[
RenameColsCB(),
ParseDateTimeCB(),
AddDepthCB(),
MeltI129CB(),
AddDetectionLimitCB(),
AddLabCB(),
SanitizeLonLatCB(),
EncodeTimeCB(),
AddSampleIDCB(col_provider="SMP_ID_PROVIDER"),
FormatStationCB()
])
tfm()
encoder = NetCDFEncoder(tfm.dfs, dest_fname=dest,
global_attrs=get_attrs(tfm))
encoder.encode()# Encode to NetCDF
encode(dest="../../_data/output/fram_strait_2022_2025.nc")
print("Fram Strait NetCDF written.")Fram Strait NetCDF written.to_csv("../../_data/output/fram_strait_2022_2025.nc")[Path('../../_data/output/fram_strait_2022_2025_SEAWATER.csv')]df = pd.read_csv("../../_data/output/fram_strait_2022_2025_SEAWATER.csv")
print(f"Shape: {df.shape}")
print(f"Columns: {df.columns.tolist()}")
print(f"\nNuclide IDs: {df.nuclide_id.unique()}")
print(f"Unit IDs: {df.unit_id.unique()}")
print(f"Sample type IDs: {df.samptype_id.unique()}")
print(f"\nStation range: {df.station.min()}–{df.station.max()}")
print(f"Date range: {df.begperiod.min()} to {df.begperiod.max()}")
print(f"Sample depth range: {df.sampdepth.min()} to {df.sampdepth.max()}m")Shape: (751, 15)
Columns: ['detection', 'lab_id', 'latitude', 'longitude', 'nuclide_id', 'salinity', 'sampdepth', 'samplabcode', 'station', 'temperatur', 'begperiod', 'uncertaint', 'unit_id', 'activity', 'samptype_id']
Nuclide IDs: [28]
Unit IDs: [12 9]
Sample type IDs: [1]
Station range: 1–415
Date range: 2022-04-23 to 2025-08-13
Sample depth range: 2.0 to 2658.0mprint(df.head()) detection lab_id latitude longitude nuclide_id salinity sampdepth \
0 = 345 78.9215 0.025167 28 34.9743 396.0
1 = 345 78.9215 0.025167 28 34.9631 247.7
2 = 345 78.9215 0.025167 28 34.9985 198.2
3 = 345 78.9215 0.025167 28 35.0144 148.6
4 = 345 78.9215 0.025167 28 35.0280 99.3
samplabcode station temperatur begperiod uncertaint unit_id \
0 1 1 2.4978 2022-09-10 45036996.0 12
1 2 1 2.9199 2022-09-10 56100070.0 12
2 3 1 3.4750 2022-09-10 36811628.0 12
3 4 1 3.9118 2022-09-10 47439108.0 12
4 5 1 4.6130 2022-09-10 51400670.0 12
activity samptype_id
0 1.807664e+09 1
1 2.280645e+09 1
2 1.477992e+09 1
3 1.938010e+09 1
4 2.091373e+09 1