Readers

Read IOM evaluation CSV exports into Evaluation records, save them as JSON, and find an evaluation and its report URL

The IOM evaluation repository exports its search results as a CSV file, from evaluation.iom.int/evaluation-search-pdf. The CSV has one row per evaluation. IOMRepoReader turns each row into an Evaluation, with an id, a list of documents and the other columns as metadata. IOMRepoReader.to_json saves the evaluations as JSON, and load_evals reads them back. find_eval looks an evaluation up by title, URL or ID, and eval_url gives the URL of its report PDF.

Core Reader Interface

source

EvalReader

def EvalReader(
    cfg:dict, # Configuration dict with field mappings and processing rules
):

Interface for a reader that turns a repository export into Evaluations

Exported source 
class EvalReader:
    "Interface for a reader that turns a repository export into `Evaluation`s"
    def __init__(self, 
                 cfg:dict # Configuration dict with field mappings and processing rules
                ): store_attr()
    def read(self): raise NotImplementedError
    def tfm(self, df): raise NotImplementedError
    def to_json(self, output_path): raise NotImplementedError
    def __call__(self):
        df = self.read()
        return self.tfm(df)

EvalReader is the interface for repository readers. A reader takes a config dict. Calling it runs read, then tfm. IOMRepoReader is the only implementation.

source

iom_input_cfg

def iom_input_cfg():

Config for reading IOM CSV exports

Exported source 
def iom_input_cfg():
    "Config for reading IOM CSV exports"
    return {
        'date_cols': ['Date of Publication', 'Evaluation Period From Date', 'Evaluation Period To Date'],
        'string_cols': ['Year'],
        'list_fields': {
            'Countries Covered': {'separator': ',', 'clean': True}
        },
        'document_fields': ['Document Subtype', 'File URL', 'File description'],
        'id_gen': {
            'method': 'md5',
            'fields': ['Title', 'Year', 'Project Code']  # fields to hash
        },
        'field_mappings': {
            'Title': 'title',
            'Year': 'year',
            # other mappings
        }
    }

source

Evaluation

def Evaluation(
    id:str, docs:list, meta:dict
)->None:

An evaluation and its documents, displayed as a summary card in notebooks

Exported source 
@dataclass
class Evaluation:
    "An evaluation and its documents, displayed as a summary card in notebooks"
    id:str    # MD5 hash of the evaluation's `Title`, `Year` and `Project Code`
    docs:list # Documents, each a dict with `subtype`, `url` and `desc`
    meta:dict # The other CSV columns, such as `Title`, `Year` and `Countries Covered`
        
    def _repr_markdown_(self):
        title = self.meta.get('Title', 'Untitled')
        year = self.meta.get('Year', 'n/a')
        org = self.meta.get('Evaluation Commissioner', 'Unknown')
        countries = self.meta.get('Countries Covered', [])
        country_str = ', '.join(countries[:3]) if countries else 'Not specified'
        if len(countries) > 3: country_str += f' (+{len(countries)-3} more)'
        
        return f"""
### {title}
**Year:** {year} | **Organization:** {org} | **Countries:** {country_str}

**Documents:** {len(self.docs)} available  
**ID:** `{self.id}`
"""

IOM Reader

source

IOMRepoReader

def IOMRepoReader(
    fname:pathlib.Path, # Path to the CSV export file
):

Read an IOM evaluation CSV export into Evaluations

Exported source 
class IOMRepoReader(EvalReader):
    "Read an IOM evaluation CSV export into `Evaluation`s"
    def __init__(self, 
                 fname:Path # Path to the CSV export file
                 ): 
        cfg = iom_input_cfg()  
        super().__init__(cfg)
        store_attr()

source

IOMRepoReader.read

def read():

Read the CSV export into a DataFrame

Exported source 
@patch
def read(self:IOMRepoReader):
    "Read the CSV export into a DataFrame"
    return pd.read_csv(self.fname)

The test export in files/test is an IOM CSV export. read loads it as it is, one row per evaluation:

#fname = 'files/test/evaluation-search-export-11_13_2025--18_09_44.csv'
fname = 'files/test/evaluation-search-export-01_27_2026--21_43_30.csv'
reader = IOMRepoReader(fname)
reader.read().iloc[0]
Title                                       EVALUATION OF IOM’S MIGRATION DATA STRATEGY
Year                                                                               2025
Author                                                           IOM CENTRAL EVALUATION
Best Practicesor Lessons Learnt                                                     Yes
Date of Publication                                                          2025-08-11
Donor                                                                               IOM
Evaluation Brief                                                                    Yes
Evaluation Commissioner                                                             IOM
Evaluation Coverage                                                              Global
Evaluation Period From Date                                                  2025-08-20
Evaluation Period To Date                                                    2025-05-31
Executive Summary                                                                   Yes
External Version of the Report                                                       No
Languages                                                                       English
Migration Thematic Areas                                 Organisational policy/strategy
Name of Project(s) Being Evaluated                                                  NaN
Number of Pages Excluding annexes                                                  44.0
Other Documents Included                                                            NaN
Project Code                                                                        NaN
Countries Covered                                                             Worldwide
Regions Covered                                                               HQ Geneva
Relevant Crosscutting Themes                                                        NaN
Report Published                                                                    Yes
Terms of Reference                                                                  Yes
Type of Evaluation Scope                                                       Strategy
Type of Evaluation Timing                                                Not applicable
Type of Evaluator                                                              External
Level of Evaluation                                                         Centralized
Document Subtype                      Evaluation report, Evaluation brief, Annexes, ...
File URL                              https://evaluation.iom.int/sites/g/files/tmzbd...
File description                      Evaluation Report, Evaluation Brief, Annex VI ...
Management response                                                                  No
Date added                                                      Thu, 08/07/2025 - 23:52
Name: 0, dtype: object
df = reader.read()
len(df)
740
dstrat = df.iloc[0]
dstrat
Title                                       EVALUATION OF IOM’S MIGRATION DATA STRATEGY
Year                                                                               2025
Author                                                           IOM CENTRAL EVALUATION
Best Practicesor Lessons Learnt                                                     Yes
Date of Publication                                                          2025-08-11
Donor                                                                               IOM
Evaluation Brief                                                                    Yes
Evaluation Commissioner                                                             IOM
Evaluation Coverage                                                              Global
Evaluation Period From Date                                                  2025-08-20
Evaluation Period To Date                                                    2025-05-31
Executive Summary                                                                   Yes
External Version of the Report                                                       No
Languages                                                                       English
Migration Thematic Areas                                 Organisational policy/strategy
Name of Project(s) Being Evaluated                                                  NaN
Number of Pages Excluding annexes                                                  44.0
Other Documents Included                                                            NaN
Project Code                                                                        NaN
Countries Covered                                                             Worldwide
Regions Covered                                                               HQ Geneva
Relevant Crosscutting Themes                                                        NaN
Report Published                                                                    Yes
Terms of Reference                                                                  Yes
Type of Evaluation Scope                                                       Strategy
Type of Evaluation Timing                                                Not applicable
Type of Evaluator                                                              External
Level of Evaluation                                                         Centralized
Document Subtype                      Evaluation report, Evaluation brief, Annexes, ...
File URL                              https://evaluation.iom.int/sites/g/files/tmzbd...
File description                      Evaluation Report, Evaluation Brief, Annex VI ...
Management response                                                                  No
Date added                                                      Thu, 08/07/2025 - 23:52
Name: 0, dtype: object

Three columns describe an evaluation’s documents. Each holds a comma-separated list, with one entry per document:

dstrat['Document Subtype']
'Evaluation report, Evaluation brief, Annexes, Annexes, Annexes, Special related reports/documents'
dstrat['File URL']
'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VI%20Case%20Study%20-%20RDH%20East%2C%20Horn%20and%20Southern%20Africa.pdf,   https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VII%20Case%20Study%20-%20RDH%20Asia-Pacific.pdf,   https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VIII%20-%20Inception%20Report.pdf,   https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Evaluation%20Brief.pdf,   https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/IOM%20MDS%20Evaluation%20Report%20-%20clean_0.pdf,   https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Migration%20Data%20Evaluation%20infographics.pdf'
dstrat['File description']
'Evaluation Report, Evaluation Brief, Annex VI Case Study - RDH East, Horn and Southern Africa, Annex VII Case Study - RDH Asia-Pacific, Annex VIII - Inception Report, Infographics'

The CSV has no ID column. _mk_id hashes the Title, Year and Project Code of a row with MD5. An evaluation gets the same ID in every export, as long as those three fields stay the same:

Exported source 
@patch
def _mk_id(self:IOMRepoReader, 
           row # DataFrame row containing evaluation metadata
          ):
    "MD5 hash of the `id_gen` fields of `row`"
    id_str = ''.join(str(row[f]) for f in self.cfg['id_gen']['fields'])
    return hashlib.md5(id_str.encode('utf-8')).hexdigest()
reader = IOMRepoReader(fname)
df_test = reader.read()
eval_id = reader._mk_id(df_test.iloc[0])
test_eq(len(eval_id), 32)
test_eq(eval_id, '9992310969aa2f428bc8aba29f865cf3')

Document Handling

Exported source 
@patch
def _mk_docs(self:IOMRepoReader, 
             row # DataFrame row with document fields
            ):
    "Pair the document columns of `row` into records"
    stypes = [s.strip() for s in str(row['Document Subtype']).split(', ')]
    urls = [u.strip() for u in str(row['File URL']).split(', ')]
    descs = [d.strip() for d in str(row['File description']).split(', ')]
    return [dict(subtype=st, url=u, desc=d) for st,u,d in zip(stypes,urls,descs) if u.strip()]

An evaluation can have more than one document, such as a report, a brief and annexes. The three document columns list them in the same order. _mk_docs pairs the entries into records with subtype, url and desc keys, and drops entries without a URL. It splits the columns on ,. A subtype, URL or description that contains , shifts the pairing.

reader = IOMRepoReader(fname)
df_test = reader.read()
reader._mk_docs(df_test.iloc[0])
[{'subtype': 'Evaluation report',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VI%20Case%20Study%20-%20RDH%20East%2C%20Horn%20and%20Southern%20Africa.pdf',
  'desc': 'Evaluation Report'},
 {'subtype': 'Evaluation brief',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VII%20Case%20Study%20-%20RDH%20Asia-Pacific.pdf',
  'desc': 'Evaluation Brief'},
 {'subtype': 'Annexes',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VIII%20-%20Inception%20Report.pdf',
  'desc': 'Annex VI Case Study - RDH East'},
 {'subtype': 'Annexes',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Evaluation%20Brief.pdf',
  'desc': 'Horn and Southern Africa'},
 {'subtype': 'Annexes',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/IOM%20MDS%20Evaluation%20Report%20-%20clean_0.pdf',
  'desc': 'Annex VII Case Study - RDH Asia-Pacific'},
 {'subtype': 'Special related reports/documents',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Migration%20Data%20Evaluation%20infographics.pdf',
  'desc': 'Annex VIII - Inception Report'}]

Data Processing

tfm cleans the columns before it builds the Evaluations. _proc_dates stores the date columns as strings, and missing dates stay missing. _proc_lists splits Countries Covered into a list of country names.

Exported source 
@patch
def _proc_dates(self:IOMRepoReader, df):
    "Store the `date_cols` of `df` as strings"
    df[self.cfg['date_cols']] = df[self.cfg['date_cols']].astype(str)
    return df
reader = IOMRepoReader(fname)
df_test = reader.read()
df_proc = reader._proc_dates(df_test)
test_eq(df_proc['Date of Publication'].dtype, 'string')
test_eq(df_proc['Evaluation Period From Date'].dtype, 'string')
test_eq(df_proc['Evaluation Period To Date'].dtype, 'string')
df_proc['Date of Publication'].iloc[0]
'2025-08-11'
Exported source 
@patch
def _proc_lists(self:IOMRepoReader, df):
    "Split each of the `list_fields` of `df` into a list of trimmed, non-empty values"
    for fname,fcfg in self.cfg['list_fields'].items():
        vals = df[fname].fillna('').astype(str).str.split(fcfg['separator'])
        df[fname] = vals.apply(lambda x: [item.strip() for item in x if item.strip()])
    return df
reader = IOMRepoReader(fname)
df_test = reader.read()
df_proc = reader._proc_lists(df_test)
test_eq(type(df_proc['Countries Covered'].iloc[190]), list)
df_proc['Countries Covered'].iloc[190]
['Austria', 'Greece', 'Italy', 'Malta', 'Poland', 'Romania', 'Spain']

source

IOMRepoReader.tfm

def tfm(
    df:pandas.DataFrame
):

Turn the raw DataFrame into a list of Evaluations

Exported source 
@patch
def _to_dict(self:IOMRepoReader, row):
    "Convert row to evaluation dict"
    meta_cols = [col for col in row.index if col not in ['id', 'docs']]
    return dict(id=row['id'], docs=row['docs'], meta={f:row[f] for f in meta_cols})
Exported source 
@patch
def _to_eval(self:IOMRepoReader, row):
    "Convert row to Evaluation object"
    meta_cols = [col for col in row.index if col not in ['id', 'docs']]
    return Evaluation(id=row['id'], docs=row['docs'], meta={f:row[f] for f in meta_cols})
Exported source 
@patch
def tfm(self:IOMRepoReader, df:pd.DataFrame):
    "Turn the raw DataFrame into a list of `Evaluation`s"
    df_proc = self._proc_lists(self._proc_dates(df.copy()))
    df_proc['id'] = df_proc.apply(self._mk_id, axis=1)
    df_proc['docs'] = df_proc.apply(self._mk_docs, axis=1)
    return [self._to_eval(row) for _,row in df_proc.iterrows()]

Calling the reader runs read, then tfm. Each Evaluation displays as a summary card:

reader = IOMRepoReader(fname)
evals = reader()
evals[0]

EVALUATION OF IOM’S MIGRATION DATA STRATEGY

Year: 2025 | Organization: IOM | Countries: Worldwide

Documents: 6 available
ID: 9992310969aa2f428bc8aba29f865cf3

Report URLs

IOM’s UNEG evaluation API also lists each evaluation’s report. get_report_urls maps each evaluation title to the URL of its first document whose name starts with Evaluation report. get_uneg_url looks up an Evaluation in that map by its Title.

source

get_report_urls

def get_report_urls():

Map each evaluation title to its report URL, from IOM’s UNEG evaluation API

report_urls = get_report_urls()
dict(list(report_urls.items())[:2])
{'IAHE Synthesis Report: A Synthesis and Meta-Analysis of Inter-Agency Humanitarian Evaluations (2015–2025)': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/iahe-synthesis-report-1.pdf',
 'WESTERN BALKANS ASSISTED VOLUNTARY RETURN AND REINTEGRATION PROGRAMME PHASE II': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/WBRR%20Phase%20II%20Final%20Evaluation%20Report%20%2827%20June%202025%29%20%281%29.pdf'}

source

get_uneg_url

def get_uneg_url(
    ev, report_urls
):

Get UNEG report URL for an evaluation

For instance:

get_uneg_url(evals[0], report_urls)
'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/IOM%20MDS%20Evaluation%20Report%20-%20clean_0_1.pdf'

source

IOMRepoReader.to_json

def to_json(
    out_path:pathlib.Path, # JSON file to write
):

Save the evaluations as JSON at out_path, with their UNEG report URLs

to_json saves the evaluations as JSON at out_path. When the UNEG API has a report for an evaluation, to_json adds it as the first document, with subtype 'Evaluation report (UNEG)'. eval_url prefers that document.

#reader = IOMRepoReader(fname)
#out_path = Path('files/test/iom_evals_test.json')
#reader.to_json(out_path)
#out_path.exists()

To use the reader:

reader = IOMRepoReader(fname)
evaluations = reader()

Then save them with to_json:

#reader.to_json('files/test/evaluations.json')

Utilities

default_config names the fields that the downloaders read from an Evaluation: id, docs, and each document’s url.

source

load_evals

def load_evals(
    json_file, # JSON file written by `IOMRepoReader.to_json`
)->fastcore.foundation.L: # The evaluations, as `Evaluation`s

Load the evaluations saved in json_file

Exported source 
def load_evals(json_file # JSON file written by `IOMRepoReader.to_json`
              ) -> L:    # The evaluations, as `Evaluation`s
    "Load the evaluations saved in `json_file`"
    return L([Evaluation(**o) for o in json.loads(Path(json_file).read_text())])
fname = 'files/test/evaluations.json'
evals = load_evals(fname)
evals[0]

EVALUATION OF IOM’S MIGRATION DATA STRATEGY

Year: 2025 | Organization: IOM | Countries: Worldwide

Documents: 7 available
ID: 9992310969aa2f428bc8aba29f865cf3

Finding evaluations

source

in_docs

def in_docs(
    ev:__main__.Evaluation, # Evaluation to search
    url:str, # Document URL
)->bool: # Whether any document of `ev` has this URL

Check whether url is one of the documents of ev

Exported source 
def in_docs(
    ev:Evaluation, # Evaluation to search
    url:str        # Document URL
    ) -> bool:     # Whether any document of `ev` has this URL
    "Check whether `url` is one of the documents of `ev`"
    return any(L(ev.docs).filter(lambda x: x['url'] == url))
url = "https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/AAP%20Evaluation%20Report_final_.pdf"
fname = 'files/test/evaluations.json'
evals = load_evals(fname)
ev = first(evals.filter(lambda x: x.id == '6c3c2cf3fa479112967612b0baddab72'))

test_eq(in_docs(ev, url), True)
test_eq(in_docs(ev, "https://fake.url/nothere.pdf"), False)

source

find_eval

def find_eval(
    evals:list, # Evaluations to search
    query:str, # Title, document URL or ID to look for
    by:str='title', # What `query` is: `'title'`, `'url'` or `'id'`
)->__main__.Evaluation: # The first matching evaluation, or `None`

Find an evaluation by title, document URL or ID

Exported source 
def find_eval(
    evals:list,    # Evaluations to search
    query:str,     # Title, document URL or ID to look for
    by:str='title' # What `query` is: `'title'`, `'url'` or `'id'`
    ) -> Evaluation: # The first matching evaluation, or `None`
    "Find an evaluation by title, document URL or ID"
    if by == 'title': return first([o for o in evals if o.meta['Title'] == query])
    if by == 'url': return first([o for o in evals if in_docs(o, query)])
    if by == 'id': return first([o for o in evals if o.id == query])
title = 'Evaluation of IOM Accountability to Affected Populations'
url = "https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/AAP%20Evaluation%20Report_final_.pdf"
test_eq(find_eval(evals, title, by='title').id, '6c3c2cf3fa479112967612b0baddab72')
test_eq(find_eval(evals, url, by='url').id, '6c3c2cf3fa479112967612b0baddab72')
test_eq(find_eval(evals, 'Nonexistent Title', by='title'), None)
test_eq(find_eval(evals, 'https://fake.url/nowhere.pdf', by='url'), None)
test_eq(find_eval(evals, '6c3c2cf3fa479112967612b0baddab72', by='id').meta['Title'], 'Evaluation of IOM Accountability to Affected Populations')
find_eval(evals, title, by='title')

Evaluation of IOM Accountability to Affected Populations

Year: 2025 | Organization: IOM | Countries: Worldwide

Documents: 5 available
ID: 6c3c2cf3fa479112967612b0baddab72

def get_sections(id, data_path='../iomeval/data'):
    "Load report and return extracted sections markdown"
    r = load_report(id, data_path)
    full_md = read_pgs(r.md_path)
    if r.selected_headings: return extract_selected(full_md, r.selected_headings)
    return full_md

source

eval_url

def eval_url(
    ev:__main__.Evaluation, # Evaluation whose report to find
)->str: # Report URL, or `None` when there is no report

URL of the evaluation report of ev, preferring the UNEG report

eval_url returns the UNEG report that to_json added, when there is one. Otherwise it returns the first document whose subtype contains evaluation report. It logs a warning when it finds no report, and when the report URL does not end in .pdf.

eval_url(find_eval(evals, title, by='title'))
'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/AAP%20Evaluation%20Report_final_.pdf'
ev = find_eval(evals, '1700c8dbadc3d87d6911a8ccc6d18b63', by='id')
ev

REVIEW OF THE USE AND FOLLOW-UP OF (EVALUATION) MANAGEMENT RESPONSE IN IOM

Year: 2025 | Organization: IOM | Countries: Worldwide

Documents: 3 available
ID: 1700c8dbadc3d87d6911a8ccc6d18b63

eval_url(ev)
1700c8dbadc3d87d6911a8ccc6d18b63: No evaluation report found