Pipeline

End-to-end pipeline for processing IOM evaluation reports

The Report class orchestrates the full evaluation pipeline:

1. Download - Fetch PDFs from IOM evaluation repository
2. OCR - Convert PDFs to markdown with heading hierarchy
3. Curate - Manually review headings and select relevant sections (via curator app)
4. Map - Identify which standardized framework themes are relevant to each report, including the Strategic Results Framework (SRF) (Enablers, Cross-cutting Priorities, and Outputs) and Global Compact for Migration objectives

from cachy import enable_cachy
enable_cachy()

Report class

The Report class wraps an Evaluation and provides methods for the pipeline stages: download → ocr → curate → map

---

source

Report

def Report(
    ev:Evaluation, # The evaluation metadata object
    pdf_url:str=None, # Optional direct URL to PDF
    results_path:str='data/results', # Path to save/load results
):

An evaluation report with full pipeline support

---

source

Report.from_url

def from_url(
    url:str, # URL of the evaluation PDF
    evals:list, # List of [`Evaluation`](https://franckalbinet.github.io/iomeval/readers.html#evaluation) objects to search
    results_path:str='data/results', # Path to save/load results
):

Create a Report by finding an evaluation matching the given URL

---

source

Report.from_title

def from_title(
    title:str, # Title to search for
    evals:list, # List of [`Evaluation`](https://franckalbinet.github.io/iomeval/readers.html#evaluation) objects to search
    results_path:str='data/results', # Path to save/load results
):

Create a Report by finding an evaluation matching the given title

---

source

Report.from_id

def from_id(
    id:str, # id to search for
    evals:list, # List of [`Evaluation`](https://franckalbinet.github.io/iomeval/readers.html#evaluation) objects to search
    results_path:str='data/results', # Path to save/load results
):

Create a Report by finding an evaluation matching the given id

Creating reports

Create a report from a URL:

evals = load_evals('files/test/evaluations.json')

url = "https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/AAP%20Evaluation%20Report_final_.pdf"
report = Report.from_url(url, evals, results_path='files/test/results')
report

Report: Evaluation of IOM Accountability to Affected Populations

Year: 2025 | Organization: IOM
ID: 6c3c2cf3fa479112967612b0baddab72

Pipeline: ✓ Curated (1 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

Or from a title:

title = 'Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa'
report = Report.from_title(title, evals, results_path='files/test/results')
report

Report: Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa

Year: 2023 | Organization: IOM
ID: 49d2fba781b6a7c0d94577479636ee6f

Pipeline: ✓ Curated (5 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

Or from an id:

id = '49d2fba781b6a7c0d94577479636ee6f'
report = Report.from_id(id, evals, results_path='files/test/results')
report

Report: Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa

Year: 2023 | Organization: IOM
ID: 49d2fba781b6a7c0d94577479636ee6f

Pipeline: ✓ Curated (5 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

Note

When creating a report from title or id (rather than URL), eval_url is used to retrieve the main evaluation report from among the documents available for each evaluation (e.g. brief, annexes, management response).

Persistence

Reports automatically save after each pipeline stage. Use load_report to resume from any checkpoint.

---

source

Report.save

def save(
    path:str=None, # Override default results path
)->Report: # Reports self for method chaining

Save report state to JSON

---

source

load_report

def load_report(
    id:str, # Report ID (hash)
    base_path:str='data', # Base directory containing pdf/, md/, results/
)->Report: # The loaded Report

Load a saved Report by id

Resuming from checkpoint

report = load_report('49d2fba781b6a7c0d94577479636ee6f', base_path='files/test')
report

Report: Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa

Year: 2023 | Organization: IOM
ID: 49d2fba781b6a7c0d94577479636ee6f

Pipeline: ✓ OCR | ✓ Curated (5 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

Pipeline Methods

The pipeline stages: download → ocr → curate → map_*

Download

Downloads the evaluation PDF from IOM’s repository.

---

source

Report.download

def download(
    dst:str='data/pdf', # Destination directory for PDFs
    force:bool=False, # Force re-download
)->Report: # Self for chaining

Download evaluation PDF to dst/eval_id/

url = "https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Abridged%20Evaluation%20Report_%20Final_Olta%20NDOJA.pdf"
report = Report.from_url(url, evals, results_path='files/test/results')
_ = report.download(dst='files/test/pdf')

Path('files/test/pdf/49d2fba781b6a7c0d94577479636ee6f').ls()

[Path('files/test/pdf/49d2fba781b6a7c0d94577479636ee6f/Abridged%20Evaluation%20Report_%20Final_Olta%20NDOJA.pdf'), Path('files/test/pdf/49d2fba781b6a7c0d94577479636ee6f/Final%20Evaluation%20Report%20Final_Olta%20NDOJA.pdf'), Path('files/test/pdf/49d2fba781b6a7c0d94577479636ee6f/HoA%20EU%20JI%20Final%20Eval%20-%20Management%20Response%20Matrix%20-%20Final.pdf'), Path('files/test/pdf/49d2fba781b6a7c0d94577479636ee6f/ISP_IOM_Case-Management-Return-Reintegr-JI-Review_final.pdf'), Path('files/test/pdf/49d2fba781b6a7c0d94577479636ee6f/Evaluation%20Learning%20Brief_Final_Olta%20NDOJA.pdf')]

OCR

Runs OCR on the PDF using Mistral’s API and converts to markdown with proper heading hierarchy.

---

source

Report.ocr

def ocr(
    dst:str='data/md', # Destination directory for markdown files
    add_img_desc:bool=True, # Whether to add image descriptions
    force:bool=False, # Force re-OCR
    progress:bool=False, # Show OCR progress messages
    fix_kwargs:dict=None, # Extra kwargs for fix_hdgs
    desc_kwargs:dict=None, # Extra kwargs for add_img_descs
)->Report: # Self for chaining

Run OCR on PDF and fix heading hierarchy

await report.ocr(dst='files/test/md', add_img_desc=False, force=False)

Report: Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa

Year: 2023 | Organization: IOM
ID: 49d2fba781b6a7c0d94577479636ee6f

Pipeline: ✓ OCR | ✓ Curated (5 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

report.md_path

Path('files/test/md/49d2fba781b6a7c0d94577479636ee6f')

report.md_path.ls()[:2]

[Path('files/test/md/49d2fba781b6a7c0d94577479636ee6f/page_21.md'), Path('files/test/md/49d2fba781b6a7c0d94577479636ee6f/page_15.md')]

Curate

Use the curator app (06_curator.ipynb) to review OCR’d headings and select relevant sections. Once complete, the report’s curation_status will be 'sections_selected' and selected_headings will contain the chosen headings.

---

source

Report.get_sections

def get_sections(
    
)->str:

Extract sections on demand from selected headings

Extracting sections

Once curation is complete, use get_sections() to extract the selected content:

report = load_report('49d2fba781b6a7c0d94577479636ee6f', base_path='files/test')
report.selected_headings

['## 1. Introduction ... page 4',
 '## 2. Background of the JI-HoA ... page 5',
 '## 3. Methodology ... page 8',
 '### 4.3. Effectiveness ... page 16',
 '## 5. Conclusions and Recommendations ... page 27']

print(report.get_sections()[:500])

# Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa ... page 1
## 1. Introduction ... page 4

In 2016, the EU and IOM launched the EU-IOM Joint Initiative for Migrant Protection and Reintegration, with as overall objective “To contribute to facilitating orderly, safe, regular and rights-based migration through the facilitation of dignified voluntary return and the implementation of development-focused and sustainable reintegration poli

Thematic mapping

Map extracted sections to IOM’s strategic frameworks (SRF and GCM). Each mapping method can be run independently after curation.

---

source

Report.ensure_sys_blocks

def ensure_sys_blocks(
    
)->None:

Ensure system blocks are available

---

source

map_single

def map_single(
    sys_blocks, # System blocks from mk_system_blocks
    theme_type, # One of: 'enbs', 'ccps', 'gcms', 'outs'
    path:NoneType=None, # Path to theme files
    model:str='claude-haiku-4-5', # Model to use for mapping
    gcm_ids:NoneType=None, # GCM IDs for output mapping
    response_format:ModelMetaclass=ThemeScores, # Pydantic model for structured output
    max_tokens:int=8192, # Max tokens for completion
    temperature:float=0, # Temperature for completion
    reasoning_effort:str=None, # Reasoning effort for completion (low, medium, high)
)->dict: # Mapping results

Map system blocks (Report) to a single theme type using appropriate prompts and formatting

Map enablers

Maps to Strategic Results Framework enablers (organizational capabilities).

---

source

Report.map_enbs

def map_enbs(
    force:bool=False, # Re-run even if already completed
    kwargs:VAR_KEYWORD
)->Report: # Self for chaining

Map report sections to Strategic Results Framework enablers

Here let’s consider we want to resume a curated report and run the mappings:

# Resuming where left
report = load_report('49d2fba781b6a7c0d94577479636ee6f', base_path='files/test')

# Mapping enablers
report.map_enbs(model='claude-haiku-4-5', force=False)

Report: Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa

Year: 2023 | Organization: IOM
ID: 49d2fba781b6a7c0d94577479636ee6f

Pipeline: ✓ OCR | ✓ Curated (5 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

sort_by_relevance(report.mappings['enbs'])[:2]

[{'theme_id': '4',
  'theme_title': 'Data and evidence',
  'relevance_score': 0.87,
  'reasoning': "This report would likely be essential for a synthesis on Enabler 4 (Data and Evidence). The evaluation explicitly examines IOM's data and evidence systems as a primary focus, with data availability and use appearing prominently in the program objectives, evaluation questions, and findings. Specific Objective 1 directly addresses 'Partner countries and relevant stakeholders developed or strengthened evidence-based return and reintegration procedures,' with substantial analysis of IOM's data production, data management systems, and evidence use. Key relevant sections include: the detailed findings on 'Data availability' (4.3.1.1) documenting that 'the JI exceeded the targets set for the number of field studies, surveys and other research conducted' and the 'increased availability of migration data...achieved mainly through the production and publication of migration data and research outputs by the Regional Data Hub'; the discussion of M&E systems (4.3.3.1) noting that 'a total of 36 planning, monitoring, learning, data collection and analysis tools were set up, implemented and/or strengthened'; and the analysis of evidence use in policymaking, with findings that '136 stakeholders reported that data produced has supported evidence-based policies, procedures, and programme design.' The report also discusses the development of the MiMOSA database, the Reintegration Sustainability Index (RSI) and RSS+ methodology, and capacity building on data collection and management. Recommendations 4 and 7 specifically address continuing support for the Regional Data Hub and building on impact evaluation methodologies. For a synthesis specialist, the detailed analysis of IOM's data systems, data production capacity, and evidence-based decision-making would be particularly valuable, especially sections 4.3.1 on data availability and capacity building, and the discussion of M&E systems implementation."},
 {'theme_id': '2',
  'theme_title': 'Partnership',
  'relevance_score': 0.79,
  'reasoning': "This report would likely contribute meaningful evidence to a synthesis on Enabler 2 (Partnership). The evaluation explicitly examines IOM's partnership model and effectiveness across the program, with substantive analysis of partnership development, coordination mechanisms, and sustainability of partnerships with national and local actors. Key relevant sections include: the discussion of the 'Regional MRC strategy with 40 partner organisations' as an important achievement; findings on the increase in 'number of stakeholders (state and non-state) involved in return and reintegration assistance...from 25 (baseline in 2017) to 180 by the end of the project in 2022'; and extensive analysis of partnerships with government entities, NGOs, and service providers. The recommendations explicitly address partnership strengthening, particularly Recommendation 1 on 'enhancing ownership and commitment of stakeholders' and Recommendation 3 on 'building partnerships with service providers who can function without (significant) funding channelled by IOM.' However, partnership is one of several operational themes covered; the report also examines program effectiveness, data systems, and reintegration outcomes with comparable depth. The content on partnership models, coordination challenges, and the analysis of how partnerships functioned across different contexts would add value to a synthesis, particularly the findings section discussing the increase in partner organizations and the recommendations on equitable partnership development."}]

Map CCPs

Maps to SRF Cross-Cutting Priorities.

---

source

Report.map_ccps

def map_ccps(
    force:bool=False, # Re-run even if already completed
    kwargs:VAR_KEYWORD
)->Report: # Self for chaining

Map report sections to Strategic Results Framework cross-cutting priorities

report.map_ccps(model='claude-haiku-4-5', force=False)

Report: Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa

Year: 2023 | Organization: IOM
ID: 49d2fba781b6a7c0d94577479636ee6f

Pipeline: ✓ OCR | ✓ Curated (5 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

sort_by_relevance(report.mappings['ccps'])[:2]

[{'theme_id': '3',
  'theme_title': 'Protection-centred',
  'relevance_score': 0.79,
  'reasoning': "This report would likely contribute meaningful evidence to a synthesis on Cross-cutting Priority 3 (Protection-centred). The evaluation explicitly examines protection-related outcomes and approaches as core components of the program. The program's overarching objective directly references 'migrant protection' and 'rights-based migration,' and the evaluation assesses 'protection and voluntary assisted return' as one of five pillars of action. Multiple findings substantively address protection-centred commitments: Section 4.3.2 ('Safe, humane, dignified voluntary return processes') evaluates IOM's effectiveness in providing protection to stranded migrants, with findings showing 95% of assisted migrants satisfied with travel arrangements and 99.6% reporting safe, well-organized travel. The report documents extensive evidence of 'life-saving benefits for beneficiaries' and notes that returnees 'faced dire situations (including abuse, violence, and exploitation)' that IOM addressed. Section 4.3.3.1 discusses psychosocial support and vulnerability assessments, with evidence that 'continuous contact between IOM and returnees ensured that returnees' needs were identified and monitored from the moment of their identification.' The conclusions emphasize that 'the EU-IOM Joint Initiative has been of crucial importance to address the needs of migrants and returnees facing dire situations.' Recommendations 4 and 6 specifically address continuation of 'direct and specialized assistance in transit' and 'safe, humane, and orderly migration pathways,' directly supporting protection commitments. However, protection-centred approaches are one of several program quality themes evaluated alongside effectiveness, efficiency, and sustainability; the report does not systematically analyze PSEAH measures, child safeguarding, or meaningful participation of affected populations in decision-making as distinct protection-centred themes. For a synthesis on protection-centred programming, specialists should pay particular attention to Section 4.3.2 on AVR processes, the findings on vulnerability identification and psychosocial support, and Recommendations 4 and 6 on continuation of protection-focused activities."},
 {'theme_id': '2',
  'theme_title': 'Equality, Diversity & Inclusion',
  'relevance_score': 0.62,
  'reasoning': "This report contains some relevant content on equality and inclusion, though these are not central evaluation themes. The report explicitly mentions gender mainstreaming efforts: Section 4.3.2.1 notes that 'IOM Ethiopia organised awareness raising on sexual and gender-based violence to migrant returnees and host community members as part of the programme's gender mainstreaming effort within the reintegration process.' The integrated approach framework (Figure 1) addresses 'individual level' initiatives responding to 'specific needs and vulnerabilities of returnees,' which implicitly encompasses attention to diverse populations. The evaluation methodology section indicates that focus group discussions included beneficiaries, suggesting some attention to participation of affected populations. However, the report does not systematically analyze IOM's performance on gender equality, disability inclusion, age-appropriate programming, or meaningful participation of marginalized groups as distinct themes. Gender appears primarily as one aspect of vulnerability assessment and awareness-raising rather than as a cross-cutting institutional commitment. The report lacks dedicated findings on gender-responsive program design, representation of women and persons with disabilities in decision-making, or analysis of how the program addressed unequal outcomes based on sex, gender, age, or disability. A synthesis specialist focused on IOM's equality, diversity, and inclusion implementation would find this report has limited substantive content, though the brief mention of gender mainstreaming and the focus on vulnerable populations could provide supplementary context. Recommend prioritizing reports with more explicit gender and inclusion analysis."}]

Map GCM objectives

Maps to Global Compact for Migration objectives.

---

source

Report.map_gcms

def map_gcms(
    force:bool=False, # Re-run even if already completed
    kwargs:VAR_KEYWORD
)->Report: # Self for chaining

Map report sections to Global Compact for Migration objectives

report.map_gcms(model='claude-haiku-4-5', force=False)

Report: Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa

Year: 2023 | Organization: IOM
ID: 49d2fba781b6a7c0d94577479636ee6f

Pipeline: ✓ OCR | ✓ Curated (5 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

sort_by_relevance(report.mappings['gcms'])[:2]

[{'theme_id': '21',
  'theme_title': 'Cooperate In Facilitating Safe And Dignified Return And Readmission, As Well As Sustainable Reintegration',
  'relevance_score': 0.92,
  'reasoning': "FUNDAMENTAL. This report would be essential evidence for a synthesis on GCM Objective 21. Safe, dignified return and sustainable reintegration are the core themes of the entire evaluation. The programme's overarching objective is 'To contribute to facilitating orderly, safe, regular and rights-based migration through the facilitation of dignified voluntary return and the implementation of development-focused and sustainable reintegration policies and processes' (page 4). The evaluation explicitly examines Specific Objective 2 ('Safe, humane, dignified voluntary return processes are enhanced along main migration routes,' pages 18-20) and Specific Objective 3 ('Returnees are sustainably integrated in host communities,' pages 20-23). Key findings show that the JI supported 9,025 migrants to return voluntarily (exceeding the target of 8,450), with 95% of assisted migrants satisfied with travel arrangements and 99.6% feeling travel was well-organized and safe (page 19). The evaluation provides extensive analysis of reintegration outcomes, including economic, social, and psychosocial dimensions, with findings that 93% of returnees reported sufficient levels of economic self-sufficiency, social stability, and psychosocial wellbeing (page 22). The report discusses the integrated approach to reintegration (individual support, community-based projects, structural-level interventions) and measures of reintegration sustainability using the Reintegration Sustainability Index (RSS+). Recommendations 4 and 5 specifically address continuation of AVR support and expansion of reintegration programming. For a synthesis specialist, the entire report would be valuable, with particular attention to: Specific Objective 2 findings (pages 18-20) on AVR effectiveness and safety; Specific Objective 3 findings (pages 20-23) on reintegration outcomes and sustainability; the discussion of the integrated approach (pages 23-24); and Recommendations 4-5 (pages 29-30) on continuation and expansion of return and reintegration support. This report provides direct evidence on implementation challenges (external factors, economic constraints, sustainability barriers) and effectiveness factors for return and reintegration programming."},
 {'theme_id': '1',
  'theme_title': 'Collect and utilize accurate and disaggregated data as a basis for evidence-based policies',
  'relevance_score': 0.88,
  'reasoning': "FUNDAMENTAL. This report would be essential evidence for a synthesis on GCM Objective 1. Data collection and utilization is a dominant theme throughout the evaluation. The report explicitly examines the Regional Data Hub's work on migration data production, capacity building, and harmonization of methodologies (Actions 1a, 1c, 1f). Key findings show the JI exceeded targets for field studies and surveys (20 vs. 19 target), with substantial analysis of data availability improvements and stakeholder capacity building on data use. The evaluation dedicates Section 4.3.1 entirely to 'Specific Objective 1: Partner countries and relevant stakeholders developed or strengthened evidence-based return and reintegration procedures,' with detailed findings on data availability, capacity of stakeholders, and use of data in policymaking. Notably, 136 stakeholders reported that data produced supported evidence-based policies (exceeding the target of 42). The report also discusses the establishment of 36 M&E tools and strengthening of 29 institutions for data collection. For a synthesis specialist, the sections on 'Data availability' (page 16), 'Capacity of stakeholders' (page 17), and findings on the Regional Data Hub's role would be particularly valuable. The report provides direct evidence on implementation challenges (staff turnover, competing priorities) and effectiveness factors for data-driven migration governance."}]

Map outputs

Maps to SRF outputs. If gcm_ids not provided, uses the top GCM objective from prior mapping.

---

source

Report.map_outs

def map_outs(
    gcm_ids:NoneType=None, # GCM IDs to filter SRF objectives
    force:bool=False, # Re-run even if already completed
    kwargs:VAR_KEYWORD
)->Report: # Self for chaining

Map report sections to Strategic Results Framework outputs

report.map_outs(model='claude-haiku-4-5', force=False)

Report: Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa

Year: 2023 | Organization: IOM
ID: 49d2fba781b6a7c0d94577479636ee6f

Pipeline: ✓ OCR | ✓ Curated (5 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

sort_by_relevance(report.mappings['outs'])[:2]

[{'theme_id': '2b63',
  'theme_title': 'Returning migrants and returning, relocated and locally integrating displaced persons receive reintegration assistance in line with their needs and those of broader community members.',
  'relevance_score': 0.89,
  'reasoning': "This report would likely be essential for a synthesis on Output 2b63 (reintegration assistance for returning migrants and displaced persons). The evaluation explicitly examines IOM's reintegration programming across the Horn of Africa, with reintegration assistance appearing prominently in the program title, evaluation objectives, and throughout the report's structure. The report dedicates substantial analysis to individual and community-based reintegration across three dimensions (economic, social, psychosocial), directly addressing this Output's core deliverables. Key findings in Section 4.3.3 provide extensive evidence on reintegration assistance delivery: 15,161 beneficiaries received reintegration support (exceeding targets), with detailed analysis of satisfaction levels (55% average), economic support effectiveness, and community-based reintegration projects (54 projects supporting 76,348 beneficiaries). The report explicitly examines how assistance aligns with beneficiary needs through focus group discussions and monitoring surveys. Particularly valuable sections include: (1) Section 4.3.3.1 on achievement of outputs and results, detailing individual and community-based reintegration with specific findings on satisfaction drivers and dissatisfaction factors; (2) the integrated approach discussion (Section 4.3.4) explaining how economic, social, and psychosocial dimensions interconnect; (3) conclusions in Section 5.1 emphasizing the importance of individualized support and community-level approaches; and (4) Recommendation 2 on strengthening community-based reintegration efforts. The report provides direct evidence on reintegration assistance effectiveness, implementation challenges, and factors affecting beneficiary outcomes—all critical for understanding this Output's delivery."},
 {'theme_id': '3a43',
  'theme_title': 'Migrants of all genders, ages, abilities and other diversities are able to cover their immediate needs upon return to their countries of origin.',
  'relevance_score': 0.85,
  'reasoning': "This report would likely be essential for a synthesis on Output 3a43 (migrants' ability to cover immediate needs upon return). The evaluation provides extensive evidence on immediate assistance provided to returning migrants, which directly addresses this Output's deliverable. Key findings demonstrate: the JI provided reintegration assistance to 15,161 beneficiaries, with 95% of assisted migrants satisfied with travel arrangements and 99.6% reporting safe, well-organized travel; emergency support on arrival (cash, medical services, mental health and psychosocial support) was deemed 'highly relevant, important, and effective' (Section 5.1); and focus group discussions confirm returnees received crucial economic support, medical assistance, and psychosocial support upon return. The report explicitly examines the adequacy of immediate assistance: returnees noted economic support was 'crucial' as they returned 'with nothing,' helping them start businesses or search for employment. However, the report also identifies a critical gap—many returnees deemed the economic support insufficient to achieve sustainable reintegration, indicating that while immediate needs were addressed, the level of support was often inadequate for longer-term stability. Particularly valuable sections include: (1) Section 4.3.2.1 on assistance to stranded migrants, detailing AVR support and satisfaction metrics; (2) Section 4.3.3.1 on individual reintegration assistance, with findings on the types and adequacy of support provided; (3) Section 5.1 conclusions emphasizing the 'life-saving benefits' of immediate assistance; and (4) Recommendation 5 on extending support scope with focus on longer-term integration. The report provides direct evidence on immediate assistance delivery, beneficiary satisfaction, and gaps in adequacy—all critical for understanding this Output's effectiveness."}]

Map all themes

Convenience method to run all mapping stages in sequence.

---

source

Report.map_all

def map_all(
    kwargs:VAR_KEYWORD
)->Report: # Self for chaining

Run all theme mappings in sequence

Run full pipeline

Run the pipeline on a single evaluation report. The pipeline runs until curation is needed, then continues after curation is complete. Re-run the same call after using the curator app to complete all stages.

---

source

should_force

def should_force(
    force, # Bool to force all steps, or set of step names to force
    step, # Step name to check
)->bool: # Whether to force the step

Check if step should be forced - handles bool or set of step names

---

source

PipelineResult

def PipelineResult(
    report, status, error:NoneType=None, step:NoneType=None
):

Initialize self. See help(type(self)) for accurate signature.

---

source

get_current_step

def get_current_step(
    report
):

Infer which step the report is at based on its state

---

source

run_pipeline

def run_pipeline(
    evals:list, # List of [`Evaluation`](https://franckalbinet.github.io/iomeval/readers.html#evaluation) objects to search
    url:str=None, # URL of the evaluation PDF
    id:str=None, # Evaluation ID
    title:str=None, # Evaluation title
    base_path:str='data', # Base directory (contains pdf/, md/, results/)
    add_img_desc:bool=True, # Whether to add image descriptions during OCR
    fix_kwargs:dict=None, # Extra kwargs for fix_hdgs in OCR
    desc_kwargs:dict=None, # Extra kwargs for add_img_descs in OCR
    force:bool | set=False, # Force re-run: True for all, or set of step names
    delete_pdf:bool=True, # Delete PDF after OCR
    verbose:bool=False, # Print progress messages
    kwargs:VAR_KEYWORD
)->PipelineResult: # Pipeline result with status and report

Run pipeline as far as possible: download → ocr → curate → map_all

evals = load_evals('files/test/evaluations.json')
id = '49d2fba781b6a7c0d94577479636ee6f'
result = await run_pipeline(evals, id=id, base_path='files/test', add_img_desc=False, 
                            force=False, verbose=True, model='claude-haiku-4-5')

__main__ - INFO - Mapping enablers...
__main__ - INFO - Mapping CCPs...
__main__ - INFO - Mapping GCM objectives...
__main__ - INFO - Mapping outputs...
__main__ - INFO - Pipeline complete!

result.status

'completed'

result.report

Report: Final Evaluation of the EU-IOM Joint Initiative for migrant protection and reintegration in the horn of Africa

Year: 2023 | Organization: IOM
ID: 49d2fba781b6a7c0d94577479636ee6f

Pipeline: ✓ OCR | ✓ Curated (5 headings) | ✓ Mappings (enbs, ccps, gcms, outs)
Curation: sections_selected

Report: View Evaluation Report

Batch

The full IOM evaluation repository, downloadable from evaluation-search-pdf, provides a list of documents per evaluation along with their type (Brief, Annex, Final Report, etc.). However, document types and URLs are currently incorrectly mapped — e.g. a URL labelled “Evaluation Report” may actually point to an annex or brief. This is a known bug being fixed by IOM.

As a temporary workaround, we use a secondary API endpoint (/json/api/evaluation) which returns the correct evaluation report URL for each evaluation, albeit with less rich metadata. We match these known-good URLs against each evaluation’s document list to identify the correct report PDF.

evals[0].docs

[{'subtype': 'Evaluation report',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VI%20Case%20Study%20-%20RDH%20East%2C%20Horn%20and%20Southern%20Africa.pdf',
  'desc': 'Evaluation Report'},
 {'subtype': 'Evaluation brief',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VII%20Case%20Study%20-%20RDH%20Asia-Pacific.pdf',
  'desc': 'Evaluation Brief'},
 {'subtype': 'Annexes',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VIII%20-%20Inception%20Report.pdf',
  'desc': 'Annex VI Case Study - RDH East'},
 {'subtype': 'Annexes',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Evaluation%20Brief.pdf',
  'desc': 'Horn and Southern Africa'},
 {'subtype': 'Annexes',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/IOM%20MDS%20Evaluation%20Report%20-%20clean_0.pdf',
  'desc': 'Annex VII Case Study - RDH Asia-Pacific'},
 {'subtype': 'Special related reports/documents',
  'url': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Migration%20Data%20Evaluation%20infographics.pdf',
  'desc': 'Annex VIII - Inception Report'}]

---

source

get_report_urls

def get_report_urls(
    
):

Get title->URL lookup dict from IOM Evaluation API

dict(list(report_urls.items())[:2])

{'WESTERN BALKANS ASSISTED VOLUNTARY RETURN AND REINTEGRATION PROGRAMME PHASE II': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/WBRR%20Phase%20II%20Final%20Evaluation%20Report%20%2827%20June%202025%29%20%281%29.pdf',
 'Co-Funding Mechanism Internal Evaluation': 'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/CoFunding%20Mechanism%20Internal%20Evaluation%20Repor_Elise%20Caroline%20Anais.pdf'}

---

source

get_eval_report_url

def get_eval_report_url(
    r, # Evaluation
    report_urls, # title -> url lut
):

Get report URL from evaluation title lookup

For instance:

url = get_eval_report_url(evals[0], report_urls)
url

'https://evaluation.iom.int/sites/g/files/tmzbdl151/files/docs/resources/Annex%20VIII%20-%20Inception%20Report.pdf'

BatchResult tracks the outcome of batch processing across multiple evaluations, grouping reports by their final status for easy inspection and resumption.

---

source

BatchResult

def BatchResult(
    completed:NoneType=None, # List of successfully completed report IDs
    awaiting_curation:NoneType=None, # List of report IDs awaiting curation
    failed:NoneType=None, # List of (report_id, error) tuples
    skipped:NoneType=None, # List of skipped report IDs (no report URL found)
):

Result of running pipeline on a batch of evaluations

---

source

filter_evals

def filter_evals(
    evals, # List of Evaluation objects
    year:NoneType=None, # Filter by year
    ids:NoneType=None, # Filter by list of IDs
):

Filter evaluations by year or list of IDs

---

source

is_completed

def is_completed(
    report
):

Check if report is completed

batch_run processes multiple evaluation reports through the pipeline, tracking results by status. It filters evaluations by year or specific IDs, skips already-completed reports (unless forced), and saves a timestamped log of results.

---

source

batch_run

def batch_run(
    evals, # List of Evaluation objects to process
    report_urls, # Title->URL lookup dict from IOM API
    base_path:str='../../data', # Base directory (contains pdf/, md/, results/)
    year:str=None, # Filter evaluations by year (e.g. '2023')
    ids:list=None, # Filter evaluations by specific IDs
    force:bool | set=False, # Force re-run: True for all, or set of step names
    stop_after:int=None, # Stop after processing N reports (not yet implemented)
    delete_pdf:bool=False, # Delete PDFs after OCR to save space
    add_img_desc:bool=True, # Whether to add image descriptions during OCR
    fix_kwargs:dict=None, # Extra kwargs for fix_hdgs in OCR
    desc_kwargs:dict=None, # Extra kwargs for add_img_descs in OCR
    kwargs:VAR_KEYWORD
)->BatchResult: # Contains completed, awaiting_curation, failed, skipped lists

Run pipeline on a batch of reports

test_ids = ['4341695461234eee3deb51ac68871109', '6c3c2cf3fa479112967612b0baddab72']

result = await batch_run(evals, report_urls, base_path='files/test', ids=test_ids,
                         add_img_desc=False, verbose=False, model='claude-haiku-4-5')
result

__main__ - INFO - 43416954... awaiting_curation
__main__ - INFO - 6c3c2cf3... skipped (already completed)

<__main__.BatchResult>