Core

Load API keys from .env, count tokens, and read the packaged prompts

Importing iomeval.core loads API keys from the .env file in the project root, the folder above the iomeval package. The extract, mapper, pipeline and curator modules import iomeval.core, and importing any of them loads the keys too. Variables already in your environment win over the file. If your environment sets ANTHROPIC_API_KEY or MISTRAL_API_KEY, iomeval.core skips the file entirely, including the other key. Without the file or python-dotenv, it loads nothing.

source

n_tokens

def n_tokens(
    text:str, # Text to count tokens in
    model:str='gpt-4', # OpenAI model whose tiktoken encoding to use
)->int: # Number of tokens

Count the tokens in text with tiktoken

txt = "This is my test text"
n_tokens(txt)
5

Anthropic models use a different tokenizer. Treat n_tokens as an estimate of their token count.

source

load_prompt

def load_prompt(
    name:str, # Prompt file name without `.md`, such as `'srf_ccps'`
    path:pathlib.Path | str | None=None, # Folder of prompt files; `None` uses the prompts shipped with the package
)->str: # Prompt text

Read the prompt name from path

print(load_prompt('srf_ccps')[:500])
### ROLE AND PURPOSE 
You are a triage assistant supporting evaluation synthesis specialists at the International Organization for Migration (IOM). Your role is to help prioritize which evaluation reports warrant in-depth review for each of the IOM Strategic Result Framework (SRF) Cross-cutting Priorities. 

Important context about your role: 
- You are suggesting relevance, not making definitive judgments 
- Human specialists will review your assessments and make final decisions 
- Your scores

The package ships five prompts. identify_core_sections uses select_sections. The framework mappings use srf_enablers, srf_ccps, gcms and srf_outputs. Pass path to use your own versions.