How trufl ranks areas

Criteria, weights and scoring methods behind each round’s ranking

trufl ranks areas by sampling priority with multiple-criteria decision-making (MCDM). It scores each area against criteria and weights that you choose, then ranks the areas by score. Rank 1 is the highest sampling priority. Why trufl explains why trufl takes this approach.

The examples on this page use the six criteria and weights of Round II in Getting started, on a state table of four areas small enough to read in full.

import pandas as pd
from fastcore.test import test_eq
from trufl.optimizer import Optimizer
from trufl.sampler import rank_to_sample

Criteria

Each column of the state that a State returns is one criterion. You choose the columns through the callbacks you pass to State. Round II uses these six:

Criterion	Callback	Type
Maximum sample value	MaxCB()	Benefit
Minimum sample value	MinCB()	Benefit
Standard deviation	StdCB()	Benefit
Sample count	CountCB()	Cost
Moran’s I	MoranICB(k=5)	Cost
Prior estimate	PriorCB()	Benefit

A high value of a benefit criterion raises the priority of an area. A low value of a cost criterion raises it. In Round II, an area with few samples therefore ranks higher than a well-sampled one. An area whose values show no spatial trend, with a Moran’s I close to 0, ranks higher than one with a clear gradient, with a Moran’s I close to 1.

The state can hold any other quantitative variable as a criterion, such as population or whether an area has been remediated.

Here is a state for four areas. Areas 0 and 1 have high values, but area 0 has two samples and area 1 has twelve. Areas 2 and 3 have low values, with three and ten samples:

state = pd.DataFrame({'Max': [9.0, 9.5, 3.0, 2.5], 'Min': [2.0, 1.5, 0.5, 1.0],
                      'Standard Deviation': [2.5, 2.0, 0.8, 0.3], 'Count': [2, 12, 3, 10],
                      'Moran.I': [0.1, 0.6, 0.2, 0.7], 'Prior': [6.0, 7.0, 2.0, 1.5]},
                     index=pd.Index(range(4), name='loc_id'))
is_benefit = [True, True, True, False, False, True]
state

	Max	Min	Standard Deviation	Count	Moran.I	Prior
loc_id
0	9.0	2.0	2.5	2	0.1	6.0
1	9.5	1.5	2.0	12	0.6	7.0
2	3.0	0.5	0.8	3	0.2	2.0
3	2.5	1.0	0.3	10	0.7	1.5

Weights

The weight vector sets the importance of each criterion relative to the others. The weights must sum to 1. If you pass w_vector=None, get_rank computes the weights from the data with w_method instead. mcdm describes the weighting methods.

With the Round II weights, area 0 ranks first. Area 1 has similar values but many more samples, and ranks second:

optimizer = Optimizer(state)
ranks = optimizer.get_rank(is_benefit_x=is_benefit, w_vector=[0.2, 0.1, 0.1, 0.2, 0.2, 0.2],
                           n_method='LINEAR1', s_method='CP')
test_eq(ranks['rank'].to_dict(), {0: 1, 1: 2, 2: 3, 3: 4})
ranks

	rank
loc_id
0	1
1	2
2	3
3	4

Raise the weight of the sample count to 0.6 and the areas with few samples move up. Area 2, with low values but only three samples, now ranks above area 1:

count_heavy = [0.1, 0.05, 0.05, 0.6, 0.1, 0.1]
ranks = optimizer.get_rank(is_benefit_x=is_benefit, w_vector=count_heavy, n_method='LINEAR1', s_method='CP')
test_eq(ranks['rank'].to_dict(), {0: 1, 2: 2, 3: 3, 1: 4})
ranks

	rank
loc_id
0	1
2	2
3	3
1	4

Normalization

The criteria have different units and ranges, such as a number of samples and a contamination level. Before scoring, get_rank rescales every criterion with n_method. Round II uses LINEAR1. It divides each benefit criterion by its maximum, and divides the minimum of each cost criterion by its values. Every criterion then lies in [0, 1], and a higher value always means a higher priority. mcdm describes the other normalization methods.

Scoring: CP and TOPSIS

Both scoring methods compare each area with the ideal area, which has the best value of every criterion. Compromise programming (CP) scores each area by its weighted distance to the ideal area. The lowest score ranks first. TOPSIS also measures the distance to the worst possible area. It scores each area by its relative closeness to the ideal area, from 0 to 1. The highest score ranks first.

With the Round II weights, both methods rank the four areas in the same order. With the count-heavy weights, they disagree on the last place. TOPSIS ranks area 1 third and area 3 last:

ranks = optimizer.get_rank(is_benefit_x=is_benefit, w_vector=count_heavy, n_method='LINEAR1', s_method='TOPSIS')
test_eq(ranks['rank'].to_dict(), {0: 1, 2: 2, 1: 3, 3: 4})
ranks

	rank
loc_id
0	1
2	2
1	3
3	4

From ranks to sample counts

rank_to_sample turns the ranks into a number of samples for each area. It takes the ranks in loc_id order. The default Weighted policy gives each area a share of the budget proportional to 1/rank. For the TOPSIS ranks above and a budget of 20 samples:

ranks_by_loc = ranks['rank'].sort_index().values
rank_to_sample(ranks_by_loc, budget=20)

array([10,  3,  5,  2])

The quantiles policy, which the tutorial uses, splits the areas into four quartiles by rank. The first quartile shares 50% of the budget, the second 30%, the third 20% and the last none. With four areas, each quartile holds one area:

rank_to_sample(ranks_by_loc, budget=20, policy='quantiles')

array([10,  4,  6,  0])

sampler describes both policies and the min parameter.