Models

1D convolutional neural networks for soil spectra

A spectrum is a 1D signal, with one value per wavenumber. The models here are 1D convolutional neural networks (CNNs) built from fastai’s ConvLayer. They take batches of shape (batch, channels, wavenumbers), usually with one channel.

from fastcore.test import test_eq

source

ConvBlock

def ConvBlock(
    ni, # input channels
    nf, # output channels
    ks:int=3, # kernel size
    stride:int=1, # stride
    pool_sz:int=3, # pooling kernel size
    act_cls:functools.partial=functools.partial(<class 'torch.nn.modules.activation.LeakyReLU'>, negative_slope=0.1), # activation
    norm_type:fastai.layers.NormType=<NormType.Batch: 1>, # normalization, e.g. `NormType.Instance`
):

Convolution, normalization and activation, then average pooling

The convolution keeps the number of positions, and the pooling divides it by pool_sz. With the default pool_sz=3, 64 positions become 21:

x = torch.randn(32, 61, 64)
block = ConvBlock(61, 128)
out = block(x)
test_eq(out.shape, (32, 128, 21))
out.shape

torch.Size([32, 128, 21])

source

FeatureExtractor

def FeatureExtractor(
    in_channel:int=1, # input channels
    nfs:tuple=(16, 32, 64, 128, 256), # output channels of each conv block
    pool_sz:int=3, # pooling kernel size of each block
    act_cls:functools.partial=functools.partial(<class 'torch.nn.modules.activation.LeakyReLU'>, negative_slope=0.1), # activation
    norm_type:fastai.layers.NormType=<NormType.Batch: 1>, # normalization, e.g. `NormType.Instance`
    conv_block_cls:type=ConvBlock, # conv block class
):

Stack of ConvBlocks, one per entry of nfs, pooled to one vector per spectrum

FeatureExtractor averages over the positions left after its last block. Each spectrum becomes one vector of nf features, whatever its length:

fe = FeatureExtractor(in_channel=61)
out = fe(torch.randn(32, 61, 243))
test_eq(out.shape, (32, fe.nf))
out.shape

torch.Size([32, 256])

source

SpectralCNN

def SpectralCNN(
    in_channel:int=1, # input channels
    nfs:tuple=(16, 32, 64, 128, 256), # output channels of each conv block
    pool_sz:int=3, # pooling kernel size of each block
    act_cls:functools.partial=functools.partial(<class 'torch.nn.modules.activation.LeakyReLU'>, negative_slope=0.1), # activation
    norm_type:fastai.layers.NormType=<NormType.Batch: 1>, # normalization, e.g. `NormType.Instance`
    conv_block_cls:Callable=ConvBlock, # conv block class
    is_classifier:bool=False, # if True, apply a sigmoid to output probabilities
    dropout:float=0.4, # dropout rate
):

1D CNN with a FeatureExtractor backbone and a linear head, for regression or binary classification

SpectralCNN returns one value per spectrum. With is_classifier=True, a sigmoid turns that value into a probability between 0 and 1:

model_reg = SpectralCNN()
model_cls = SpectralCNN(is_classifier=True)

x = torch.randn(32, 1, 1701)
pred_reg = model_reg(x)
pred_cls = model_cls(x)

test_eq(pred_reg.shape, (32,))
assert 0 <= pred_cls.min() and pred_cls.max() <= 1
pred_reg.shape, pred_cls.min().item(), pred_cls.max().item()

(torch.Size([32]), 0.38777458667755127, 0.6036885380744934)

MirzaiCNN builds the architecture from Albinet et al. (2022), Prediction of exchangeable potassium in soil through mid-infrared spectroscopy and deep learning: From prediction to explainability, Artificial Intelligence in Agriculture 6, 230–241. It is a SpectralCNN with five BatchNorm blocks and AvgPool1d(3), sized for MIR spectra of about 1,700 wavenumbers.

source

MirzaiCNN

def MirzaiCNN(
    in_channel:int=1, # input channels
    is_classifier:bool=False, # if True, model will output probabilities
    dropout:float=0.4, # dropout rate
)->__main__.SpectralCNN:

SpectralCNN with the architecture of Albinet et al. (2022)

The paper’s model has 131,857 parameters:

mirzai = MirzaiCNN()
n_params = sum(p.numel() for p in mirzai.parameters())
test_eq(n_params, 131_857)
n_params