toolsrtm.deep_learning

Deep-learning trait inversion: dense (“Hidden-layers”) and 1D-CNN Keras architectures with a configurable optimizer. Direct port of ToolsRTM::getMLmodel/getMLmodel.withRetrain.

Note

Optional – needs the dl extra: pip install "toolsrtm[dl]" (TensorFlow). Not required for the rest of the package; toolsrtm.inversion’s scikit-learn-based dispatcher covers most trait-inversion needs without it.

Quick example

import numpy as np, pandas as pd
from toolsrtm import foursail
from toolsrtm.deep_learning import get_ml_model

rng = np.random.default_rng(2)
rows = []
for _ in range(600):
    Cab, LAI = rng.uniform(10, 80), rng.uniform(0.5, 6)
    inputLUT = dict(N=1.5, Cab=Cab, Car=8, Anth=1, Cbrown=0, EWT=0.01, LMA=0.009, alpha=40,
                     LIDFa=-0.35, LIDFb=-0.15, TypeLidf=1,
                     LAI=LAI, hspot=0.01, tts=30, tto=0, psi=0)
    sail = foursail(inputLUT, np.full(2101, 0.15), leaf_model="PROSPECT-D", spectrum_all=True)
    row = {"Cab": Cab, "LAI": LAI}
    for wl in (490, 560, 665, 705, 740, 783, 842, 865, 1610, 2190):
        row[f"R{wl}"] = sail.rsot[wl - 400]
    rows.append(row)
df = pd.DataFrame(rows)
band_cols = [c for c in df.columns if c.startswith("R")]

result = get_ml_model(df, dep_var="Cab", model="Hidden-layers", n_epochs=500, n_times=3, seed=2)
print(result.stats["r2"])   # held-out R2

Input                              get_ml_model()            Output
---------------------------        ----------------------    ---------------------------
df   [n rows]  LUT: predictor                                 result.model     fitted Keras model
               bands + dep_var     -------------------->      result.x_scaler  fitted StandardScaler
dep_var  = trait to invert                                     (re-apply to new X before
model = "Hidden-layers"/"CNN"                                    predict() -- see the R
n_epochs, n_times, seed                                          Tutorial 13 scaling-bug story)

Note

result.x_scaler must be applied to any new predictor data before calling result.model.predict(...) – training happens in scaled space, so predicting on raw reflectance directly produces silently wrong (often catastrophically bad) results. This is exactly the bug documented and fixed in ToolsRTM Tutorial 13.

Deep-learning trait inversion: dense (“Hidden-layers”) and 1D-CNN Keras models with a configurable optimizer, matching R’s getMLmodel / getMLmodel.withRetrain.

Needs the optional dl extra (pip install toolsrtm[dl]: tensorflow). Like toolsrtm.inversion, nothing here is imported by toolsrtm/__init__.py and TensorFlow is imported lazily inside get_ml_model(), so a plain import toolsrtm never requires it.

Unlike R’s own (non-reproducible, GPU/BLAS-order-dependent) Keras training, this is not verified to floating-point precision against R – what’s verified is that both architectures train to convergence and produce sane held-out R^2/RMSE on synthetic data (see tests/test_deep_learning.py), the same standard already used for Scripts/Python/*/3_inversion_dl.py/4_inversion_dl.py, which this module formalizes into an installable, tested package function.

class toolsrtm.deep_learning.MLModelResult(model: 'object', history: 'dict', stats: 'dict', predictions: 'dict', x_scaler: 'object')

Bases: object

Parameters:

• model (object)

• history (dict)

• stats (dict)

• predictions (dict)

• x_scaler (object)

model: object

the fitted keras.Model

history: dict

per-epoch training history (keras.callbacks.History.history)

stats: dict

.., “rmse”:..} on the held-out validation split

Type:

{“r2”

predictions: dict

np.ndarray, “y_pred”: np.ndarray} on the held-out validation split

Type:

{“y_true”

x_scaler: object

fitted sklearn.preprocessing.StandardScaler for the predictors

toolsrtm.deep_learning.get_ml_model(dataset, dep_var, model='Hidden-layers', optimizer='adam', batch_size=125, n_epochs=100, prop_split=(0.8, 0.2), n_layers=3, n_neurons=64, n_times=1, seed=123, verbose=0)

Train a dense or 1D-CNN Keras regression model to predict dep_var from every other column of dataset.

Python port of getMLmodel/getMLmodel.withRetrain (R). Predictors are standardized (sklearn.preprocessing.StandardScaler) before training, matching R’s own data.trans='preProcess' default; the response is left on its original scale (matching R’s own depVar.trans=FALSE default).

Parameters:

• dataset – pandas.DataFrame containing dep_var and predictor columns.

• dep_var (str) – name of the column to predict.

• model (Literal['Hidden-layers', 'CNN']) – "Hidden-layers" (dense MLP: n_layers hidden layers of n_neurons units, ReLU, dropout 0.1 after the first hidden layer, matching R’s 3-layer 64/32(dropout)/16 default when n_layers=3, n_neurons=64) or "CNN" (1D convolution over the predictor vector: conv(64,k=4) -> pool -> conv(32,k=2) -> pool -> dense(16) -> dropout(0.1) -> output).

• optimizer (str) – one of "adam", "adadelta", "adagrad", "adamax", "nadam", "rmsprop", "sgd" (same learning rates/momenta as the R defaults for each).

• batch_size (int) – training batch size.

• n_epochs (int) – maximum training epochs (early stopping on val_loss, patience 5, restores best weights – matches R).

• prop_split (tuple[float, float]) – (train_fraction, val_fraction).

• n_layers (int) – number of hidden layers for "Hidden-layers" (ignored for "CNN").

• n_neurons (int) – units in the first hidden layer for "Hidden-layers" (subsequent layers halve down to a floor of 8; ignored for "CNN").

• n_times (int) – fit this many times with different random initializations and keep the run with the lowest validation loss (matches getMLmodel.withRetrain’s n.times).

• seed (int) – random seed for the train/val split and Keras initialization.

• verbose (int) – Keras fit() verbosity (0, 1, or 2).

Returns:

MLModelResult.

Return type:

MLModelResult