toolsrtm.inversion

Trait-inversion tools: CARS-PLS and VIF predictor selection, LUT nearest- neighbour (“merit function”) matching, and a 12-algorithm ML dispatcher (PLSR/SVM/RF/GB/NN/Bayesian/AdaBag/BRNN/xGB/RVM/qLASSO/Ensemble) built on scikit-learn/xgboost. Direct port of ToolsRTM::carspls/get.cars.pls, getVIF, get.inversionOpt, get.inversion, hybrid_inversion/ hybrid_inversionE.

Note

Needs the optional ml extra: pip install "toolsrtm[ml]". Nothing in this module is imported by toolsrtm/__init__.py’s own import chain – a plain import toolsrtm never requires scikit-learn/xgboost.

Note

R’s get.inversion/hybrid_inversion dispatch to specific caret methods (bartMachine, rqlasso, rvmLinear, AdaBag, brnn, …). See ALGORITHMS for exactly which scikit-learn/xgboost estimator each algorithm name maps to, and, where there’s no direct equivalent, what was substituted and why.

Quick example

import numpy as np, pandas as pd
from toolsrtm import foursail
from toolsrtm.inversion import get_inversion

rng = np.random.default_rng(1)
rows = []
for _ in range(200):
    Cab, LAI = rng.uniform(10, 80), rng.uniform(0.5, 6)
    inputLUT = dict(N=1.5, Cab=Cab, Car=8, Anth=1, Cbrown=0, EWT=0.01, LMA=0.009, alpha=40,
                     LIDFa=-0.35, LIDFb=-0.15, TypeLidf=1,
                     LAI=LAI, hspot=0.01, tts=30, tto=0, psi=0)
    sail = foursail(inputLUT, np.full(2101, 0.15), leaf_model="PROSPECT-D", spectrum_all=True)
    row = {"Cab": Cab, "LAI": LAI}
    for wl in (490, 560, 665, 705, 740, 783, 842, 865, 1610, 2190):
        row[f"R{wl}"] = sail.rsot[wl - 400]
    rows.append(row)
df = pd.DataFrame(rows)
band_cols = [c for c in df.columns if c.startswith("R")]

result = get_inversion(df, dep_var="Cab", inputs=band_cols, algorithm="PLSR", n_samples=200, seed=1)
print(result.statistics["test"]["r2"])   # held-out test R2

Input                              get_inversion()           Output
---------------------------        ----------------------    ---------------------------
df   [n rows]  LUT: predictor                                 result.model      fitted estimator
               bands + dep_var     -------------------->      result.statistics  train/test R2, RMSE
dep_var  = trait to invert (e.g. "Cab")                       result.predictions test-set predicted
inputs   = predictor column names                                                 vs. observed
algorithm = "PLSR"/"RF"/"SVM"/... (see ALGORITHMS)

Trait-inversion tools: CARS-PLS feature selection, VIF-based collinearity pruning, LUT nearest-neighbour (“merit function”) inversion, and a multi-algorithm ML dispatcher.

Python port of ToolsRTM/R/carspls.R / get.cars.pls.R, getVIF.R, get.inversion.R, hybrid_inversion.R / hybrid_inversionE.R, and get.inversionOpt.R. Needs the optional ml extra (pip install toolsrtm[ml]: scikit-learn, xgboost) – none of these functions are imported by toolsrtm/__init__.py at import time, and each imports its own ML dependencies lazily so a plain import toolsrtm never requires scikit-learn/xgboost to be installed.

R’s get.inversion/hybrid_inversion dispatch by name to specific caret methods (bartMachine, rqlasso, rvmLinear, AdaBag, brnn, …). Several of those have no direct scikit-learn/xgboost equivalent; where that’s the case the docstring of the relevant function says exactly which estimator was substituted and why. Unlike the pure radiative-transfer math ported elsewhere in this package, none of this module is verified to floating-point precision against R – caret’s own cross-validated tuning is stochastic, so a Python port using different (but comparable) estimators and search grids will not reproduce R’s numbers bit-for-bit even in principle. What’s verified instead: each algorithm runs end-to-end on held-out data and produces sane, comparable-magnitude accuracy metrics (see tests/test_inversion.py).

class toolsrtm.inversion.CarsPlsResult(coef, n_var, rmsecv, num_lv, optimal_iteration, min_error, selected_variables)[source]

Bases: object

Result of carspls(). Mirrors the R CARS list 1:1 except selected_variables is 0-indexed (Python) instead of 1-indexed (R).

Parameters:

coef (ndarray)
n_var (ndarray)
rmsecv (ndarray)
num_lv (ndarray)
optimal_iteration (int)
min_error (float)
selected_variables (ndarray)

coef: ndarray: (n_vars, iteration) coefficient path

n_var: ndarray: (iteration,) number of retained variables per iteration

rmsecv: ndarray: (iteration,) cross-validated RMSE per iteration

num_lv: ndarray: (iteration,) best number of latent variables per iteration

optimal_iteration: int: 1-indexed iteration with the lowest RMSECV (matches R)

min_error: float

selected_variables: ndarray: 0-indexed column positions into the original X

toolsrtm.inversion.carspls(X, y, n_lv=2, fold=10, scale_pretreat=True, iteration=50, partition_type='interleaved', verbose=False)[source]

Competitive Adaptive Reweighted Sampling for PLS variable selection.

Python port of carspls/get.cars.pls (R, original algorithm by Yizeng Liang & Hongdong Li, MATLAB->R port by Hongdong Li 2009). At each of iteration rounds: fits a PLS model on the currently-retained variables, cross-validates it to get an RMSECV curve over 1..n_lv components, records the coefficient-magnitude-ranked variable importance, and forcibly eliminates the lowest-ranked variables via an exponentially decreasing retention schedule (Monte-Carlo/EDF sampling). The iteration with the lowest RMSECV gives the final selected variable set.

Parameters:

X (ndarray) – (n_samples, n_vars) predictor matrix.
y (ndarray) – (n_samples,) response vector.
n_lv (int) – number of PLS latent variables (components) to fit/tune over.
fold (int) – number of cross-validation segments.
scale_pretreat (bool) – if True, scale (not just center) each predictor.
iteration (int) – number of CARS-PLS elimination rounds.
partition_type (Literal['interleaved', 'consecutive', 'random']) – cross-validation fold assignment: "interleaved" (round-robin), "consecutive" (contiguous blocks), or "random".
verbose (bool) – print progress per iteration (matches R’s own screen output).

Returns:

CarsPlsResult.

Return type:

CarsPlsResult

toolsrtm.inversion.get_vif(frame, columns=None, thresh=10.0, trace=True)[source]

Backward-elimination variable selection by Variance Inflation Factor.

Python port of getVIF (R, VIF function originally from https://beckmw.wordpress.com/2013/02/05/collinearity-and-stepwise-vif-selection/). Iteratively regresses each remaining variable on all others; drops the variable with the highest VIF (1 / (1 - R^2)) as long as any VIF exceeds thresh.

Parameters:

frame (ndarray) – (n_samples, n_vars) array, or a pandas.DataFrame.
columns (Sequence[str] | None) – variable names, required if frame is a bare array; ignored (and taken from frame.columns) if frame is a DataFrame.
thresh (float) – VIF threshold above which a variable is flagged as collinear.
trace (bool) – print each elimination step (matches R’s own console output).

Returns:

names (or 0-indexed positions, if columns is None and frame is a bare array) of the retained variables.

Return type:

list[str] | list[int]

class toolsrtm.inversion.InversionOptResult(rfl_best: 'np.ndarray', lut_best: "'pd.DataFrame'")[source]

Bases: object

Parameters:

rfl_best (np.ndarray)
lut_best (pd.DataFrame)

rfl_best: np.ndarray: (n_obs, n_wave) best-matching (n_opt-averaged) simulated spectra

lut_best: pd.DataFrame: (n_obs, n_lut_columns) n_opt-averaged LUT parameters per observation

toolsrtm.inversion.get_inversion_opt(rfl_sensor, rfl_rtm, lut, wave=None, method='merit-RMSE', n_opt=1, custom_stat=None)[source]

LUT (look-up table) inversion by nearest-neighbour spectral matching.

Python port of get.inversionOpt (R). For each observed spectrum in rfl_sensor, ranks every simulated spectrum in rfl_rtm by a merit (error) function and averages the n_opt best matches’ LUT parameters and reflectance. Fully vectorized (broadcasts each observation against the whole LUT at once) rather than R’s nested per-row loop – same algorithm, no numerical differences expected for the built-in merit functions (verified against a hand-computed reference below).

Parameters:

rfl_sensor (ndarray) – (n_obs, n_wave) observed/sensor reflectance.
rfl_rtm (ndarray) – (n_lut, n_wave) simulated reflectance from the LUT.
lut – (n_lut, n_params) pandas.DataFrame of the LUT’s input parameters.
wave (Sequence[float] | None) – wavelengths corresponding to columns of rfl_sensor/rfl_rtm (only used to name the returned reflectance columns; optional).
method (str) – one of "merit-RMSE", "merit-NRMSE", "merit-MAE", "merit-NMB", "merit-FGE", or "merit-custom.metric" (requires custom_stat).
n_opt (int) – number of best-matching LUT rows to average per observation.
custom_stat (Callable[[ndarray, ndarray], ndarray] | None) – optional f(sim, obs) -> error merit function, broadcast over the last axis exactly like the built-in ones; overrides method.

Returns:

InversionOptResult.

Return type:

InversionOptResult

toolsrtm.inversion.ALGORITHMS = {'AdaBag': "sklearn AdaBoostRegressor over shallow DecisionTreeRegressor stumps (matches caret method 'AdaBag')", 'BRNN': "sklearn MLPRegressor with strong L2 (alpha) regularization -- substitute: approximates 'Bayesian regularization' via explicit weight decay, not R's brnn Gauss-Newton/Levenberg-Marquardt fit", 'Bayesian': "sklearn BayesianRidge -- substitute: no BART implementation in sklearn/xgboost; BayesianRidge is a Bayesian *linear* model, not R's bartMachine (Bayesian additive trees)", 'Ensemble': "sklearn StackingRegressor(GB + SVR + MLP, final_estimator=LinearRegression) (matches caretEnsemble::caretStack(..., method='glm'))", 'GB': "sklearn GradientBoostingRegressor (matches caret method 'gbm')", 'NN': "sklearn MLPRegressor, single hidden layer (matches caret method 'nnet')", 'PLSR': "sklearn PLSRegression, n_components tuned by 5-fold CV (matches caret method 'pls')", 'RF': "sklearn RandomForestRegressor, max_features tuned (matches caret method 'rf')", 'RVM': "sklearn BayesianRidge -- substitute: no Relevance Vector Machine in sklearn/xgboost; BayesianRidge shares RVM's sparsity-favouring linear-Bayesian character", 'SVM': "sklearn SVR(kernel='rbf'), gamma/C tuned by grid search (matches caret method 'svmRadial' via e1071::tune.svm)", 'qLASSO': "sklearn QuantileRegressor(quantile=0.5, solver='highs'), L1-penalized (matches caret method 'rqlasso')", 'xGB': "xgboost XGBRegressor(booster='gblinear') (matches caret method 'xgbLinear')"}: R’s caret-method name each algorithm dispatches to, and, where scikit-learn/ xgboost has no direct equivalent, the substitution actually used here.

class toolsrtm.inversion.InversionResult(model_label: 'str', model: 'object', predictions: 'dict', statistics: 'dict', importance: 'dict | None')[source]

Bases: object

Parameters:

model_label (str)
model (object)
predictions (dict)
statistics (dict)
importance (dict | None)

model_label: str

model: object

predictions: dict

np.ndarray, “test”: np.ndarray}

Type:: {“train”

statistics: dict

{“r2”:.., “rmse”:.., “mae”:..}, “test”: {…}}

Type:: {“train”

importance: dict | None

importance}, or None if not available for this algorithm

Type:: {input_name

toolsrtm.inversion.get_inversion(data, dep_var, inputs, algorithm='PLSR', seed=123, n_samples=500, test_size=0.3)[source]

Fit and evaluate a plant-trait inversion model with one of 12 algorithms, on a held-out train/test split.

Python port of get.inversion (R). See ALGORITHMS for exactly which scikit-learn/xgboost estimator each algorithm name dispatches to, and, for the 4 algorithms with no direct equivalent (Bayesian, BRNN, RVM – and note AdaBag/xGB/qLASSO do have close matches), what was substituted and why. Unlike R’s version, tuning here is a single small grid search per algorithm (not caret’s full repeated-CV search), to keep this runnable as a demo rather than a multi-hour job – the same design choice already used by Scripts/Python/*/2_inversion_ml.py.

Parameters:

data – pandas.DataFrame containing dep_var and all inputs columns.
dep_var (str) – name of the response column to predict.
inputs (Sequence[str]) – names of the predictor columns.
algorithm (str) – one of the keys of ALGORITHMS.
seed (int) – random seed for the train/test split and any stochastic estimator.
n_samples (int | None) – if given and less than len(data), randomly subsample this many rows before splitting (matches R’s own tuning-sample-size argument).
test_size (float) – fraction of (sub-sampled) data held out for testing.

Returns:

InversionResult.

Return type:

InversionResult

class toolsrtm.inversion.HybridInversionResult(model: 'object', keep_variables: 'list[str]', statistics: "'pd.DataFrame'", predictions: 'dict')[source]

Bases: object

Parameters:

model (object)
keep_variables (list[str])
statistics (pd.DataFrame)
predictions (dict)

model: object

keep_variables: list[str]: predictor columns actually used, after pattern/collinearity selection

statistics: pd.DataFrame: rows “train”/”test” (+ “field” if field_data given), columns r2/rmse/mae

predictions: dict

np.ndarray, “test”: np.ndarray[, “field”: np.ndarray]}, original (untransformed) scale

Type:: {“train”

toolsrtm.inversion.hybrid_inversion(lut, input, split=0.8, seed=None, method=None, collinearity=None, pattern=None, trans=True, field_data=None, acron=None)[source]

Fit a single-algorithm trait-inversion model with optional predictor selection (by name pattern, then optionally VIF or CARS-PLS pruning) and an optional log-transform of the response.

Python port of hybrid_inversion (R). method dispatches to the same 5 estimators as get_inversion()’s SVM/RF/GB/NN/ Ensemble ("nnet" here maps to "NN" there, matching R’s own caret method name) – see ALGORITHMS for what each one is. Note: R’s train/test split uses caret::createDataPartition (percentile- stratified on the response); this port uses a plain random split via scikit-learn, which is not percentile-stratified – a documented approximation, not expected to change results materially for the LUT-sized (typically hundreds of rows) datasets this is meant for.

Parameters:

lut – pandas.DataFrame with the response column input and candidate predictor columns.
input (str) – name of the response column to predict.
split (float) – train-fraction of the train/test split (R’s own convention; note this is the train fraction, unlike get_inversion()’s test_size).
seed (int | None) – random seed.
method (str | None) – one of "SVM", "RF", "GB", "nnet", "Ensemble". Defaults to "SVM" (matches R’s own default).
collinearity (Literal['VIF', 'CARS'] | None) – None (use every pattern-matched column), "VIF" (prune via get_vif()), or "CARS" (select via carspls()).
pattern (str | None) – substring that predictor column names must contain (e.g. "B" for reflectance-band columns named B1, B2, …); if None, every column except input is a candidate.
trans (bool) – log-transform input before fitting (matches R’s own default); predictions/statistics are reported back on the original scale.
field_data – optional pandas.DataFrame of field observations to validate against, in addition to the LUT’s own test split.
acron (str | None) – suffix appended to input to find the observed column in field_data (e.g. acron="_obsv" looks for f"{input}_obsv"). Required if field_data is given.

Returns:

HybridInversionResult.

Return type:

HybridInversionResult

toolsrtm.inversion.hybrid_inversion_ensemble(lut, input, split=0.8, seed=None, collinearity=None, pattern=None, field_data=None, acron=None)[source]

hybrid_inversion() with method="Ensemble" fixed.

Python port of hybrid_inversionE (R) – the R function is hybrid_inversion with the model choice hardcoded to the 3-model (GB + SVM + neural net) stacking ensemble and no trans/log-transform option, which this wrapper matches (trans=False).

Parameters:

input (str)
split (float)
seed (int | None)
collinearity (Literal['VIF', 'CARS'] | None)
pattern (str | None)
acron (str | None)

Return type:

HybridInversionResult