How a classifier scores a panel
A whole genome usually carries thousands of SNVs, and its signatures can be read off its spectrum directly. A targeted panel carries a handful, often fewer than ten, and most of the 96 channels are empty. SigMA2 asks a different question of a panel: not "which signatures compose this spectrum" but "does this sparse spectrum look like the panel view of a tumor of this type that carries the signature, or of one that does not". The method is that of the SigMA paper (Gulhan et al., Nature Genetics 2019); SigMA2's own measurements are in its manuscript, in preparation.
Three features from one sparse spectrum
For each tumor type, SigMA2 holds the typical spectra of its genomes grouped by signature composition, the reference spectra. Breast genomes with Signature 3 look one way, breast genomes with APOBEC another, and clock-like breast genomes a third. A new panel is compared with these rather than with the bare COSMIC signatures, because a Signature 3 tumor's spectrum is Signature 3 plus everything else that co-occurs in that tissue.
| Feature | Question | Why it works at low counts |
|---|---|---|
| Likelihood | How probable are these few mutations under each reference spectrum? | A multinomial likelihood is defined for five mutations; it uses every mutation, not a shape |
| Cosine similarity | How close is the spectrum to the signature? | Familiar, but unstable at low counts: one mutation moves the angle |
| NNLS exposure | How much of the spectrum does the signature explain, among the tumor type's signatures? | Restricting the decomposition to the tissue's signatures keeps it from assigning foreign signatures to noise |
The tumor type sets two of the three: the reference spectra, and the signature set the decomposition may use. That is why tumor_type is required, and why a wrong one gives a wrong answer without an error.
The learner
The features, with the SNV count, go into a boosted learner trained per cell: one tumor type, one signature, one platform. The learner's output is <soi>_mva, a score in [0, 1]. Most shipped cells use a monotone learner, a sign-constrained logistic baseline with monotone boosted trees: more cosine, more exposure or a higher signature-present likelihood can only raise the score, which keeps the ranking in order in the sparse tail, where a few training samples would otherwise let a flexible model learn shapes that contradict the features' meaning. The others use scikit-learn's gradient-boosted trees, which carry no such constraint; type(sg2.model(name).estimator).__name__ names a cell's learner (MonotoneLogitBoost, GradientBoostingClassifier or HistGradientBoostingClassifier).
Training on simulated panels
No panel cohort carries ground truth for a signature. The training data are whole genomes with known exposures, each reduced to the panel's territory: the mutations inside the panel's capture regions, topped up to a realistic burden by resampling the genome's own spectrum, then thinned to the depth of a real cohort on that panel. Every genome becomes what the panel would have seen of it, and its label comes from its genome-wide exposure. One genome gives one simulated panel per platform; the same labels train the msk, fo, op, seqcap and tcga_mc3 cells.
Labels are bins of the exposure, not a threshold at zero: genomes with a clear exposure are positives, genomes with none are negatives, and the band between can be left out of training when it is not judgeable on a panel. The card's label_curation records what was done; Curate training labels covers the how.
Two cutoffs on held-out negatives
The score ranks the samples; the calls come from two cutoffs set in cross-validation: the loose one at a 10% false-positive rate on the held-out negatives, the strict one at 1%. For most shipped cells, cross-validation is repeated with new splits and the cutoff is the median over repeats, so it does not depend on one fold layout; the card's cutoff_rule and cv_repeats record this for each cell. Floor, cutoffs and catalog has the rules; Release status has the gate that decides whether the cutoffs are trusted at all.
The detection law and a cell's ceiling
What a cell can call is bounded by the burden its assay delivers. For every cell, training leaves an out-of-fold score for each genome's simulated panel, and the released loose cutoff decides which positives were called. The package relates those calls to one quantity per positive, its burden on the panel (sigma2.tools.detection):
- expected burden, the signature's share of the genome's mutations, from the whole-genome fit, times the SNVs the simulated panel holds: what the assay should have delivered, never zero for a positive;
- assigned burden,
exp_<soi>, the mutations the decomposition attributed to the signature on the panel: what the assay did deliver, and zero for a large share of weak positives.
On the positives, the probability of a call rises with expected burden as a logistic in its logarithm, fitted by maximum likelihood with two parameters:
P(call | λ) = 1 / (1 + exp(−(a + b · log10 λ)))
Two points on it are the limits of detection: LoD50, the burden at which a call is as likely as not, and LoD90, where it becomes dependable; a bootstrap over the positives gives their intervals. The fit is on expected burden because assigned burden is zero when detection failed, and a law fitted on it would be conditioned on the outcome it predicts. This is the limit-of-detection regression of assay validation (the probit or logistic fit of the hit rate against log concentration in CLSI EP17-A2), with the expected count of the signature's mutations in the place of a concentration. A cell whose fit has no slope, because its positives sit on one side of the cutoff or because the trend declines, is reported as such and its limits are not read.
The law is specific to the cell: it moves with the tumor type's background, since a signature that adds little shape to that background needs more of its own mutations for the same call, and with the platform's depth. What the cell can reach follows from where its positives' burdens fall on the curve. Drawn on the law's axis, the burden density of the positives can be read off the curve: a vertical line from the density gives the probability of a call at that burden. The share of positives below LoD50 is the share no classifier at this false-positive rate calls as often as not, and the share below LoD90 is where calls stop being dependable. That share is the ceiling: the sensitivity at the operating point, and with it the AUC, cannot exceed what the burden distribution permits, and a cell whose positives all sit above LoD90 is at its ceiling with every one of them called. The package's evaluation uses it this way: learning curves show such a cell is not limited by training data, more genomes do not lift it, and an oracle given each tumor's background reaches the same bound; what moves the curve left is per-mutation discrimination against the tissue's background, and what moves the positives right is the assay's depth. Beside the cell's AUC, the AUC that SNV count alone gives says how much of the separation is burden rather than spectrum.
The floor is a cut on the total count, and λ is a share of it, so a positive at or above the floor can still sit far below LoD50. On your own training, the law, its limits and the share of positives under them come from the tuning result:
from sigma2.tools import detection
report = detection.detection_report(models, n_boot=200)
cols = ["signature", "seq_type", "n_positive", "median_lambda_expected",
"undetectable_fraction", "lod50", "lod90", "law_note"]
report[cols]
sigma2.plot.dtc(models) draws the law, one burden density per assay and the AUC markers; compare= overlays a second run of the same cohort so a training change reads as a movement of the curve.
HRD and MMRD/POLE
SigMA2's HRD model is two steps around a Signature 3 score: the score, likelihood and exposure of SigMA v1's Signature 3 classifier on cosmic_v2_inhouse, as SigMA2's port computes them (what the port leaves out), become features, joined by copy-number instability (HRD-LOH, TAI, LST from allele-specific segments, or fraction of genome altered and a log-ratio LST from panel segments) and two indel features (microhomology-mediated and non-homologous end-joining deletions). A pan-cancer tree ensemble and a tumor-type ensemble score them; the call is HRD when both exceed 0.5, HRP when neither does, and HRD-low when they disagree. What a call means states what the call supports. A tumor type has its own model when it had more than ten positives in the first training step, its BRCA1/2-deficient tumors plus three neighbors of each; the others share a pooled model. Detect HRD.
MMRD/POLE is a multiclass tree model per platform on SigMA v1's features plus microsatellite indel counts. Its call is the most probable of the three classes; a per-class cutoff set at a 1% false-positive rate in cross-validation is also required, but at its values it never overrules the most probable class. Detect MMRD or POLE.
Consequences
A call on a panel with a few mutations comes with a false-positive rate that was measured, and a status that says when the measurement is not good enough. In return, a classifier exists only for a cell that was trained, a sample below the floor gets no score, and a panel of a different design is scored as the closest bundled platform. The bundled classifiers page lists what exists; Train a classifier adds a cell.