Floor, cutoffs and catalog

Get the example file

curl -LO https://raw.githubusercontent.com/parklab/SigMA/master/inst/extdata/examples/test_mutations_50sample.maf

The SNV floor

A sample with fewer SNVs than the floor is not scored by any classifier. Below it, the 96-channel spectrum is too sparse for the likelihoods, the cosine and the exposures to carry information.

Which floor applies, in order:

1. snv_ctf= when you pass it;
2. else the floor the classifier was trained at, recorded on its card as snv_cutoff;
3. else the tumor type's floor: 5, with prostate at 4 and osteosarcoma and pancreatic neuroendocrine tumors at 3.

Every shipped signature classifier records the floor it was trained at, and every one was trained at its tumor type's floor, so cases 2 and 3 agree for the shipped cells. HRD and MMRD/POLE apply the tumor type's floor. sg2.describe() heads each tumor type's table with the floor its classifiers apply.

import sigma2 as sg2

out = sg2.run(
    input="test_mutations_50sample.maf",
    ref_genome="hg19",
    tumor_type="breast",
    seq_type="msk",
    soi="SBS3",
)
out8 = sg2.run(
    input="test_mutations_50sample.maf",
    ref_genome="hg19",
    tumor_type="breast",
    seq_type="msk",
    soi="SBS3",
    snv_ctf=8,
)
(out.snv_floor, out.snv_ctf, len(out.df)), (out8.snv_floor, out8.snv_ctf, len(out8.df))

((5, None, 38), (8, 8, 28))

Below the floor: the sample is dropped from out.df and stays in out.input_matrix, for every input format; verbose=2 lists it; out.snv_floor records the floor applied; with readjust=True it does not count toward the cohort's median, as it did in SigMA v1; predict_mmrd_ppd leaves it out of its table; an HRD run keeps it with NA Signature 3 features and a call that is not an assessment.

Changing the floor moves the classifier off its validated operating point. A lower floor scores sparser samples with features the model did not see in training at those depths; a higher one discards samples the model could score. Pass snv_ctf= to reproduce SigMA v1's flat 5 on prostate (Reproduce SigMA v1), or for a reason you can state. Training at a different floor is a tune option (Train a classifier). What a sample above the floor can show depends on its signature share as well as its count: the detection law and a cell's ceiling.

The two operating points

A classifier's score is a number in [0, 1]. Two cutoffs on it were set in cross-validation at two false-positive rates on the classifier's held-out negatives:

A score above both is a high-confidence call; above the loose one only, a low-confidence call; below both, negative. A cohort study that wants sensitivity reads pass_mva; a report that wants few false positives reads pass_mva_strict.

No strict cutoff. Some classifiers made no call at the 1% tier in cross-validation. Such a classifier has a loose operating point only: pass_mva is the call, pass_mva_strict is NA, and confidence is negative below the loose cutoff and NA above it. The NA is not a failed strict call.

The required motif. One card adds a rule on top of the cutoffs: lung APOBEC on msk requires at least one TCW>T or TCW>G mutation. A lung panel without one is not called APOBEC-positive, however high its score; motif_count reports the count.

Readjust

The cutoffs were set on simulated panels at the training cohort's depth. A cohort sequenced to a different depth has a different score distribution, and readjust=True re-derives every cutoff for the batch from the classifier's held-out negatives at the batch's depth:

rj = sg2.run(
    input="test_mutations_50sample.maf",
    ref_genome="hg19",
    tumor_type="breast",
    seq_type="msk",
    soi="SBS3",
    readjust=True,
    verbose=1,
)
rj.df["cutoff_readjusted"].iloc[0]

MVA::readjust breast/SBS3/msk: cutoffs readjusted to this cohort's median of 11.0 SNVs (trained at 9.0):
FPR 10% cutoff 0.4477 -> 0.4417

Four cases fall back to the shipped cutoffs, each with a warning: a card without the table, too few effective negatives, a batch outside the negatives' depth range, and legacy models.

Caution Use it for whole cohorts sequenced off the training depth, never for a single patient. The rule reads the batch's own depth distribution, and the package enforces no minimum batch size: a batch of one would shift the cutoff to its own depth. Leave it off by default, and do not readjust with fewer than about 30 samples. The card's readjust_recommended says whether the classifier's authors saw a need for it.

The catalog

A classifier's features are exposures and likelihoods of a catalog's signatures, and its cutoffs hold for those numbers only. The catalog is the classifier's:

warned = sg2.run(
    input="test_mutations_50sample.maf",
    ref_genome="hg19",
    tumor_type="breast",
    seq_type="msk",
    soi="SBS3",
    catalog="cosmic_v2_inhouse",
    verbose=1,
)
warned.catalog_name

catalog cosmic_v2_inhouse is not the one the classifier was trained on (cosmic_v3p2_inhouse). Its scores
and calls are those of a classifier fed exposures it was not trained on, not the validated ones; leave
catalog= out to use cosmic_v3p2_inhouse.

A catalog SigMA2 picks itself that differs from the classifier's is refused as a bug.

The catalogs are cosmic_v2_inhouse, cosmic_v3_inhouse, cosmic_v3p2_inhouse and cosmic_v3p2: COSMIC's SBS catalogs, where "inhouse" marks SigMA's curated copy with the signature sets per tumor type the decomposition is restricted to. The package also knows cosmic_v2, COSMIC's v2 catalog without those sets, which a run with a tumor type refuses. Names match in any case. Vocabulary lists which tumor types each catalog has a set for.

Summary

sg2.run(
    input="prostate.maf", ref_genome="hg19", tumor_type="prost", seq_type="msk", soi="SBS3",
    # snv_ctf=None  -> the classifier's floor (4 for prostate)
    # catalog=None  -> the classifier's catalog
    # readjust=False
)

The defaults are the validated run. The output records what applied (out.snv_floor, out.catalog_name, cutoff_readjusted) and any override (out.snv_ctf).