Curate training labels

A classifier learns what you label. annotate() cuts each signature's exposure distribution into bins; a sample in a band of weak exposure is a positive the panel may not be able to see, and a sample at zero exposure that carries a trace of the signature is an imperfect negative. Curation decides, per signature, which samples train and as what. The blocks on this page are templates on a built Add object a (Train a classifier).

See where the labels come from

a.annotate("gauss")

Add.annotate is called with its defaults. All parameters of Add.annotate

- Annotation -
+------+-------+-------+----------------------+
|      |   Pos |   Neg |         Bins         |
+======+=======+=======+======================+
| SBS3 |    15 |    25 | P0=25, P60=9, P100=6 |
+------+-------+-------+----------------------+

Bin 0 is negative; every other bin is a band percentile of the positive exposures and is positive. A Gaussian mixture places the bands (BIC chooses two to five components; n_components= fixes it). Then cluster() groups samples within each band, and the cluster centroids become the reference spectra the likelihood features compare against.

a.label_report()

Add.label_report is called with its defaults. All parameters of Add.label_report

Curate in two ways

Curate in sigscape studio

The exposure editor in sigscape studio reads the object's JSON export, shows bins and clusters per signature over the spectra, and lets you move samples between bins, merge or split clusters, and drop samples. The edited JSON goes back into the object:

a.save("breast.json")             # to the editor
# ... curate in the studio, download the edited JSON ...
a.apply("breast_curated.json")    # bins and clusters replaced; simulations and models invalidated
a.simulate()
a.tune(
    estimator="monologit",
    cross_fit=True,
    cv_repeats=20,
    cutoff_rule="median_repeat",
    output_dir="results",
)

Add.save is called with its defaults. All parameters of Add.save

Add.apply is called with its defaults. All parameters of Add.apply

Add.simulate is called with its defaults. All parameters of Add.simulate

apply(path, only=["APOBEC"]) takes one signature's clusters from the JSON and leaves the others. In sigscape studio describes the editor.

Curate by rule, in code

The shipped classifiers were curated by rules applied to the exposure bands, recorded on each card as label_curation. The breast SBS3 card, for instance, records that a band of moderate SBS3 fraction, mostly SBS5-dominant, was left out of training as not judgeable on a panel, so positives are the top band only and the classifier is not intended to call weak SBS3.

The rules live in the authors' release scripts, not in a package API. Use their vocabulary when you write your own, so that a card reads the same whoever trained it:

The model axes beside them: estimator (monologit, gbm, hist), feature_set, cluster method, n_components.

In code, the same moves are annotate() options (min_exposure, below="exclude", n_components), edits to a.annotated through apply(), and a.drop_duplicates() for genomes present twice. Record what you did in the cards' vocabulary, and set it on your model's card before you share it.

Keep the rules

• Recluster after any label change. The reference spectra are per band; moved samples change them.
• Do not label from reconstructed spectra. An object built from an exposure matrix and a signature matrix (Add.load((H, W))) has spectra that are W·H, so a label set from them restates the exposure that made them. Label from genomes.
• Keep the negative class clean. The gate certifies the false-positive rate on the distinct negatives. A negative class padded with ambiguous samples lowers the measured rate and raises the real one.
• Watch the sparse share. a.sparse_report() says how many simulated samples per bin fall below the SNV floor and would be discarded at inference. A positive band that is mostly sparse on a panel is a band the panel cannot call.
• Measure stratification before acting on it. export_threshold_table(stratifier="APOBEC") splits a signature's discrimination into within-stratum and between-stratum parts, so you know whether a classifier reads its signature or the presence of another. tune(stratify_by=...) acts on it.

a.sparse_report()

Add.sparse_report is called with its defaults. All parameters of Add.sparse_report

Optional: supplement from another cohort

a.supplement_from(...) adds donor genomes from another cohort to a thin class. In the authors' measurements it harmed more often than it helped, and nothing predicted when it would help. If you use it, run a seed sweep and a negative-donor control, and record the donors on the card.

Optional: extend an object

a.extend("SBS8")                 # add a signature to a built object, with its own annotation
# simulations made elsewhere, {panel}.csv with the object's sample IDs
a.inject_simulations("sims/")
a.merge(other)                   # two objects of the same tumor type

Add.extend is called with its defaults. All parameters of Add.extend

Add.inject_simulations is called with its defaults. All parameters of Add.inject_simulations

Add.merge is called with its defaults. All parameters of Add.merge

Train a classifier is the pipeline these fit into; Files and formats describes the JSON the editor reads.