Manage models and the store
List, load, verify, share and delete classifiers, in the package's store or your own.
Before you begin, you need the package installed.
A classifier is a directory. The package ships its classifiers in a store, you add your own to it or to a store of your own, and run() finds a cell by its tumor type, signature and platform. The Input blocks on this page need only the installed package.
List what is installed
import sigma2 as sg2
from sigma2 import classifiers
sg2.describe() # one table per tumor type, every status, the floors
sg2.models() # every name, as 'tumor_type/signature/platform'
sg2.models(tumor_type="breast") # filtered
sg2.models(soi="SBS3", seq_type="msk")
classifiers.cards() # the same as data: one Card per cell
describe is called with its defaults. All parameters of describe
models is called with its defaults. All parameters of models
classifiers.cards is called with its defaults. All parameters of classifiers.cards
A Card has tumor_type, classifier, seq_type, kind (signature, hrd, mmrd), status (released, weak, experimental), verified, trained (a date) and snv_cutoff, the floor the classifier was trained at.
for c in classifiers.cards(tumor_type="breast", seq_type="msk"):
print(c.classifier, c.status, c.snv_cutoff)
SBS18 released 5
APOBEC released 5
SBS8 released 5
SBS3 released 5
HRD released 5
MMRD released 5
describe() and cards() include models in your own store as well as the package's. To list the package's alone, point SIGMA2_DATA_DIR at an empty directory.
Load, verify and delete a model
m = sg2.model("breast/SBS3/msk") # by name, any case
m = sg2.model(tumor_type="breast", soi="SBS3", seq_type="msk") # by parts
# True, or ValueError on a checksum mismatch
sg2.verify_model("breast/SBS3/msk")
model is called with its defaults. All parameters of model
verify_model is called with its defaults. All parameters of verify_model
sg2.delete_model("mytype/SBS3/msk", force=True)
| Parameter | Description | Default |
|---|---|---|
| forcebool | Delete without a confirmation prompt. | False |
A loaded model runs with its reference spectra:
curl -LO https://raw.githubusercontent.com/parklab/SigMA/master/inst/extdata/examples/test_mutations_50sample.maf
out = sg2.run(
input="test_mutations_50sample.maf",
ref_genome="hg19",
tumor_type="breast",
seq_type="msk",
soi="SBS3",
model=m,
)
| Parameter | Description | Default |
|---|---|---|
| inputpath, list of paths, folder or DataFrame | The mutations: a MAF path, a VCF path, a list of VCF paths, a folder of VCFs, a CSV, or a DataFrame of SBS-96 counts. | Required |
| ref_genomestr or path | The build of the coordinates (hg19, hg38, GRCh37, GRCh38) or a FASTA path; unused for a matrix input. | 'hg19' |
| tumor_typestr | A tumor type of the vocabulary, in any case (the package lists 39 names; msi and pole among them are signature sets, not tumor types); it sets the signature set, the reference spectra and the classifier. | Required |
| seq_typestr | The platform (msk, fo, op, seqcap, tcga_mc3, wgs); "auto" chooses by SNV burden and reports the choice. | 'auto' |
| soistr or CompositeSignature | The signature of interest: a COSMIC name, a composite preset (APOBEC, SBS7, SBS10, SBS17, CLOCK), an alias (UV, POLE), or HRD. | Required |
| modelstr or GBMModel | Which classifier runs: "auto" for the released cell, "legacy" for SigMA v1's, or a model loaded with sg2.model(). | 'auto' |
Read the card
m.config["catalog"] # 'cosmic_v3p2_inhouse'
m.config["snv_cutoff"] # 5
# where the floor came from: the training summary, the tune result, or the tumor type's default
m.config["snv_cutoff_source"]
m.config["evidence"] # 'informative' or 'weak'
m.config["label_curation"] # how the training labels were curated
m.config["training_depth"] # how the simulated panels were matched to real depth
The public fields:
name is an attribute of the model, status, verified and kind are fields of the cell's Card (classifiers.cards()), and the rest are keys of m.config:
| Field | Meaning |
|---|---|
name | <tumor_type>/<soi>/<seq_type>, with experimental/ in front for a withheld cell |
status | released, weak, experimental (Release status) |
verified | the weights match their recorded SHA-256 |
kind | signature, hrd, mmrd |
catalog | the catalog the features were computed on; run() uses it |
snv_cutoff, snv_cutoff_source | the floor and where it came from |
training_depth | the depth the simulated panels were matched to |
evidence, evidence_reason | what a positive call is worth, and the gate's reason |
label_curation | the label curation (full on PCAWG cards, a sentence on the breast and colorectal SBS3 cards, absent on older cards) |
readjust_recommended, readjust_note | whether the card suggests readjust=True for cohorts off the training depth |
required_motif | a motif a call requires, on the cells that have one (lung APOBEC on msk) |
The card also records the gate's inputs, the cross-validation settings and performance figures. This documentation does not quote them.
Find the store on disk
<data dir>/sysdata/models/
breast/SBS3/msk/
model.joblib the learner
imputer.joblib the imputer the features pass through
centroids.csv the tumor type's reference spectra the likelihoods are computed against
metadata.json the card
meta.json SHA-256 of the files, for verify_model
experimental/
prost/SBS3/op/ a withheld cell
retired.json weights a release replaced
<data dir> is $SIGMA2_DATA_DIR or the per-user application directory (Install). The package's classifiers are seeded there from the wheel on first use; the HRD and MMRD/POLE models ship inside the package (sg2.hrd_model("msk").path shows where), and HRD models you train go to sysdata/hrd/. When the directory is not writable, the package reads its classifiers from the wheel.
Make a store of your own
Add.export(output_dir) writes the same layout to any directory, which is then a store:
a.export("my_store") # <tumor_type>/<soi>/<seq_type>/ per model
sg2.models(root_dir="my_store")
m = sg2.model("breast/SBS3/msk", root_dir="my_store")
sg2.run(
input="panels.maf",
ref_genome="hg19",
tumor_type="breast",
seq_type="msk",
soi="SBS3",
model=m,
)
Add.export is called with its defaults. All parameters of Add.export
Add.publish() writes into the data directory instead, behind the release gate: a cell that fails goes to experimental/, where run() refuses it without force=True. export() applies no gate. Train a classifier compares the two.
Sharing a model is copying its cell directory into another store and loading it with root_dir=. The directory carries the weights, the imputer, the reference spectra, the card and the checksums.
Optional: migrate retired weights
A package release can retire weights. retired.json in the package lists them by the hash of their weights, and a store that still holds a retired copy is moved by
sg2.models.migrate(dry_run=True)
| Parameter | Description | Default |
|---|---|---|
| dry_runbool | True reports what would change; False moves the store's retired weights. | True |
Optional: read the reference tables
catalog = sg2.get("sig_catalogs/cosmic_v3p2_inhouse")
tumor_types = sg2.get("indices/tumor_types")
sets = sg2.get("sigs_per_tissue")
get is called with its defaults. All parameters of get
sg2.set and sg2.clear write your own tables to sysdata/custom.h5.