Files and formats

What SigMA2 writes and reads on disk: a model's directory, a training run's result tree, the Add object's two save formats, and the files the sigscape runner adds. Input formats (MAF, VCF, matrix, segments) are on Prepare your input files.

A model directory

<tumor_type>/<soi>/<seq_type>/
  model.joblib        the learner (scikit-learn pickle)
  imputer.joblib      the imputer the features pass through
  centroids.csv       the tumor type's reference spectra, 96 rows, one column per cluster
  metadata.json       the card
  meta.json           SHA-256 of the files; verify_model checks them

The same layout under the data directory's sysdata/models/ (the store), under experimental/ for a withheld cell, and in any directory Add.export() wrote. Manage models and the store.

metadata.json, the card. Public fields: name, tumor_type, soi, seq_type, catalog, snv_cutoff, snv_cutoff_source, evidence, evidence_reason, label_curation, training_depth, readjust_recommended, readjust_note, required_motif, created_at. The gate's inputs (gate_releasable, gate_reason, release_gate, certifiable_fpr, operating_point_stable), the cross-validation settings (cv_repeats, cutoff_rule, cross_fit, cv_grouping, thinned_to, burden_clip, result_tree), the cutoffs (cutoff_loose, cutoff_strict, dynamic_cutoffs) and the performance fields are internal: the package reads them, this documentation does not quote them. A nested meta block holds the tuner's own record of the same, with score_only and curation.

The result tree

Add.tune(output_dir=...) writes one tree per run:

<output_dir>/
  README.md
  <tumor_type>/
    manifest.json                     what was run, with versions and hashes
    training_log.csv                  one row per signature and platform trained
    summary.pdf, auc_heatmap.pdf, roc_combined.pdf, *_roc_comparison.pdf
    <seq_type>/
      training_log.csv, roc_comparison.pdf
      <soi>/
        training_summary.json         the gate's verdict and the operating point (below)
        cv_scores.tsv                 out-of-fold scores per sample, per repeat (cv_score_r<r>, fold_r<r>)
        eval_scores.tsv               scores on a held-out split, when split() was used
        roc_curve.tsv                 the vertical average of the per-repeat ROC curves
        roc_curve_mean_score.tsv      the ROC of the repeat-averaged scores
        roc.pdf, cv.pdf, confusion.pdf
        training_data.tsv             the feature table the learner saw
        training_data_crossfit.tsv    with cross_fit
        filtered_samples.tsv          samples below the training floor, with their truth
        panel_simulations.tsv         the simulated spectra
        sample_assignments.tsv        bins and clusters per sample
        confusion_samples.tsv

User-facing: training_summary.json, cv_scores.tsv, eval_scores.tsv, roc_curve.tsv, roc.pdf, confusion.pdf. The rest is working material.

training_summary.json keys to read: releasable, release_reason, release_rule, certifiable_fpr, operating_point_stable, snv_cutoff, cv_repeats, cutoff_rule, the operating point and its counts, capture_fidelity when requested.

export_threshold_table() adds threshold_auc_table.tsv, threshold_curve.tsv, sigma2_thresholds.json, threshold_auc_table.meta.json, and stratified_auc.tsv with a stratifier.

The Add object

Two save formats, chosen by extension:

The JSON export's blocks: df, obs, bins, clusterInfo (required on load), metadata, signatures, samples, clusteredSignatures, mutations, optional simulations and tuning (a manifest and a training_log of per-<seq_type>/<soi> tables). There is no schema version yet; the core blocks and metadata are stable in practice, tuning is provisional. This JSON is what sigscape studio's exposure editor reads and writes, and what Add.apply() takes back.

Add.load((H, W)), with an exposure matrix and a signature matrix, reconstructs an object whose spectra are W·H: good for annotate, cluster and the editor export, invalid for simulate and tune, which warn.

Simulations from elsewhere

Add.inject_simulations(sims) takes {panel}.csv files or a {panel: path} dict, each a sample × 96 table whose sample IDs equal the object's. Mismatched IDs are refused naming both sides; keep handles duplicate IDs; panel_aliases maps names (mc3 to tcga_mc3 is built in).

The data directory

$SIGMA2_DATA_DIR  or the per-user application directory
  reference/<build>_genome.fa (+ .fai)
  repeat_bed_files/merged_bed_*.bed
  sysdata/models/, sysdata/hrd/, sysdata/add/, sysdata/custom.h5
  cache/

Inside the package: var/sysdata/data.h5 (catalogs, signature sets per tumor type, reference spectra, SigMA v1's cutoffs and burden tables), var/panels/*.bed.gz (the five bundled capture designs, hg19), the legacy weights, and var/registry.json (the downloadable assets with their checksums).

What the sigscape runner adds

A Score run in sigscape studio (In sigscape studio) writes into its run folder:

These two JSON files are defined by the runner's driver, not by the package, and carry no schema version yet either.