Tutorial 4 — Inspecting NMF convergence

This tutorial shows how to use the convergence diagnostics returned by multi_sample_nmf to assess whether the chosen rank range and number of initialisations are sufficient.

Dataset — Peter Sims lab DMSO vs. panobinostat scRNA-seq cohort.

0 · Imports and data preparation

import kaleidocell
import scanpy as sc
import matplotlib.pyplot as plt

adata = sc.read_h5ad('../data/petersims_example.h5ad')
print(adata)

1 · Run NMF and capture the convergence object

multi_sample_nmf returns a tuple (results_nmf, nmf_convergence). The second element is a dict mapping each sample to a DataFrame with one row per rank containing the best Frobenius reconstruction error across all initialisations.

results_nmf, nmf_convergence = kaleidocell.multi_sample_nmf(
    adata,
    batch_key='Patients',
    test_ranks=[4, 5, 6, 7, 8, 9],
    n_initializations=10,
    seed=42,
)

# nmf_convergence is a dict {sample_name: DataFrame(rank, error)}
list(nmf_convergence.keys())

2 · Inspect reconstruction error per rank

A large drop in error between consecutive ranks suggests that the higher rank captures a genuinely new signal. An elbow in the curve is a common heuristic for rank selection.

fig, axes = plt.subplots(1, len(nmf_convergence), figsize=(4 * len(nmf_convergence), 3), sharey=False)

for ax, (sample, df) in zip(axes, nmf_convergence.items()):
    ax.plot(df['rank'], df['error'], marker='o')
    ax.set_title(sample)
    ax.set_xlabel('Rank')
    ax.set_ylabel('Frobenius error')

plt.tight_layout()
plt.show()

3 · Derive meta-programs and inspect quality metrics

After selecting a rank based on the elbow above, proceed with derive_nmf_metaprograms and review the per-MP quality metrics to confirm the consensus is stable.

results_mp = kaleidocell.derive_nmf_metaprograms(results_nmf)

# Quality metrics: sampleCoverage, silhouette, meanSimilarity, nPrograms, nGenes
results_mp['metrics']