Demo data¶

Every tutorial on this site runs on the same small dataset, so you can work through them in order without re-deriving anything.

It is one exemplar image from an mcmicro run: 11,201 cells and 9 markers, with cell centroids and morphology. Small enough that a whole analysis takes seconds, big enough that the spatial statistics mean something.

Getting it¶

The data is not shipped inside the package — the raw image alone is over GitHub's 100 MB file limit — so example_data/ is downloaded separately and lives at the root of the repository checkout.

example_data/
├── adata_scimap.h5ad        the prepared cell table (3.9 MB)
├── markers.csv              channel names for the image
├── manual_gates.csv         hand-picked gates, one column per image
├── phenotype_workflow.csv   the hierarchical gating table
├── quantification/          the raw mcmicro output
│   ├── quant.csv
│   └── quant_1.csv
├── registration/            the registered OME-TIFF
└── segmentation/            the cell masks

In [1]:

from pathlib import Path

import matplotlib.pyplot as plt
import pandas as pd

import scimappro as sp

# The demo data is not shipped with the package (it is 247 MB). Point DATA at
# wherever you unpacked it; this looks in the repository root first, then in the
# path the docs are built from.
DATA = next(
    path for path in (Path("example_data"), Path("../../../example_data"))
    if path.exists()
)
DATA

from pathlib import Path import matplotlib.pyplot as plt import pandas as pd import scimappro as sp # The demo data is not shipped with the package (it is 247 MB). Point DATA at # wherever you unpacked it; this looks in the repository root first, then in the # path the docs are built from. DATA = next( path for path in (Path("example_data"), Path("../../../example_data")) if path.exists() ) DATA

/Users/aj/Partners HealthCare Dropbox/Ajit Nirmal/nirmal lab/softwares/dev/scimappro/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Out[1]:

PosixPath('../../../example_data')

What is in the prepared object¶

adata_scimap.h5ad is what you get after running [sp.pp.mcmicro_to_scimap][scimappro.pp.mcmicro_to_scimap] on quantification/quant.csv — the next tutorial builds it from scratch.

In [2]:

import anndata as ad

adata = ad.read_h5ad(DATA / "adata_scimap.h5ad")
adata

import anndata as ad adata = ad.read_h5ad(DATA / "adata_scimap.h5ad") adata

Out[2]:

AnnData object with n_obs × n_vars = 11201 × 9
    obs: 'X_centroid', 'Y_centroid', 'Area', 'MajorAxisLength', 'MinorAxisLength', 'Eccentricity', 'Solidity', 'Extent', 'Orientation', 'CellID', 'imageid'
    var: 'index'
    uns: 'all_markers', 'gates'
    layers: 'log', None (.X)

Nine markers, in var_names:

In [3]:

list(adata.var_names)

list(adata.var_names)

Out[3]:

['ELANE', 'CD57', 'CD45', 'CD11B', 'SMA', 'CD16', 'ECAD', 'FOXP3', 'NCAM']

And per-cell metadata in obs — the two centroid columns are what every spatial function reads, and imageid is what they group by.

In [4]:

adata.obs.head()

adata.obs.head()

Out[4]:

	X_centroid	Y_centroid	Area	MajorAxisLength	MinorAxisLength	Eccentricity	Solidity	Extent	Orientation	CellID	imageid
quant_1	1767.692308	257.290598	117	12.402944	12.006487	0.250814	0.959016	0.812500	-1.146733	1	quant
quant_2	1107.173913	665.869565	92	11.874070	9.982065	0.541562	0.948454	0.696970	-0.435290	2	quant
quant_3	1116.413793	671.068966	58	10.113305	7.629922	0.656364	0.878788	0.585859	1.221658	3	quant
quant_4	982.728625	677.029740	269	25.433196	15.183300	0.802251	0.835404	0.531621	-0.705293	4	quant
quant_5	1141.071078	680.125000	408	26.604670	19.759781	0.669604	0.937931	0.739130	-0.711002	5	quant

In [5]:

adata.obs["imageid"].value_counts()

adata.obs["imageid"].value_counts()

Out[5]:

imageid
quant    11201
Name: count, dtype: int64

.X holds the log-transformed intensities and .raw.X the originals, which is the convention [sp.pp.mcmicro_to_scimap][scimappro.pp.mcmicro_to_scimap] establishes.

In [6]:

print("X     ", adata.X.shape, adata.X.min().round(2), adata.X.max().round(2))
print("raw.X ", adata.raw.X.shape, adata.raw.X.min().round(2), adata.raw.X.max().round(2))
print("layers", list(adata.layers))

print("X ", adata.X.shape, adata.X.min().round(2), adata.X.max().round(2)) print("raw.X ", adata.raw.X.shape, adata.raw.X.min().round(2), adata.raw.X.max().round(2)) print("layers", list(adata.layers))

X      (11201, 9) 0.0 1.0
raw.X  (11201, 9) 0.0 60800.4
layers ['log', None]

Where the cells are¶

In [7]:

fig, ax = plt.subplots(figsize=(5, 5))
ax.scatter(adata.obs["X_centroid"], adata.obs["Y_centroid"], s=0.5, alpha=0.5)
ax.set_aspect("equal")
ax.invert_yaxis()
ax.set_xlabel("X_centroid")
ax.set_ylabel("Y_centroid")
plt.show()

fig, ax = plt.subplots(figsize=(5, 5)) ax.scatter(adata.obs["X_centroid"], adata.obs["Y_centroid"], s=0.5, alpha=0.5) ax.set_aspect("equal") ax.invert_yaxis() ax.set_xlabel("X_centroid") ax.set_ylabel("Y_centroid") plt.show()

The two CSVs you will need¶

manual_gates.csv supplies the positive/negative boundary for each marker. The first column must be called markers; every other column is named after an image id in obs['imageid'].

In [8]:

pd.read_csv(DATA / "manual_gates.csv")

pd.read_csv(DATA / "manual_gates.csv")

Out[8]:

	markers	quant
0	DNA_6	6
1	ELANE	6
2	CD57	6
3	CD45	6
4	DNA_7	6
5	CD11B	6
6	SMA	6
7	CD16	6
8	DNA_8	6
9	ECAD	6
10	FOXP3	6
11	NCAM	6

phenotype_workflow.csv is the hierarchical gating table. Column 1 is the parent group, column 2 the cell type to assign, and the rest are markers holding gating keywords.

In [9]:

pd.read_csv(DATA / "phenotype_workflow.csv").fillna("")

pd.read_csv(DATA / "phenotype_workflow.csv").fillna("")

Out[9]:

	Unnamed: 0	Unnamed: 1	ELANE	CD57	CD45	CD11B	SMA	CD16	ECAD	FOXP3	NCAM
0	all	ECAD+							pos
1	all	Immune			pos
2	all	SMA+					pos
3	Immune	NK cells		allpos		neg		allpos
4	Immune	Other myeloid cells				pos
5	Immune	Treg								pos
6	Other myeloid cells	Dendritic cells		allneg				allneg

Next¶

• Build the cell table — from the raw mcmicro CSV.
• Prior-knowledge phenotyping — using the two CSVs above.