Build the cell table¶

Turning raw mcmicro output into the AnnData that the rest of scimappro works on.

In [1]:

from pathlib import Path

import matplotlib.pyplot as plt
import pandas as pd

import scimappro as sp

# The demo data is not shipped with the package (it is 247 MB). Point DATA at
# wherever you unpacked it; this looks in the repository root first, then in the
# path the docs are built from.
DATA = next(
    path for path in (Path("example_data"), Path("../../../example_data"))
    if path.exists()
)
DATA

from pathlib import Path import matplotlib.pyplot as plt import pandas as pd import scimappro as sp # The demo data is not shipped with the package (it is 247 MB). Point DATA at # wherever you unpacked it; this looks in the repository root first, then in the # path the docs are built from. DATA = next( path for path in (Path("example_data"), Path("../../../example_data")) if path.exists() ) DATA

/Users/aj/Partners HealthCare Dropbox/Ajit Nirmal/nirmal lab/softwares/dev/scimappro/.venv/lib/python3.12/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Out[1]:

PosixPath('../../../example_data')

What mcmicro gives you¶

One CSV per image: a row per cell, marker intensities first, then morphology and position.

In [2]:

raw = pd.read_csv(DATA / "quantification" / "quant.csv")
raw.shape

raw = pd.read_csv(DATA / "quantification" / "quant.csv") raw.shape

Out[2]:

(11201, 22)

In [3]:

raw.head()

raw.head()

Out[3]:

	CellID	DNA_6	ELANE	CD57	CD45	DNA_7	CD11B	SMA	CD16	DNA_8	...	NCAM	X_centroid	Y_centroid	Area	MajorAxisLength	MinorAxisLength	Eccentricity	Solidity	Extent	Orientation
0	1	17449.786325	1179.529915	161.410256	708.111111	5956.128205	378.692308	246.256410	332.769231	6960.008547	...	853.205128	1767.692308	257.290598	117	12.402944	12.006487	0.250814	0.959016	0.812500	-1.146733
1	2	18053.913043	1124.728261	177.217391	790.271739	11803.586957	910.434783	423.445652	649.989130	13424.717391	...	902.489130	1107.173913	665.869565	92	11.874070	9.982065	0.541562	0.948454	0.696970	-0.435290
2	3	8418.017241	1249.500000	156.637931	791.189655	5159.948276	1034.827586	477.362069	599.482759	5636.793103	...	936.741379	1116.413793	671.068966	58	10.113305	7.629922	0.656364	0.878788	0.585859	1.221658
3	4	22444.189591	1159.676580	187.431227	853.814126	13773.092937	816.914498	2221.639405	375.884758	15566.044610	...	891.988848	982.728625	677.029740	269	25.433196	15.183300	0.802251	0.835404	0.531621	-0.705293
4	5	8076.987745	1568.526961	162.904412	318.497549	4991.470588	827.286765	362.482843	366.365196	5639.095588	...	896.955882	1141.071078	680.125000	408	26.604670	19.759781	0.669604	0.937931	0.739130	-0.711002

5 rows × 22 columns

Note where the marker columns stop and the metadata begins — that boundary is what the split parameter names. Here it is X_centroid.

In [4]:

list(raw.columns)

list(raw.columns)

Out[4]:

['CellID',
 'DNA_6',
 'ELANE',
 'CD57',
 'CD45',
 'DNA_7',
 'CD11B',
 'SMA',
 'CD16',
 'DNA_8',
 'ECAD',
 'FOXP3',
 'NCAM',
 'X_centroid',
 'Y_centroid',
 'Area',
 'MajorAxisLength',
 'MinorAxisLength',
 'Eccentricity',
 'Solidity',
 'Extent',
 'Orientation']

Reading it in¶

[sp.pp.mcmicro_to_scimap][scimappro.pp.mcmicro_to_scimap] does the whole conversion: it splits markers from metadata at split, drops the nuclear counterstain channels, and puts a log1p copy in .X with the raw values kept in .raw.X.

In [5]:

adata = sp.pp.mcmicro_to_scimap(
    str(DATA / "quantification" / "quant.csv"),
    removeDna=True,
    log=True,
    split="X_centroid",
    verbose=False,
)
adata

adata = sp.pp.mcmicro_to_scimap( str(DATA / "quantification" / "quant.csv"), removeDna=True, log=True, split="X_centroid", verbose=False, ) adata

Total execution time: 0:00:00

Out[5]:

AnnData object with n_obs × n_vars = 11201 × 9
    obs: 'X_centroid', 'Y_centroid', 'Area', 'MajorAxisLength', 'MinorAxisLength', 'Eccentricity', 'Solidity', 'Extent', 'Orientation', 'CellID', 'imageid'
    var: 'index'
    uns: 'all_markers'
    layers: None (.X), 'log'

imageid comes from the file name, which is what keeps cells from different images apart once several are concatenated.

In [6]:

adata.obs["imageid"].value_counts()

adata.obs["imageid"].value_counts()

Out[6]:

imageid
quant    11201
Name: count, dtype: int64

Several images at once¶

Pass a list and they are read in parallel and concatenated. uniqueCellId=True prefixes each cell name with its image id, so names stay unique.

In [7]:

both = sp.pp.mcmicro_to_scimap(
    [str(DATA / "quantification" / "quant.csv"),
     str(DATA / "quantification" / "quant_1.csv")],
    removeDna=True,
    uniqueCellId=True,
    verbose=False,
)
both.obs["imageid"].value_counts()

both = sp.pp.mcmicro_to_scimap( [str(DATA / "quantification" / "quant.csv"), str(DATA / "quantification" / "quant_1.csv")], removeDna=True, uniqueCellId=True, verbose=False, ) both.obs["imageid"].value_counts()

Total execution time: 0:00:00

Out[7]:

imageid
quant      11201
quant_1    11201
Name: count, dtype: int64

In [8]:

both.obs_names[:3].tolist(), both.obs_names[-3:].tolist()

both.obs_names[:3].tolist(), both.obs_names[-3:].tolist()

Out[8]:

(['quant_1', 'quant_2', 'quant_3'],
 ['quant_1_11199', 'quant_1_11200', 'quant_1_11201'])

Tidying marker names¶

mcmicro column headers often carry a mask suffix — CD45_cellRingMask and the like. removeStringFromName strips it while reading, so you never have to rename var_names afterwards.

adata = sp.pp.mcmicro_to_scimap(
    "quantification/quant.csv",
    removeStringFromName="_cellRingMask",
)

Prototyping on a subsample¶

A whole-slide quantification can run to millions of cells. randomSample gives you a workable object to develop against.

In [9]:

small = sp.pp.mcmicro_to_scimap(
    str(DATA / "quantification" / "quant.csv"),
    randomSample=2000,
    verbose=False,
)
small.shape

small = sp.pp.mcmicro_to_scimap( str(DATA / "quantification" / "quant.csv"), randomSample=2000, verbose=False, ) small.shape

Total execution time: 0:00:00

Out[9]:

(2000, 9)

Already have an .h5ad?¶

Then there is nothing to prepare. Every scimappro function takes the path directly, so you never need to load it first:

sp.tl.spatialDistance("analysis.h5ad", phenotype="phenotype", outputDir="results")

Saving¶

In [10]:

outputDir = Path("tutorial_output")
outputDir.mkdir(exist_ok=True)
adata.write(outputDir / "prepared.h5ad")
sorted(p.name for p in outputDir.iterdir())

outputDir = Path("tutorial_output") outputDir.mkdir(exist_ok=True) adata.write(outputDir / "prepared.h5ad") sorted(p.name for p in outputDir.iterdir())

Out[10]:

['prepared.h5ad']

Next¶

• Prior-knowledge phenotyping — assign cell types from a gating table.
• Unsupervised clustering — let the data group itself.