Case-level computational pathology

PathoSynVLM

Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation

Zhiyuan Yang^*, Jiahao Cheng^*, Vincent Quoc-Huy Trinh, Mahdi S. Hosseini

^* Equal contribution

CSSE, Concordia University, Montréal, Canada · Centre de recherche du CHUM, Université de Montréal, Montréal, Canada · IRIC, Université de Montréal, Montréal, Canada · Mila – Québec AI Institute, Montréal, Canada

Quick Start

# clone and install the research code
git clone https://github.com/AtlasAnalyticsLab/PathoSynVLM
cd PathoSynVLM
conda create -n pathosynvlm python=3.11 -y
conda activate pathosynvlm
pip install -e .

# inspect the multi-WSI report-generation entry point
python scripts/generate_case_report.py --help

Inference requires prepared CONCHv1.5 H5 embeddings and a local weight package. Follow the paper pipeline for data, training, and evaluation.

Abstract

Generating clinically useful pathology reports for pathology cases from whole-slide images (WSIs) is challenging due to gigapixel resolution, long visual-token sequences, and the complexity of case-level reasoning, where a single case may contain multiple WSIs with heterogeneous tissues and ambiguous findings. We present a simple token-efficient vision–language model for case-level synoptic report generation that remains practical under constrained GPU memory. Our architecture follows a minimal three-component design: a frozen pathology patch encoder, a lightweight two-layer MLP vision-language aligner, and a large language model decoder, with an explicit WSI marker token to separate slides within a case. Training proceeds in two supervised stages: (1) aligner-only WSI captioning using heterogeneous WSI-text pairs, and (2) case-level supervised fine-tuning on case-report pairs for structured report generation. To reduce sequence length, we represent each slide using 512 × 512 patches at 5× magnification, which reduces the average sequence length by up to 64× compared to commonly used 20× patches. Combined with efficient training techniques, we enable practical training with only half an NVIDIA H100 GPU. Across both training stages, our approach achieves high ROUGE-L/METEOR/BLEU-4 scores while being substantially more efficient in memory and runtime. In AI-based evaluations, the model performs strongly against several baselines, while the paper also documents settings where a competing system is preferred. Extensive ablations characterize performance-efficiency trade-offs and identify simple choices that improve robustness in multi-WSI settings. Overall, this work provides a reproducible baseline for efficient pathology report generation, lowering the barrier to multi-WSI VLM research under limited compute.

Two-Stage Training Pipeline

Stage 1 aligns frozen pathology patch features with a frozen language model through a trainable MLP projector. Stage 2 fine-tunes case-level report generation from multiple whole-slide images. — **Stage 1:** a two-layer MLP learns visual-language alignment while the pathology encoder and language model remain frozen. **Stage 2:** the model learns case-level structured generation from one or more WSIs under a constrained visual-token budget.

Training Data

Bar charts showing retained HistGen and REG2025 WSI-text pairs for Stage 1 and retained HISTAI cases by group for Stage 2. — Dataset composition after metadata standardization and integrity filtering.

Stage 1 retains 7,637 HistGen and 7,378 REG2025 WSI-text pairs for visual-language alignment. Stage 2 uses case-report pairs from HISTAI across mixed and organ-specific groups.

The second-stage distribution is intentionally broad but imbalanced, with mixed and skin cases accounting for most retained examples. The public repository documents the preprocessing and deterministic split procedure without redistributing restricted source data.

Data guide Embedding format

Results

Reported paper values are shown directly below. The Stage 2 headline row corresponds to Table 3 and the B1 prompt-repetition setting in Table 7; WSI-marker variants are evaluated separately in the ablation section of the paper.

Headline text-generation and field-level results from paper Table 3
Stage	ROUGE-L	METEOR	BLEU-4	BERTScore F1	Diagnosis exact	Diagnosis relaxed	Certainty
Stage 1	0.4743	0.4810	0.1247	0.4253	—	—	—
Stage 2	0.2495	0.1988	0.0525	0.3018	0.1667	0.3333	0.9000

Stage 1 is single-WSI captioning. Stage 2 is case-level structured reporting; overlap metrics are computed on the Conclusion field.

Stage 1 efficiency under single-WSI inputs from paper Table 4
Setting	Patches per input	Preprocessing seconds	Inference seconds
Baseline · 5×	160.04	45.98	1.87
20× baseline	12,247.74	196.84	4.62

Low-magnification patching reduces the visual sequence and the dominant preprocessing cost while retaining practical report generation.

Diagnosis: ...
Certainty: ...
Conclusion: ...

The fixed schema supports field-level evaluation while retaining a free-form conclusion for case-level findings.

Style-sensitive stress tests

Preference results depend on reporting style and domain specialization.

These are separate evaluations of the Stage 2 model with different eligible case sets and reference targets. Read each comparison independently rather than as a pooled leaderboard.

AI preference

LLM-as-judge · ours / tie / comparator

Ours Tie Comparator

vs WSI-LLaVA 515 cases

vs PRISM 186 cases

vs HistoGPT^* 100 skin cases

Under these separate protocols, the evaluated Stage 2 model was more often preferred against WSI-LLaVA and PRISM; HistoGPT was more often preferred in the skin-only comparison.

^*HistoGPT is specialized for dermatopathology and was evaluated only on skin cases. Counts are reported in the paper's AI-preference comparison table.

Pathologist audit

67 overlapping skin cases · 0–5 composite scale

Ours2.49

HistoGPT2.24

Consistency / 3

0.85 ours vs 0.61 HistoGPT

Pairwise preference

20 ours 39 tie 8 HistoGPT

Reviewed by one senior certified pathologist across 67 overlapping skin cases.

How to read these comparisons

The AI comparisons use ChatGPT 5.4 to select the better output and score case agreement from 0 to 10, with different output constraints and reference targets: WSI-LLaVA and HistoGPT are compared against microscopic-protocol content, while PRISM is compared on diagnosis content. Preference is sensitive to report style and is not the same as diagnostic accuracy. Approximately 93% of the full Stage 2 corpus is categorized as skin or mixed. The pathologist audit covers one senior certified pathologist and 67 overlapping skin cases; no statistical significance test was reported in the presentation. These results are research evidence rather than clinical validation, and the presentation concluded that neither audited system was clinically adequate.

Qualitative Results

Multi-WSI cases are packed into one visual sequence and decoded into the required Diagnosis, Certainty, and Conclusion fields.

Three Stage 2 examples showing whole-slide image inputs, ground-truth structured reports, and PathoSynVLM baseline predictions. — Stage 2 examples from HISTAI. Each row pairs the input WSI evidence with the reference report and baseline prediction. These examples illustrate both close matches and clinically meaningful omissions or disagreements.

Citation

@misc{yang2026simpletokenefficientvisionlanguage,
  title         = {Simple Token-Efficient Vision-Language Model for
                   Case-level Pathology Synoptic Report Generation},
  author        = {Zhiyuan Yang and Jiahao Cheng and
                   Vincent Quoc-Huy Trinh and Mahdi S. Hosseini},
  year          = {2026},
  eprint        = {2605.30716},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2605.30716}
}

Research use only. PathoSynVLM is not a clinical diagnostic device. Generated reports may be incorrect or incomplete and must not be used for patient care without appropriate validation, regulatory review, and expert oversight.