cngvng/ct2rep-radgenome-ct-finetune

CT2Rep fine-tuned on RadGenome-ChestCT

Full training (from scratch, per the paper's own recipe) of a CT2Rep-architecture 3D CT report generator, trained on the RadGenome-ChestCT benchmark's train split (22,942 volumes), evaluated on its held-out test split (1,564 volumes). 8 epochs (paper's own default is 100 -- reduced for feasibility on a single A100-40GB at batch_size=1/240x480x480 input resolution).

Evaluation (RadGenome-ChestCT test split, n=1564)

Scored with this project's shared Clinical Efficacy + NLG metric suite (src/metrics/report.py::MetricsReport) -- the same pipeline used for every other baseline in this comparison.

Metric	Value
Clinical Efficacy F1 (macro)	0.0000
Clinical Efficacy F1 (micro)	0.0000
BLEU-1	0.4529
BLEU-4	0.2773
ROUGE-L	0.3566
METEOR	0.4225
CIDEr-D	0.2927
BERTScore F1	0.8904

Known issue: severe prediction mode collapse (disclosed, not hidden)

Clinical Efficacy is exactly 0.0 across every metric. Direct inspection of the 1564 test predictions found only 3 unique generated reports across the entire test set (68.3% / 28.7% / 3.0% of cases), all "everything normal" boilerplate that never asserts any of the 18 CT-RATE abnormality categories. This is why Clinical Efficacy is exactly zero: the model never predicts a positive finding, so there are no true positives to score, regardless of the reference report's actual content.

The NLG scores above (BLEU/ROUGE/METEOR/CIDEr/BERTScore) look strong in isolation because RadGenome-ChestCT reference reports are themselves dominated by shared "normal finding" boilerplate sentences that a templated near-constant output happens to overlap with heavily -- this is exactly the reward-hacking pattern this project's own metric methodology (Clinical Efficacy reported alongside NLG, never NLG alone) is designed to catch. Do not cite the NLG numbers above as evidence of report-generation quality without this context.

Not yet root-caused (open follow-up, not this checkpoint's last word): possible causes include the paper's own block_trigrams/beam-search decode settings interacting poorly with this dataset's vocabulary distribution, insufficient training epochs (8, vs. the paper's own 100) for the vision-conditioning signal to overcome a strong "normal report" prior in the training distribution, or a genuine architecture/training deviation introduced during reproduction (see docs/baselines/ct2rep.md in the source repository for the full deviation list and audit trail).

Files

• checkpoint_latest.pth -- final fine-tuned model state (epoch 8/8, 1.76GB).

Per-epoch checkpoints (checkpoint_epoch1.pth through checkpoint_epoch7.pth) and optimizer/scheduler resume state are intentionally not included here (training-resume-only, not needed to use or reproduce this checkpoint's outputs).