Skip to content
khamirdesaiPublic

About

RetViM: Vision Transformer and Mamba for retinal OCT classification. Research implementation and trained checkpoint.

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

11 Commits

Folders and files

Repository files navigation

RetViM

Sequential Hybrid Vision Transformer with MedMamba for Retinal Disease Classification

Accepted for presentation at IEEE GSCon 2027.

Khamir Desai · Mayuri A. Mehta · Sree Saicharan Vadapalli

RetViM combines a Vision Transformer with convolution/state-space blocks for four-class retinal OCT classification: CNV, DME, DRUSEN, NORMAL.

Model weights · Checkpoint details · Citation

Installation

Python 3.10+; Python 3.12 is recommended.

git clone https://github.com/khamirdesai/RetViM.git
cd RetViM
pip install -e .

Inference

retvim predict --image /path/to/oct.png

The first run downloads the pinned Hugging Face checkpoint. For local weights:

retvim predict --image /path/to/oct.png --weights /path/to/model.safetensors

Dataset and reported result

Kermany retinal OCT dataset (Kaggle): CNV, DME, DRUSEN and NORMAL. Original dataset record.

Model Test images Accuracy (paper-reported)
RetViM 968 99.90% (967/968)

This is the highest accuracy among the methods compared in our manuscript. It is not an independently verified overall leaderboard ranking; this release has not rerun the full benchmark.

Training and evaluation

pip install -e '.[train,eval]'
retvim prepare-data --data /path/to/OCT2017 --output runs/data/manifest.json
retvim train --manifest runs/data/manifest.json --output runs/retvim --device cuda
python scripts/evaluate.py --manifest runs/data/manifest.json \
  --weights runs/retvim/best.safetensors --output runs/retvim/test.json --device cuda

Training exports best.safetensors and last.safetensors, both compatible with retvim predict --weights. Data preparation records hashes, keeps the supplied test set fixed, and rejects patient/file overlap. See training settings and validation scope. Download dataset files separately under their own terms.

Checkpoint

Backbone Hybrid blocks Parameters Input Format
ViT-B/16 2 101.22M RGB, 224 × 224 safetensors

The supplied checkpoint uses native 1D Mamba. Its scan backend and classifier differ from the manuscript's summarized SS2D configuration; this implementation preserves the trained tensors. See checkpoint details. The full paper benchmark has not been independently rerun by this release.

Citation

@unpublished{desai2027retvim,
  title  = {RetViM: Sequential Hybrid Vision Transformer with
            MedMamba for Retinal Disease Classification},
  author = {Desai, Khamir and Mehta, Mayuri A. and
            Vadapalli, Sree Saicharan},
  year   = {2027},
  note   = {Accepted for presentation at IEEE GSCon 2027}
}

Acknowledgments

Mentor: Prof. Mayuri A. Mehta. We acknowledge MedMamba, VMamba, Mamba, and timm.

License

Original implementation: MIT. Model weights: license pending; rights reserved. Intended for research; no prospective clinical validation is claimed.

The manuscript, paper figures, and publisher PDF are not distributed here. See copyright scope and license scope.

About

RetViM: Vision Transformer and Mamba for retinal OCT classification. Research implementation and trained checkpoint.

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages