Specification

A human-in-the-loop image annotation system with continuous training.

Install

You need to install:

• the packages in requirements.txt (you can install uv and do uv venv --python 3.13 and then uv pip install -r requirements.txt)
• dinov3
  • by cloning the repo cd src/ml/ && git clone https://github.com/facebookresearch/dinov3
  • and downloading the weights you wanna use to src/ml/weights/ (we support small and large for now)

Run

edit db_init.py to configure what images you will load

you must launch both

webapp (local):

gunicorn src.backend.main:app --bind 127.0.0.1:8001 --reload

and the AI, which is one of the following:

• classification training (timm arch):

  python -m src.ml.fastai_training --arch vit_medium_patch16_reg4_gap_256.sbb_in12k_ft_in1k
  

• classification training (frozen dinov3 + linear):

  python -m src.ml.dinov3_classification
  

• segmentation training (frozen dinov3 + linear):

  python -m src.ml.dinov3_training 
  

changelog

• 25-09-22:
  • make the router show both options if task is not set in config and show only the selected task if it is, and fail if going directly (using url) to one of the tasks if it's not the selected task, and if it is the right one and no task was set, set it. essentially, if the task is not set and we hit the router, show both tasks. once we move to a task, if the task is unset, we set it. if one moves to a task that was set and is different, show an alert that says something like 'the current task is X, you can't change tasks, if you want to do another task, reset the app' (resetting means rm / renaming current session dir and relaunching the apps)
  • check cls frontend works
  • fix classif undo
• 25-09-30:
  • remove the live accuracy things in the seg frontend
  • update live acc thing in classif frontend
  • add skip button for classif
  • filter images via name
  • try classif app in qc
  • show image id in frontend
  • check if claimed is removed by time
  • integrate mask training (done)

WIP

• expose way to add more images if needed <- needs trying
• make using the keyboard easier (should work everywhere in the app) <- needs trying
• allow the user to jump to a specific image id <- needs trying

to-do

• expose way to load external annotations
• when removing a class show warning, remove the annotations for that class, and reset the model
• add next image strategy (random, sequential, select class) for seg frontend
• make it possible to start from trained model (maybe self-train?)
  • add a self-training weight, when non-zero, a self-training loop is run in the ml backend (use predictions as targets)
• show both saved mask and prediction