Fine tune Gemma 4 12b to extract bibliographic info

[kaldan007]

Description

To automate bibliographic information extraction, we are going to fine tune a Gemma 4 12b because it hits the optimal balance between language skills, structured output precision, and compute efficiency. Unlike smaller 4B edge models—which struggles on complex non-Latin scripts—the 12B parameter backbone possesses a significantly richer multilingual embedding space and context retention needed to parse Tibetan syntax.

Subtask

• Fine tune with standard LORA
• Fine tune with QLORA
• Evaluate the results on benchmark
• If not satisfied do full fine tuning
• Document the experiments and upload models on hugging face