Run evalutation of model on Dharmabench QUDT benchmark

[karmatai]

Description

The current finetuned scores really low on quotation detection. As I found there already exist a quotation benchmark developed by Dharmabench, it would be better to see how good our models are doing before moving into further implementations.

Subtask

• Load the current model checkpoint.
• Load the Dharmabench QUDT benchmark.
• Run evaluation script
• Document the result

Reviewer

• @kaldan007

[karmatai]

aclanthology.org/2025.ijcnlp-long.114

[kaldan007]

Add the description about the data format of the benchmark dataset.

[karmatai]

DharmaBench QUDT Performance for model: mmbert_base_v1.3

Decoder | MUC Type F1 | MUC Exact F1 | IoU@0.5 F1

argmax | 0.5501 | 0.5074 | 0.0376
viterbi_penalty_4 | 0.6534 | 0.5700 | 0.1590

[kaldan007]

prepare a graph with our model score on the benchmark with other model