Configs for reproducing the evaluation

[PT-10]

Hi,
Thank you for the work. I am looking to reproduce the reported results from the paper. I understand that the evaluation on DL3DV uses the same setting as RayZer. However, running the evals on dl3dv_even using rayder_l.pt gives metrics close to reported Rayder-B metrics but not exact.

NVS Evaluation

Can you confirm the exact counts for input & target views at eval time for each dataset. Is it the same 16 context - 8 target for DL3DV like RayZer?
What are the exact frames chosen as input and target?

It would be if you could share these. Thanks

[stefan-baumann]

Hi,

I assume you're talking about Tab. 6 (in the arxiv v1)? If so, that table (as mentioned in the caption) is a B-scale RayDer model that is trained in the same setting as the baselines -- i.e., a model not directly comparable with the publicly released checkpoint. The exact relevant differences are listed in Tab. B.3 -- some relevant major changes vs. the setting in which the public checkpoint was evaluated include:

• Train dataset: SpatialVid -> DL3DV
• Total views: 8 -> 24
• Time between views: 0.5s -> 0.67s
• Train loss: MSE -> MSE + 0.2 * LPIPS
  These changes were necessary to make the comparison maximally fair, since all the baselines were trained with them, and obviously significantly influence performance in this eval.

This should directly explain the difference you're seeing, where the released, much more general (and sparser-view-optimized), RayDer-L checkpoint does not significantly outperform the tailored RayDer-B-DL3DV checkpoint on DL3DV.

Regarding the evaluation setting, it's the exact RayZer one using their benchmark data that you can obtain from https://github.com/hwjiang1510/RayZer#data.

I hope this helps!

Stefan