Step Loss Trend Issue in Training

[lucasxu777]

Hey do you guys seem to have this issue too? After plotting the step loss, I think there is no effective learning going on here; however, their winning rate is indeed increased when evaluating on 500 prompt examples from pickscorev2 dataset, matching to what they put in their paper. I am having the same issue with DSPO, which is a newer paper built upon this paper.

[youyuanyi]

Can u share pickscorev2 dataset? It can not be accessible now.

Hey do you guys seem to have this issue too? After plotting the step loss, I think there is no effective learning going on here; however, their winning rate is indeed increased when evaluating on 500 prompt examples from pickscorev2 dataset, matching to what they put in their paper. I am having the same issue with DSPO, which is a newer paper built upon this paper.

[xw3981-web]

Hey do you guys seem to have this issue too? After plotting the step loss, I think there is no effective learning going on here; however, their winning rate is indeed increased when evaluating on 500 prompt examples from pickscorev2 dataset, matching to what they put in their paper. I am having the same issue with DSPO, which is a newer paper built upon this paper.

I met exactly the same problem after many tries. The loss is almost the same as yours. Have you solved the problem?