hey there, sadly i dont clearly understand how the wiki.train.tokens and vocab.txt file is produced. followingly could you please show me the steps on how to reproduce the same train.tokens and vocab.txt file on a custom dataset ? (i am trying to implement this on a different language)
hey there, sadly i dont clearly understand how the wiki.train.tokens and vocab.txt file is produced. followingly could you please show me the steps on how to reproduce the same train.tokens and vocab.txt file on a custom dataset ? (i am trying to implement this on a different language)