January 2024
1 item. Back to the archive.
-
Generation configurations: temperature, top-k, top-p, and test time compute
ML models are probabilistic. Imagine that you want to know what’s the best cuisine in the world. If you ask someone this question twice, a minute apart, their answers both times should be the same.…