Microsoft Releases 1.3 Bn Parameter Language Model, Outperforms LLaMa
Open link in next tab
Microsoft Releases 1.3 Bn Parameter Language Model, Outperforms LLaMa
https://analyticsindiamag.com/microsoft-releases-1-3-bn-parameter-language-model-outperforms-llama/
Microsoft Research has upped the game with an even smaller model. phi-1 is a transformer based model with just 1.3 billion parameters.
I wonder if higher quality datasets are the future rather than using tons of internet scraped texts. Either way, neat model!