We build stuff and share it with the community.

That's pretty much it. And yes, "stuff" was the right word to use.

FromZero is inspired by Re:Zero - Starting Life in Another World. Just like the main character, all of our models begin from zero.

Thankfully, they don't spend their time dying repeatedly or having mental breakdowns.

Training is already painful enough.

Well, there isn't actually a definitive goal we're chasing. I mean, yeah, we want to be the greatest. But so does everyone else.

We're just here to build weird models, share them with the community, and grow side-by-side with everyone.

Syn-2.6M: A 2.6M-parameter Tiny Language Model (TLM) trained only on synthetic data (tiny-textbooks, tiny-stories-v2, longpage, and finephrase).

Orez - Tiny (1.3M), Small (5M), Medium (12.4M), Large (30M): A family of Small Language Models ranging from 1.2M to 30M parameters. Each of them will be massively overtrained on a minium of 1000+ tokens per parameter.
Unknown.com: A website where you can test all of our models for free.

Join our Discord community: FromZero.