Bytedance Trains 10-Trillion-parameter AI Model, Three Times Larger Than Moonshot's Kimi

The ByteDance model is being pre-trained, a stage that typically takes three to six months, before it is fine-tuned and released if all goes well, one of the people said. The exact model size would only be determined at a later stage. ByteDance's efforts to train one of the world's largest AI models show Chinese labs' ambition to not only catch up but also outperform their US peers in the most advanced level of AI.

Anthropic doesn't disclose the size of its models, but industry estimates say its most advanced Mythos 5 has about 8 trillion parameters and Fable 5 about 5 trillion. While parameter count sets the fundamental capacity or memory limits for the models to store information, actual capability also depends on other factors such as data quality and training methods.