Mark Austin, At&t's Head of AI, Said Open-source Models Match Some Closed Models in Performance and Even Outperform Them on Certain Tasks

AT&T is turning to open-source models to keep its OpenAI and Anthropic spending in check as artificial intelligence usage surges, shifting most employee requests to options from Nvidia, Meta and Google. About 40% of employee requests are now routed to Nvidia, Meta and Google models, with a target of raising that share to 60-70% within a few years, the company said. Mark Austin, AT&T's head of AI, said open-source models match some closed models in performance and even outperform them on certain tasks.

The company's Ask AT&T platform, launched in 2023, initially relied entirely on OpenAI models. It has since evolved into a multi-model system working with different providers. The platform processes 45 billion tokens daily and routes requests to the most suitable model based on task complexity.

Using LiteLLM for model routing, AT&T cut costs by up to 56% on some code-generation tasks while keeping output quality loss at around 2%. The company also uses Nvidia Nemotron, Meta Llama and Google Gemma models. We expect open-source models to continue improving, Austin said.

While AT&T still uses powerful closed models for complex coding work, it assigns lighter tasks like summarization to cheaper options. This approach aims to grow usage volume while keeping OpenAI and Anthropic spending stable. The company wants to avoid its AI bill ballooning rapidly.