AI Pioneer Rich Sutton Calls Synthetic Data Reliance 'A Big Mistake'

In an episode of Sequoia's podcast released Tuesday, the Canadian computer scientist and Turing Award winner said the AI industry is headed in the wrong direction because of its dependence on synthetic training data. "That's just a big mistake, he said when asked about using synthetic data to keep scaling large language models. "Maybe it's the next big lesson.

Synthetic data is information artificially produced by algorithms or AI models rather than collected from original, real-world sources. It has become increasingly attractive as AI companies scour the internet and hunt for new sources of training data. Examples include computer-generated images of cars for self-driving training or fake bank records for AI fraud detection.

On some level, Big Tech agrees with Sutton. Tech giants are going to increasingly extraordinary lengths to find more of the real thing.