What comes next after a moment when companies sell existential risk but pursue public funding and DIY regulation, even as software developers question their life choices? Peter Norvig, distinguished education fellow at Stanford HAI and former Google research director, used his opening keynote at The AI Conference in San Francisco on Wednesday to speculate on the answer.

Almost certainly, it's another AI conference – on Thursday in fact – given the two or three dozen of them that have bubbled up in San Francisco this year or are scheduled in the remaining three months.

To get to the present, he took a detour through the past. He acknowledged the milestones of AlexNet and ImageNet in 2012, the Transformer architecture for machine learning in 2017, the debut of ChatGPT in 2022, and the emergence of reasoning and coding agents in 2024.

He also touched on the sudden competency of AI models for math, noting that two years ago, GPT-4 couldn't count the number of "r"s in "strawberry." Yet as of August this year, he observed, 25 percent of math preprint papers on arXiv acknowledged AI assistance.

Norvig went on to cite Linux kernel creator Linus Torvalds' changing attitude toward AI. "Six months ago he was a skeptic, he became a cautious adopter, and now he's a dogmatic advocate," said Norvig.

So what now? Norvig says the process of software engineering needs to change.

The tech industry has changed many times before, he said, pointing to transitions from plugging wires into mainframes, to assembly language, to higher level programming languages. 

"At every step you have to change the way you think about what software engineering is," he said. And now we're going to have to change it all again. How do we do specifications and documentation? How do we capture this theory of the evolving program that we're not capturing well now?"

To illustrate that point, Norvig recounted some of his recent work. 

"I did an embarrassing thing the other day, right? So I'm using Codex, [and I] write some stuff that seems to work," he said.

He told Codex to send the pull request for review, and his colleague asked why he was checking in 6,000 temp files.

"And I said, 'Well, I didn't notice that,'" Norvig explained. "And sure enough, I had written 6,000 temporary files, and Codex thought that it wanted to check those in. So I told [Codex], 'No, don't check them in, just delete them.' At first I thought, shame on me. I should have looked before I asked for a code review on that. And I just skipped that step of looking. Maybe I should be doing that."

Then, said Norvig, he thought that the GPT model should have known that he didn't want to save 6,000 temporary files, so as not to burden the reviewer. And then he mused that if you have enough storage and memory, maybe it would be useful to have those files to have a more thorough history of how the codebase changed.

"So the expectations of what's good practice and how that's going to change, that's all going to be different," he said.

And that's just the start.

"We gotta get the security and privacy and data pipelines and supply chains, those are all gonna be different in how they interact with each other," he said. "This idea of continuous integration and recursive self-improvement. How much do we let the systems go off on their own and how are we monitoring them?"

Judging by the steady drip of incidents where AI models have hacked into third-party websites, discovered only after someone scrutinized log files, we're not monitoring them.

So yeah, there's work to be done. ®