The hard part of programming is building and maintaining a useful mental model of a complex system. The easy part is writing code. They’re positioning this tool as a universal solution, but it’s only capable of doing the easy part. And even then, it’s not able to do that part reliably. Human engineers will still have to evaluate and review the code that an AI writes. But they’ll now have to do it without the benefit of having anyone who understands it. No one can explain it. No one can explain what they were thinking when they wrote it. No one can explain what they expect it to do. Every choice made in writing software is a choice not to do things in a different way. And there will be no one who can explain why they made this choice, and not those others. In part because it wasn’t even a decision that was made. It was a probability that was realized.
This post also has a really good explanation of how large language models work.
There may be real, productive uses for these kinds of tools. There may be ways to build and deploy them ethically and sustainably. But that’s not the situation with the instances we have. AI, as it’s been built today, is a tool to sell out our collective futures in order to enrich already wealthy people. They like to frame it as being akin to nuclear science. But we should really see it as being more like fossil fuels.