Testers Will Thrive After The AI Tsunami

If you're reading this article, there's a good chance Bertrand Meyer knows more about software development than you do. For sure he knows more than I do! Since you can see further standing on the shoulders of giants, let's learn from what this giant of software has to say.

In his recent article "AI Does Not Help Programmers," Bertrand explains:

AI in its modern form, however, does not generate correct programs: it generates programs inferred from many earlier programs it has seen. These programs look correct but have no guarantee of correctness. (I am talking about "modern" AI to distinguish it from the earlier kind—largely considered to have failed—which tried to reproduce human logical thinking, for example through expert systems. Today's AI works by statistical inference.)

Fascinating as they are, AI assistants are not works of logic; they are works of words. Large language models: smooth talkers (like the ones who got all the dates in high school). They have become incredibly good at producing text that looks right. For many applications that is enough. Not for programming.

Despite the utility of generative AI tools, experienced developers and testers know better than to trust AI-generated code verbatim. AI helps developers write more code faster, but more code is not necessarily better code.

That said, quantity has a quality all its own. With the speed advantages AI offers, experienced software developers are among its greatest adopters. Surveys of ChatGPT users suggest that programming assistance is the most popular use of ChatGPT and 90% of US developers are using an AI-enhanced tool at work.

The ability to generate code without having to learn a programming language has also created opportunities for many people. Combined with the increased adoption of no-code software tools, AI-generated software is enabling people around the world to create customized software solutions to address their own needs.

Regardless of its source, the sheer quantity of AI-generated code will expand the need for skilled software testers who can create tools to keep up with the onslaught. As Jason Arbon writes:

In the near term, generative AI will literally generate orders of magnitude more software and general software output that needs to be tested. The testing community largely still produces test cases sequentially, whether automated or not. We need the emergence of testing systems based on AI to have the hope of keeping up with AI itself.

All this AI code will need to be tested, but who will judge its quality? Humans will. Even if AI begins to write a greater share of code, humans will define the problems the code is meant to solve. As a result, human testers who embrace AI will have a critical role to play in evaluating the quality of the solutions AI helps to create.

AI-assisted software development will redefine the software industry, and perhaps initially outpace software testing. Testers who rise to this challenge, however, will survive the initial wave of change to explore a new world of opportunities.