In one of those landmark moments that will doubtless pass most of us by, but ought to have coders sitting up and taking notice, Facebook’s Android app recently became one of the first in the world to run software debugged by Artificial Intelligence (AI).
Called SapFix, the company describes it as an “AI hybrid tool” that can be used in conjunction with the Sapienz automated Android testing tool originally developed by university researchers but taken in-house by Facebook some time ago.
Sapienz finds the bugs in the code that might cause something like a crash or perhaps even a simple security vulnerability – and this is the new bit – SapFix fixes them. Beams Facebook:
To our knowledge, this marks the first time that a machine-generated fix – with automated end-to-end testing and repair – has been deployed into a codebase of Facebook’s scale.
How does AI do this?
From Facebook’s description, the workflow begins by trying to revert the code back to the state it was in before the bug that caused the problem was introduced.
If it’s a more complex issue, SapFix looks at a collection of “templated fixes” built up from those made by human developers over time.
If even this won’t work, SapFix sets about what Facebook calls a “mutation-based fix” whereby it starts making small code modifications to the problem statement until it thinks the bug has been mitigated.
Finally, it creates several versions of the fix to see whether each solves the issue by running them through the separate Sapienz testing tool. Then, and only then, the system sends its solutions to a human being for review.
So far, SapFix is only in the proof-of-concept phase, which is why no fixes are implemented without a human making that decision. But it seems to work:
Since we started testing SapFix in August, the tool has successfully generated patches that have been accepted by human reviewers and pushed to production.
This sounds more like automated problem-solving for humans than true AI, which would be autonomous – presumably why Facebook describes it as a “hybrid” of both worlds.
The question is how far the AI decision making could be pushed. Logically, the next step would be to allow whatever SapFix turns into to make bigger decisions – the first fateful step on the road to the machines-programming-machines of science fiction paranoia.
This won’t happen soon because it might change the nature of the human accountability that is still important in software management. And if programmers aren’t doing simple grunt work like this, will they stop understanding the software they are designing?
It’s a future that might see programmers becoming the people who simply build the AI systems that do the real work. Or perhaps even those will be built by AI too.
But let’s not get carried away. The company hopes to offer SapFix to other developers on an open source basis, which could give the underlying tech a big bump. For now, this is not the beginning of Skynet, just a faster way to get Facebook’s Android app out the door.
Ralph Hartwell
Which raises the most interesting possibility of automated malware changes and updates on individual computers as the AV programs try to defeat the malware. Spy vs. Spy!
delayedthoughtengineering
Indeed, with malware frequently using some sort of software mutator to avoid one-time classification and quarantine, creating entire families of related malware, it’s only a matching step that anti-malware and bug-hunting software incorporate automation on the same level. This bug-hunting automation is a key element of DevOps, a relatively new software development method that is focused on heavy automation and baked-in security.
Paul Ducklin
The anti-virus industry has used automated techniques to mop up automatically generated malware for very many years. (Let the machines do the repetitious work so the humans can concentrate on proactive detection so that the automated systems have less to mop up in future so the humans get even more time…and so on.)
There’s a big difference here in that generating permuted versions *of what is effectively the same software* – which is what malware authors do with varying degrees of reliability, for all that they care – is not at all the same thing as automodifying software to have carefully different behaviour specifically to avoid bugs.
The malware authors’ mutation engines [a] sre doing a much more predictable sort of transformation that changes the look but not the function of the code [b] are churning out lots of different versions, so occasional bugs affect very few people, making the process feel more reliable than it really is [c] are part of a piece of malicious software that’s already designed to screw over your computer, so the author couldn’t care less if you get hit by a dud mutation.
So I hear your analogy, but I don’t think it quite fits in this case.
Hui Rong
Would it be accurate to say that Facebook implemented a version of genetic algorithm to perform bug fixes based on a fitness function developed by Facebook developers?
Anonymous
Of course it’s skynet.