An in-depth study of reported bugs has produced a list of the top 25 bug categories in software today. Those topping the list are decades old, showing us that we’ve a long way to go in the journey to creating quality software.
The Common Weakness Enumeration (CWE) project analysed reams of bug data from the Common Vulnerabilities and Exposures (CVE) database as part of its research. The CVE gets thousands of new bugs each year, and the CWE classifies them to help guide software analysis and testing.
In 2005, it began collecting these bugs into categories, building on internal work by MITRE (the company which began the CVE list). The idea was to publish a standard list of common software security weaknesses, giving developers and tools vendors a framework to work from when assessing software for security bugs.
This is the first CWE top 25 since 2011, and we were hoping for some analysis of the key movers, Top-Of-The-Pops style. Sadly, that’s not really possible because CWE changed its approach this time around. It remapped the CVEs to a broader list of categories. It also took a more data-focused approach by mining the National Data Vulnerability (NVD) database. The 2011 study used surveys and personal interviews with developers, security analysts, and vendors.
Still, there are some interesting findings. Buffer flaws (categorised as ‘Improper Restriction of Operations within the Bounds of a Memory Buffer’) topped the list this time around. This covers a range of evils including buffer overflows, which can lead to arbitrary code execution, and out-of-bounds reads, which can crash a system or access sensitive data (out of bounds read also gets its own entry in fifth place).
Cross-site scripting was second on the list. That this term, first coined by Microsoft in 2000, is still featuring heavily in real-world bugs shows how much work is left to do in teaching developers how to avoid it. HackerOne’s recent list of total report volumes and bounty payments per weakness type showed XSS leading the pack by far.
Third came improper input validation. This is a common problem for developers who don’t think about all the wrong ways that people could enter information into a system to manipulate it (such as entering negative numbers into an ecommerce shopping cart and crediting their account).
Fourth came information exposure, in which a program accidentally reveals valuable data to a user. This could be private messages or program configuration files that are supposed to be private and which could give attackers a way into the system. This category of bug came third on the Hacker One list.
The CWE list has its own weaknesses. The research project omits vulnerabilities found and fixed in online or bespoke internal services before a public release. Nevertheless, its sheer breadth and data-driven analysis make it a good litmus test for the kinds of bugs causing people the biggest security headaches today.