Why You Want to Measure Software Quality and Which Metrics You Should Measure

Software quality is ostensibly straightforward to define—but is it? It is, as long as the software conforms to its design and works as needed. Underneath that simple definition lies a host of complex factors that influence the quality of software you develop.

And what is software quality? Quality is more complex than it first appears, and it is not practical or possible to measure quality with a single value.
Software quality influences company revenue (for companies that develop software), so it's helpful to know how to measure and improve on it. There are standards that teams can abide by to ensure quality, such as the CISQ code standards. Software quality metrics also play a vital role in providing feedback on any software developed.

Furthermore, the shift towards Agile development promises to deliver working software faster and more often. It's important for Agile teams to ensure that quality remains high in a fast-paced development environment, and they can do this by using metrics.

In this article, I'll look at some vital metrics that can inform you on different aspects of software quality for a more holistic picture.

Measuring Different Software Quality Aspects

Since it's not feasible to provide a single measurement of quality, you can focus on metrics that provide insight on the four factors used in the CISQ code standards. Before you dive in, take a look at this article by SeaLights with a broad list of software quality metrics they recommend to measure.

Reliability

Reliability is the likelihood that the software operates without failing in a specified environment for a given duration. Since software that fails tends to make end users unhappy, it's important to measure reliability to avoid releasing software that ends up breaking down due to defects or malfunctions.

Measurements of Reliability

• Cyclomatic complexity—the more complex a codebase is, the greater the risk of errors slipping through to production which break the software because extremely complex code is difficult to test. Cyclomatic complexity is the number of linearly independent paths through a program's source code, and it is tied closely to reliability.
• Production incidents—you can judge a system's reliability by measuring the number of critical bugs found after the software's release date.

Efficiency

Efficiency refers to how well the application responds and uses its resources under different conditions. Good software must be able to operate under high load conditions and still respond with minimal latency to avoid frustrating end users.

Measurements of Efficiency

• User feedback—simulate a heavy software load and ask real customers (or ask people within the company to act as a customer proxy) how frustrated they feel when using the software under different load conditions. A simple pain scale from 1 to 10 can quantify user feedback.

Security

Security is an aspect of quality that indicates how well an application's coding and architectural practices protect information and data from security breaches. High-quality software does not experience serious security breaches.

Measurements of Security

• Source code analysis—use a tool to process code and check for known security vulnerabilities, such as SQL injections or cross-site scripting in web apps. Fewer vulnerabilities (ideally none) mean better software.
• You can also examine vulnerabilities by category to determine whether you need to implement a specific strategy for preventing certain defects. For example, the below image shows a software project that needs a strategy for preventing buffer overflows.

Maintainability

In fast-paced modern development teams, you should aim to build software that is easy to transfer between development teams and easy to adapt to other purposes. Software is better quality if you can adapt it to changing user requirements quickly and cost-efficiently.

Measurements of Maintainability:

• Lines of code—the number of lines of code directly affects software maintainability because more code means less maintainability. High-quality software is as concise as possible given its requirements.
• Maintainability index—this metric is a compound metric that includes cyclomatic complexity, total lines of code, and comments ratio (ratio of code to comments) to measure maintainability.

Closing Thoughts

Software quality encompasses several factors, and organizations that develop software must conduct automated and manual tests which attempt to measure all aspects of quality.

Keeping track of useful software quality metrics provides the necessary insight to continuously improve the quality of the software your company releases.

However, it's important to remember that with software quality there are many contributing factors, and keeping track of all underlying metrics and tests can quickly become overwhelming. There is a need for a centralized dashboard that holistically displays all relevant software quality information.