As part of my “7 Habits of Successful Programmers” series, today I’ll discuss some ways to ensure that static analysis is effective and sustainable.

strategy

To start from scratch, it’s best to start with your static analysis strategy. Some people wonder why this is important, but it is actually critical to a successful static analysis project. Your policy will cover things like which rules you should run, when to ignore them and when to fix them, how to suppress them, and so on. You also need to decide what code needs to be analyzed — all your legacy code that is sitting idle, which we’ll discuss shortly.

Without a consistent strategy, static analysis can quickly become an occasional Bugfinder, reducing the value you get from it. If you’re in a compliance business like healthcare or automotive, you have to do static analysis anyway, so you might as well do it right to get some value out of it.

Tool selection

It’s important to get the right tools. The tool needs to work in your code editor (IDE), in the language you are using, and on the operating system you are using. The tool should support both server and in-IDE execution (for developers) and Web-based reporting (for managers).

You need to be able to configure the tool to execute the rules you want (not just the rules the tool vendor wants you to run). Mature tools come with out-of-the-box configurations that allow you to use industry initiatives such as MISRA, OWASP, CWE, and PCI.

Being able to extend tools with your own rules, ideas, and best practices is also important for your long-term success. And don’t forget the tech support — can someone look around the noisy rules and help you solve them? Will you use open source and fix it yourself? Do you usually use vendor services to help get your projects off the ground and moving in the right direction?

These are all things to consider when choosing a tool. Don’t fall into the trap — it provides little useful information about whether a tool really does what you need it to do, static analysis tools just work differently.

The rules to choose

As mentioned above, it’s easy to fall into the trap of simply opening any rules in your static analysis tool. Some vendors actively encourage this, and some even require it! Everyone’s software testing needs are different. A default configuration will get you started, but to succeed, you have to make it your own.

A few things to do include turning off noisy rules and matching the severity level to what you actually do (you don’t want a tool to say something is critical when you think the severity is low). One not-so-obvious hint is to turn off rules that you like but aren’t going to fix today. Open the rule when you have time to resolve the violation, not before. Ignoring them when reported only teaches the developer that not all rules are important, leading to frustration and bad habits.

The rules you’re running should relate to issues you encounter in the field, issues you might reasonably expect, safety issues, things you find during code reviews, and any compliance programs — rules you have to run.

Inhibition of

Even if you have exactly the right rules, sometimes the context makes a particular static analysis not important in one part of the code but still important in another. You have several options here, the most valuable of which is to turn off the rule in frustration. If you know that the rule can provide value in some area, this is a bad choice. If your tool does not allow you to configure and suppress correctly, you have chosen the wrong tool – go back to step 2 above.

Another trap some people fall into is to do some manual management of the results and pass on the results they think are important. It’s labor intensive, it’s not scalable, it’s not maintainable, and you end up fixing the less important stuff and missing out on the more important stuff. You need a data-driven process that helps you assess which violations need to be fixed and which can safely be ignored, and then have a way to flag them so they don’t come back. Ideally, this can be done directly by the developer in the IDE where the code resides, or by managers, architects, and so on in a Web-based UI.

Some people put these inhibitions in a separate system. This has the advantage of never changing the code, but there is a risk that you will have to rework it when you branch or refactor, or have some synchronization problems. Others make changes to the code in the form of special comments. While this does require code changes, it’s persistent, accurate, and records suppression — which is great if you’re being audited. Consider your needs and choose the approach that best suits your environment. If your tool does not support either approach, go back to Step 2.

Legacy code

Legacy code is related to suppression. Deciding what to do with old code is important. Again, it depends on your needs, but I’ve described a few methods below.

Some organizations choose to fix static analysis violations in legacy code only if there is an unfinished field bug-report that touches those lines of code — nothing else in the file should be touched. If your code is really old, or the risk of change is high, this is a perfectly reasonable policy. It minimizes the risk.

Another idea is that anyway, when you’re in a file everything has to be fixed. This carries some risk of change, but if you have a good unit test suite (which we’ll cover next time), then you don’t have to worry so much. On the positive side, any code you touch will be up to date on required code standards and best practices. Stale code is less stale than the original code.

You can also take a hybrid approach, or simply ignore all old code prior to a certain date and just statically analyze the way forward. Figure out what you can achieve, and then do it. Just think about the risks and benefits of your proposed strategy.

Static analysis is one of the most valuable tools in your software quality/security Arsenal. Mastering it can bring tremendous value, but get it wrong and a generation of coders in your organization will be condemned to outdated quality and security processes. If you have any questions, let me know in the comments below, and I’d be happy to explore how static analysis can work.