Posted on

Improving Our Software With 5 Lightweight Processes You Can Adopt This Month

29 March 2018 by Phillip Johnston • Last updated 28 June 2022

This article comes from the March 2018 edition of our embedded systems newsletter.

We typically use the newsletter as a way to share interesting embedded news, summaries of new parts, libraries we’ve been investigating, and other miscellanea. However, we thought that this topic was worth sharing with a wider audience.

It is our sincere hope that your teams adopts at least one new process to help improve code quality.

Improving Our Software With 5 Lightweight Processes You Can Adopt This Month

It’s that time of year when the Barr Group releases their yearly Embedded Systems Safety & Security Survey results. Last year’s results were eye opening, as nearly 50% of respondents reported not using static analysis and 36% reported that they do not perform code reviews. The 2018 results were no better:

  • 38% of safety-critical products don’t comply with a formal safety standard
  • 43% of teams working on safety critical products don’t perform code reviews
  • 41% of of teams working on safety critical devices don’t perform regression testing (54% for IoT product teams)
  • 33% of teams working on safety-critical products don’t perform static analysis (49% for IoT product teams)

These numbers are even more alarming given the fact that 25% of the reported “internet-connected devices” could kill or injure people if hacked. 22% of respondents mentioned that security for connected devices wasn’t even on their to-do list. We’re becoming increasingly connected, but our standards for safety, testing, and verification are not keeping pace.

I want to be clear: this is unacceptable.

Many teams skip crucial development processes and justify it with scheduling pressure or by blaming the boss. There will always be scheduling pressure, so we must adjust our approaches or we will continue to flounder. Bugs are expensive in time, money, and morale. Debugging accounts for 40–50% of most project costs and schedules. We all make mistakes. Anything we can do to keep bugs out of our code or catch them as early as possible will save time and money. Our goal is to identify and resolve defects as early as possible. As McConnell points out, the longer a defect remains, the more expensive it is to correct.

I’ve selected five simple quality improvement processes that your team can adopt over the next month. These processes are cheap or free, apply across languages and platforms, and best of all – they work. While each process requires a bit of time to get up and running, there is little-to-no maintenance involved in continuing to use them.

Here are five lightweight processes for improving code quality and identifying problems early:

  1. Fix All of Your Warnings
  2. Set Up Static Analysis Support
  3. Measure and Tackle Complexity
  4. Create Auto-formatting Rules
  5. Have Your Code Reviewed

Fix All of Your Warnings

The first thing I do when working on a new project is fix all of the compiler warnings. It’s amazing to me how developers will ignore warnings or rationalize their presence. Occasionally you even run into a team who will fight tooth and nail to prevent you from fixing them!

The compiler knows the programming language better than you ever will. You should not ignore the compiler when it is alerting you to an issue. The way you are using the language is dangerous and likely has unintentional side effects. Depending on the warning, you might be introducing undefined behavior into your software. That’s not our idea of quality software.

If you have warnings in your code base, fixing them is one of the fastest ways to improve quality. You will fix bugs and flaws in your program, regardless of whether or not they are currently problematic.

For more on compiler warnings:

Set Up Static Analysis Support

Static analysis tools provide us with even better feedback than the compiler. Your compiler may happily allow some problematic cases which are legal in language, such as out-of-bounds pointer accesses or missing initialization values. Your analyzer will catch these problems, and also report red flags such as unused or redundant code. When used throughout the development cycle, your static analysis tool will help you catch and prevent latent problems even earlier than your testing cycle.

Some governmental and industrial organizations now require static analysis data for certification processes. Companies such as PRQA provide tools that can check for compliance with safety critical standards in a variety of industries.

There are many free static analysis tools available and their commercial counterparts are also inexpensive (most are less than $1000). At Embedded Artistry, we use Clang’s static analyzer alongside clang-tidy.

Here are some resources you can use to find a static analysis tool that fits your needs:

Measure and Tackle Complexity

By the time you’ve eliminated warnings on your project and cleaned up glaring problems exposed by your static analysis tool, you’ve already made significant progress with software quality. The next goal is to measure complexity in your software. Because highly complex functions tend to be hard to understand, test, and maintain, these functions are prime candidates for refactoring and simplification.

By using a metric to measure complexity, we have a quantitative way to evaluate our code and identify pieces that need special attention. We can see how our changes are impacting the code base over time and trigger automatic alerts and reviews whenever a threshold is exceeded. We can focus our code reviews on functions with high complexity scores, making sure they get the most of our limited attention. Metrics aren’t perfect, but they increase our insight into our software quality.

These are the simplest and most popular metrics for measuring code complexity:

  • Lines of code (LOC): a count of the non-blank, non-comment source lines in a function, module, or project
  • McCabe cyclomatic complexity (MCC): provides a complexity score based on the number of branches (e.g. conditional statements)
  • Strict cyclomatic complexity (SCC, CC2): expands MCC by considering the number of conditions within each branch, which provides an approximation for the number of test cases needed for full coverage

These free tools will calculate complexity metrics for C/C++. We currently use Lizard at Embedded Artistry.

For more on software complexity:

Create Auto-formatting Rules

Automated code formatting might seem like a strange recommendation to put into the top five, but it serves three purposes:

  1. Automated formatting reduces a programmer’s cognitive load by eliminating an entire category of details and decisions they need to keep in mind
  2. Automated formatting improves the quality of our peer code reviews (the next recommendation) by eliminating arguments about style
  3. Automated formatting is the first step toward implementing and enforcing a coding standard

Every team that I’ve encountered with a written style guide inevitably ignores those guidelines, and multiple programming styles run rampant. Instead of relying on developers to constantly keep an arbitrary set of rules in mind, we can automate the process to make it simple and impersonal. At the very least, it’s worth eliminating the pointless, time-wasting arguments that cause friction within our teams.

We use clang-format on Embedded Artistry projects. Uncrustify and Astyle are other popular code formatting tools.

For more on automated code formatting:

Have Your Code Reviewed

Writers accept the fact that first drafts are generally garbage and need heavy editing. Before I send this newsletter out into the world, it has usually gone through 2–3 self-editing sessions and 1–2 peer reviews. Along the way, the newsletter is trimmed and restructured. The result is a much better product than the initial draft.

Yet, for some reason, programmers seem to think that perfect code is produced on the first try. The 2018 Barr Group survey results show that 54% of IoT product teams don’t perform regular code reviews. The survey results also show that a painfully scary 43% of teams working on safety-critical software don’t perform regular reviews.

Perfect code on the first try might be possible if you’re a prodigious programmer. But remember: even the famed author Ernest Hemingway had an editor. A second set of eyes can identify flaws that you missed in your first pass. Another developer may have different experiences that provide insight into the merits or risks of your approach. The architect on your team probably has input on how a module should interface with other pieces of the program. The “ego effect” also comes into play: knowing that our code will be presented and reviewed by another human can dramatically improve the overall quality. We will spend time cleaning up and checking the logic before putting code up for judgment.

Code reviews can waste time and become unproductive if poorly implemented. Best Practices for Peer Code Review provides some excellent tips for getting started. Notably, a lightweight review process is more efficient and practical than long, in-depth reviews with multiple developers. Even performing reviews on only 20–33% of the submissions provides benefits due to the “ego effect”. While 20% may seem low, remember that we are aiming for achievable: reviewing 20% of the source code is definitely better than none.

I highly recommend implementing peer code reviews after setting up automated code-formatting. This helps constrain code review discussions by preventing them from devolving into style nit-picking. If you’ve set up static analysis, make sure the analysis tools are used prior to code reviews.

For more information on code reviews:

The Keys to Success When Adopting New Processes

When adopting new processes, it’s important to focus on implementing one at a time. Adopting new processes in stages ensures that you have time to correctly implement each new technique before moving on to the next one. By implementing too many changes at once, you are likely to overwhelm your team and evoke a mutiny.

If you’re leading a team, it helps to find someone who is excited and can help you champion the idea. Empower that person so they can demonstrate the benefits of the new process to your team. Back them up when there is pushback. Change is always hard. Expect resistance, but don’t let it stop you.

The key to making these new processes stick is to make them as automated as possible. There is never a case where it isn’t worth the time it takes to automate a development process. Automation ensures that the process is easy to follow and always happens, rather than trusting individual contributors to remember to follow a process. Automation also makes the process less personal. The rules are clearly defined and are being enforced by a tool. De-personalization helps us view the situation dispassionately, rather than as an attack on our abilities.

To recap, when you implement new processes:

  1. Adopt one new process at a time
  2. Empower a process champion on your team
  3. Automate!

Newsletter

If you enjoyed this article, sign up for our monthly newsletter to receive interesting embedded tidbits, articles I’ve enjoyed reading, summaries of interesting new parts, companies looking for engineers, and other miscellanea.

Share Your Thoughts

This site uses Akismet to reduce spam. Learn how your comment data is processed.