Welcome to the September 2018 edition of the Embedded Artistry Newsletter! This is a monthly newsletter of curated and original content to help you build superior embedded systems. This newsletter is intended to supplement the website and covers topics not mentioned there.
This month we'll cover:
- Testing Embedded Systems
- Embedded articles from around the web
- Embedded Artistry website updates
Testing Embedded Systems
Embedded Artistry was founded with the goal of creating reliable, safe, and well-tested embedded systems. The sad fact is that most embedded software that we've encountered is low-quality and untested. Perhaps this holds true for most of the software industry - the continued procession of hacks, flaws, and errors is discouraging.
We can talk about why this is all day long, instead we want to focus on ways that we can all improve at our craft.
We'll cover three facets of testing embedded systems:
- Unit testing
- Debugger-based testing using metal.test
- Phil Koopman's lectures on embedded software quality and testing
This is not an exhaustive list, and there is much more to say about software quality that we’ll continue to cover in future articles and newsletters. For today, focusing on testing is a great place to start.
Most developers know they should be writing unit tests as they develop new features. If you're unfamiliar with the concept, unit testing focuses on testing the finest granularity of our software, the functions and modules, in an independent and automated manner. We want to ensure that the small pieces operate correctly before we combine them into larger cooperating modules. By testing at the finest granularity, we reduce the total number of test combinations that are needed to cover all possible logic states. By testing in an automated manner, we can ensure that any changes we make don't introduce unintended errors.
"A key advantage of well tested code is the ability to perform random acts of kindness to it. Tending to your code like a garden. Small improvements add up and compound. Without tests, it's hard to be confident in even seemingly inconsequential changes." -Antonio Cangiano (@acangiano)
Unfortunately, at Embedded Artistry we’ve only worked on a handful of projects that perform unit testing.
The primary reason for this is that many developers simply don’t know where to start when it comes to writing unit tests for embedded systems. The task feels so daunting and the schedule pressures are so strong that they tend to avoid unit testing all together.
After receiving a request for more resources on Test Driven Design (TDD) with embedded systems, we decided to share our favorite unit testing & TDD resources.
James Grenning and TDD
James Grenning has put a tremendous amount of effort into teaching embedded systems developers how to adopt TDD. He published an embedded systems classic, Test-Driven Development for Embedded C, and regularly conducts TDD training seminars.
James has written extensively about TDD on his blog. Here are some of our favorite posts:
- TDD Guided by ZOMBIES - walks through testing a circular buffer
- Physics of Test Driven Development
- Manual Test is Unsustainable
- I've got integration and system tests, why do I need unit tests?
- Preventing Brittle Tests
You can also watch these talks for an introduction into the how and why of TDD:
- Webinar - Test-Driven Development for Embedded Software
- NDC conference talk: Test Driven Development in C/C++
Phillip is signed up to take James Grenning's TDD webinar, which begins today. If you're interested in taking a training class, look into these links:
- TDD & Agile: Power Techniques for Better Embedded Software Development - scheduled for 1-3 October 2018 near Washington, DC
Matt Chernosky and Electron Vector
Matt Chernosky, who runs the Electron Vector blog, is an invaluable resource for embedded unit testing and TDD. If you are having a hard time getting started with unit testing and TDD, Matt's articles provide a straightforward and accessible approach.
Here are some of our favorite articles from Matt's blog:
- Mocking hardware interfaces using Ceedling and CMock
- Practice writing code without the hardware
- When you want to unit test, create well-defined software units
- When you want to unit test, abstract the hardware
- Event-based interfaces for testability
- Avoiding mocks by enqueueing events
- Test-driving with mocks instead of hardware
Throw the Switch
Aside from creating testing tools, Throw the Switch maintains a library of test-related articles and a course called Unit Testing & Other Embedded Software Catalysts. Additional courses related to unit testing for embedded systems are being developed.
Unit Testing Frameworks
Listed below are frequently recommended unit testing frameworks for C and C++. There are more unit testing frameworks in existence than we can ever review, so again this list is not exhaustive. If nothing jumps out at you in this list, keep looking to find one that fits your team’s development style.
Cmocka is the C unit testing framework we started with. Cmocka ships with built-in mock object support and operates similarly to Unity & CMock. The framework is built with C standard library functions and works well for embedded systems testing.
Catch appears to be the most popular C++ unit testing framework. Catch is a header-only library which supports C++11, C++14, and C++17.
Doctest is the unit test framework we use for EA’s C++ embedded framework project. Doctest is similar to Catch and is also header-only. Our favorite attribute of Doctest is that it keeps the test code alongside the implementation code. Doctest also enables you to write tests in headers, which Catch does not support.
GoogleTest is Google's C++ unit testing framework. GoogleTest is one of the few C++ frameworks with built-in mocking support.
CppUTest is a C++ test suite that was designed with embedded developers in mind. This framework is featured in James Grenning's book Test-Driven Development for Embedded C. C++ features within the framework are kept to a minimum enabling it to be used for both C and C++ unit testing.
Embedded Debugger-Based Testing
We always want to run as many tests as possible on a host PC when unit testing embedded systems code. However, we can’t test every aspect of our system on a host machine. Before shipping the final system, we need to evaluate the target compiler, issues which only present themselves on the target (e.g. endianness, timing), and the actual target hardware and hardware interfaces.
Metal.test is a framework which can help us with on-target testing. This project is maintained by Klemens Morgenstern, an independent contractor and consultant. Metal.test enables automated execution of remote code using debugger hardware, such as J-LINK or ST-Link. The project currently supports
lldb support is planned for a future release.
- I/O Forwarding
- Code Coverage
- Unit testing
- Call tracing
- Function Stubbing at link-time
- Extensive documentation (in the GitHub repository Wiki)
Metal.test also includes a plugin system. While it's not essential, plugin support enables developers to extend the functionality to support any use case that a debugger can support.
Klemens is looking for feedback on metal.test. Don't hesitate to reach out with questions, issues, or other feedback.
Phil Koopman on Testing & Software Quality
Phil has produced an immense and invaluable body of work, much of it focused on embedded software quality. The lecture notes for his embedded systems courses are available online, and he regularly posts lecture videos on his Youtube channel.
Here's a selection of his lectures that are related to testing and software quality:
- Embedded Software Quality: Why is it so terrible? What can we do about it?
- Unit Testing
- Integration Testing
- Testing Quality
- System Level Testing
- SQA isn't testing
Around the Web
NAND makers are striving for larger storage capacities. There are announcements of 512 Gbit, 1 Tbit, 1.33 Tbit capacities. SK Hynix is also working on a package with a whopping 8 Tbit capacity.
Jack Ganssle has started a blog at the end of July. Jack is also publishing a weekly series on embedded.com which expand upon his "Top 10 Reasons Embedded Projects Get Into Trouble". So far, he's worked his way up to number 7:
- 10: Not Enough Resources Allocated to a Project
- 9: Jumping into Coding Too Quickly
- 8: The Undisciplined Use of C and C++
- 7: Bad Science
You should also check out Phil Koopman’s slides for Avoiding the Top 43 Embedded Software Risks. How many risks has your team taken on?
We've added a list of our favorite software-related books to the Support page. If you're looking for a new programming or architecture book, you can support the website and newsletter by using the Amazon links found there.
Our popular Circular Buffers in C/C++ article was re-written based on learnings and feedback received over the past year.
The Embedded Artistry Technology Radar has received its first update with new techniques, libraries, and tools.
Three of our Resource pages have a brand-new look:
Our Glossary is always expanding and has been updated with new terms.
These articles were published on our website in August:
- Building a Team that Delivers Business Value
- Timeless Laws of Software Development
- EMB2: A C/C++ Framework for Multi-core and Multi-chip Embedded Systems
- Embedded Artistry's Technology Radar
These were our most popular articles in August:
- Circular Buffers in C/C++
- Jenkins: Configuring a Linux Slave Node
- Installing LLVM/Clang on OSX
- An Overview of C++ STL Containers
- Jenkins: Running Steps as sudo
- C++ Casting, or: "Oh No, They Broke Malloc!"
- Implementing an Asynchronous Dispatch Queue
- Migrating from C to C++: NULL vs nullptr
- A Simple Consulting Services Agreement
Thanks for Reading!
Have any feedback, questions, suggestions, interesting articles, or resources to recommend to other developers? Simply reply to this email!