Tag: Design for Change

27 July 2023 by Phillip Johnston

Another source of embedded software tight coupling is external libraries and frameworks. Using existing external libraries and frameworks significantly reduces your up-front development efforts, but it can also create problems for you in the future if you do not manage how your software interacts with these dependencies.
Some problems that can arise from tightly coupling your software to external libraries and frameworks are:

• Tight coupling to external dependencies can make code difficult (or impossible) to test in isolation
• External dependencies that use non-standard language features force you to use specific compilers or operating environments
• External dependencies may have dependencies of their own – transitive dependencies may end up tightly coupling your hardware to a specific platform
• Using dependencies that are coupled to a target processor architecture or SDK further couples your software to that environment
  • Some dependencies will be tightly coupled to a specific environment, such as the Arduino SDK
  • Some dependencies may assume a 32-bit processor
  • You may use a dependency like CMSIS, which provides an abstraction layer specifically targeted for ARM Cortex-M and Cortex-A5/A7/A9 processors
• Frameworks are “intrusive”. Applications built using a framework will usually become tightly coupled to the framework
• External dependencies are developed and maintained by third parties who are not aligned with your vision
  • APIs can change or become deprecated, forcing you to update the interacting code when updating dependency versions
  • Changes in the upstream project may cause you to change dependencies
  • Maintenance or development may also stop altogether, forcing you to change

You must pay careful attention to the libraries and frameworks you select for your systems. Libraries targeted just for one platform, such as Arduino-specific libraries, may do exactly what you need, but they will tightly couple your implementation to a single platform. Ideally, you will select libraries that are already designed for use on multiple platforms. These libraries often provide an abstraction layer that you must implement to get the library working on your target system. But no matter how portable the dependency is, ensure that your software is not tightly coupled to external dependencies so that they can be easily swapped out in the future. Based on the systems we have worked on, odds are low that all of your external dependencies will remain fixed throughout your system’s entire lifetime.

References

• Tight Coupling Makes Change Difficult
• Musings on the Tight Coupling Between Firmware and Hardware
• Prototyping and Design for Change: Lightweight Architectural Strategies

27 July 2023 by Phillip Johnston

While a large amount of bare metal embedded software is still being developed, the vast majority of projects we’ve worked on have involved an operating system (typically an RTOS), such as FreeRTOS, ThreadX (now Azure RTOS), or Zephyr. Software will often directly use the OS APIs and types throughout the program, which results in tight coupling to the OS.

While these RTOSes provide similar capabilities, each one is different. Migrating from one RTOS is more difficult than a simple find-and-replace operation, since the functionality, configuration options, error conditions, parameters, and API style often differ between RTOSes.

Coupling to an OS still has a significant impact on your software system’s ability to respond to change:

• If a processor change occurs, you may find that the RTOS you are using does not support the new processor. Then you must switch to a new OS (changing all OS API/type references) or go through the effort of porting the OS to the new platform.
  • You may also find that the vendor’s SDK is based on a different OS (e.g., Zephyr), in which case, you may be forced to switch OSs for practical reasons.
• When a module directly uses OS API calls, you can only reuse it with another OS after a porting effort.
• Code that depends on an RTOS is much more difficult to test independently and to test off-hardware, since the OS calls need to be mocked. Testing may be difficult to do, so teams often leave OS-dependent code untested (or only tested on the target).

Directly using OS API calls and types in your software results in tight coupling to the OS. Eliminating this source of coupling makes software resilient to changes in the underlying OS, improves its portability and reusability, and positions us to write automated tests off-target.

References

• Processor dependencies make embedded software difficult to change

• Tight Coupling Makes Change Difficult

• Musings on the Tight Coupling Between Firmware and Hardware

• Prototyping and Design for Change: Lightweight Architectural Strategies

• 5 Tips for Developing an RTOS Application Software Architecture by Jacob Beningo

  One problem that I often see in RTOS based software architectures is that developers select their RTOS and then build their entire software architecture around it. While at first glance this doesn’t seem like a bad idea, it can become quite a headache to maintain or port that code to other applications where a different RTOS may be used. Think about it, there is no RTOS interface standard that RTOS developers follow (even though Arm does have CMSIS-RTOSv2). Instead there are about 100 different RTOSes each with their own API. Switching RTOSes requires every RTOS API call to be found and changed which is probably time consuming.

  A better way to design the architecture is to make the application agnostic to the RTOS. Most RTOSes provide similar functionality anyways, so the putting the RTOS behind an Operating System Abstraction Layer (OSAL) allows the specific RTOS selected to be deferred until much later, decreases application RTOS dependence and improves the architectures flexibility.

• Building Software for Portability by Greg Blackham (1988)

  Different operating systems provide different services to an application. This is a veritable Pandora’s box of problems for which there are no easy solutions.

  Each operating system has a distinct set of possible error conditions that can be encountered by an application. Many of these extend across all environments. For example, every operating system returns a “file not found” error when an “open” system call fails because the file does not exist. Each environment may also have its own unique set of messages. On multiuser or networking systems, an “open” might also fail because the user does not have rights to access the file, or because the file is in use by another user. Error-handling code must be flexible to handle the various exceptions appropriately.

27 July 2023 by Phillip Johnston • Last updated 15 August 2023

Processors aren’t the only hardware to which embedded software becomes tightly coupled – you must also look out for tight coupling to a specific hardware platforms (i.e., your custom PCB).

Some forms of coupling to hardware platforms are explicit. Embedded software often interacts directly with the hardware components that are installed in the system by invoking driver APIs that are specific to a component. For example, most embedded developers will make direct API calls to a BMA280 or LIS3DH accelerometer driver instead of a generic accelerometer interface. If a particular component ever changes, all code interacting with its driver must be updated to use the new component.

Coupling to a hardware platform can also be subtle – you may develop your software based on assumptions about the current hardware design. For example, you might write your code in such a way that it expects there will be exactly four temperature sensors. But these types of details can change in future designs: your team might realize that four sensors are too many and cut that down to two, or your team might realize that one sensor is not enough and add more sensors to the system. Hard-coding these assumptions into your software can cause far-reaching modifications when the initial assumption changes.

Embedded software can also become tightly coupled to specific PCB revisions. Development hardware is expensive to produce, with limited availability, so your software typically must support multiple hardware revisions in a single code base. Luckily, you can drop support for the array of development hardware versions once you ship the product to customers. However, you might make hardware changes after the initial release, and you must support all production versions for the remaining life of the product. Improperly supporting multiple revisions, such as with #ifdefs and conditional logic statements scattered throughout the code base, leads to an explosion in complexity.

Embedded software will always involve software components that are tightly coupled to the underlying hardware. But when you do not properly manage coupling, it becomes increasingly difficult for your software to respond to changes in the underlying hardware design. If your product survives long enough, you will encounter hardware changes: a new board revision, a component that becomes obsolete, or a long-term component shortage that sends your team scrambling for replacement parts. If your software is tightly coupled to the hardware, these changes will trigger extensive rework in the software design.

References

• Tight Coupling Makes Change Difficult
• Processor Dependencies Make Change Difficult
• Musings on the Tight Coupling Between Firmware and Hardware
• Prototyping and Design for Change: Lightweight Architectural Strategies
• A Procedure for Designing Abstract Interfaces for Device Interface Modules by Parnas, Britton, and Parker
  

  Problems in confining change are caused by three types of errors:

  1. The device interface module allows user programs to exploit special characteristics of a particular device so that user programs must be revised if the device is replaced.
  2. The virtual device lacks essential capabilities, so that user programs must access the actual device directly; again user programs must be revised if the device is replaced.
  3. Programs that are not necessarily device-dependent are included in the device interface module. As a result, the device interface module may need to be changed if the requirements change even if the device is not changed. Furthermore the module will be harder to change when the device is changed.

27 July 2023 by Phillip Johnston • Last updated 15 August 2023

Processor dependencies are the most common form of tight coupling we find in embedded software, and it is also the most problematic.

Coupling to the processor manifests in these ways:

1. Using processor vendor SDK APIs or processor-specific headers and types throughout the project
2. Making direct register accesses throughout the project
3. Using libraries and frameworks that are target architecture- or vendor SDK-specific
4. Making assumptions about endianness
5. Making assumptions about data alignment
6. Making assumptions about word size
7. Use of special machine instructions

Processor coupling starts innocently enough. Processor vendors have invested heavily in SDKs to encourage developers to pick their chips by dangling the carrot of “rapid prototyping and development.” Embedded teams want to get started with development as quickly as possible, and vendor SDKs allow them to do just that.

In the most common cases, teams will develop software using a vendor’s SDK without an intermediary abstraction layer. Processor-level function calls are commonly intermixed with application logic and driver code, ensuring that the software becomes tightly coupled to a specific processor. The rapid, early progress feels good, especially since the team has saved a significant amount of time by leveraging the vendor’s development efforts.

There are many reasons processors change. The longer your product is maintained, the more likely you are to run into one of these scenarios:

1. The team is required to migrate to another processor or vendor. Coupling to a specific vendor’s SDK will trigger a rewrite of most of the system.
   • Migration can happen for many reasons: insufficient supply to meet production requirements, processor EOL, reducing power consumption, increasing processing power, requiring additional peripherals, accessing new features, or supporting multiple platforms/customers/environments.
2. The processor vendor updates their SDK, making subtle changes that break the existing software, causing the team to make adjustments throughout the program when updating SDK versions (or, in an even worse outcome, causing the team to avoid updates altogether).
3. The processor vendor changes their entire framework, making it impossible to update SDK without rewriting the software.
   • An example of this can be seen with Nordic’s deprecation of the nRF5 SDK, which is no longer actively maintained. Users must migrate to the newer nRF Connect SDK, which is based on Zephyr.

In other cases, software becomes dependent upon the underlying processor architecture. Your embedded software may work on an ARM system, but not be readily portable to PIC, AVR, or x86_64. It can also subtly manifest in the types used for interfaces and data structures: developers who are working on 32-bit processors write 32-bit code, which causes problems or inefficiencies when porting the software to a 64-bit processor (or, perhaps less commonly nowadays, porting to an 8-bit processor).

A subtler form of processor coupling can occur even when using abstraction layers. Teams can create abstractions which depend on a specific feature, an operating model particular to a single vendor, or a processor architecture-specific interaction.

You can also couple your software to the processor using architecture-specific libraries and abstractions, which we’ll cover when we discuss library and framework dependencies.

Directly calling a processor vendor’s SDK or HAL in your application is an anti-pattern. De-coupling firmware from dependencies on the underlying processor is one of the most important activities when designing your embedded systems for change.

References

• Tight Coupling Makes Change Difficult

• Library and framework dependencies make embedded software difficult to change

• Musings on the Tight Coupling Between Firmware and Hardware

• Prototyping and Design for Change: Lightweight Architectural Strategies

• Building Software for Portability by Greg Blackham (1988)

  Architecture issues arise from the instruction set available on the specific processor on which the application will run. Each processor has certain idiosyncrasies that you must consider when you write portable code. [PJ: Such issue might include endianness, address sizes, word sizes]

• Portability by Design by Michael Ross

  • Notes on things that change across processors:
    • Alignment of data.
    • Special machine instructions, such as transcendentals.
    • Size of various fundamental data types.
    • Endianness
    • Floating-point format. [PJ: perhaps not as much anymore – we’ve standardized pretty well on IEEE 754]

  Your code should contain safety checks to prevent you from relying too much on a particular word size or floating-point representation.

  Alignment of data is a problem mainly when your application depends on transmitting binary data across platforms, or uses different language processors on the same platform. Aside from the obvious endianness difficulties, there are variations in field alignment. This often necessitates representing the binary data in textual form or having a data broker perform the necessary transformations between the two architectures. Consider the “bad” C code in Example 1(a), which was intentionally written to demonstrate alignment problems. The alignment of the various structure fields will vary from one compiler to another and from one architecture to another. Imagine trying to write this to a binary file and read it back on another platform! Even with the same language processor, this structure will vary in size, due to memory “holes”; for example, on SPARC sizeof(mystruct)=24 bytes, on 80386 sizeof(mystruct)=16 bytes, and on HP-PARISC sizeof(mystruct)=24 bytes.

27 July 2023 by Phillip Johnston

One of the most pernicious problems in software engineering is tight coupling, and it contributes significantly to making software difficult to change.

Coupling is a qualitative measure of how much one component or module depends on other components or modules in the system. When changing one component requires corresponding changes in another software component, those two components are tightly coupled. When changes in one component rarely (or never) requires changes in another component, those two components are loosely coupled.

Tight coupling has many observable consequences for your software:

• Individual modules and components are more difficult to reuse. To reuse a tightly coupled piece of code, all of its dependencies must be satisfied in the next system. This can introduce a sufficiently high barrier for reuse, making it more expedient to re-implement the functionality than reuse the proven solution.
• Individual modules and components are more difficult to test in isolation – you must bring in all the dependencies in order to test the module, and testing without hardware becomes difficult or impossible. When testing is difficult, you’ll likely skip it. Without automated tests, you cannot change and refactor your system with confidence.
• It makes the program more difficult to reason about, since changes in one module can affect multiple modules within a system in unexpected ways.
• Whenever there is a change in a tightly coupled dependency, you must propagate that change to all other coupled modules.
• Changing requirements, design decisions, and hardware components will typically trigger large or wide-ranging changes in the system.

Over time, these consequences add up and result in reduced system maintainability and flexibility of the system, quickly leading to software aging.

Of course, tight coupling has been the status quo in embedded software and it occasionally works out. But it is always a gamble. You’re betting on the idea that you won’t need to make fundamental changes to your system – and in our experience, that rarely holds true, especially as the complexity of the systems we build increases. In order for a tightly coupled system to avoid schedule delays, you must make a significant up-front time investment to ensure that the choices made at the beginning of the development lifecycle will be suitable for the product’s entire lifetime. Any changes in the system, even long-term, become much more expensive and difficult to make when dealing with tight coupling. Unfortunately, in today’s agile world, up-front design has been wholly de-emphasized. Tight coupling arises out of expediency, not deliberate planning and decision making.

Minimizing coupling allows components and modules to be used, changed, swapped, and tested independently from other components or modules in the system. These software properties enable true agility.

There are four common coupling sources that present challenges specific to embedded software development:

1. Processor dependencies
2. RTOS dependencies
3. Hardware dependencies
4. Library and framework dependencies

Even developers who pay attention to coupling in their own modules commonly ignore these coupling sources. However, as we’ll show in the following lessons, if you don’t aggressively resist these sources of coupling, they will eventually wreak havoc on your embedded software.

References

• Coupling
• Dependencies Make Reuse Difficult
• Musings on the Tight Coupling Between Firmware and Hardware
• Prototyping and Design for Change: Lightweight Architectural Strategies
• Timeless Laws of Software Development by Jerry Fitzpatrick
• Maintainability
• The complexity of our systems is increasing, but we are creating products with smaller teams