Public Entry

Callback Function [CB]

16 May 2022 by Phillip Johnston • Last updated 24 May 2022

A callback is a reference to a function (or function-like object) that is passed as an argument to another module. The expectation is that the supplied reference will be executed whenever a specific condition occurs (typically, a completed event).

Callbacks can be categorized as synchronous (aka “blocking”) or asynchronous (aka “deferred”), which refers to how the callback function is invoked.

  • Synchronous callbacks are invoked before a function returns and in the same thread context as the calling function.
  • Asynchronous callbacks are often added to a queue of some time, and (depending on the system’s properties) are likely to execute after the invoking function returns, possibly in another thread of execution.

Callbacks involve the following primary components:

  • Storage for the callback
    • This can be a variable that holds a single value, a static array of values with a maximum size, or a dynamic list of values.
    • In some cases, you will encounter callback functions that are defined via weak linkage, or that are declared but not defined. In these cases, you are expected to provide a function with the same name.
  • A function that invokes the registered callback(s) at the proper point
  • A mechanism for registering the callback
    • This is typically a dedicated function, a constructor/initialization function parameter, or a field in a configuration struct.
  • A concrete implementation of the callback function
    • This code is application-specific and supplies the “missing” behavior.
  • Registration of the concrete callback implementation with the proper module

Callback Use Cases

Callbacks provide a mechanism for connecting modules together while reducing coupling between them. Callbacks are commonly used to separate application specific logic (related to the completion of an event or operation) from code that could otherwise be kept generic, such as a device driver, library, or communication stack. The calling code only knows about the callback abstraction and invokes stored values at the appropriate location(s). In this sense, callbacks are commonly conceptualized as providing decoupling between layers, since “lower” layers in a software stack can invoke actions in a “higher” layer without the need to know anything about the higher-level module.

Specific applications for callbacks in embedded software include:

  • Registering for a notification when a specific event happens (e.g. starting to record video, power state change, button pushed), enabling the application to take a specific action in response to the event.
  • Registering for a notification when an asynchronous event has completed (e.g. asynchronous USB transfer complete callback, DMA transfer complete callback), enabling the application to execute the next step without needing to monitor for the completion of the asynchronous event.
  • Registering callbacks as handlers for interrupt, timer, and other OS services.

Handling “Default” Cases

For code that uses callbacks, the implementer needs to consider the case where the users has not supplied a callback. Depending on the purpose of the module, any one of these options are suitable:

  • Only invoke callback(s) if they are registered, otherwise do nothing.
  • Execute a default operation if a user has not registered a callback.
  • Force an assertion if a user-supplied callback is required for proper operation.

Diving Deeper into Callbacks

Frameworks and Libraries

  • The Embedded Template Library provides reusable infrastructure for supporting callbacks in C++ applications. Basic support is provided by the Callback Service (associated tutorial). The ETL also provides a Delegate class that works as a static memory alternative to std::function (allowing you to register lambda functions, class member functions, static functions, and other functors as callbacks), as well as a Delegate Service that operates similarly to the Callback Service that uses Delegates.
  • Callbacks are typically used to implement the Observer Pattern
  • Callbacks operate similarly to the Template Method Pattern. For both patterns, tight coupling between modules can be handled externally by supplying an implementation for an optional customizable step. Callback operations are conceptually focused on customizing what happens when an operation is completed or event occurs, whereas the Template Method pattern is conceptually focused on customizing what happens during an operation. Given this, callbacks can be viewed as an application of Template Method.

References

  • Callback (computer programming) – Wikipedia

    In computer programming, a callback or callback function is any reference to executable code that is passed as an argument to another piece of code; that code is expected to call back (execute) the callback function as part of its job. This execution may be immediate as in a synchronous callback, or it might happen at a later point in time as in an asynchronous callback. Programming languages support callbacks in different ways, often implementing them with subroutines, lambda expressions, blocks, or function pointers.

    There are two types of callbacks, differing in how they control data flow at runtime: blocking callbacks (also known as synchronous callbacks or just callbacks) and deferred callbacks (also known as asynchronous callbacks). While blocking callbacks are invoked before a function returns (as in the C example below), deferred callbacks may be invoked after a function returns. Deferred callbacks are often used in the context of I/O operations or event handling, and are called by interrupts or by a different thread in case of multiple threads. Due to their nature, blocking callbacks can work without interrupts or multiple threads, meaning that blocking callbacks are not commonly used for synchronization or for delegating work to another thread.

  • Improving Your Callback Game

    Callbacks abound in embedded system design. You can find all sorts of use cases:

    • Registering for a notification when a specific event happens (e.g. starting to record video).
    • Registering for a notification when an asynchronous event has completed (e.g. asynchronous USB transfer complete callback, DMA transfer complete callback).
    • Registering callbacks as handlers for interrupt, timer, and other OS services.
  • Embedded Basics – Callback Functions | Beningo Embedded Group

     A callback function is a reference to executable code that is passed as an argument to other code that allows a lower-level software layer to call a function defined in a higher-level layer(10). A callback allows a driver or library developer to specify a behavior at a lower layer but leave the implementation definition to the application layer.

     A callback function is a reference to executable code that is passed as an argument to other code that allows a lower-level software layer to call a function defined in a higher-level layer(10). A callback allows a driver or library developer to specify a behavior at a lower layer but leave the implementation definition to the application layer.

    A callback function at its simplest is just a function pointer that is passed to another function as a parameter. In most instances, a callback will contain three pieces:

    • The callback function
    • A callback registration
    • Callback execution

    The code that will invoke the callback function within the module is often called the signal handler.

  • Increasing code flexibility using callbacks – Embedded.com by Jacob Beningo

    A callback is a function that is executed through a pointer where the function being pointed to contains application specific code. Callbacks are normally passed into a function and provides the developer with the flexibility to change the callback code without having to modify the calling function. For example, callbacks are often used in low-level drivers to access application specific behaviors in the application layer. The driver may not know what the timer interrupt should do so it makes a call through a callback to application level code that defines what the interrupt should do. This allows the driver code to stay the same and for the application level code to define its behavior.

  • Aaron’s Pithy Thoughts on Architecture

    Use Callbacks (or Generate Events)

    Another area where it is common to violate acyclicity is where the lower-level module needs to provide some signal back to the higher-level module. If there is only one higher-level dependent, it may be tempting to have the lower-level module call the higher-level one directly. This creates cyclical awareness. Don’t do this.

    Design the lower-level modules such that callbacks can be registered for events. Or, alternatively, design them to generate events to which higher-level modules can subscribe.

    Consider the nrfx_uart driver in the Nordic SDK. A high level module can initiate an RX or TX operation using functions like nrfx_uart_tx(). But … it gets notified of completion via RX_DONE and TX_DONE events. The nrf driver knows nothing about the clients using it.

    One good question to ask yourself when designing the lower-level layers is, “How easily could I pull this out and use it in another project?” If the answer is “not easily,” that is a sign of tight coupling and poor design.

Message Passing

9 May 2022 by Phillip Johnston • Last updated 11 July 2025

Message Passing is a technique for invoking behavior by sending messages to communicate what you want done, in contrast to the default approach of directly invoking methods (functions) on objects. While originally a property of object-oriented languages like SmallTalk and Objective-C, the paradigm is now used more generally for communication between objects, inter-process/thread communication, and communication. However, the paradigm has spread beyond that and is used for internal system communications, inter-process communications, and communications in distributed systems.

The general concept involves using messages to share data and communicate between different threads, objects in a system, or subsystems in a distributed system. Typically, messages are sent through an intermediary message queue. These messages may be read from the queue directly, sent across a network through a broker, or processed and dispatched by an event loop or message pump. 

Note

Sending messages via an intermediary queue may be referred to as “indirect” message passing. This is how message passing has been handled on most of our projects. “Indirect” message passing can be contrasted with “direct” message passing, where messages to be sent directly from one task to another, without the use of an intermediary queue. Some RTOSes provide this support.

Table of Contents:

  1. General Concept Overview
  2. Known Uses
    1. Patterns
    2. Eliminating Shared Global Data
    3. Messages as a Mutual Exclusion Mechanism
  3. References

General Concept Overview

An excellent resource for learning about message passing is this set of training slides from D. Kalinsky Associates: Messages for Intertask Communication (direct pdf access). This presentation provides a detail overview for message passing between threads, covering topics such as:

  • Indirect (using queues) vs direct (not using queues) message passing
  • Synchronous vs asynchronous message passing
  • Problems with message queues
  • How to use message passing even in places where semaphores have been traditionally used
  • Why you should do all of our communication with messages

Known Uses

In general, message passing is used for communication between objects, processes, or subsystems. It is also used to enable asynchronicity – that is, to eliminate synchronous/blocking aspects in subsystems by using a queue to submit requests. The asynchronicity aspect also helps ensure that requests are processed on the proper thread, and also enables repeated requests to be aggregated.

Patterns

Message passing is a core component of other patterns and frameworks:

  • Publish-Subscribe systems often use message passing through a central broker for communication.
  • Active Objects use message passing and an event loop for inter-object communication.
  • Event-driven architectures typically utilize message passing and event loops.
    • Miro Samek’s QP Framework relies on an event-driven paradigm with active objects: instead of invoking a method on another object directly, you post events to an object’s internal queue, which are processed whenever the object’s thread is given processing time by the scheduler.

Eliminating Shared Global Data

One of the most common reasons for using global data in an embedded program is the need to share data between different tasks, or between an ISR and the main program. Instead of using shared data (and the required locks or semaphores), you can keep data localized by passing information around with messages.

Further reading

The Problems with Global Variables explores other approaches that can be taken to avoid the use of global data.

As Charlie Munger famously said, “Take a simple idea and take it seriously.” Using messages to communicate between tasks is a simple idea that can be applied to practically any system you’re working on with beneficial impacts on the design of your program, including eliminating global data, decoupling modules from each other, and making modules easier to test in a standalone manner.

Messages as a Mutual Exclusion Mechanism

Messages for Intertask Communication provides an overview of how messages can be used as a mutual exclusion mechanism, avoiding the need for manual locking and unlocking:

Messages can also be used as a “mutual exclusion” mechanism — to ensure that 2 or more tasks do not access a “critical resource” at the same time. This can be seen in the diagram on this page. Here we see two tasks on the left, that would like to share the printer at the far right. When one task is printing, we don’t want the other task to join in and print at the same time — because that would trash the printer output. 

A traditional solution for this problem would be to set up a (binary) semaphore between the 2 tasks on the left. Each task would be required to “lock” the semaphore for itself before starting to print; and would “unlock” the semaphore when it finished printing. If a task would attempt to lock an already-locked semaphore, it would be made to wait until the semaphore became unlocked — thus ensuring mutual exclusion or exclusive access. This solution works, with only small problems. 

An alternative solution for this design would be to tell both tasks on the left that they’re forbidden to access the printer at all. Instead, if they want something printed they need to send a print request message to the task shown toward the right side of the diagram (in blue). This kind of task is sometimes called a “Resource Monitor Task”, since its job is to hide the detailed workings of the printer while at the same time performing services at the printer at the request of other tasks. Mutual exclusion and exclusive access are taken care of, since the “Resource Monitor Task” handles one printing request (message) at a time, even if a substantial queue of messages has developed. So one print job at a time will be printed, without interference from other print jobs. 

Which of these 2 solutions, the semaphore solution or the messaging solution, is more general? Answer: The semaphore solution is good in single-processor and/or single memory-partition environments. But the messaging solution works equally well in those environments and in multi-partitioned memory environments, as well as in multi-CPU multi-core and fully distributed environments. 

References

Non-Virtual Interface Pattern

24 March 2022 by Phillip Johnston • Last updated 5 April 2022

The Non-Virtual Interface (NVI) pattern controls how methods in a base class are overridden. Base classes use public, non-virtual functions that can be called by clients. Overridable methods are defined as protected, virtual members.

Problem

When creating an abstract base class, there is a tendency to make public methods virtual or abstract, allowing them to be overridden by subclasses.

When a base class method is overridden by a subclass, the subclass’s implementation may optionally call the base class method. However, this is not mandatory, and it is not easily enforced, so it is possible that key functionality implemented in the base class is not used in a subclass. This makes these classes harder to use correctly. Wouldn’t it be better if there was a way to enforce the use of key functionality in the base class?

Solution

The NVI pattern separates the core functionality of the base class from the public interface of the base class.

  • Public methods of the base class are neither virtual nor abstract, and cannot be overridden. Functionality that must be maintained in subclasses is contained in public members of the base class.
  • Public methods call protected, virtual members in the base class. These functions contain core functionality that may be overridden by subclasses, or they may be pure abstract functions that must be implemented by the derived class.

This pattern results in two distinct interfaces:

  1. The public interface used by clients
  2. The private interface, which is used, extended, and/or implemented by subclasses.
Note

While this pattern is usually used with class interfaces, the same ideas apply to library and framework interfaces.

Consequences

  • This pattern mitigates the Fragile Base Class interface problem.
  • This pattern requires minimal overhead to implement, but represents a reorganization of the base class’s functionality.

Known Uses

References

Generation Gap Pattern

24 March 2022 by Phillip Johnston • Last updated 5 April 2022

Separate generated code from non-generated code through the Template Method pattern. This ensures that customizing the generated code does not require users to modify generated code.

Note

This pattern was originally documented by John Vlissides, one of the four authors of the Design Patterns book. However, this pattern is not described in that work.

Aliases

This is a specialization of the Template Method pattern.

Problem

Generated code should never be directly modified by users. If generated code is modified, it cannot be safely regenerated without the user’s changes being lost. Instead, there needs to be a way for users to supply customizations or missing details without directly modifying generated code.

Solution

John Vlissides proposed that we use inheritance (or, as more generally interpreted, a Template Method) to create separation between the generated code and handwritten code that will modify the generated code. Generated code would be the base class, with modifications in subclasses.

We can generalize this suggestion to treating generated code as a Template Method, with modifications being made by users overriding customizable steps or implementing required steps.

Consequences

This pattern enables modifications to remain separate from generating code, allowing generated code to be re-generated safely whenever necessary.

However, this pattern is dependent upon the generator providing a suitable structure for users to leverage. Some generators do not provide this type of decoupling strategy. We recommend avoiding them whenever possible.

References

Template Method Pattern

24 March 2022 by Phillip Johnston • Last updated 16 September 2024

The Template Method pattern defines the skeleton of an algorithm or operation in high-level steps. Users or subclasses can override or implement the behavior of specific steps within the algorithm, but are not able to modify the general algorithm flow itself.

This design pattern is categorized as a “behavioral class” pattern in the Design Patterns book by Gamma et al., who summarize the pattern in this way:

““
Quote

Define the skeleton of an algorithm in an operation, deferring some steps to subclasses. Template Method lets subclasses redefine certain steps of an algorithm without changing the algorithm’s structure.

Table of Contents:

  1. Context
  2. Problem
  3. Forces
  4. Solution
  5. Consequences
  6. Known Uses
    1. Examples
  7. Variants
  8. Related Patterns
  9. References

Context

The intent of this pattern is to define and enforce the overall structure of an operation or algorithm, while allowing users or subclasses to refine, redefine, or implement specific steps. Users are not able to modify the overall algorithm or its structure (for that, you would use the Strategy pattern).

This pattern makes use of Inversion of Control. The library, framework, or base class controls the operation, but it invokes functions in user code or subclasses which implement individual steps.

Problem

The Template Method pattern can be used to address two common situations that arise in software development.

As you develop a system, you will often naturally develop functions or classes that are largely duplicated in their functionality, but varying in small individual implementation details. This often happens as system grows because a specific operation or algorithm needs to be extended or customized for different specific use cases. In other cases, it happens because existing library or SDK code needs to be adjusted to be better suit your system’s needs. In any case, wouldn’t it be preferable for the common code to be kept in a single place, eliminating the duplicated behavior across all of the implementations, while still preserving the ability to customize individual operational steps for specific use cases?

Note

That is the original problem that the Template Method pattern was targeted at.

Embedded software developers deal with the same problem described above, but also have another common challenge. Every embedded device is essentially unique in its combination of hardware and software dependencies: different processors, vendor SDKs, OSes, libraries, frameworks, hardware components, and PCB layouts contribute to completely unique products. A large amount of embedded code could be portable and reusable, but implementations are often tightly coupled to a unique platform due to the need to account for the unique hardware and software makeup of that platform. Wouldn’t it be better to structure embedded software in such a way that users could implement or override specific steps (such sending and receiving data over a SPI or I2C bus) to support their unique hardware/software dependencies, allowing the implementation to remain independent of a specific platform?

Forces

In these scenarios, you are trying to balance control with customization while minimizing duplication. You need to define an overall operation, but users or subclasses need the ability to vary individual steps in the operation. The naive approach is to customize the operation by making a copy of it and varying the steps as necessary. However, this leads to duplication of the common steps of the algorithm, and if a change has to be made it must be propagated to all of the implementations. There is also no enforcement of the operation across the duplicate implementations: they can deviate in behavior over time.

With the Template Method pattern, you can enforce control of the operational flow while giving clients the ability to customize individual steps of the operation. Common code can be kept in a single location, eliminating duplication. Thanks to the eliminated duplication and customizability, you increase the possibility of reusing the code for new purposes.

Reuse is also improved by the reduction of coupling that this pattern enables, which is specifically beneficial to the embedded scenario above. Embedded software is often tightly coupled to the underlying hardware/software platform. However, we can leverage the customization of individual steps to allow users to implement functionality specifically for their system outside of the primary implementation. This also improves the portability of the software.

Solution

In a base class, library, or framework, define a Template Method which contains the invariant code (the common code) for the overall algorithm or operation. The operation is broken down into a series of steps. The distinct steps, especially those that can or must be customized, are factored into individual functions. Individual steps may be controlled completely by the implementation, or they may be customizable by the user.

The customizable steps (sometimes called “helper methods” or “customization points”) of a Template Method can be further categorized in the following way:

  • Abstract steps, which do not supply a default implementation and must be provided by every subclass or application
  • Optional steps, which supply a default implementation but can be overridden if desired
  • Hooks, which are optional steps with an empty body that provide additional extension points for an algorithm
    • Hooks are often placed before and after crucial steps of an operation to provide additional extension points

In order to use the Template Method, subclasses and application code will supply any abstract steps and optionally override any optional steps. This mechanism ensures that the overall operation or algorithm flow is enforced while granting users the ability to override specific details.

The mechanisms used to implement this pattern depend on the chosen language. Common choices include:

  • Function pointers
  • Weakly linked functions
  • Inheritance
  • C++ Templates

Consequences

The Template Method pattern eliminates duplication across implementations by keeping the common operational code in a single location. The pattern also gives the base implementation control over the general algorithm while giving clients the ability to customize specific implementation steps. This improves reusability of the code, as users can modify steps to suit the situation. The pattern can also be used to decouple the operation from a specific hardware or software platform, since implementation details can be supplied through abstract steps that the user is required to implement.

Template Methods make use of Inversion of Control – the “lower-level” code (the library, framework, or base class) invokes “higher-level” code (an application or subclass). While useful, this can make the processing flow difficult to trace for those not familiar with the software, as some implementation details in the higher-level code will appear to be present but not used by the application. When debugging, this can lead to a “ping pong” or “yo yo” effect, where you are tracing back-and-forth between the code that defines the Template Method and the code that implements individual steps.

Users must be able to identify which operations they are required to implement (“abstract operations”) and which are optional. This is best addressed in the interface documentation. Users can also be aided by using a naming convention to help identify optional and required steps to implement. Failure to document these details reduces the likelihood that the code will be reused.

Gamma et al. point out (and we agree) that an important design goal when using Template Methods is to minimize the number of operations that a user must implement. The more operations that need to be overridden, the more tedious it is to use the code. Note that this is different from giving users the option to override specific steps that have a default implementation.

The indirection introduced by the use of this pattern will incur a slight performance penalty. In nearly all cases, this will have no impact on the system’s operation. However, if a Template Method is invoked frequently (i.e., in a hot loop), this pattern may introduce enough overhead that a direct function call would be preferred. The system should be measured to confirm the cause of the performance bottleneck.

Known Uses

  • This pattern is fundamental and is often used by abstract class hierarchies, libraries, frameworks, and SDKs to provide customization of default behavior or to provide methods that must be implemented according to the specific use case.
  • Template Methods can be used to decouple software from external dependencies, such as specific hardware components or operating systems. Users can supply platform-specific implementation details through template methods, allowing the original code to be decoupled from any particular platform.
  • Template Methods are often used within generated code to allow users to customize behavior without needing to directly modify the generated code. This is often called the Generation Gap pattern.
  • Template Method can be used to refactor classes or functions with a significant amount of code duplication, but variation in individual steps. Martin Fowler captures this process in the Form Template Method refactoring pattern.
    1. Identify the differences in the existing code.
    2. Separate the differences into new operations.
    3. Replace the differing code with a Template Method that calls the new operations.

Examples

Variants

  • The Generation Gap pattern is a specialization of the Template Method pattern to ensure users do not directly modify generated code.
  • Factory Methods can be viewed as a specialization of the Template Method pattern.
  • The Strategy Pattern can be viewed a larger-scale extension of the ideas behind the Template Method: Strategy varies an algorithm in its entirety, where Template Method is used to vary individual steps of an algorithm while enforcing an overall algorithm structure.
    • Others distinguish these two patterns by saying that Template Method uses inheritance while Strategy uses delegation (delegating the algorithm to another section of code). We find this particular distinction as artificially limiting, since the idea behind Template Method can be applied even without inheritance and can also be viewed as delegation.
  • The Non-Virtual Interface (NVI) Pattern is a way to implement the Template Method pattern
  • Factory Methods and Template Methods can often be found together, with a Factory Method being used as a step within a larger Template Method.
  • Callback Functions operate similarly to the Template Method Pattern. For both patterns, tight coupling between modules can be handled externally by supplying an implementation for an optional customizable step. Callback operations are conceptually focused on customizing what happens when an operation is completed or event occurs, whereas the Template Method pattern is conceptually focused on customizing what happens during an operation. Given this, callbacks can be viewed as an application of Template Method.
  • Martin Fowler describes a Form Template Method refactoring pattern. When you notice that two related methods perform similar steps in the same order, yet the steps are different, you can refactor to a Template Method pattern.
    • Gamma et al. describe refactoring using Template Method as well.

References

  • Design Patterns by Gamma et al.

  • Making Embedded Systems: Design Patterns for Great Software, 2nd edition, by Elecia White

    A related pattern is the template pattern. A template provides a skeleton of an algorithm but allows the steps to change (without changing the algorithm’s structure). Usually these aren’t function pointers; the steps are part of the organization of the algorithm. In our data-driven system, we could make a template that looked like the following:

    […]

    Even though each of these functions has the same prototype, they aren’t interchangeable, unlike the strategy pattern. Instead, they provide an outline of how the system works. An instance of this template for the system described would have the ADC sampling the data, the data being amplified, and the DAC outputting the result. The instantiation of a template may override certain (or all) parts of the default implementation. For instance, if the default implementation sets up the order “ADC-process-DAC,” a test version of the class may override the sample function to read data from a file while leaving the other two functions in place.

    […]

    Object-oriented software has the concept of inheritance, where an instance is-a concrete version of an object. Template patterns are like that (the template is a series of these steps). An alternative approach is composition, where the software has-a concrete version of an object. Strategy patterns are more like that (they offer a function to call). Composition is more flexible than inheritance because it is easier to switch what things have than what they are. On the other hand, building (composing) a system at runtime might not be a good use of limited resources. Balance the trade-offs for your system.

  • Template method pattern – Wikipedia

  • Design Patterns VS Design Principles: Template Method – Fluent C++

  • The Template Method – Modernes C++ by Rainer Grimm

  • Form Template Method – Refactoring Patterns

    You have two methods in subclasses that perform similar steps in the same order, yet the steps are different.

    Get the steps into methods with the same signature, so that the original methods become the same. Then you can pull them up.

Leaky Abstraction

26 January 2022 by Phillip Johnston • Last updated 27 January 2026

An abstraction is “leaky” when it exposes details about the underlying implementation to the users that should ideally be hidden away.

The term was coined by Joel Spolsky in The Law of Leaky Abstractions, where he states:

All non-trivial abstractions, to some degree, are leaky.

Abstractions fail. Sometimes a little, sometimes a lot. There’s leakage. Things go wrong. It happens all over the place when you have abstractions.

The existence of leaky abstractions means that abstractions do not always simplify our work in the intended ways. While we can often operate with the abstractions, we are not free from understanding the implementation beyond the abstraction. Eventually, a problem will appear, and we will need to look behind the curtain and learn the underlying details.

As Spolsky points out, one implication of this is that abstractions save us time working, but they don’t save us time learning. One tradeoff here is that we can build more complex systems more quickly, but debugging problems that leak from the abstractions can be a lengthy process.

Examples of Leaky Abstractions

Spolsky cites a number of examples in his article. For embedded developers, the most relevant is the idea that memory is abstracted as a big flat address space, but often this abstraction leaks. As Spolsky points out, iterating over a large two-dimensional array can have different performance characteristics depending on how you iterate due to potentials for page faults / cache invalidations or other underlying processor performance implications.

Many embedded systems abstractions designed around hardware are leaky because they encode information in the abstraction that is not actually generally applicable across different hardware components. When a component is swapped, the designers may realize that their abstractions leaked some hardware details that do not apply to the new component, resulting in changes to the interface and/or application.

References

  • Wikipedia: Leaky Abstraction

    A leaky abstraction is an abstraction that exposes details and limitations of its underlying implementation to its users that should ideally be hidden away. Leaky abstractions are considered problematic, since the purpose of abstractions is to manage complexity by concealing unnecessary details from the user.

  • Towards a New Model of Abstraction in Software Engineering by Gregor Kiczales

  • The Law of Leaky Abstractions by Joel Spolsky

    That is, approximately, the magic of TCP. It is what computer scientists like to call an abstraction: a simplification of something much more complicated that is going on under the covers. As it turns out, a lot of computer programming consists of building abstractions. What is a string library? It’s a way to pretend that computers can manipulate strings just as easily as they can manipulate numbers. What is a file system? It’s a way to pretend that a hard drive isn’t really a bunch of spinning magnetic platters that can store bits at certain locations, but rather a hierarchical system of folders-within-folders containing individual files that in turn consist of one or more strings of bytes.

    Back to TCP. Earlier for the sake of simplicity I told a little fib, and some of you have steam coming out of your ears by now because this fib is driving you crazy. I said that TCP guarantees that your message will arrive. It doesn’t, actually. If your pet snake has chewed through the network cable leading to your computer, and no IP packets can get through, then TCP can’t do anything about it and your message doesn’t arrive. If you were curt with the system administrators in your company and they punished you by plugging you into an overloaded hub, only some of your IP packets will get through, and TCP will work, but everything will be really slow.

    This is what I call a leaky abstraction. TCP attempts to provide a complete abstraction of an underlying unreliable network, but sometimes, the network leaks through the abstraction and you feel the things that the abstraction can’t quite protect you from. This is but one example of what I’ve dubbed the Law of Leaky Abstractions:

    All non-trivial abstractions, to some degree, are leaky.

    Abstractions fail. Sometimes a little, sometimes a lot. There’s leakage. Things go wrong. It happens all over the place when you have abstractions.

    One reason the law of leaky abstractions is problematic is that it means that abstractions do not really simplify our lives as much as they were meant to. When I’m training someone to be a C++ programmer, it would be nice if I never had to teach them about char*’s and pointer arithmetic. It would be nice if I could go straight to STL strings. But one day they’ll write the code “foo” + “bar”, and truly bizarre things will happen, and then I’ll have to stop and teach them all about char*’s anyway. Or one day they’ll be trying to call a Windows API function that is documented as having an OUT LPTSTR argument and they won’t be able to understand how to call it until they learn about char*’s, and pointers, and Unicode, and wchar_t’s, and the TCHAR header files, and all that stuff that leaks up.

    The law of leaky abstractions means that whenever somebody comes up with a wizzy new code-generation tool that is supposed to make us all ever-so-efficient, you hear a lot of people saying “learn how to do it manually first, then use the wizzy tool to save time.” Code generation tools which pretend to abstract out something, like all abstractions, leak, and the only way to deal with the leaks competently is to learn about how the abstractions work and what they are abstracting. So the abstractions save us time working, but they don’t save us time learning.

    And all this means that paradoxically, even as we have higher and higher level programming tools with better and better abstractions, becoming a proficient programmer is getting harder and harder.

    Ten years ago, we might have imagined that new programming paradigms would have made programming easier by now. Indeed, the abstractions we’ve created over the years do allow us to deal with new orders of complexity in software development that we didn’t have to deal with ten or fifteen years ago, like GUI programming and network programming. And while these great tools, like modern OO forms-based languages, let us get a lot of work done incredibly quickly, suddenly one day we need to figure out a problem where the abstraction leaked, and it takes 2 weeks.

Dependency Inversion Principle

6 August 2021 by Phillip Johnston • Last updated 5 July 2022

One fact you can’t escape from: some parts of your system are always going to be specific to the underlying system, whether it’s the hardware setup, an OS, a given processor, a particular library, or an external system’s interface. For a specific example, at some level, you do need to declare an IMU driver for a specific part. But most of your system probably doesn’t care about the specific IMU being used: it simply needs IMU samples provided within a specified time frame and in a specific data format. From this perspective, you can switch the IMU as needed as long as you can satisfy your program’s expectations.

Your goal is to isolate these types of dependencies. You can do this by using abstract interfaces to hide the implementation details so that the rest of the program can be more generic and not worry about these details (i.e., write code against YOUR abstractions, not theirs). By interacting with abstract interfaces, other modules can remain loosely coupled. This ensures you can easily modify and swap out the implementations of those interfaces without needing to change other parts of our system. If we are directly interacting with the implementations, our code has a higher chance of becoming tightly coupled to the implementation, especially if the implementation provides capabilities that are not exposed in the abstract interface.

Note

This even applies to our IMU driver example. It likely needs to communicate over I2C or SPI – but it should not be tightly coupled to the processor’s specific driver for that protocol. Instead, there should be an abstraction that sits between them so the IMU driver is independent of the processor it runs on

This idea is formally known as the “dependency inversion principle”, which states that:

  1. High-level modules should not depend on low-level modules. Both should depend on abstractions and interfaces
  2. Abstractions and interfaces should not depend on details. Details (concrete implementations) should depend on abstractions.
Note

The reason for the name is that the principle dictates that both high-level and low-level objects must depend on the same abstraction – inverting the way that most people naturally think about software, where high-level modules depend on low-level modules.

When this principle is applied to source code, it results in the dependency inversion pattern.

References

  • Wikipedia: Dependency inversion principle
  • C2 Wiki: Dependency Inversion Principle

    A. High level modules should not depend upon low level modules. Both should depend upon abstractions.
    B. Abstractions should not depend upon details. Details should depend upon abstractions.

    We wish to avoid designs which are:

    • Rigid (Hard to change due to dependencies. Especially since dependencies are transitive.)
    • Fragile (Changes cause unexpected bugs.)
    • Immobile (Difficult to reuse due to implicit dependence on current application code.)
  • Playbook: Writing Portable Embedded Software

    Portable code must be kept “generic” by using these generic interfaces, standard language features, and portable libraries. Portable code should never directly reference any platform-specific modules or details. This includes vendor SDK driver interfaces, RTOS-specific APIs, direct register accesses, and platform-specific debugging functionality. These items prevent us from easily porting our code from one platform to another and belong behind a barrier of some kind.

  • Use of Abstract Interfaces in the Development of Software for Embedded Computer Systems by David Parnas

    In summary, the procedure that we are discussing can be formulated in yet another way:

    1. Specify an Abstract interface embodying all the information shared by all of the possible interfaces
    2. Procure programs to meet this abstract interface
    3. Procure additional programs in order to meet the actual interface.

Adapter Pattern

21 July 2021 by Phillip Johnston • Last updated 29 August 2022

The Adapter pattern allows the interface of an existing module to be used as/with another interface. This is done by adding in a thin “adapter” module that maps the desired interface onto the existing interface.

Context

The adapter pattern helps us address the following problems:

  • How can a class be reused that does not have an interface that a client requires?
  • How can classes that have incompatible interfaces work together?
  • How can an alternative interface be provided for a class?

Problem

We want to reuse existing source code to reduce the burden of creating a system. This can be done quite easily when use decoupling techniques.

However, even when modules stand on their own, their interface often provides a challenge. When using an implementation’s interface directly, the system becomes tightly coupled to that specific implementation. To address that, the system may define its own abstractions for a given component or concept. However, existing implementations will not support this interface. How can we reuse existing code without becoming tightly coupled to a specific implementation through its interface?

Forces

Adapter balances the need to have system-specific abstractions and interfaces with the desire to reuse code.

Solution

Create an Adapter that simply performs an interface adjustment operation. The Adapter’s interface will match your system’s expectations. The Adapter implementation will be a “simple shim” that converts the internal API to the one provided by the target implementation.

Consequences

  • Adapters can enable reusability by giving us the ability to adapt existing code to work in our system even if there is a difference in interface requirements.
  • Adapters prevent “generic” system code from becoming coupled to specific implementations.
  • Adapters add a layer of indirection.
  • The Facade Pattern is similar to the Adapter in that it presents a unified interface to a subsystem. Facade can be used to adapt multiple components to a single interface, while Adapter is usually focused on a 1:1 adaptation.

References

Single Responsibility Principle

21 July 2021 by Phillip Johnston • Last updated 30 December 2025

The Single Responsibility Principle (SRP) states that a given module or component should have a single responsibility. Often, the SRP is described alternatively as “having only a single reason to change”, in accordance with Dave Parnas’s paper On the Criteria to be Used in Decomposing Systems into Modules:

We have tried to demonstrate by these examples that it is almost always incorrect to begin the decomposition of a system into modules on the basis of a flowchart. We propose instead that one begins with a list of difficult design decisions or design decisions which are likely to change. Each module is then designed to hide such a decision from the others.

Robert Martin provides another wording for the SRP:

Gather together the things that change for the same reasons. Separate those things that change for different reasons.

The SRP is a metric that can be used to judge the cohesion and coupling of a given module or component. You can easily identify components or modules that do not embody the SRP by looking for mixed metaphors.

Note

The SRP is the “S” in the acronym SOLID.

  • The SRP is an application of Separation of Concerns taken to the extreme.
  • High cohesion and low coupling is a realization of the SRP. As Robert Martin says:

    We want to increase the cohesion between things that change for the same reasons, and we want to decrease the coupling between those things that change for different reasons.

  • Robert Martin said that he came up with the SRP to combine Parnas’s idea of Information Hiding and Dijkstra’s idea of Separation of Concerns

References

  • SOLID
  • The Single Responsibility Principle (paper) by Robert Martin
  • Design Principles and Design Patterns by Robert Martin
  • The Single Responsibility Principle by Robert Martin

    And this gets to the crux of the Single Responsibility Principle. This principle is about people.

    When you write a software module, you want to make sure that when changes are requested, those changes can only originate from a single person, or rather, a single tightly coupled group of people representing a single narrowly defined business function. You want to isolate your modules from the complexities of the organization as a whole, and design your systems such that each module is responsible (responds to) the needs of just that one business function.

    However, as you think about this principle, remember that the reasons for change are people. It is people who request changes. And you don’t want to confuse those people, or yourself, by mixing together the code that many different people care about for different reasons.

  • Paper: On the Criteria to Be Used in Decomposing Systems into Modules by David Parnas
  • Patterns in the Machine : A Software Engineering Guide to Embedded Development by John Taylor and Wayne Taylor

    Remember that layers are your friends when it comes to unit testing, especially when implementing automated unit testing. It is also the key to decoupling code and preparing your software for future reuse. The Single Responsibility Principle is a tactical best practice, but it enables strategic best practices such as unit testing and platform-independent code.

  • One Responsibility Rule | C2 Wiki

    From BertrandMeyer‘s ObjectOrientedSoftwareConstruction, there was the statement (quoting from memory):

    A class has a single responsibility: it does it all, does it well, and does it only.

    […]

    One of the criteria I use is to try to describe a class in 25 words or less, and not to use “and” or “or”. If I can’t do this, then I may actually have more than one class. 

Separation of Concerns

12 July 2021 by Phillip Johnston • Last updated 5 January 2026

“Separation of concerns” is a software design principle that advises us to divide our application into distinct modules or sections, each of which addresses a separate “concern” or “feature”. The goal is to have as little overlap in functionality and scope between the divided concerns. A concern is typically considered to be separated if it is localized in a single component of a system, whether that is a file, class, container, or function. A concern is typically considered to be separated if it is localized in a single component of a system, whether that is a file, class, container, or function.

The goal of applying separation of concerns to our software design is to have as little overlap in functionality and scope as possible among our divided concerns. We typically say that a concern is separated if it is localized within a single module/component/file/class/function (henceforth simplified to “component” for readability).

History

The term “separation of concerns” originated in Djikstra’s essay “On the role of scientific thought“. Dijkstra states that you must be willing and able to study pieces of your subject matter (whatever you define that to be) on their own while still knowing that you’re occupying yourself with just one of the aspects of the overall subject. Were you to tackle all parts of your subject matter at once, you would simply be overwhelmed. Separating concerns (that is – focusing your attention on a single aspect) and analyzing them on their own, while not perfectly possible, allows you to effectively order your thoughts.

Collectively, we have since extended this idea to how we structure source code. You will have an easier time reasoning about your software if you are able to reason about distinct “concerns” separately, rather than being forced to reason about the entire system (or multiple concerns) all at once. Of course, as Dijkstra points out, this is not perfectly possibly: we still have to contend with interacting and dependent concerns as well as the behavior of the system-as-a-whole.

What are concerns?

A “concern” is commonly defined along the lines of “any particular interest in the system” or “a set of information that affects the code of a computer program”. This is vague, so it helps to consider examples. Concerns may be general, such as “the details of the hardware for this application” or “the details of how databases interact”. They may also be specific, such as “specify the name of a class to instantiate” or “perform calculation X”. We may divide up concerns based on any number of criteria, including:

  • purpose
  • functionality
  • structure
  • behavior
  • cost
  • supportability
  • safety
  • interoperability

We might also separate interactions at the system’s boundary from those at the application core, use domain analysis to suggest abstractions and separations, and separate concerns based on system responsibilities.

Benefits

In one sense, separation of concerns is an organizational principle that can help us evaluate the structure of a given piece of code. It helps produce a program that is modular. “Good” separation of concerns also allows each component to be simpler and more easily understood. The ideal is to achieve high cohesion and low coupling in our separations. Separating concerns can also enable independent development of program modules.

Separation of concerns is also viewed as a form of abstraction. Separation of concerns is achieved by encapsulating information inside of a section of code or a module that has a well-defined interface. By hiding implementation details within a module, we can modify a single concern’s implementation without knowing the details of or modifying other concerns. This provides us with an improved ability to modify, test, and reuse our modules.

Tradeoffs

  • How we decompose our systems matters. Separating concerns at the wrong boundaries can result in high coupling and complexity between features/concerns, even if the contained functionality within a feature does not cause significant overlaps with another concern
  • As with most form of abstractions, there is a cost in the addition of code, both in terms of interfaces and in machinery to support them. This introduces a potential execution penalty, although in many cases this is negligible.

Examples

  • You can often identify poor separation of concerns by looking for mixed metaphors.
  • Layered architecture involves separating a system into layers, each which usually represents a significant concern of the system (e.g., a hardware abstraction layer or board support package in embedded software encapsulates the hardware-specific concerns of the system)
  • A hexagonal architecture also models separation of concerns. This architectural style separates the application core from the components and services that interact with the outside world.
  • Event-Driven and Publish-Subscribe architectures often result in high separation of concerns due to the interaction through events and messages. Collaborators only know about the exchanged events, not the implementations sending or receiving them.

References

  • Wikipedia: Separation of Concerns
  • Wikipedia: Concern (computer science)
  • Real-Time Software Design for Embedded Systems by Hassan Gomaa

    Structuring a concurrent system into tasks results in a separation of concerns about what each task does from when it does it. This usually makes the system easier to understand, manage, and develop.

  • Microsoft Application Architecture Guide, Chapter 2: Key Principles of Software Architecture

    Separation of concerns. Divide your application into distinct features with as little overlap in functionality as possible. The important factor is minimization of interaction points to achieve high cohesion and low coupling. However, separating functionality at the wrong boundaries can result in high coupling and complexity between features even though the contained functionality within a feature does not significantly overlap.

  • Use of Abstract Interfaces in the Development of Software for Embedded Computer Systems

    There are additional benefits. This process leads toward what is sometimes referred to as a cleaner structure of the software – one in which there is a good separation of concerns, allowing each component to be simpler and more easily understood. Further, those components that are not cognizant of the real-world details of the interface can be more elegant and subject to a more mathematical analysis. Elegance is not a property of systems that must deal with ugly real world facts, but it can be obtained in those components that are separated from the real world b y the use of well-defined interfaces.

  • On the role of scientific thought by E.W. Djikstra

    Let me try to explain to you, what to my taste is characteristic for all intelligent thinking. It is, that one is willing to study in depth an aspect of one’s subject matter in isolation for the sake of its own consistency, all the time knowing that one is occupying oneself only with one of the aspects. We know that a program must be correct and we can study it from that viewpoint only; we also know that it should be efficient and we can study its efficiency on another day, so to speak. In another mood we may ask ourselves whether, and if so: why, the program is desirable. But nothing is gained —on the contrary!— by tackling these various aspects simultaneously. It is what I sometimes have called “the separation of concerns”, which, even if not perfectly possible, is yet the only available technique for effective ordering of one’s thoughts, that I know of. This is what I mean by “focusing one’s attention upon some aspect”: it does not mean ignoring the other aspects, it is just doing justice to the fact that from this aspect’s point of view, the other is irrelevant. It is being one- and multiple-track minded simultaneously.

  • Revisiting Information Hiding – Reflections on Classical and Nonclassical Modularity

    A concern is separated if its code is localized in a single component of a system, such as a file, a class, or a container.

  • The Art of Readable Code by Dustin Boswell and Trevor Foucher

    Key Idea Code should be organized so that it’s doing only one task at a time.

  • From Software to Systems: The Newsletter – Communication Patterns: Decoupling edition

    There are many reasons to create a separation of concerns. Perhaps the front end is constrained by the technology selected for the back end. Building with front end frameworks, like Angular or React, can generate more-performant software with its own logic. The front end team can design their own software and push changes quicker, without getting entangled in a monolith.

    Meanwhile, back end software and services can focus on creating, structuring and publishing content that will be consumed by other software.

Design one part so that it does not need to know anything about the other part, except when it does . Usually, this is done by events. Caution: it’s very easy to build a distributed monolith in the cloud, rather than actually decoupling. If your parts aren’t independently deployable, you might still have tight coupling.