How Our Approach to Abstract Interfaces Has Changed Over the Years

11 July 2022 by Phillip Johnston • Last updated 29 July 2022

For many years, we were convinced that abstract interfaces were the primary idea behind a design for change approach. This view was informed by the work of David Parnas, positive examples from early projects we worked on, and the frequent difficulties that arose with tight coupling between an embedded application and the underlying hardware. While working on the Designing Embedded Software for Change course, we realized that our views on abstract interfaces shifted enough that it triggered a major course rewrite. It was fun for us to take the chance to go back and trace the evolution of these ideas through the years.

Table of Contents:

1. The Starting Point: Full-Fledged Abstract Interfaces
2. A New Idea Emerges: Minimal Abstractions
3. Design for Change Course Development: Minimal Abstractions Dominate
4. Dedicated Abstract Interfaces Are Still a Useful Tool
5. References

The Starting Point: Full-Fledged Abstract Interfaces

If you looked at systems we developed, you would find heavy use of “virtual devices” – modules that interact with external hardware through an abstract interface. For example, we would create an abstract interface for a SPI bus that could support many different implementations.

// SPI Master Abstract Class
class SPIMaster
{
  public:
    virtual void configure(spi::baud_t baud) noexcept = 0;
    virtual spi::mode mode() const noexcept = 0;
    virtual spi::mode mode(spi::mode mode) noexcept = 0;
    virtual void start() noexcept = 0;
    virtual void stop() noexcept = 0;
    virtual comm::status transfer(void* data, size_t length) noexcept = 0;
};

// A derived class, which is a SPI peripheral driver that 
// implements the abstract class interfaces
class SomeSPIDriver : public SPIMaster
{
    // Implementation of each abstract function 
    // + possibly some extras that won't be in the generic interface
};

// The C alternative is to define a struct of function pointers.
struct spi_intf
{
    void (*configure)(spi_baud_t);
    spi_mode_t (*get_mode)();
    void (*set_mode)(spi_mode_t);
    void (*start)();
    void (*stop)();
    comm_status_t (*transfer)(void*, size_t);
};

// And map those to specific functions for each bus 
// (or generic functions with an optional parameter)
struct spi_intf spi0 = {
    &configureSPI0,
    &getModeSPI0,
    &setModeSPI0,
    &startSPI0,
    &stopSPI0,
    &transferSPI0
};

Each driver that communicated over SPI would interact with the bus through this abstract interface, and the driver would take a reference to the specific SPI bus it was attached to. This strategy enabled us to swap out SPI implementations without needing to change any of the modules that depended on SPI. This approach works well, and we have portable device drivers that can run on any system that supplies the necessary interface implementations. However, this is time consuming work, since creating abstract interfaces that can support multiple implementations requires research and experience; it’s easy to accidentally create an abstraction that is coupled to a specific implementation in some way.

A New Idea Emerges: Minimal Abstractions

While working on a client project back in 2019, we discovered an AX5043 driver project that took a completely different approach to abstraction. Instead of relying on any particular SPI interface, this driver defined its own SPI abstraction. In order to use the driver with your system, you simply needed to implement the spi_transfer function.

// The driver required a"spi_transfer" function implementation.
// All other hardware interactions are built on top of this interface.
void (*spi_transfer)(unsigned char*, uint8_t);

// --- Example Use ---

#define SPI_CHANNEL_RADIO_0 0
#define SPI_CHANNEL_RADIO_1 3

// Define a function that implement the API
void radio0_spi_transfer(unsigned char* data, uint8_t length)
{
    wiringPiSPIDataRW(SPI_CHANNEL_RADIO_0, data, length);
}

// Or multiple, in the case of two instances.
// A single implementation could be supported with the use of an optional
// user-configurable input parameter, but this API did not provide for that.
void radio1_spi_transfer(unsigned char* data, uint8_t length)
{
    wiringPiSPIDataRW(SPI_CHANNEL_RADIO_1, data, length);
}

int main(void)
{
    // ...

    ax_config radio0_config;
    ax_config radio1_config;
	
	// Here, we assign the function pointer for each radio instance
	// to the corresponding implementation above.
    radio0_config.spi_transfer = radio0_spi_transfer;
    radio1_config.spi_transfer = radio1_spi_transfer;
	
	// More configuration...

    ax_init(&radio0_config);
    ax_init(&radio1_config);

	// ...
}

At the time, we noted that the AX5043 driver’s approach has two major benefits:

1. It represents a better separation of concerns, meaning that the AX5043 driver is only exposed to those SPI capabilities that it actually needs (transferring data). Other responsibilities, such as configuring the SPI bus and managing resource contention, are handled outside of this module.
2. It provides a minimal abstraction approach: only one abstract function needs to be defined (c.f. a fully fleshed out abstract SPI interface).

We also noted a downside with the approach we had been using:

Our radio driver, for better or worse, is dependent upon the SPI driver abstract interface definition. We are not omniscient nor omnipotent, and eventually we will need to modify our abstract interfaces. The unfortunate impact is that every module which is dependent upon those abstractions must change. For something as fundamental as a SPI driver, the impact can be significant.

Looking back at the AX5043 article now, it’s almost amusing that was the major downside we identified. Any interface change can be painful, abstract or not. The more exposure our system has to a specific interface, the more painful changes are. The real downside, we now believe and wrote about a year later in our demonstration of real-world portable drivers:

However, our drivers are still tightly coupled to the underlying ecosystem, which in this case is the Embedded VM. For software developed by your company, this isn’t a problem: your company will continue to refine and reuse the abstract interfaces you’ve defined.

For building drivers that can be reused on a wide scale, however, this is still a limited approach, as our driver depends on a specific interface in order for it to function. If you want your sensor driver to be usable across different I2C interfaces, we need to take a different approach.

In the next article, we will take a different approach to associating device drivers. Our approach will be similar to that taken by the C-based radio driver that we reviewed in the past.

We had already proven out the use of abstract interfaces for achieving portability, but we ran into a roadblock where we wanted to use one of our drivers on a project that did not make use of our framework. It turns out that we had only insulated ourselves against change in some ways – we still couldn’t easily take a driver or subsystem we created and use it with a completely different framework without major modification. This kind of thing happens all the time in our experience: you find a driver or library that looks useful, but it’s written against the Arduino SDK and you’re not using that for your project, so now you have to rewrite the driver to work with your system (or try to find an alternative). Our focus on developing drivers against our own abstract interfaces blinded us to this common scenario.

Design for Change Course Development: Minimal Abstractions Dominate

That “next article” mentioned in the quote above was never written, because we started working on Designing Embedded Software for Change. The AX5043 driver’s approach to abstraction was certainly an important lesson that we wanted to highlight in the course, but it still did not factor significantly into the original outline. Instead, it was relegated to a single lesson. During the initial development, we once more focused heavily on creating and using abstract interfaces, and one of the first technical modules we completed walks through the process of creating abstract interfaces for modules in your system.

As development continued, we realized that many of the ideas we were presenting kept landing us back at the “minimal abstraction” approach that was introduced to us by the AX5043 driver. Some examples include:

• Loose coupling – should we tightly couple ourselves to an external abstract interface if we could otherwise avoid doing so?
• Separation of concerns – should we give modules access to capabilities provided by an interface that they don’t require?
• Minimizing external dependencies– externally defined abstract interfaces are also dependencies!
• Constrain tight coupling to specific locations – how can we move tight coupling out of our modules and into an external “connective tissue” area?

We also wanted to present programming techniques that could be applied to module implementations to make them more easily configured, ported, and reused without needing to change the implementation. We were surprised when we realized that all of these tactics focus on a module’s interface, and many of them can be used to eliminate the need to depend on another module’s abstract interface.

1. Use the Template Method pattern to enable applications to customize specific behaviors.
   • This is how we categorized the AX5043 technique.
2. Use callbacks and the Observer Pattern to connect modules together from the outside.
3. Have cooperating modules communicate through queues.
   • A shared data format is another type of “minimal abstraction”.
4. Use configurable settings to externally control module behavior.

When we combined the key ideas with the tactics, we reached a totally different conclusion about how to best design modules and their interfaces to support change:

1. Modules should ideally:
   1. Define their own abstractions that users are required to implement (as well as a detailed description of how they are expected to behave).
   2. Build up the implementation on top of the module’s base abstractions.
   3. Provide additional abstractions and configuration parameters that users can optionally override to control module behaviors.
2. “Fundamental” modules, such as device drivers and standalone libraries, should be structured so that they are isolated from as much of the rest of the system as is humanly possible (i.e., no dependencies on other modules). This way, they can be easily reused and ported. They can also be easily tested in isolation.
3. The implementations for each module’s abstractions (and the connections between modules) should be externalized and kept in some application or configuration specific area (instead of spread throughout the system). Changing the system to support a new platform requires a new implementation of this tightly coupled module, but the rest of the system can remain unchanged. Multiple configurations can be supported by conditionally compiling the proper file(s).

As a result of having modules define their own required abstractions, you need to define fewer abstractions. It is easier to get these abstractions “right” the first time due to the reduced scope (you are creating abstractions for a single module, not for an entire class of modules). If you don’t get them right, the impact of a change is minimal: they only impact the module itself and the tightly coupled connective code. You can also use the module on any system that can supply the necessary implementations, not only on systems that adhere to a given framework or SDK. These are significant improvements over a custom SPI abstraction that is used by all SPI implementations and SPI dependent modules.

Defining abstractions in this way also makes implementation much faster. In a perfect world, we would still use our custom driver abstractions and implementations for everything – many vendor SDK implementations are subpar and can be implemented better with a little bit of work. But creating these fundamental abstractions for the full set of processor peripherals is time-consuming, and often we find our time is better invested in other areas. We can easily get a module working with a new SDK or framework by mapping each module’s abstractions to the appropriate SDK APIs and instance handles. This makes it much faster to get up and running on new platform, and you can easily support multiple configurations (e.g., a desktop simulator and an on-device embedded application; two different embedded processors) by having each configuration use a different implementation file for the connective code.

This approach also led us to add a new “Key Principle” late in the development of the course: the Open-Closed Principle – software entities should be open for extension, but closed for modification. We had completely overlooked this idea during our initial development efforts, but we realized toward the end of version 1.0 development that this was an essential idea underlying our latest approach to design for change: as much as possible, we want to structure our modules so that we can change their behavior from the outside instead of needing to modify the module internals. Whereas the original course outline was focused heavily on information hiding and separation of concerns, the final course material has ended up more heavily focused on loose coupling and the Open-Closed Principle.

Dedicated Abstract Interfaces Are Still a Useful Tool

Reconsidering where we define the abstractions used by our modules has changed how we approach designing software that is resilient to change with great benefit. Of course, this doesn’t mean that the idea of dedicated abstract interfaces should be discarded. Instead, our goal has shifted to create minimal, module-specific abstractions whenever feasible.

The usage of a given module type may dictate whether or not a dedicated abstraction is useful. For example, device drivers for external components may only require a spi_transfer function to successfully operate, since that is all they need to complete their responsibilities. But if you have a command shell that provides a spi command, having an abstract interface for a SPI device ensures that the shell implementation can remain fixed while supporting multiple SPI implementations. You may also encounter a mixed model, such as a time-of-flight sensor driver that requires a spi_transfer function implementation, while the sensor driver itself implements a ToFSensor abstract interface that is used elsewhere in the system.

Sometimes complexity drives the outcome. Perhaps the abstractions needed are too complex or too great in number to be easily specified by the module itself, and instead are better off being placed in another module and accessed through the abstract interface. Or maybe you need something like the State Pattern, where you define a common abstract interface and switch between implementations based on the current state. In other cases, a module may represent a complex, coordinated application-specific activity (e.g., for power control, persistent user configuration parameters) that depends on other system components, and is not likely to be used outside of the current application or framework. Ideally, this subsystem or activity will interact with other system components through abstract interfaces.

Having a standardized interface approach helps us reduce cognitive load when working our systems, since we aren’t having to guess at different interface variations. Standardized interfaces allow us to easily swap out implementations without having to adjust dependent modules. These are real benefits of their own that cannot be overlooked.

Finally, keep in mind that all of the lessons and insights about defining and documenting abstract interfaces that are unlikely to change apply regardless of whether you are defining standalone abstract interfaces or minimal module-specific abstract interfaces. These must be thought out to ensure they meet the needs of the module and can support the possible set of implementations that you may need to support now and in the future. Module-specific abstract interfaces must be documented as thoroughly as any other abstract interfaces.

References

• Designing Embedded Software for Change
• Virtual Devices in Embedded Systems Software
• Real-World Portable Driver Examples
• Practical Decoupling Techniques Applied to a C-based Radio Driver
• Musings on Tight Coupling Between Firmware and Hardware