Course Review

What I Learned from James Grenning's Remote TDD Course

Test Driven Development (TDD) is an important software development practice which is typically foreign to embedded teams. James Grenning has put a tremendous amount of effort into teaching embedded systems developers how to adopt TDD. He published an embedded systems classic, Test-Driven Development for Embedded C, and regularly conducts TDD training seminars.

Admittedly, TDD is one of those concepts that I've heard about but never actually got around to studying and implementing. After seeing a tweet about a remote TDD training class, I decided to sign up and see if it was really all it's cracked up to be.

If you're looking to grow as an embedded developer, I recommend taking a TDD class with James - it has transformed my development approach. TDD helps us to decouple our software from the underlying hardware and OS, as well as to develop and test embedded software on our host machines. We've all felt the pain of the "Target Hardware Bottleneck" - this class shows you how to avoid the pain and to adapt to sudden requirements changes.

Aside from getting hands-on experience with TDD, I learned many valuable lessons from James's course. Below I will recount my experience with James's remote TDD training, review my lessons learned, and share my thoughts on taking the course vs reading the book.

Table of Contents:

  1. Why I Took the Course
  2. Course Structure
  3. Lessons Learned
  4. Course vs Book
  5. In Conclusion
  6. Further Reading

Why I Took the Course

I've come to believe that the common approach for developing software, especially embedded systems software, must be dramatically overhauled. I see far too many projects which skimp out on design, testing, code reviews, continuous integration, or other helpful practices which can improve code quality and keep our projects on schedule.

I've also noticed that I spend too much time with "debugging later programming", as James calls it. I write a bunch of code, get it to compile, and then deploy it and test on the target. The debugging time often ends up being much longer than the coding time - there must be a better way to approach development. Furthermore, why do I need to flash to the target to do most of my testing? Can't I build my programs in such a way that I can test large pieces of them on my host machine, where I have an extensive suite of debugging tools on hand?

When I was a junior embedded engineer, I believed other developers when they told me that unit tests weren’t useful or feasible for embedded systems due to our dependence on hardware. After studying architecture, design principles, and experiencing sufficient pain on multiple projects, I realize that there is immense value in changing our current approach to building and testing embedded systems. James’s course is the perfect way to dive head-first into TDD and unit testing.

Course Structure

I signed up for the remote training course, which consists of three five-hour days of training. The training is conducted with a suite of web-based tools:

  • Zoom meeting for video/audio
  • CyberDojo for programming exercises
  • A central course website with links to resources & exercises
  • A "question board" where we could post questions as we thought of them without interrupting the flow of the class

The course follows this pattern each day:

  • Discuss theory
  • James performs a TDD demo
  • Class members perform a hands-on programming exercise (~2 hrs long) while receiving live feedback from James (~2 hours each day)
  • James answers questions, reflects on the exercise, and discusses more theory

We used the CppUTest framework throughout the training, which is the same test framework featured in his book. I had not used CppUTest before the course, so it was great to get experience with a new test framework.

Day 1

Day 1 started with introduction into TDD. James opened with a discussion about the impact of the typical Debug-Later programming style and the value propositions of TDD. He introduced us to the TDD cycle:

  • Write a test
  • Watch it not build
  • Make it build, but fail
  • Make it pass
  • Refactor (clean up any mess)
  • Repeat cycle until work is finished

The cycle is directly related to Bob Martin’s TDD rules which we continually referred to throughout the course:

  • Do not write any production code unless it is to make a failing unit test pass
  • Do not write any more of a unit test than is sufficient to fail; and compilation failures are failures
  • Do not write any more production code than is sufficient to pass the one failing unit test

We also discussed a TDD-based development cycle for embedded systems, which involves writing code on the host machine first, then incrementally working up to running the code on the target hardware. This development cycle enables embedded software teams to prototype, create modules, and test driver logic before target hardware is available.

We followed the TDD cycle with “design for testability” concepts, which are the same general design concepts we should already be applying:

  • Data hiding
  • Implementation hiding
  • Single responsibility principle
  • Separation of concerns
  • Dependency inversion (depend on interfaces not implementations)

After this introduction to TDD, we dove right in with live programming exercises. James performed a demo where he used TDD to create and test a circular buffer library in C. After showing us the TDD approach, he set us loose to write our own circular buffer library. The exercise took 2 hours, and James gave each of us direct feedback as we worked through the exercise.

We ended the day with a discussion of the next day’s exercise, which involved creating a light scheduler for a home automation system. He gave us optional homework to write a “spy” for a light controller, which took me around 15 minutes to complete.

Day 2

Day 2 started with a discussion of “spies”, “fakes”, and strategies for testing modules in the middle of a hierarchy. We reviewed TDD strategies, focusing on how to write a minimal number of tests and how each new test should encourage us to write new module code.

We quickly moved to the programming exercise, which involved TDD for a Light Scheduler. The tests were written using the Light Controller Spy that we created prior to class as homework, and demonstrated how to apply spies in our testing process. As with the circular buffer exercise, James monitored our progress and offered live feedback while we worked.

After the exercise was completed, James performed a refactoring demo, showing how we can use our unit tests to maintain confidence while performing major changes to our code base. We also discussed code coverage tools and ran gcov on our unit tests.

At the end of class, James gave a brief introduction to CppUTest’s mocking support. Our homework on day 2 was to play around with the mocking functions to get a feel for how the framework functions and how expectations can be used during testing.

Day 3

Day three opened with a discussion of test doubles, mocking, and run-time substitution. After a brief introduction, we started the day’s programming exercise: writing and testing a flash driver off-target using mocking.

After finishing the exercises, we recapped the lessons we had learned up until that point and reviewed the value propositions for TDD.

After the review, we moved into a discussion on refactoring. James covered general refactoring theory, code smells, design principles, and refactoring strategies. He introduced a method for refactoring legacy code (“Crash to Pass”), and pointed us to resources to help us test and refactor our existing code.

After the refactoring discussion, we had one last general Q&A session and then wrapped up the training course.

Lessons Learned

There were more lessons packed into the workshop than I can reasonably relate here. Many of them are simple one-offs to guide you as you develop your TDD skills:

  • Use a test harness that will automatically find your test cases and run them, saving you the headache of manual registration
  • Write the minimal amount of code you need to exercise your program paths (aka "don't write too many tests")
  • Ruthlessly refactor your tests whenever they are passing to keep the tests maintained and understandable
  • Even though we are incrementally building our modules, we want to try to invent the full parameter list up-front (TDD will show you exactly how painful it is to update APIs)
  • Mocking can be a refactoring code smell, as it identifies coupling within your system

Aside from these practical tidbits, here are some of the deeper lessons learned during the course:

Feedback Loop Design: Work in Small Steps

In system design, I've been struck by the importance of feedback loops. Donna Meadows frequently touches on their importance and impact on undesirable behavior:

Delays in feedback loops are critical determinants of system behavior. They are common causes of oscillations. If you’re trying to adjust a system state to your goal, but you only receive delayed information about what the system state is, you will overshoot and undershoot. Same if your information is timely, but your response isn’t.

One of the key challenges with building embedded products is that there are numerous delayed feedback loops in play. Firmware engineers are writing software before hardware is available, hardware issues aren't identified until it's too late for another spin because the software wasn't ready yet, critical bugs aren't discovered until integration or acceptance testing starts, and the list goes on.

Shortening our firmware engineers' feedback cycles can dramatically impact a program lifecycle. With TDD, developers get immediate feedback when errors are introduced. We can correct these errors right away, one at a time, and stay on track.

TDD also helps us keep our modules decoupled and testable, allowing firmware to be increasingly developed and tested on a host machine. We can make full use of debugging tools and avoid hundreds of time-consuming flashing steps. We can also utilize mocking, spies, and fakes to develop interfaces, modules, and higher-level business logic before hardware is available.

If you're not getting the system behavior that you want, you likely need to adjust your feedback loops and feedback delays. TDD is one approach to improving feedback loops for embedded systems development.

TDD Feels Slower, But I Programmed Faster

TDD certainly feels like it is more work and that you're moving slower. However, this was merely an illusion in my experience. By working in small steps and addressing problems as they arise, we can stay engaged, move forward continually, and avoid many of those intense debugging sessions.

Let's consider the circular buffer exercise, which I finished in 1 hour and 20 minutes. One of the most popular articles on this website is Creating a Circular Buffer in C and C++. It took me at least 4 hours to get my libraries implemented correctly thanks to debugging tricky logic errors. That's quite a difference!

You might say that I had an advantage in the exercise, having written such a library before. Sadly, I will admit that I made the same mistakes that I struggled with in my initial implementation - some logic errors are just easy to make. However, with the TDD approach I noticed the flaws immediately, rather than having them pile up at the end.

As the popular military maxim goes, "Slow is smooth, smooth is fast". James repeatedly emphasizes this point with his own motto: "Slow down to go fast".

Trust the Process

If you read my website, you know that I am a great believer in processes. We can turn much of our operation over to autopilot, allowing us to allocate our brain’s valuable critical thinking resources to the problem at hand.

When I’m deep in thought, it’s maddening to be interrupted, as the house of cards in my mind comes tumbling down. I always took this as “Just The Way It Is”, but TDD showed me that it doesn’t have to be that way. By working in small steps through a defined process, we know exactly where to jump back in if we get interrupted. We are kept from being overwhelmed because we know what the next step is. We can enter a state of flow more easily - small steps and continual progress keep us moving forward and helps me feel more productive.

Having a defined process also helps when you are stuck. You’re never really wondering what to do next - simply move on to the next step in the process.

Keep a Test List

There's no need to worry about writing all of your unit tests at once. Maintain a test list for each module that describes any work which still needs to be completed. The best place to store this list is inside of the test source code itself, e.g. as a block comment at the top of the file.

If you think of a new test to write, make a note. Then you never need to worry about remembering all of the tests.

TDD is Not the Holy Grail

James emphasizes throughout the course that while TDD reduces the errors that are introduced into our programs, TDD is not sufficient for proving that our programs are bug-free. The best that TDD can do for us is to show us that our code is doing what we think it should do. This does not equate to correctness - our understanding may still be incomplete or incorrect.

TDD only helps us ensure that our code is working on purpose. You still need design and code reviews, integration testing, static analysis, and other helpful developmental processes.

Course vs Book

If you have James's Test Driven Development for Embedded C book, you may be wondering whether the course is still worth taking. I respond with an emphatic yes. I recommend the course in conjunction with the book for one simple reason: the course requires you to actually program in the TDD style. Practice makes perfect.

 
 

During the course, you'll work through multiple hands-on programming exercises and receive direct feedback. Whenever I skipped steps or started writing code without tests, James noticed and helped me get back on track. Without this feedback, I would not have been successful at noticing and breaking my existing development habits.

When reading a book, we commonly acquire knowledge but never take the time to apply it. By getting a chance to try out the method for yourself, you're more likely to feel the benefits and adopt the process. Once you have experience with TDD, the concepts in the book can be easily connected to real experiences. You will be much more likely to make connections in your mind and apply the concepts in practice.

There's one more reason I recommend the course in addition to the book: when you are a beginner, you have many questions. It's hard to get help if you don't know what, how, or where to ask questions. James is willing to answer your testing questions and provides you with plenty of resources and forums for finding answers. Even better, once the course is finished, you have access to email support from James. As long as your questions aren't easily Google-able, you will always have a resource to help guide you.

In Conclusion

I really enjoyed James's remote TDD training and think can help developers at any skill level (in fact, most of the attendees were experienced programmers). The hands-on programming exercises were unexpected and enjoyable. The direct and immediate feedback from James was an invaluable aid for adopting the process and correcting our default behaviors.

If you're interested in taking the TDD class, you can find the course options and schedules on James's website.

I adopted TDD immediately after completing the course. I spent a day setting up my development environment so I can compile and run tests with a keystroke, just like we did in CyberDojo. The process is addictive - writing new tests and getting them to pass is a continual reward cycle that keeps me focused on programming for much longer periods of time.

I've already found myself refactoring and updating my code with increasing confidence, since I have tests in place to identify any glaring errors which are introduced.

A key advantage of well tested code is the ability to perform random acts of kindness to it. Tending to your code like a garden. Small improvements add up and compound. Without tests, it's hard to be confident in even seemingly inconsequential changes.
--Antonio Cangiano (@acangiano)

Further Reading

James's book, Test Driven Development for Embedded C is an excellent starting point for TDD, especially for embedded systems developers. Again, I recommend this book in conjunction with the online training course. You can find the courses and schedules on James's website.

These talks by James provide an introduction into the how and why of TDD:

James has written extensively about TDD on his blog. Here are some of my favorite posts:

Other TDD-related links:

Related Posts

What I Learned at Bredemeyer Consulting's Software Architecture Workshop

I've become increasingly interested in software architecture and visual modeling as I've progressed in my career. I chanced upon a Bredemeyer Consulting article, "What is Software Architecture?", which put the idea of becoming an architect within my reach. When Bredemeyer Consulting announced an open-enrollment workshop I immediately signed up.

I attended four days of architecture training hosted by Ruth Malan (@visarch). The course was easily worth 2-3x the amount that I paid, and I walked away with over 20 pages of written notes (not including annotated slides which are covered in scribbles and dog-eared pages).

Course Structure

The four-day course follows a structure which iterates across the design space, refining architecture views:

  1. What is software architecture? What do architects do?
  2. Understanding the high-level strategy, business needs, and user needs
  3. Setting design direction: explore value and set the scope of the project
  4. System Structuring Pass 1: create the architectural strategy and set technical direction
  5. Validation Pass 1: validate the architecture strategy
  6. Design system capabilities & qualities
  7. System Structuring Pass 2: create the conceptual architecture (focusing on identifying components, responsibilities and relationships)
  8. Reflect on Pass 2: find weaknesses in the conceptual architecture
  9. System Structuring Pass 3: create the logical architecture (explore system behavior and define interfaces)
  10. Reflect on Pass 3: reflect on and improve the logical architecture
  11. Design Pass 4: create execution architecture requirements
  12. System Structuring Pass 4: create the execution architecture
  13. Using and evolving the architecture (considerations in varying organizational contexts)

We started the four-day workshop with an in-depth discussion about what software architecture is and what software architects do. Ruth illustrated her concepts with stories pulled from all over the world: we heard about Madison's orchestration of the Continental Congress, how the Federalist papers helped make the Constitution a reality, Chuang-tzu's parable about butchering an ox (slow down at the tricky spots!), and how Walt Disney used specific rooms for brainstorming, evaluating, and criticizing ideas.

In order to ground our software architecture discussions, Ruth had the group select a software product idea that we would design over the duration of the course. Our class decided to work on a supply chain management and traceability system for agriculture.

We started our architecture efforts by exploring the business value of the system. We performed a variety of high-level exercises, such as Competitive Landscape Maps, Rich Pictures, evaluating stakeholder goals, and creating stakeholder empathy maps. Ruth emphasized that architects are important members of these business-level meetings, and stressed that we do have perspective on the industry and what technology makes possible (though we may need to suppress our technologist’s tendency to focus on flaws too early).

The results of the business-level analysis served as a starting point for brainstorming our system's initial requirements, responsibilities, and non-functional qualities. We continued iterating, guided by the architecture decision model, with each new piece of information feeding into later steps. For example:

  • Our initial list of responsibilities was used to generate our initial set of components
  • Our initial set of components fed into the first-pass high-level structuring for the system
  • Our business values and initial structuring was used to generate qualities and principles to guide our architecture & development teams
  • Our stakeholder analyses were used to generate use cases and user stories, and non-functional qualities were tied to the use cases
  • We used our use cases, qualities, principles, and responsibilities to review our architecture and fill in missing pieces
  • Our architecture and use cases were used to generate activity diagrams, sequence diagrams, and communication diagrams
  • An "outside architect" reviewed our design, evaluating whether or not the architecture sufficiently addressed the responsibilities, qualities, principles, use cases, and user stories

At each step in the process, the class noted two common experiences:

  1. When we looked at the examples, we thought "this will be easy". Putting the ideas onto paper in a sensible way was harder than it looked, and usually required 2-3 drafts before we settled on a reasonable approach
  2. Each new analysis caused us to refine our design in some way. We always found an aspect of the system or design that was missing in another view.

At the end of the workshop, you will have a well-supplied architectural toolkit, but you need to continue to practice and apply the concepts on your own. We were exposed to a wide array of diagrams, exercises, and exploratory concepts that can be used during the architectural process. But the self-study must continue, especially in identifying patterns, models, and processes that are most valuable to your software domain.

Lessons Learned

The following lessons stood out over the course of the workshop:

  1. Architecture can be quick and light-weight.
  2. Visual modeling skills have atrophied, but are the best tool to quickly communicate and identify problems
  3. Architects are the bridge between the business team and the development team (others can be too, but the architect must be)
  4. You must understand the system in context to make effective decisions
  5. Inspiration, wisdom, and design lessons can be drawn from a wide variety of fields
  6. We are limited by our cognition and biases - take a different view and invite outside opinions
  7. We need to continually move up-and-down the architectural hierarchy, adjusting our learnings as we go

Architecture Can Be Lightweight

Outside of the defense industry, I have never worked on a team that performed any real architecture work. With Agile, the industry has largely moved away from any sort of architecture or up-front design work.

We don't have to convert to waterfall to gain value from architecture efforts. We can produce a "minimum viable architecture": tackle the riskiest areas first, and think about how to properly organize behaviors that reach across the system. We can fill in the minor details later, but it's much harder to rework the keystones in our design.

Visual Modeling

Our visual modeling skills have atrophied greatly. In the past ten years, I haven't seen a single UML or SysML diagram used in any company that I've worked with. I've even admitted to myself and others that I am a purely verbal thinker, as I spend most of my time writing and programming.

I was repeatedly struck by how many problems jumped out at us when we looked at a picture - coupling, god classes, big ball of mud architecture. Had we started with the code, these properties would have emerged and we would not notice the problem until it was too late.

We can also communicate more clearly with a picture than we can with words. Complex interactions can be grasped at a glance. Parallel processes can be more easily understood when drawn. Our words leave so much room for interpretation, missing details, and force a sequential analysis (you read word-by-word).

Architects Are the Bridge

Architects are the primary bridge between the business team and the development team. Architects must understand the business needs and translate them into system requirements and developer actions. Architects also need to translate technical challenges into business-speak so that issues and risks are clearly communicated.

The job is highly personable, and you must understand how to deal with people and communicate value appropriately. Understand the business tools. Understand how to identify and draw out value. Understand how to translate that value to the development team.

Understand the Context

You cannot make proper decisions unless you are armed with all of the necessary information, including overall business strategy. We need to write down all of the decisions, alternatives that were considered, and why we choose our specific path. That information should be on record for ourselves and for our team.

The context for our decisions is more important than the details of the code. It's hard to recreate the context from the source code. Don't skip the documentation step.

Be Widely Read

We can draw architectural insights from all over. Many wide-ranging examples are used during the course:

  • The Articles of Confederation & the Federalist Papers
  • Chuang-tzu's Parable of Butchering an Ox
  • Using metaphors and patterns evident in nature
  • Applying concepts from other engineering disciplines
  • Take visual notes like XKCD's Thing Explainer
  • We can find clues in surprising places:
    • Sift for principles and heuristics like Bud did in the children’s story "Bud not Buddy"
    • The children’s story “The Wheel on the School” covers the creation of a shared team vision and dynamic teaming
  • Cognitive biases

Keep learning - but don't just focus on the technical side. Develop your communication abilities. Understand our cognitive limitations as humans. Study philosophy. Explore nature. Learn about the world. Anything can be a source of inspiration for our designs.

We Need Different Perspectives

As human beings, we are limited by our cognition and biases. Work on taking different perspectives (the dreamer, the critic, the realist), and be sure to share your ideas and invite outside reviews. We are all subject to biases and blind spots, so we must rely on others to evaluate our designs and help us identify what we missed.

Architecture is an Iterative Process

We need to continually move up-and-down the architectural hierarchy, adjusting our learnings as we go. A new detail from one form of analysis will expose a hole in a different model. By adjusting our architectural view and creating different types of diagrams, we are forcing ourselves to take a different perspective in looking at the system. Just be sure to capture these new learnings and adjust your designs as you identify flaws.

The course structure mirrors this iterative point. As we dived into behavioral views and mappings to the physical system topology, we identified problems with the previous views that required adjustments. The higher-level adjustments then drove further changes in the lower-level models. We were continually learning and adapting as we explored the architecture.

In Conclusion

I thoroughly enjoyed Bredemeyer's architecture workshop and would recommend it to anyone with an interest in software architecture. You will gain exposure to the full architectural lifecycle. You will be armed with a wide array of diagrams, exercises, and techniques. You will be actively applying these techniques throughout the workshop while brainstorming, modeling, and creating an architecture.

You will need to continue to practice these concepts on your own once you leave, and you will need to explore the architectural areas appropriate to your own software domain. And don't forget to stay curious, keep learning, and draw inspiration from everything you encounter.

Further Reading