Aaron’s Advice to Up-and-Coming Embedded Developers

Today we have a guest post from Aaron Fontaine.  Aaron is an embedded software developer with a passion for the craft and a commitment to ongoing learning.  He likes understanding things not just from the engineering perspective, but from the customer and management perspectives as well.

You can find Aaron on GitHub and LinkedIn.


This article is a response to a question sent to Embedded Artistry by a college student.  The question was sent out to the entire Embedded Artistry community for all of us to offer answers.

My biggest question would be if you were graduating right now, what kind of advice would you give yourself to best prepare for a career in embedded systems development?

Obviously, the journey is continuous, but I’m trying to learn as much as I can this final year while I can still take advantage of university resources (O-Scopes are expensive!).

My Response

The main thing I would say is stay focused on that “continuous journey.”  A college degree shows you have some level of maturity and responsibility, but it’s very unlikely that college has prepared you well for the industry.  The learning isn’t so much about the equipment you have.  You’ll get a lot more out of learning architecture and fundamentals.  I feel that most of what I know I’ve gained through personal exploration and learning, and through job experience.

As for making the most out of your final year in college, I would say to focus on a senior project and make the most out of it.  Any projects you can talk about in an interview that show a solid demonstration of your skills are good to have.

Expanding Your Domain Knowledge

All embedded developers should have an understanding of computer organization, build systems, and how to work with compilers and linkers.  The nuts and bolts, so to speak.  Some of these topics may have been covered in college, and others may not have been.  It helps to have an avid interest.

You will need to keep learning new technologies, toolchains, SDKs, and languages, even if your primary language continues to be C/C++.  Bash and Python are valuable for scripting. I’ve also created numerous C# forms applications as internal utilities and test/debug aids on projects.

Almost every project I’ve worked on has required some learning curve.  The following are things I’ve had to ramp up on for my current project: Atmel QTouch library, ST’s LSM6 IMU, NFC and NXP’s PN7150, Apple iAP and MFi coauth chip, BLE, USB Android app development, Nordic SDK, GainSpan SDK, Express Logic ThreadX RTOS, Green Hills uVelocity RTOS.

Every project has a similar list of new technologies I have needed to learn. But that is no problem: I have always been able to deliver.

Back in 2011, Patrick McKenzie declared, “You are not defined by your chosen software stack.”

I recently asked via Twitter what young engineers wanted to know about careers.  Many asked how to know what programming language or stack to study.  It doesn’t matter.  There you go.
[…]
Talented engineers are rare — vastly rarer than opportunities to use them — and it is a seller’s market for talent right now in almost every facet of the field.  Everybody at Matasano uses Ruby.  If you don’t, but are a good engineer, they’ll hire you anyway.  (A good engineer has a track record of — repeat after me — increasing revenue or decreasing costs.)  Much of Fog Creek uses the Microsoft Stack.  I can’t even spell ASP.NET and they’d still hire me.

McKenzie is talking about fundamentals, and fundamentals do not change.  I can tell you that the ability to learn, adapt, and most importantly prove a consistent ability to deliver regardless of the circumstances makes you far more valuable than any technology of the moment.

Architecture

One of the best things you can do for yourself is to learn the principles of good architecture.  Software development strays into the realm of complexity far too easily and much of what we do is keep that complexity at bay.  Learning to write modular software that is adaptable, resilient, and flexible is a skill learned over time.  It takes discipline.  In my experience, it is one of the most important things to ensuring project success.  Poor (or missing) architectural design at the start of a project can lead to significant costs and delays down the line.

Where to start?  I highly recommend the principles of SOLID.  Understand what they are and why they’re important.  Often, we’ll get stuck trying to think about exactly how to break a system into subcomponents.  In these cases, I have found it useful to ask myself clarifying questions for each component:

  • What is the (single) responsibility of this module?
  • What awareness, if any, should it have of other components?
  • As a standalone module, what should this module’s interface be?
  • What data is owned by this module?
  • What subcomponents is this module composed of? Are they abstract?  Swappable at runtime?
  • What signals need to occur between this and other modules to achieve the desired system behavior? How would that look in implementation?

It is important to think about architecture and consider these questions through all stages of development

Along with the fundamentals, it is important to learn to think non-linearly.  Think of your design as a system of independent, interacting objects.  Objects are often state-based and event driven.  The processing of events may be invoked from a main loop, execute on separate threads of an RTOS, execute on separate CPU cores, or operate under some other, custom scheme.  All of this means that you need to understand the concurrency model of your system.

Learning UML is valuable.  With a modeling language, you can visually represent aspects of your architecture that would be very lengthy to explain through text.  I find it best to keep architectural descriptions short and rely heavily on diagrams.  The diagrams I find most useful recommend are state charts, sequence diagrams, data flow diagrams, and activity diagrams.

Finally, you should study design patterns.

[M]any real-time and embedded (RTE) systems have important safety-critical and high-reliability requirements because they are often used in systems such as avionics flight control, nuclear power plant control, life support and medical instrumentation.  The creation of RTE systems to meet these functional and quality of service requirements requires highly experienced developers with decades of experience.  Yet, over the years, these developers have encountered the same problems over and over – maybe not exactly the same problems but common threads.  The very best developers abstract these problems and their solutions into generalized approaches that have proved consistently effective.  These generalized approaches are called design patterns.  They are often best applied at the level of system or software architecture – the sum of design decisions that affect the fundamental organization of the system.[1]

One should learn general design patterns such as those described by the Gang of Four, but it also useful to learn patterns specific to embedded development.  All are useful.

Tools

In my experience, the logic analyzer has consistently proven the most useful in debugging.  I would argue that a scope is not as important, although the EE Times 2019 Embedded Markets Study disagrees.  The logic analyzer (along with other tools such as WireShark or the Beagle USB analyzer) can tell you exactly what’s happening on an interface.  It will confirm that what you expect to be happening is actually happening.

A logic analyzer, however, will not tell you about voltage ripples, stuck pins, and the like.  For these, you typically do need a scope. It may be possible to get by with an analyzer that has analog functionality, which can tell you whether voltage levels and level transitions are clean.  Other tools I recommend are a power profiler for measuring the exact current draw of your device, a decent multimeter, and a decent power supply.

Learn to solder.  If you feel weak in this area, actually watch some videos and practice it.  It’s good to have a decent solder station and wire stripper at home.  I’ve often found myself working on bare production PCBs with fine-pitch (30 or 32 gauge) wire soldered to them for test and debug.  Those wires pop off very easily, and it’s nice to be able to repair them myself.

If you work for an embedded design house, you won’t need to spend too much on your own equipment as you should be able to borrow equipment when working from home.  My minimal set would be a good soldering gun, a multimeter (doesn’t need to be Fluke), and a logic analyzer.

Schedule Pressure

It’s always better to think about architecture than not, regardless of the circumstances.  You will be the one experiencing the debug churn if things aren’t working.  Bear in mind that a non-functional deliverable is the same as no deliverable in the client’s eyes.  Also, try to understand deadlines from your client’s and leadership’s perspective. That helps you understand where exactly you can negotiate and be flexible, and where you have hard requirements.  If you cut corners or do not follow a process, you will open yourself (and the client) up to more risk down the line.  It is better for you and the client to negotiate deliverables and manage risk intelligently.

Another issue worth mentioning here is cognitive load.  We’ve all run into our limit at some time or another.  The mind gets foggy.  You cannot focus.  If you are pushing long hours to meet a deadline and start running into mental fog, that’s a red flag.  You’re at risk of making oversights and design compromises that may seriously impact the project.  Every situation is different, of course, and you may not always have a choice, but you should at least have a sense of when you are hitting your limit.

The reason I bring this up is for personal awareness.  There comes a point where adding more hours may no longer lead to a positive return and may even start producing negative returns.  Many engineers implicitly understand this (e.g., “Just spent the day debugging the issue I introduced last night at 9pm… I need a break.”).  Some managers understand this.  Some may not.  Some schedules are reasonable, some can be achieved with a little push, and some are impossible regardless of the effort.  If you are consistently experiencing the latter, it is time to look for a new job.

Debugging

Sometimes debugging feels like more art than science, yet I am deeply aware that there are principles and methods to what I do.  When I sit down and think about it, three of the principles I run by are:

  • Understand the system.
  • Find a consistent way to reproduce the problem.
  • Gain visibility.

There is much I could write on each of these and this list is by no means complete.  Fortunately, plenty has already been written about the art of debugging.  I strongly recommend the Debugging section under Foundational Skills in Embedded Artistry’s Field Atlas.

One more note: When it comes to issues brought to you by others, don’t take what they say at face value.  You have to verify exactly what’s happening.  If they’re remote, have them video conference and show you the problem.  If it’s another developer (e.g., for the application or backend services), try to verify the communication links (at a low level) between your component and theirs.

General

For each project you work on, try to learn as much about the product as possible.  Really understand its use cases and the business need behind it.  Ask questions if you need to.  At the end of the day we are here to solve somebody else’s business problem.  Doing that properly starts with a good understanding of their needs.  Put yourself in your client’s shoes.  Then put yourself in the end user’s shoes.  Think about the overall experience of the product.  Don’t be afraid to ask questions or offer suggestions.  Some of most rewarding projects I’ve had involved high collaboration with the client.

Ongoing Learning

As far as continuing to learn, it’s more about an avid interest and exploration than pushing yourself.  It may feel like a lot, but keep in mind your coworkers have their own things going on and are also trying to find balance in their lives.  They may have family and social obligations, community education, or volunteering.  They may be raising kids.  They may have hobbies and personal interests unrelated to work.  It may seem like you’re falling behind if you don’t push yourself, but such perceptions are often more projection than reality.  People go years without putting any effort into professional development.  My belief is that if you keep your interest up and take time to read up on things every now and then and you’ll be fine.

Final Point About College Resources

As far as “college resources,” I would recommend that what your college has to offer in terms of industry connection and networking is far more important than lab equipment.  Degree programs that prepare people directly for an industry often have strong ties within that industry.  It may be worthwhile to talk with professors and alumni associations to see who they know and who would be willing to talk with you.  You may be able to set up informational interviews that help you better understand a career path.  The resources are there, but they are kind of hidden and it really is on you to take advantage, especially in your final year.

Best luck,
Aaron

[1] Douglass, Bruce Powel, Real-Time Design Patterns, Preface

2 Replies to “Aaron’s Advice to Up-and-Coming Embedded Developers”

  1. Such a great post! I love the emphasis on the “soft skills” also: managing yourself, learning how to learn, being able to add value to a team instead of just being a “coder”. I want this to be an interview so there could be lots of follow-up questions.

Share Your Thoughts

This site uses Akismet to reduce spam. Learn how your comment data is processed.