Product Development

A Look at Ten Hardware Startup Blunders, Part 4: Conclusion

At Embedded Artistry, we've spent countless hours meeting with founders, working at startups, and swapping war stories with fellow engineers. After enough iterations, we started to notice a repeated set of critical strategic missteps made by new hardware companies. These missteps result in compromised designs, increased development costs, and continual schedule delays.

We've selected ten mistakes regularly observed at new hardware companies which fall into three general categories: process, focus, and team. This article is the final article in our four-part series

  1. A Look at Ten Hardware Startup Blunders, Part 1: Process
  2. A Look at Ten Hardware Startup Blunders, Part 2: Schedule and Focus
  3. A Look at Ten Hardware Startup Blunders, Part 3: Team

A Review of Our Top Ten

Mistakes made at the beginning of a company's or product's life can have long-reaching effects.

The beginning is the most important part of the work. -- Plato

The impact of the mistakes we highlight are not immediately apparent. Like cancer, the damage slowly builds up over time until there is no going back. Your company will have to undertake painful exercises to get back on course, and if the damage is too great your team may not fully recover.

When the problems are finally big enough that your team must face them head on, they are often so far removed from the causes that it feels like an inevitable consequence of building a new product. Keep an eye out for signs of these common blunders and make a course correction before it becomes too painful:

  1. Failure to separate research and development phases
  2. Failure to consider data collection requirements
  3. Lack of defined internal processes
  4. Skipping software architecture
  5. Prioritizing mechanical design
  6. Setting the schedule based on hardware milestones
  7. Heading straight to China for manufacturing
  8. Committing to a schedule before having the execution team
  9. Hiring junior or mid-level engineers and expecting senior-level work
  10. Attempting to build a full-stack engineering team

None of these challenges and missteps are new. You can read about some of these observations in engineering classic written in the 1940s. So why are we repeatedly encountering the same basic problems?

Jack Ganssle, an embedded systems consultant, highlights our lack of learning while reflecting on "NASA's lost software engineering lessons":

And today, 49 years later, we still haven't learned those lessons. Large software projects are routinely plagued with the same sort of problems we "learned" to avoid half a century ago.

Jerry Fitzpatrick, a software consultant and author of The Timeless Laws of Software Development has this to say:

Collectively, we don’t seem to be learning from the past. I don’t know if this is due to an education shortfall, the influx of so many new developers, the acceleration of business expectations, or other factors. I suppose if the reasons were obvious, there would be fewer mistakes.

After all this time, the only thing that has changed is that we expect our engineers to produce faster and faster, even though we haven't fixed our fundamental attitudes and processes. We can change the trend and learn from companies who have gone before us.

Further Reading

Here are classic works related to software project management, scheduling, requirements, and estimation:

  1. The Mythical Man Month, by Fred Brooks
  2. Applied Software Project Management by Andrew Stellman and Jennifer Greene
  3. Software Requirements by Karl Wiegers and Joy Beatty
  4. Software Estimation: Demystifying the Black Art by Steve McConnell
  5. The Economics of Software Quality by Capers Jones

For a straightforward approach to software project estimation, try the Wideband Delphi technique described in the Karl Wiegers's paper "Stop Promising Miracles".

We frequently write about product development on Embedded Artistry:

The following websites provide excellent product development and manufacturing advice that can benefit new hardware teams:

Jack Ganssle, a veteran of the embedded systems industry, shared his own "Top 10 List for How Embedded Projects Run Into Trouble". He's also published a series on embedded.com which covers each point in further detail:

Thanks for Reading

Thanks for following this series. We hope that you will be able to identify and avoid these common blunders on your next project. We must learn from our mistakes to avoid repeating them endlessly. Wouldn't you rather solve new and interesting problems instead of rehashing the same old ones?

Would you tell me, please, which way I ought to go from here?”
“That depends a good deal on where you want to get to,” said the Cat.
“I don’t much care where–” said Alice.
“Then it doesn’t matter which way you go,” said the Cat.
“–so long as I get SOMEWHERE,” Alice added as an explanation.
“Oh, you’re sure to do that,” said the Cat, “if you only walk long enough.”

  • Alice’s Adventures in Wonderland, Chapter 6

A Look at Ten Hardware Startup Blunders, Part 3: Team

At Embedded Artistry, we've spent countless hours meeting with founders, working at startups, and swapping war stories with fellow engineers. After enough iterations, we started to notice a repeated set of critical strategic missteps made by new hardware companies. These missteps result in compromised designs, increased development costs, and continual schedule delays.

We've selected ten mistakes regularly observed at new hardware companies which fall into three general categories: process, focus, and team. This article is the third in our four-part series and focuses on team-related mistakes.

Team-related Mistakes

Building a team which can execute on your company’s business goals is a challenging task. Team composition can make or break a new company due to the serious cash flow constraints they face. Companies must focus carefully on hiring to ensure they have a mix of senior and junior engineers while also building a balanced team focused around the core business value. Hiring recklessly or staffing up too quickly can become a serious drain on company resources and morale.

The following three mistakes related to team composition are among the most common we encounter at new hardware companies:

  1. Committing to a schedule before having the execution team
  2. Hiring junior or mid-level engineers and expecting senior-level work
  3. Attempting to build a full-stack engineering team

Committing to a Schedule Before Having the Execution Team

In the previous article, we discussed how teams can build an unworkable schedule by focusing exclusively on hardware development milestones. Another major scheduling mistake is committing to a schedule before hiring an engineering team to execute your plan.

Hardware startups often base their schedules around market timing or investor expectations. The schedules are entirely fantastical, since teams construct them without input from the engineers who will be designing and building the product. Fundamentally, the company has agreed to deliver a product before having the ability to produce that product.

Teams making this kind of overly-aggressive schedule also exhibit unrealistic expectations for the time it takes to hire engineers, especially firmware developers. Young teams often think they can fill an engineering position in 2-6 weeks. Reality is not so kind: in the San Francisco Bay area, we find that it takes between 3-6 months to hire for each firmware engineering position. If a team’s schedule is dependent on quick hiring, reality will expose the plan as impractical.

When the leadership team sets the schedule without input from the actual engineers doing the work, and the hiring takes longer than expected, the new engineers that join the team are immediately put under pressure. The firmware is almost always “late” by the time the first engineer joins the team, and everyone else on the team is anxious to see the system working. This situation leads to conflicts, cutting corners, unhappy engineers, and delayed releases.

Your schedule is ultimately dictated by the capabilities of your team and the staff you have on hand at the present moment. You need your team’s input when creating the plan. Keep these facts in mind and create a schedule that matches your team’s realistic capabilities.

Hiring Junior or Mid-Level Engineers and Expecting Senior-Level Work

The most common mistake we see young hardware companies make is hiring junior or mid-level engineers and expecting them to produce senior-level work. This can occur because the CEO is not aware of a senior engineer’s value, because the schedule “necessitates” beginning work without senior engineering guidance, or because the team gave up on finding a senior engineer after a long search.

Hiring a senior engineer is essential for the success of a project, especially in today’s world of highly interconnected systems. These companies simply must start with someone who has shipped products before and led the whole process first hand. Junior engineers lack the experience, perspective, and design skills necessary to architect and build new systems. When someone with underdeveloped engineering and design skills builds a system, the most common result is a hard-to-maintain system held together with duct tape and prayers. What’s under the hood often resembles a shanty town, not the 5-star hotel the CEO envisioned. New features become increasingly difficult (or impossible) to implement due to the fragility of the system. When you add multiple interacting systems into the mix, the resulting design problems can multiply and prevent you from shipping your product.

Senior engineers have the experience and foresight to set requirements and identify critical design problems before it’s too late. They can provide guidance to junior engineers and ensure that they aren’t overtaxed. Senior engineers are also crucial when working with contractors, design houses, and other external suppliers. Junior engineers may not have the experience or perspective to review the work products, judge whether the quality is sufficient, and determine if the design satisfies the requirements. Having a senior engineer to manage the work of external teams acts as an insurance policy against integrating low-quality work into your system.

If you want to avoid schedule delays and hard-to-maintain systems, you must have senior engineers on staff.

Attempting to Build a Full-stack Engineering Team

Many new hardware companies focus on building a full-stack team that can handle all aspects of product development. As my friend Saket Vora pointed out, companies should instead build a team that focuses on their core business value.

Most small companies outsource accounting, bookkeeping, and payroll activities to external firms, since they do not directly contribute value to the business. For most new hardware teams, they can likewise outsource the bulk of firmware and hardware development to consultants, contractors, and design firms. Given the difficulty in hiring senior engineers at early stage companies, utilizing external experts can also help alleviate the problem of expecting senior-level work from junior engineers.

Unless your company’s focus is innovative firmware or electrical development, writing firmware and designing circuit boards does not add direct value to your business. Your team will need one experienced engineer who can integrate and review the outsourced work. Your remaining full-time engineers should be fully focused on feature development, integrating with external services, and improving the customer experience.

Another aspect to consider is whether you can justify the employment of your full-stack team. We’ve repeatedly seen companies hire multiple full-time firmware and electrical engineers and successfully keep the team busy while shipping the first product. As efforts shift from new product development to feature improvements, marketing, and customer support, these engineers find there is no work for them to do. If you don’t immediately have the second product on deck, the under-utilized engineers get laid off, leave the company because they’re bored, or simply hang around and serve as a drain on your runway.

Headcount is a precious commodity, and it can quickly turn into an anchor on your company. Make sure you carefully build your team to deliver on your core business value propositions. You don’t get style points for doing it all in house, and the customer can’t tell the difference.

Next Up

That’s it for common team mistakes. Next week, we’ll summarize our list of ten common blunders and provide resources for teams that wish to learn more about product development and project management.

A Look at Ten Hardware Startup Blunders, Part 2: Schedule and Focus

At Embedded Artistry, we've spent countless hours meeting with founders, working at startups, and swapping war stories with fellow engineers. After enough iterations, we started to notice a repeated set of critical strategic missteps made by new hardware companies. These missteps result in compromised designs, increased development costs, and continual schedule delays.

We've selected ten mistakes regularly observed at new hardware companies which fall into three general categories: process, focus, and team. This article is the second in our four-part series and focuses mistakes related to the team's focus.

Focus-related Mistakes

Building a new hardware product can be an overwhelming endeavor. There are many facets to consider: mechanical hardware, electrical hardware, firmware, app software, server software, packaging, supply chain, manufacturing, marketing, and more.

Many new hardware teams go astray by focusing heavily on specific aspects of product development while ignoring others. A hardware company must design an entire ecosystem, and each aspect of the system must be properly considered for all the pieces to fall into place. Teams that focus too heavily on a subset of product development efforts while ignoring others will encounter late-stage re-design efforts and schedule delays.

Let's look at four focus-related mistakes that send hardware companies astray:

  1. Skipping software architecture
  2. Prioritizing Mechanical Design
  3. Setting the schedule based on hardware milestones
  4. Heading straight to China for manufacturing

Skipping Software Architecture

Teams will regularly focus on industrial, mechanical, and electrical prototyping and design efforts before beginning software engineering and design work. When it comes to software, most new companies bypass software prototyping and architecture altogether. Teams dive right into coding, even before they understand the system requirements and product functionality.

The popular view of software is that it's infinitely changeable and cheap to develop. This view ignores the fact that software, especially firmware, is one of the most expensive aspects of creating a new product. Software development efforts for most products will cost between $15-$45 per line of code. More complex engineering projects like the space shuttle may reach up to $1000 per line. For many projects, a mere 1000 lines of code will end up costing as much as a kilogram of gold. Simple projects might get away with a design involving 25,000 lines of code, while more intensive products will have hundreds of thousands.

Without performing software architecture and software design processes, startups significantly increase the risk of schedule delays due to design flaws. Hardware ecosystems are complicated and typically involve multiple interactive software components. Your firmware design and electrical design are tightly coupled, and selecting incompatible processor/storage/RAM which can’t support the true needs of the product will lead to significant redesign efforts.

Prototyping efforts are invaluable when building new products. Teams can quickly try out solutions, check the feasibility of features, and create product demos. These prototypes are quickly thrown together with little thought given to system integration, interface design, or interactions with other software systems. Time and time again we have seen prototypes transform into the shipping design, inevitably causing teams to enter into a never-ending cycle of patching holes in a sinking ship.

Dedicate time to intentionally designing the shipping software architecture for your product. Your architecture efforts should consider the hardware product itself, as well as the application software and server software that it will interact with. Once you have an architecture for the software, you can create a more effective product development schedule - you know what your team needs to build, and you can identify pieces that can be parallelized, outsourced, purchased, or licensed.

Working systems don’t come together by accident, and software architecture efforts are a necessary activity to ensure your system intentionally meets your design goals. Failure to spend time on software architecture will result in continual schedule delays due to redesign and debugging efforts.

Prioritizing Mechanical Design

Many teams begin their product development by exclusively focusing on the industrial and mechanical design. Teams want their product to look good, and mechanical design is satisfying: it exists in the physical world, and you can show off the prototypes to your investors and entice prospective employees. Often, mechanically focused teams only begin to consider electrical and software engineering after they lock the industrial and mechanical design. Locking mechanicals before considering software and electrical constraints will create design challenges for your team - some of these constraints will inevitably impact the mechanical design.

The mechanical design places constraints on MLB size and component selection. With today's trend of miniaturization, our electrical designs must fit into smaller and smaller packages. This comes with a tradeoff in increased complexity of the electrical design, manufacturability concerns, and limits on component sizing. Cramming our electrical designs into small packages requires specialized layout expertise and often comes with an increase in signal integrity and desense problems. Teams may also find that the components that are needed to support the product's features simply don't fit into the package, leading to feature cuts or requiring the software team to mimic hardware features. Your product's performance becomes limited by the components which fit within a limited size range. Perhaps the perfect component is larger than one your mechanical design will support.

The product's thermal performance, another common design issue, impacts how the software and electrical hardware function and how long the product can operate. Thermal problems are best resolved with a mechanical design change, although many teams choose to implement difficult workarounds: software must support load monitoring and dynamic throttling, layout gymnastics must be performed to shuttle heat around the PCB, or features are limited because there isn't enough thermal budget.

When teams lock mechanical and industrial design before considering the electrical and software requirements, a laundry list of mechanical change requests inevitably results. Management and mechanical engineers meet these change requests with significant pushback because the changes often necessitate new tooling or modifications to the look and feel of the product. This pushback creates team friction, requires feature cuts, and often results in schedule delays.

Your team should not lock down your mechanical design until they understand the full product requirements. Teams must design and prototype the full electro-mechanical-software system before committing to expensive tooling purchases. In many systems, thermal constraints are a significant design problem requiring mechanical updates. Oftentimes the perfect components for your design may require a larger form factor, or selecting smaller components results in an increased BOM cost or difficulty securing the parts.

Lock down your mechanical design only after the team understands the full system requirements and you have prototyped the production intent product. All members of the engineering team must have the opportunity to weigh in and confirm their ability to design to the requirements within the given the mechanical constraints.

Setting the Schedule Based on Hardware Milestones

Hardware companies tend to set their development schedule based purely on hardware milestones. The universality of the New Product Introduction (NPI) process and the defined build milestones make it easy for teams to throw together a schedule. The team picks dates for these milestones based on market timing (e.g. “We want to be in stores by Christmas”) instead of basing the dates on engineering accomplishments (e.g. “We’ve implemented the critical features and are ready to refine the design and ship to customers.”).

Teams with aggressive hardware schedules end up spacing development builds based on hardware lead-times. The team, typically small at a startup, focuses primarily on building the system. They do not have time to sufficiently review data, validate, debug, and improve the design before the next hardware build. Hundreds of thousands of dollars worth of development hardware is produced, most of it destined to become paperweights.

Aside from purely mechanical designs, hardware companies today are building complex ecosystems involving multiple interdependent systems:

  • Electrical hardware
  • Mechanical hardware
  • Firmware
  • Application software
  • Server/backend software
  • Packaging
  • Accessories

When companies rush through the various hardware build milestones, they generally learn that the ship date must be delayed because the software isn’t ready. Millions of dollars in capital is tied up in inventory that cannot be sold to customers while the developers are still completing the software. Weeks and months pass by while the developers finalize the software, and CM payment penalties start piling up in the form of storage and shipment delay fees.

Teams must base the product schedule on the development of the entire ecosystem, not just the EE and ME components. Your team needs sufficient time to test the designs, identify improvements, and incorporate changes for the next milestone.

In almost every case of a ship date delay that we’ve encountered, software readiness was the culprit. If you base your schedule on your total ecosystem and team’s capabilities, you will spread out your hardware builds. Your hardware team will have time to validate and refine the design, and the quality of your hardware will increase. The hardware team can also benefit from time to design and validate the accessories between major system builds.

Heading Straight to China for Manufacturing

New hardware startups tend to head straight to China for manufacturing their products. We even encounter teams that focus on building manufacturing relationships in China without even having a working prototype or list of product requirements.

In most cases, we believe that manufacturing in China is the wrong choice for new hardware companies. We suspect that companies head straight to China because they want to want to be like the big guys - Apple, Google, Amazon, etc..

Manufacturing in China places a significant burden on your company:

  • Your team does not benefit from the lessons learned during small manufacturing runs in a timely and effective manner due to language barriers and the long distance between your engineers at home base and your engineers in China
  • You must pay NRE costs for setting up the manufacturing process
  • You must pay for labor during development builds
  • You must pay for your employees to fly to China and stay in hotels
  • You must commit to build dates and build quantities, and if you do not meet those commitments you will still pay for the builds and parts ordered
  • You have to worry about customs issues for importing and exporting parts and products
  • If you build too few development units, you must wait until the next build event to produce more
  • If you build too many development units, you waste capital
  • Employees spend significant time traveling to/from China, taking away from product development time
  • Employee effectiveness drops due to jet lag and exhaustion
  • Employees frequently do not have access to the tools they need
  • Employee morale suffers due to frequent travel, long trips, and time away from family
  • Teams constrain their hiring to those employees who are willing to travel to China frequently

Until you have a product design that is ready for production at scale, finding a CM in China will only create unnecessary stress and financial burden. It's best to focus on your piece part supplier relationships first.

Before you get there, take that NRE money and invest it in local design houses, prototyping shops, and fabricators to finalize your design. Plenty of local small-scale turnkey producers exist, and many of these producers can provide quick turnarounds on small orders. Instead of waiting two months to test your next design, you can have parts in hand within days or weeks. While your cost per unit will be higher, you will spend less money overall by getting immediate design feedback to your team. And your employees will be able to work toward the end goal more efficiently.

In the early stages, your team should be assembling your product in house. Your engineers will gain valuable learnings from assembling the product and seeing the problems first hand. You can build continuously to quickly iterate on the design - if there is a required change, you can get parts built and integrated without waiting for a full manufacturing event.

Start looking for a CM partner only when your team has finalized the product design and is ready for large-scale manufacturing. If you are still designing your product, there is no sense in committing to manufacturing deadlines and production quantities. With a final design you are much better armed to know your timelines and quantities and can focus on pricing negotiation. It's difficult and costly to do both design and negotiation in parallel.

Next Up

That’s it for common focus mistakes. Next week, we’ll take a look at mistakes related to startup team composition.