Proto Stage [Proto]

The Proto stage in the NPI Process involves creating prototype devices and finalizing the product design. Prototypes are tested and evaluated to ensure they meet the requirements and specifications as outlined in Product Requirements Specification. During this stage, there will often be several adjustments and modifications to the design.

The Proto stage often involves creating a small number of units (1-10). Only rarely will complete units be built – and if they are, they are often non-enclosed devices (NEDs), wire-wrapped prototypes, or systems assembled out of existing off-the-shelf development kits. Mechanical components are often quickly prototyped, such as with a 3D printer. “Looks-like” prototypes may be created, but lack any actual functionality, while the “works-like” prototypes are often too large to fit into the target enclosure at this stage.

There is often little-to-no testing that happens at the Proto stage. If there are tests, they are usually for informational purposes only, and not for preventing units from leaving the line.

NPI Process Flow

  • The Proto stage begins once there is a Product Requirements Specification and engineering resources are assigned.
  • The Proto stage is completed once there is enough confidence in the design specifications and approach.
  • After the Proto stage is complete, EVT begins.

References

  • NPI Process
  • Hardware engineers speak in code: EVT, DVT, PVT decoded – Instrumental by Anna-Katrina Shedletsky

    The Proto build is a small test run of key product concepts to gain confidence that they can work — potentially a combination of different form factors including looks-like and works-like.

    Purpose: to understand risks around specific modules or designs, usually with multiple variants in low quantities, such as:

    • Fragility of coverglass in drop test with different adhesives, perhaps done on dummy housing bucks
    • Waterproofness of five different button seal designs

    Typical Quantities: 10 or fewer, sometimes no “full systems” are even built

    • Parts may be “stand-ins” or rapidly prototyped (which may change results for better or worse)
    • Sub-modules do not have to be integrated — units may be “works like” or “looks like”

    Things that Go Wrong:

    • Part quality is poor, resulting in incorrect dimensions or an interference was missed in the CAD (3D model), so parts do not fit together and have to be modified by hand
    • Pin 1s on connectors were not correctly mapped, so things do not electrically work even when plugged together
    • The intended design fails miserably during testing and needs to be redesigned

    Exit Criteria: one design concept for the product that the team has reasonable confidence is three major iterations or less from a mass-production worthy design

Ramp

“Ramp” refers to the NPI process stage following PVT. The main goal from ramp is transition from PVT to Mass Production (MP) output volumes. In practice, Ramp is a distinct stage, but in description it is often lumped together as a “subset” of MP.

Ramp can involve a number of activities:

  • Additional assembly lines are brought up to increase production volumes.
  • Processes may be sped up until they reach an optimum in the tradeoff between throughput and quality.
  • Test limits may be adjusted to increase the throughput on the line.

Ramp also faces a number of challenges:

  • The pressure to increase throughput and output can result in poor-quality parts being allowed onto the line, causing an increase in failures.
  • One or more components can gate the transition to mass production due to quality problems or late deliveries.
  • As production quantities increase, the absolute number of failed units that needs to be inspected by the engineering team increases, leading to potential delays in identifying and addressing critical problems.

NPI Process Flow

  • Ramp immediately follows PVT.
  • Ramp transitions into Mass Production once assembly lines have been brought up, quality/throughput have stabilized, and you have sufficient build material in house and on the way to manufacture the desired number of devices.

References

Production Validation and Test [PVT]

PVT is a stage in the NPI process. During PVT, no further design changes are expected. You are focused on working out the final kinks in the manufacturing process before entering Mass Production. The goal of the engineering team is to produce one “golden line” that operates with desired yield and throughput. This golden line can then be replicated by the operations teams to scale up production.

Qualities of PVT

  • Volumes are typically 5-10% of the initial mass-production run.
  • Production-intent processes and parts are in place.
    • Custom tooled parts are used: no more milling, printing, soft molds, etc.
    • Some tools and processes may be introduced at PVT; they are in effect being qualified before entering mass production.
    • In practice, there may still be some experiments going on at PVT, but this is not the ideal situation and you should strive to avoid it.
  • The units that are built at PVT are revenue-able (can be sold to customers).
    • If this is not the case, you’re not ready for PVT, but having a DVT build.
  • The focus is on improving yield and throughput to hit mass production goals
    • The build will often start “slow” and speed up while moving through gated phases (‘red’, ‘yellow’, ‘green’ being common phase descriptors) that reflect operator training level, throughput, and yield levels.
    • Test station software and manufacturing firmware are improved to reduce retest rates and cycle time.
    • Process flaws are addressed to improve yield and throughput.
    • Cosmetic fallout caused by activities on the manufacturing line is addressed.
      • PVT is often heavily focused on cosmetic yield.
  • The packaging flow is perfected.
  • Outgoing Quality Control (OQC) and/or Final Quality Control (FQC) processes roll out in force.
  • Proceeding into the next PVT build phase (or on to Ramp and Mass Production) is often gated by a problematic vendor or three, whether due to yield problems, insufficient build quantities, late deliveries, or other problems.

Uses of PVT Units

PVT units are used for:

  • Sale
    • Units at this stage should ideally be “revenue-able” – able to be sold to customers.
    • Often, cosmetic flaws are the reason that units from PVT will not be sold. You can still use these for other purposes. If not terribly egregious, you can offer them to friends and family at a significant discount.
  • Internal development
  • Beta testing
  • “Golden units” (ideal devices) are used for GR&R activities, test station validation, and manufacturing firmware validation at the CM

NPI Process Flow

  • The PVT stage begins after DVT has been completed with:
    • Sufficient confidence in addressing yield loss issues
    • Certifications have been completed
    • Packaging is ready
    • Reliability and environmental testing show acceptable results
  • A significant change in the design at the PVT stage should move the product back to DVT. In practice, you are more likely to see “Pre-PVT” or “PVT-2” builds than a reset to DVT.
  • PVT is completed when:
    • There are acceptable yields and throughput for mass production on at least one manufacturing line (the “golden line”, which will be replicated to other lines).
    • You have sufficient build material in house and on the way to manufacture the desired number of devices.
  • After the PVT stage is completed, production begins to ramp to Mass Production levels.

References

  • Hardware engineers speak in code: EVT, DVT, PVT decoded by Anna-Katrina Shedletsky

    PVT is the “last build” — the units you are building are supposedly intended to be sold to customers, if they pass all of your test stations. PVT typically transitions directly into Ramp and Mass Production, or a Pilot build with no time gap.

    Purpose: to verify mass production yields at mass production speeds

    • Validate and qualify additional tools needed to support quantities for early ramp
    • No parallel experimental units allowed (I have never seen this actually happen, but it is a goal that should be driven to for as long as possible)

    Typical Quantities: 1K to 20K

    • All units are intended to be sold to customers
    • The build is potentially phased — red, yellow, green is common — indicating “maturity” of the production process, which includes a combination of operator training level, line speed, and line yield

    Things that Go Wrong:

    • There is almost always at least one issue that is still outstanding at the start of PVT — this is likely the item at highest risk of impacting your schedule
    • There is usually at least one vendor whose yields are way lower than expected, and because they cannot produce at the quantities promised, input is gated by their deliveries
    • If you have a high cosmetic standard, your cosmetic yield likely starts at 0%. Unless you decide to loosen your standard, the conventional way to improve it is to knowingly input units to a 0% yield line and painstakingly seek places where damage occurs and improve them. This process can take weeks and hundreds or thousands of units. An Instrumental system can streamline and significantly accelerate this process

    Exit Criteria: mass production yields at mass production speeds on at least one line, and replication to other lines already started.

  • The different engineering validation stages in a nutshell | EVT, DVT, PVT | by Chris Boucher | Medium
  • Overview of the hardware product development stages: POC – EVT – DVT – PVT explained
    • PVT objectives:
      1. Verify mass-production yields;
      2. Finalize DF-X with the help of CM aiming to minimize waste and make assembly more efficient;
      3. Make the first pilot production run and ensure the product quality adheres to your expectations;
      4. Weed out the last design flaws during the pilot production run;
    • PVT prototype quantities typically range between 50 and 500 in order to verify mass-production yields and provide product samples.
    • Technologies: Industrial technologies suitable for volume production only;
    • Outputs / Deliverables: Final product produced in a limited quantity by using the tools for mass-production. Electronic layouts and components are revisited using PCB stencils for soldering components. Mechanical DFM is finalized and plastic parts are manufactured by using 2nd generation moulds.
    • Duration: 3-6 months in general.
    • Limitations: The time required to design and produce custom tools is generally long.

New Product Introduction [NPI Process]

The New Product Introduction (NPI) process is a framework for taking a new product from design to manufacturing through a series of structured phases. The goal of the process is producing a functional, reliable, manufacturable, and cost-effective product.

The NPI process is not a standard that is set-in-stone, but more of a general pattern. The implementation and interpretation of the process and its stages varies from company to company. We learned the NPI process at Apple, which certainly colors our interpretation of the process. You will find descriptions of the process similar to ours from other ex-Apple engineers, such as Anna-Katrina Shedletsky of Instrumental.

Stages

  1. Product Requirements Specification (also called the Product Requirements Document)
  2. Proto
  3. EVT
  4. DVT
  5. PVT
  6. Ramp
  7. Mass Production

References

Mass Production [MP]

Mass production (MP) is the final stage in the NPI and represents sustaining production of the new product in meaningful quantities.

In MP, the bulk of the responsibility shifts from engineering to the CM and operations team.

Ideally, at this stage, there will be no changes to the product or process. In practice, there will often be ongoing efforts to reduce costs and improve yields (even if only by loosening limits). New tools or vendors will likely need to be qualified at some point (through a Post Ramp Qualification) – often due to cost advantages, supply problems, or end-of-life issues. Minor design changes and test coverage improvements may occur, especially as a result of Early Field Failure Analysis (EFFA) activities.

NPI Process Flow

  • Ramp transitions into Mass Production once sufficient assembly lines have been brought up and quality/throughput have stabilized.
  • Eventually, engineering resources cycle off of the project and the factory manages successive mass production runs. Quality usually degrades when the factory is left unsupervised.
  • At some point, the project will reach end-of-life (EOL).

References

  • Manufacturing
  • Hardware engineers speak in code: EVT, DVT, PVT decoded by Anna-Katrina Shedletsky

    PVT flows immediately into the phase of the program called Ramp, where parallel assembly lines are being brought up to increase daily output volume. Mass Production is a superset of Ramp and the sustaining production that follows.

    Purpose:

    • Bring up multiple lines in parallel to support high volume
    • Continue to improve ongoing yield
    • Qualify additional tools or vendors
    • Make design changes based on returns, Early Field Failure Analysis (EFFA), or cost down efforts

    Things that Go Wrong:

    • Vendors change processing parameters or take down tools for maintenance, resulting in dimensional or quality shifts that can cause line failures
    • Parts from unqualified tools are allowed on the line and cause failures
    • A single-sourced part becomes the supply gate, usually due to ongoing yield issues
    • Quality tends to decrease as engineering is pulled away and factory is left unsupervised

Engineering Validation and Test [EVT]

EVT is a stage in the NPI process. EVT units are intended to test the functionality of your product against its requirements. In some sense, EVT is a “feasibility study” of the design.

EVT builds will often be the first time that a proper Form Factor Engineering Prototype device is built – one that both works like and looks like the intended product. However, it is still common to produce Non-Enclosed Devices. Some materials, such as housings, may be in short supply. NEDs are also useful for software development and EE teams, as they provide easier access to the components for debugging and measurement purposes.

True yields at the EVT stage are quite low. There will be manufacturing process errors, out-of-spec components, and other problems. This is useful, however, as the engineering team will investigate failures and improve the design and manufacturing processes to improve yield (though significant improvements may not be seen until future builds).

EVT units should meet the requirements outlined in the Product Requirements Specification before proceeding DVT. There will often be significant design changes that need to be made at this stage, requiring at least another EVT event (“EVT-2”). We have rarely worked on products that made it past EVT with a single build event.

Qualities of EVT

  • EVT builds produce a small quantity of units. Of course, what “small” means varies according to company resources and product cost.
    • Small may be “5-25 units”, built in 1-5 unit batch sizes
    • Small may be “50-100 units”, built in 5-10 unit batch sizes
    • Small may be “500-1000 units”, built in 50-100 unit batch sizes
  • EVT builds often involve several configurations (e.g., distinguished by using different component vendors)
  • Production-intent materials are used, however:
    • Cosmetics are almost always ignored at this stage
    • Production-intent materials may not be available (e.g., tooling has not been kicked off), so 3D printing, soft tooled parts, or milled parts may be used in place of dye cast or molded parts
  • Testing happens, but is often a secondary concern
    • Often, you’re bringing up test stations for the first time at EVT.
    • Software may not yet be stable, requiring frequent software updates to address issues.
    • At a minimum, test stations should be collecting data. Limits may be wide open, or selected to catch only egregious failures.
    • If parametric test limits exist, you will often still pass every unit through the line even when out-of-spec.
      • Units that fail in some particular way can still be useful for other teams, e.g., for software development.
  • Manufacturing process steps are refined as the team works through the assembly, testing, and repair processes

Uses of EVT Units

EVT units are used for:

  • Internal development
  • Validation of the design
  • Identification of issues that need to be fixed
  • Comparing alternative configurations (e.g., different component vendors)

NPI Process Flow

  • EVT follows the Proto Stage, when the team has a path forward on a design that is worth the manufacturing effort.
  • The EVT stage is completed when there is at least one production-worthy product configuration that meets the requirements outlined in the Product Requirements Specification. If this has not happened, another EVT event will be scheduled, incorporating improvements from the previous design.
    • Some companies will set yield targets for exiting EVT, but these will usually be quite low (e.g. 60% yield).
  • After EVT is completed, the DVT stage begins.

References

  • Hardware engineers speak in code: EVT, DVT, PVT decoded by Anna-Katrina Shedletsky

    The EVT build is the first time you combine looks-like and works-like into one form factor, with production intent materials and manufacturing processes.

    Purpose:

    1. To select the production intent design, sometimes from a build matrix of options
    2. To identify all of the issues that need to be fixed with that design

    Typical Quantities: 100 to 1000

    • Units must be fully functional and testable, made from the intended materials and with the intended manufacturing process, but may be from soft-tools (if you’re using 3D printed parts, it’s not EVT!)
    • All functional test stations must be present and collecting data

    Things that Go Wrong:

    • A new revision of an intended design does not work after reliability testing
    • Tighter than expected (or capable) tolerances are needed to meet the intended performance specifications — such as with an antenna element
    • Depending on product complexity, up to ~40% of the units built may fail for a variety of functional or performance reasons and need to be analyzed
    • Engineering has started the battle to get glue processes, hand-soldering, environmental seals, and other tricky steps under control

    Exit Criteria: one production-worthy configuration that meets all of the product requirements for functionality, performance, and reliability

  • The different engineering validation stages in a nutshell | EVT, DVT, PVT | by Chris Boucher | Medium
  • Overview of the hardware product development stages: POC – EVT – DVT – PVT explained
    • The objective of the EVT is to combine look-alike and work-like subsystem prototypes made of intended components to meet the functional requirements in the form factor as per your PRD (product requirements Document).
    • EVT prototype quantities: 3-50 units, depending on the design complexity and BOM cost. On average, 5-12 prototypes are required to complete the EVT.
    • Technologies: 3D printing, laser cut/milled PCBs, soft tooling (silicon molds), professional hardware development kits (HDK), rapidly cut/milled parts;
    • Outputs / Deliverables: fully-functional prototype with key components performing as intended.
    • Limitations: Prototypes delivered throughout the EVT phase may look somewhat ugly, raw and have a lack of beautiful cosmetic finish. The EVT prototype can also miss some non-key mechanical features such as handles, curves in enclosure, painting, etc.

Design Validation and Test [DVT] [DVT]

DVT is a stage in the NPI process. DVT units should represent, as much as possible, the final production-intent design. No major future design changes should be expected at the start of DVT (otherwise, you should have another EVT build).

The goal of the DVT is to validate that the MP-intent production process can build production-intent units at sufficient quality. Unlike EVT, DVT enforces test limits. Fallout rates are often high, especially early in the build, requiring engineering engagement to correct the problems and bring yields. up.

At the end of DVT, you should be confident that any issues causing unacceptable yield losses have been (or will be) corrected. If yields are not at an acceptable level, or resolutions are uncertain, another DVT event is warranted (“DVT-2”).

Qualities of DVT

  • Units are produced at “medium” quantities: 2-5x EVT quantities.
    • This often means 250-2500 units are produced (and in larger batch sizes than at EVT).
  • Units are produced in fewer configurations than EVT – ideally, one per SKU. This is rarely adhered to, however.
    • There are often challenges such as your production-intent supplier producing lower-than-expected yields, requiring you to evaluate an alternative.
    • Additional configurations may be created as cost-down experiments.
    • Keep in mind that additional configurations add significant costs. You need to build each configuration in sufficient quantity to prove that the design is suitable for production.
      • If you are building multiple small-quantity configurations, you are probably not at the DVT stage.
    • There may be experiments or “DOEs” to evaluate different process parameters: different glue vendors, varied glue curing times, modified assembly orders, etc.
  • Production-intent components should be used
    • Devices are all form factor units
    • Components should come from production processes (e.g., using hard tools, not soft tools or prints or mills)
      • This is often the first time that hard tools are used at a build and thus represents qualification for those tools.
      • Economic reasons may still require the use of, e.g., milled parts instead of hard-tooled parts, but this should be minimized, as it represents a significant risk if you see these parts for the first time at PVT.
    • Cosmetics may still not be at the desired quality level from the supplier
    • Capabilities like dust- and water-proofing should work at DVT
  • Manufacturing test stations are enforcing realistic limits, allowing you to understand (and improve) actual process yield.
    • Since there is new
    • Failures may still be waived, depending on how egregious the failure is. This often involves setting “continue-on-fail” policies, allowing you to track your true process yield while still producing units that are good enough for development or testing purposes.
  • Packaging is typically introduced, and packaging processes evaluated
  • Additional checks on the manufacturing line are added: e.g., cosmetic inspection, OQC
    • For cosmetics, there will often be an effort to track down cosmetic fallout introduced on the line, but this is often difficult when input components are not cosmetically sound

Uses of DVT Units

DVT units are used for:

  • Development
  • Certification efforts (FCC certification, UL certification, Bluetooth certification, etc.)
  • Reliability and environmental testing
  • Internal and external beta testing
  • Test station software and manufacturing firmware validation at the CM

NPI Process Flow

  • The DVT stage begins once there is at least one EVT configuration that meets the requirements outlined in the Product Requirements Specification
  • The DVT stage is complete when:
    • Yield loss problems have been addressed (or there is high confidence that corrective actions put in place after DVT will address the yield problems)
    • Certifications have passed (design modifications to address certification failures may be significant enough to warrant another DVT event, though some teams plunge into PVT anyway)
    • Reliability and environmental testing have yielded acceptable results (design modifications to address failures may be significant enough to warrant another DVT event, though some teams plunge into PVT anyway)
    • Packaging for the device is finalized
    • As a reminder, the DVT units must still meet the Product Requirements Specification.
  • After the DVT stage is complete, PVT begins.

Exit Criteria: high confidence in all corrective actions for any issue that causes unacceptable yields on units using mass production parts made from mass production tools.

  • Manufacturing Test limits are enforced at DVT and used to fail units from the line. However, test limits may still be wider than expected in future build stages.
  • Outgoing Quality Control is often introduced at this stage for process development and feedback on manufacturing

References

  • Hardware engineers speak in code: EVT, DVT, PVT decoded by Anna-Katrina Shedletsky

    The DVT build is supposed to be one configuration of your production-worthy design, made of components from production processes (and hard tools) and on a line following production procedures. I believe very few companies actually stick to this requirement — because even if miraculously there are no outstanding issues, there may be parallel efforts to cut cost or increase yields that create additional configurations to build.

    If you do have functional, performance, or reliability issues that are driving Plan B and Plan C configurations at this stage, it can be costly because each of those alternates needs to be built in “full quantity” to ensure that design can be fully mass-production qualified by the end of the build. I believe that’s the real test for whether you are at DVT or not: if you are running side configurations of 20 units, you are fooling yourself, and should call it EVT2.

    Purpose:

    1. To verify mass production yields with one production-worthy design (one configuration for each shipping SKU)
    2. To qualify the first hard tool for every part in the assembly

    Typical Quantities: 300 to 2000

    • All parts should be from hard tools or mass production capable processes
    • All functional test stations must be present with limits in place to understand true yields

    Things that Go Wrong:

    • High functional fallout rates — requiring the need for fast failure analysis and corrective actions
    • Cosmetic yields are 0% — there may be an effort to try to track down and fix cosmetic aggressors, but it is usually fruitless because your cosmetic part suppliers are likely still shipping scratched parts (and you are having to waive them)
      • DOEs (there’s another one! Design of Experiments, mentally replace with “experiments”) are run with alternate glues or curing parameters
      • there are nightly calls with vendors demanding support or giving updates to hardware company executives

    Exit Criteria: high confidence in all corrective actions for any issue that causes unacceptable yields on units using mass production parts made from mass production tools.

  • The different engineering validation stages in a nutshell | EVT, DVT, PVT | by Chris Boucher | Medium
  • Overview of the hardware product development stages: POC – EVT – DVT – PVT explained
    • The objective of the DVT is to fix the design (i.e. dimensions, weight, materials, finish, moving mechanical parts) and rationalize the final product’s features.
      1. At this stage you should carefully revise and consider features vs product quality/finish vs production and BOM cost vs production volume.
      2. Complete the necessary certifications;
      3. Develop and finalize boxing and packaging
      4. Commence to request RFQs from mass-producers and devise plans for logistics.
    • DVT prototype quantities: typically 20-200 units, depending on the design complexity and BOM cost. The prototypes will be used for various reasons: certification lab tests, “beta tests” with early customers/testers.
    • Technologies: 3D printed + gel-coated enclosures with the finish “as from the factory”, rapidly cut/milled parts; industrial equipment (e.g. injection moulding) and 1st generation tooling (e.g. “quick moulds”).
    • Outputs / Deliverables: a [batch of] functional prototypes ready for mass-production with BOM and a design documentation package. Boxing and Packaging design completed. Estimate mass-production yields
    • Limitations: The DVT prototypes and documentation is nearly final and can be slightly changed further in development. Some mechanical parts and electronic components may not be final due to economic reasons (e.g. it is cheaper to CNC mill some metallic parts instead of using dye casting).