“The process of design applied to products that are to be manufactured through techniques of mass production.”
Interface Control Document [ICD]
A systems or software engineering document that provides a record of all interface information generated for a project.
Failure Analysis [FA]
“Failure analysis is the process of collecting and analyzing data to determine the cause of a failure, often with the goal of determining corrective actions or liability.”
Engineering Validation and Test [EVT]
EVT is a stage in the NPI process. EVT units are intended to test the functionality of your product against its requirements. In some sense, EVT is a “feasibility study” of the design.
EVT builds will often be the first time that a proper Form Factor Engineering Prototype device is built – one that both works like and looks like the intended product. However, it is still common to produce Non-Enclosed Devices. Some materials, such as housings, may be in short supply. NEDs are also useful for software development and EE teams, as they provide easier access to the components for debugging and measurement purposes.
True yields at the EVT stage are quite low. There will be manufacturing process errors, out-of-spec components, and other problems. This is useful, however, as the engineering team will investigate failures and improve the design and manufacturing processes to improve yield (though significant improvements may not be seen until future builds).
EVT units should meet the requirements outlined in the Product Requirements Specification before proceeding DVT. There will often be significant design changes that need to be made at this stage, requiring at least another EVT event (“EVT-2”). We have rarely worked on products that made it past EVT with a single build event.
Qualities of EVT
- EVT builds produce a small quantity of units. Of course, what “small” means varies according to company resources and product cost.
- Small may be “5-25 units”, built in 1-5 unit batch sizes
- Small may be “50-100 units”, built in 5-10 unit batch sizes
- Small may be “500-1000 units”, built in 50-100 unit batch sizes
- EVT builds often involve several configurations (e.g., distinguished by using different component vendors)
- Production-intent materials are used, however:
- Cosmetics are almost always ignored at this stage
- Production-intent materials may not be available (e.g., tooling has not been kicked off), so 3D printing, soft tooled parts, or milled parts may be used in place of dye cast or molded parts
- Testing happens, but is often a secondary concern
- Often, you’re bringing up test stations for the first time at EVT.
- Software may not yet be stable, requiring frequent software updates to address issues.
- At a minimum, test stations should be collecting data. Limits may be wide open, or selected to catch only egregious failures.
- If parametric test limits exist, you will often still pass every unit through the line even when out-of-spec.
- Units that fail in some particular way can still be useful for other teams, e.g., for software development.
- Manufacturing process steps are refined as the team works through the assembly, testing, and repair processes
Uses of EVT Units
EVT units are used for:
- Internal development
- Validation of the design
- Identification of issues that need to be fixed
- Comparing alternative configurations (e.g., different component vendors)
NPI Process Flow
- EVT follows the Proto Stage, when the team has a path forward on a design that is worth the manufacturing effort.
- The EVT stage is completed when there is at least one production-worthy product configuration that meets the requirements outlined in the Product Requirements Specification. If this has not happened, another EVT event will be scheduled, incorporating improvements from the previous design.
- Some companies will set yield targets for exiting EVT, but these will usually be quite low (e.g. 60% yield).
- After EVT is completed, the DVT stage begins.
Related Concepts
- Manufacturing Test and Manufacturing Firmware often make their debut at EVT
- Engineering Prototype
- Non-Enclosed Device
- Form Factor
References
- Hardware engineers speak in code: EVT, DVT, PVT decoded by Anna-Katrina Shedletsky
The EVT build is the first time you combine looks-like and works-like into one form factor, with production intent materials and manufacturing processes.
Purpose:
- To select the production intent design, sometimes from a build matrix of options
- To identify all of the issues that need to be fixed with that design
Typical Quantities: 100 to 1000
- Units must be fully functional and testable, made from the intended materials and with the intended manufacturing process, but may be from soft-tools (if you’re using 3D printed parts, it’s not EVT!)
- All functional test stations must be present and collecting data
Things that Go Wrong:
- A new revision of an intended design does not work after reliability testing
- Tighter than expected (or capable) tolerances are needed to meet the intended performance specifications — such as with an antenna element
- Depending on product complexity, up to ~40% of the units built may fail for a variety of functional or performance reasons and need to be analyzed
- Engineering has started the battle to get glue processes, hand-soldering, environmental seals, and other tricky steps under control
Exit Criteria: one production-worthy configuration that meets all of the product requirements for functionality, performance, and reliability
- The different engineering validation stages in a nutshell | EVT, DVT, PVT | by Chris Boucher | Medium
- Overview of the hardware product development stages: POC – EVT – DVT – PVT explained
- The objective of the EVT is to combine look-alike and work-like subsystem prototypes made of intended components to meet the functional requirements in the form factor as per your PRD (product requirements Document).
- EVT prototype quantities: 3-50 units, depending on the design complexity and BOM cost. On average, 5-12 prototypes are required to complete the EVT.
- Technologies: 3D printing, laser cut/milled PCBs, soft tooling (silicon molds), professional hardware development kits (HDK), rapidly cut/milled parts;
- Outputs / Deliverables: fully-functional prototype with key components performing as intended.
- Limitations: Prototypes delivered throughout the EVT phase may look somewhat ugly, raw and have a lack of beautiful cosmetic finish. The EVT prototype can also miss some non-key mechanical features such as handles, curves in enclosure, painting, etc.
Engineering Prototype [EP]
“Also known as the ‘looks-like / works-like’ prototype.”
Engineering Change Order [ECO]
An engineering change order (ECO) is a documentation packet that outlines the proposed change, lists the product or part(s) that would be affected and requests review and approval from the individuals who would be impacted or charged with implementing the change.
Design Validation and Test [DVT] [DVT]
DVT is a stage in the NPI process. DVT units should represent, as much as possible, the final production-intent design. No major future design changes should be expected at the start of DVT (otherwise, you should have another EVT build).
The goal of the DVT is to validate that the MP-intent production process can build production-intent units at sufficient quality. Unlike EVT, DVT enforces test limits. Fallout rates are often high, especially early in the build, requiring engineering engagement to correct the problems and bring yields. up.
At the end of DVT, you should be confident that any issues causing unacceptable yield losses have been (or will be) corrected. If yields are not at an acceptable level, or resolutions are uncertain, another DVT event is warranted (“DVT-2”).
Qualities of DVT
- Units are produced at “medium” quantities: 2-5x EVT quantities.
- This often means 250-2500 units are produced (and in larger batch sizes than at EVT).
- Units are produced in fewer configurations than EVT – ideally, one per SKU. This is rarely adhered to, however.
- There are often challenges such as your production-intent supplier producing lower-than-expected yields, requiring you to evaluate an alternative.
- Additional configurations may be created as cost-down experiments.
- Keep in mind that additional configurations add significant costs. You need to build each configuration in sufficient quantity to prove that the design is suitable for production.
- If you are building multiple small-quantity configurations, you are probably not at the DVT stage.
- There may be experiments or “DOEs” to evaluate different process parameters: different glue vendors, varied glue curing times, modified assembly orders, etc.
- Production-intent components should be used
- Devices are all form factor units
- Components should come from production processes (e.g., using hard tools, not soft tools or prints or mills)
- This is often the first time that hard tools are used at a build and thus represents qualification for those tools.
- Economic reasons may still require the use of, e.g., milled parts instead of hard-tooled parts, but this should be minimized, as it represents a significant risk if you see these parts for the first time at PVT.
- Cosmetics may still not be at the desired quality level from the supplier
- Capabilities like dust- and water-proofing should work at DVT
- Manufacturing test stations are enforcing realistic limits, allowing you to understand (and improve) actual process yield.
- Since there is new
- Failures may still be waived, depending on how egregious the failure is. This often involves setting “continue-on-fail” policies, allowing you to track your true process yield while still producing units that are good enough for development or testing purposes.
- Packaging is typically introduced, and packaging processes evaluated
- Additional checks on the manufacturing line are added: e.g., cosmetic inspection, OQC
- For cosmetics, there will often be an effort to track down cosmetic fallout introduced on the line, but this is often difficult when input components are not cosmetically sound
Uses of DVT Units
DVT units are used for:
- Development
- Certification efforts (FCC certification, UL certification, Bluetooth certification, etc.)
- Reliability and environmental testing
- Internal and external beta testing
- Test station software and manufacturing firmware validation at the CM
NPI Process Flow
- The DVT stage begins once there is at least one EVT configuration that meets the requirements outlined in the Product Requirements Specification
- The DVT stage is complete when:
- Yield loss problems have been addressed (or there is high confidence that corrective actions put in place after DVT will address the yield problems)
- Certifications have passed (design modifications to address certification failures may be significant enough to warrant another DVT event, though some teams plunge into PVT anyway)
- Reliability and environmental testing have yielded acceptable results (design modifications to address failures may be significant enough to warrant another DVT event, though some teams plunge into PVT anyway)
- Packaging for the device is finalized
- As a reminder, the DVT units must still meet the Product Requirements Specification.
- After the DVT stage is complete, PVT begins.
Exit Criteria: high confidence in all corrective actions for any issue that causes unacceptable yields on units using mass production parts made from mass production tools.
Related Concepts
- Manufacturing Test limits are enforced at DVT and used to fail units from the line. However, test limits may still be wider than expected in future build stages.
- Outgoing Quality Control is often introduced at this stage for process development and feedback on manufacturing
References
- Hardware engineers speak in code: EVT, DVT, PVT decoded by Anna-Katrina Shedletsky
The DVT build is supposed to be one configuration of your production-worthy design, made of components from production processes (and hard tools) and on a line following production procedures. I believe very few companies actually stick to this requirement — because even if miraculously there are no outstanding issues, there may be parallel efforts to cut cost or increase yields that create additional configurations to build.
If you do have functional, performance, or reliability issues that are driving Plan B and Plan C configurations at this stage, it can be costly because each of those alternates needs to be built in “full quantity” to ensure that design can be fully mass-production qualified by the end of the build. I believe that’s the real test for whether you are at DVT or not: if you are running side configurations of 20 units, you are fooling yourself, and should call it EVT2.
Purpose:
- To verify mass production yields with one production-worthy design (one configuration for each shipping SKU)
- To qualify the first hard tool for every part in the assembly
Typical Quantities: 300 to 2000
- All parts should be from hard tools or mass production capable processes
- All functional test stations must be present with limits in place to understand true yields
Things that Go Wrong:
- High functional fallout rates — requiring the need for fast failure analysis and corrective actions
- Cosmetic yields are 0% — there may be an effort to try to track down and fix cosmetic aggressors, but it is usually fruitless because your cosmetic part suppliers are likely still shipping scratched parts (and you are having to waive them)
- DOEs (there’s another one! Design of Experiments, mentally replace with “experiments”) are run with alternate glues or curing parameters
- there are nightly calls with vendors demanding support or giving updates to hardware company executives
Exit Criteria: high confidence in all corrective actions for any issue that causes unacceptable yields on units using mass production parts made from mass production tools.
- The different engineering validation stages in a nutshell | EVT, DVT, PVT | by Chris Boucher | Medium
- Overview of the hardware product development stages: POC – EVT – DVT – PVT explained
- The objective of the DVT is to fix the design (i.e. dimensions, weight, materials, finish, moving mechanical parts) and rationalize the final product’s features.
- At this stage you should carefully revise and consider features vs product quality/finish vs production and BOM cost vs production volume.
- Complete the necessary certifications;
- Develop and finalize boxing and packaging
- Commence to request RFQs from mass-producers and devise plans for logistics.
- DVT prototype quantities: typically 20-200 units, depending on the design complexity and BOM cost. The prototypes will be used for various reasons: certification lab tests, “beta tests” with early customers/testers.
- Technologies: 3D printed + gel-coated enclosures with the finish “as from the factory”, rapidly cut/milled parts; industrial equipment (e.g. injection moulding) and 1st generation tooling (e.g. “quick moulds”).
- Outputs / Deliverables: a [batch of] functional prototypes ready for mass-production with BOM and a design documentation package. Boxing and Packaging design completed. Estimate mass-production yields
- Limitations: The DVT prototypes and documentation is nearly final and can be slightly changed further in development. Some mechanical parts and electronic components may not be final due to economic reasons (e.g. it is cheaper to CNC mill some metallic parts instead of using dye casting).
- The objective of the DVT is to fix the design (i.e. dimensions, weight, materials, finish, moving mechanical parts) and rationalize the final product’s features.
Design-for-X [DFX]
“X can be any generic trait or attribute which the team is designing for. At Microsoft, this phrase can mean ‘Design for Excellence’.”
Design for Variability [DFV]
Design to account for and support variability, where variability refers to the unpredictability, inconsistency, unevenness, and changeability associated with a given nuance.
Design for Cost [DFC]
Designing for cost is the conscious use of engineering process technology to reduce life cycle cost.
