Predicting Reliability at the Prototype Phase

Reliability engineering is often thought of as an exercise undertaken when a product is far enough along to be ready for manufacturing.  Conversely, it’s always helpful to ask early on:  Is the promise of your technology linked to high reliability in its mature product form?  What’s behind this assumption and what can you learn about it by testing early prototypes?

The answer often lies in gathering and understanding reliability data for off-the-shelf components, combined with testing any custom parts and then the integrated prototypes, always from the perspective of the product’s intended use.  This means considering the product environment, duty cycles, and performance requirements as defined in the Product Requirements Document, and testing that’s informed by the Risk Management Plan, which helps to identify the most likely failure modes.

To the extent practical, testing can be completed to gain confidence that the design as currently prototyped may yield a reliable product.  However, many factors conspire to confound reliability projections and the answers are often far from perfect.  Short development schedules and prototype heterogeneity typically don’t allow for reliability datasets large enough to draw statistically significant conclusions.  In addition, differences between the design and fabrication processes for prototypes and production products blur the applicability of the results.  In fact, designers may even dismiss prototype failures as artifacts of the prototype processes. This is often true, but every observed failure tells a story and we designers should strive to understand them all.  Thoughtful testing with prototypes can contribute to confidence (or lack thereof) in the design approach, and this is always valuable input for the next prototype iteration.  Here are some basic tips for performing useful reliability tests with prototypes:

If you only have one or two prototypes, don’t test for reliability until a later phase when you have more, unless you have a clear, well-defined reliability concern (eg fatigue) and you are fairly certain you can test without breaking an expensive prototype.

  • Before you start testing prototypes, make sure you understand whether they are all of the same configuration or whether they contain differing subcomponents or assembly methods that may affect the statistics coming out of reliability testing.
  • There are typically a multitude of issues that limit the reliability of prototypes.  Without making a project out of it at this stage, strive to understand the root cause of all observed failures.  Investigate potential design changes in response to the root causes regardless of whether you’re certain they’ll propagate into the commercial product.  Design changes at this stage are much easier to make than in later iterations.
  • If your product is a system consisting of a durable device and a single-use consumable, for example, a diagnostic instrument with an assay cartridge, or an infusion device with a disposable line set, you are likely to have extremely limited single-use consumables at this development stage, and they may not offer the fidelity to do any system level testing.  Recognize that it is not possible to draw meaningful conclusions on datasets with less than ~50 consumables for most failure modes and others may require hundreds or thousands be tested.  If this is the case, don’t even start system-level reliability testing at this stage.  It will be a waste of time and money.
  • When you perform testing, it can help to categorize failures in order to find the best path forward:
    •  High probability design issues – happens on all prototypes – pay particular attention to these.
    • “Bad actors” – one or two of your prototypes fails an order of magnitude more often than the others.  Look for differences to give clues and make sure your next design iteration accommodates what you find.
    • Use errors or modifications the users are inadvertently making – since testing is not being performed by end users.  Ignore these – it’s not relevant to test for these at this stage.

Reliability testing and prediction at the prototype phase can be imprecise and far from perfect, but doing a limited scope of thoughtful testing with prototypes can yield productive design improvements when they’re still easy to make.  To find out more about early reliability testing, or if you’d like us to take a look at your prototype performance, contact us here.

Jenny Regan

Every challenge is different – Tell us about yours.