Learn How to Assess the Type and Usability of Manufacturer-Supplied Reliability Data

Manufacturer-supplied reliability data can be a valuable source of information if it is properly understood and applied. Manufacturer data is especially important to reliability engineers who must estimate the reliability of little known or unique components. However, the source and quality of manufacturer data varies from company to company.

Oftentimes, reliability engineers who are assessing overall system reliability need to use manufacturer data when performing their analyses. If this data is not adequately understood, the MTBF (Mean Time Between Failure) values generated from it are likely to be overly optimistic; this can result in inappropriate or incorrect decisions being made. It is important, therefore, that manufacturer data be interpreted and applied correctly. This article provides general how-to information to help reliability practitioners to better assess and use manufacturer data.

Evaluating Manufacturer Data

Manufacturer data is typically, and most appropriately, used to augment a reliability prediction or internal test plan. When manufacturer data is to be used for this purpose, some effort must be put forth to ensure that it is used correctly.

The flowcharts in Figures 1 and 2 provide processes for evaluating test and field data. To begin, you must determine the source of your manufacturer data. Test data is typically accompanied by a list of the tests performed. If you cannot tell whether the manufacturer data is from tests or field operation, you should call and ask the manufacturer.

Figure 1. Evaluation Process for Test Data

Figure 2. Evaluation Process for Field Data

Evaluation of Test or Laboratory Data

What type of test is performed?

Test data is likely to be derived from operating life and/or accelerated life test results. Because both operating life and accelerated life tests are conducted to simulate customer use, the load used in these tests will be the load that the manufacturer anticipates that the component will encounter in the field. This type of data is the preferred source of test data used in reliability predictions.

Test data may also be derived from acceptance test and burn-in test results. Because failures are less likely to occur during these tests, the Chi-Squared distribution is often used to determine the MTBF. (For additional information about the Chi-Squared distribution, refer to Calculating MTTF When You Have Zero Failures.) While acceptance test and burn-in test results can be used as upper and lower bounds on the MTBF values used in reliability predictions, it is best to avoid using the MTBF values calculated from test and burn-in test results in your reliability predictions.

Does the operating profile used for the test resemble your operating profile?

Because the operating profile that is used for the test greatly affects the failure rate that is calculated, it is important that you understand it. The two examples below highlight how understanding the operating profile for the text can help you to avoid incorrect use of the manufacturer data.

• Example 1: If a solid-state sensor is tested using vibration levels similar to ground benign operation, it is very likely that this sensor will possess a much higher failure rate if used in a more stressful environment, such as in a fighter airplane.
• Example 2: The MCTF (Mean Cycles To Failure) numbers supplied by a gear manufacturer may be based on intermittent operation and dynamic loading conditions, especially in the case of plastic gears, which are not intended for constant use. Applying a MCTF number that was derived from intermittent testing to a continuously operating gear would significantly underestimate the actual failure rate.

Are the test results compromised by faulty assumptions?

Assumptions made during tests can influence the usefulness of the test data. For instance, a modified Arrhenius equation is often used to predict the failure rate of a component at a certain temperature given an activation energy. Increasing the temperature or decreasing the activation energy increases the failure rate. Accelerated life tests provide information about the effect of temperature increases on failure rates. However, some manufacturers assume that the activation energy value of their component is 0.7. If the activation energy value deviates from 0.7, this error is compounded in the MTBF resulting from the accelerated life tests.

Does the product make use of a new technology?

New technology may introduce unexpected failure modes that remain undetected until the product reaches the field. If you decide to use the data, you should do so with caution. It might be best to perform Bayesian analysis of this data rather than simply using it in your reliability predictions. Additional information about Bayesian analysis can be found in the last section of this document.

Evaluation of Field Data

Are failed items returned to the manufacturer?

Some manufacturers have excellent visibility into field failures whereas others, such as manufacturers of most passive components, do not. In some cases, failed components are sent back to their manufacturers. In most cases, however, failed items are simply replaced with new components, and the failed items are discarded rather than returned. For example, a manufacturer of radiation-hardened FPGAs (Field-Programmable Gate Arrays) would be likely to receive failed units as returns, while a manufacturer of thick-film resistors would not.

Did the manufacturer analyze the data correctly?

It is important to determine if the manufacturer has performed accurate field FIT (Failure In Time) analysis. The effectiveness of the manufacturer's FRACAS (Failure Reporting and Corrective Action System) program impacts the accuracy of field MTBF values. A FRACAS tracks field failures and can provide good metrics for field evaluation if the appropriate data is collected and analyzed. For example, companies who have good visibility of field failures are more likely to have a FRACAS that will generate useful field failure rates. However, not all manufacturers have an effective FRACAS in place. Knowledge about a manufacturer's capabilities will assist in determining the validity of field failure rate numbers.

Do you use the component in a unique way?

If you use a component in a dramatically different fashion than that for which it was intended, the field data may not apply. You should only use this data if you know the operating profiles are similar.

Using Manufacturer Data

Different reliability prediction models have different ways of accounting for manufacturer data. If you have concluded that it is appropriate to use manufacturer data in your prediction, there are several ways to incorporate the information.

• Use the manufacturer-supplied failure rates.
  If the data appears to be truly representative of your conditions, stresses, and temperatures, replace your predicted failure rate with the failure rate specified in the manufacturer data. When you do this, be sure to consider how this failure rate is to be combined with the failure rates from other sources. For instance, manufacturer MTBFs are typically based on a 60% confidence interval, which means that 60% of the components will meet or exceed the given MTBF. However, reliability predictions are commonly based on a 90% confidence interval on a part-by-part basis. To accommodate for the difference between a 60% and 90% confidence interval, the manufacturer MTBF (provided at the 60% confidence) must typically be divided by at least 2.1 to arrive at a 90% confidence MTBF. The actual divisor depends upon the number of failures analyzed. Because this number can vary greatly, even major manufacturers put cautions regarding the use of their data on their web sites.
• Use the manufacturer's activation energy value in the Arrhenius equation to determine the failure rate.
  If the data appears to have been gathered in such a way that the conditions are similar but the operating temperature is different, use the manufacturer's activation energy value in the Arrhenius equation to determine the failure rate. Replace your predicted failure rate with this calculated failure rate. For additional information about the Arrhenius equation, refer to the Relex help or consult reliability textbooks such as Practical Reliability Engineering (Patrick O'Connor) or Reliability: A Practitioner's Guide (Intellect and Relex Software Corporation). You can also refer to MIL-STD-338.
• Use Telcordia Case II, Telcordia Case III, or PRISM Bayesian analysis to incorporate the field experiences into reliability predictions.
  If the data is gathered in such a way that neither the conditions nor the temperature closely match your situation, then use Telcordia Case II, Telcordia Case III, or PRISM Bayesian analysis to incorporate the field experiences into reliability predictions. Bayesian statistics modify the predicted failure rate value based on the supplied lab or field data. When a component is well defined by the reliability prediction standard in use, this method is generally preferred because the calculation model provides for the consideration of the operating environment, stress, and temperature. The Bayesian analysis then modifies the predicted failure rate to reflect the manufacturer's test or field data. The Bayesian method is also valuable for analyzing components that you plan to re-use and may be more beneficial than using the manufacturer-supplied failure rate. When you use a component in two different designs, it is likely that the operating temperature and stresses will vary. Using the prediction model with Bayesian data will allow the reliability engineer to specify different stresses and temperatures while using the "lessons learned" to modify the predicted results.

Relex tools provide support for incorporating manufacturer supplied-failure rate data into your predictions. For additional information, please contact your Relex Application Consultant or send an e-mail request to info@relex.com.