Avoiding the pitfall of over testing
13 March 2013
Some designers tend to err on the side of caution by over-testing products, wasting money through unnecessary testing costs and slowing the time to market.
The benefits of ensuring the continued compliance of electronic products by testing them, both at the pre-production stage and as part of an ongoing audit process, are obvious. Indeed, the high levels of safety and reliability that end-users now expect from products are because testing often goes beyond just meeting regulatory or contractual requirements. However, testing can also be duplicated unnecessarily to ‘cover all bases’.
Over-testing often occurs because customers’ demands reflect a poor understanding of the product’s intended environment and the specifications that need to be used to qualify if it should be tested, or what tests should be applied. Consequently, some designers tend to err on the side of caution by over-testing products. This not only wastes money through unnecessary testing costs, but often slows the time to market for new product launches in a highly competitive space.
What price reputation?
Brand loyalty can be quickly lost if a product is defective, unreliable, or in the worst case scenario it injures someone. Protecting a brand’s reputation is therefore a major factor not only in requiring that tests are done correctly, but also in explaining why products are frequently over-tested.
Linked with brand reputation is the issue of post-sale warranty returns costs. This is another key driver for ensuring that the correct tests are done to help reduce product failures. It’s also obvious that customers will have more confidence in their suppliers if they can prove that they have an effective test strategy in place. Being able to demonstrate to customers that you are an extremely diligent supplier, even if they do not require it contractually, will inevitably increase their confidence in all of the products delivered.
So the legacy of over testing within an organisation often exists if product reliability problems in the past have had a negative impact on brand, warranty or customer satisfaction. This may make the business over-cautious in terms of the amount of testing that is done in order to try and ensure that such a poor product is not produced again and that the shockwaves of such a failure never impact the wider organisation.
To test or not to test
While there are good reasons for testing, these often lead to a blind spot in terms of how companies approach testing and indeed actually over-test. There are in fact many considerations that should first be taken into account before embarking on a set of expensive tests.
Firstly, it is important to check whether the product in question has been tested to another specification. There are a huge variety of tests and standards that have very similar specifications, meaning that taking a ‘read across’ approach may be appropriate. This is where test specialists compare two standards, identify where they differ and create an argument for partial testing or not testing at all.
It is also vital to consider if the product in question is very similar to others that have already been tested, as a ‘qualification by similarity’ may be possible. If there is a lot of similarity between the current and a previous product it may also be possible to ‘read across’ from the previous product’s test results to avoid some re-testing. For example, this would apply to product upgrades where it is the same product but with some additions. Likewise for a full product range, it may be possible simply to test the basic and top-end product without testing all of the model variations in between.
If an existing product is sold into a new market, do not ignore the value of good field evidence of its current application. While it may not satisfy a very particular requirement in a completely new market, such as a significant change in temperature from its current use, such evidence may have some value and indeed a new customer may be willing to accept that field record so that the product does not need to be re-tested.
Even if a product is vastly different from a previous model there may be constructional similarities which could reduce the requirement for embarking on specific tests such as contamination. This could be the case where the materials used are the same as those on previous products. If this is the situation there will be no need to run the very specific requirements of contamination testing again. Likewise, if the material’s performance characteristics are already known and are intrinsically resistant to the known contaminants and their conformance can be proven, there may be no reason to test. An example would be that we know that plastics do not rust when they get wet. This might seem obvious, but we do regularly see such testing time and money wasted.
Often, manufacturers test their products repeatedly to similar but slightly different standards in order to meet country specific requirements. Despite the existence of the CB Scheme, an international scheme operated by the IECEE (IEC System for Conformity testing and Certification of Electrotechnical Equipment and Components), we see many companies still applying for testing and certificates with a plethora of certification bodies to gain access to individual export markets.
The CB Scheme is the world's first international system for the mutual acceptance of test reports and certificates for electrical and electronic components, equipment and products. Before the CB Scheme, manufacturers’ only option was to have their products tested and certified by many different national testing laboratories/certification bodies - a difficult, time-consuming and expensive process.
Essentially a CB test report is a technical passport which helps a product gain individual country approval. It does not completely eliminate the need for additional “in-country” approval or testing but it does get you 85 per cent of the way there. The multilateral agreement reduces significantly the need for duplicate testing, is operational in 53 countries, and is being used by more than 15,000 manufacturers worldwide.
If you are selling products internationally and not currently benefiting from this, then the CB Scheme is for you, as many countries will now accept CB test reports and certificates without the need for local certification. The ability to carry out one test programme to, in effect, gain many national marks, faster and at a lower cost surely means that more manufacturers should be taking advantage of it?
The customer is not always king
Frequently, we find that a customer’s original requirements are flawed because of:
? Poorly understood environment – while the customer is aware of the intended environment, they do not necessarily capture all of the environments for every possible use, often concentrating on operational environments and neglecting transportation.
? Poorly defined test requirements –the test standards and methods that are defined by the customer show a misunderstanding and tests are incorrectly specified. A good example is that a customer once wanted to specify a gritty sand environment but actually mistakenly specified an environment simulated using soft fine talcum powder.
? Exaggerated test parameters – there is always a temptation to be ‘safe’ with test parameters, involving rounding up and adding safety margins. Be sure that this has only happened once, not once for each person involved!
If it transpires that your product does require testing, very often the duration of tests can be cut to reduce costs. Historically, climatic tests were run overnight starting at 5PM and ending a 9AM, so a 16 hour dwell soon became a standard. If a product stabilises more quickly why would you pay for unnecessary laboratory time? Stabilisation plus two hours is now accepted and the common sense approach.
It is also important to consider if combined tests satisfy two requirements. For example, an altitude test may satisfy both an altitude and a temperature requirement, thus significantly reducing test times and laboratory costs. Where there is already confidence that the product is sufficiently robust, storage and operational tests may be combined as a sequence, offering a reduction in time over the tests performed individually.
Quite simply, when a manufacturer demands that certain tests are done, it is advantageous for both them and their test house to reassess if these tests are required, or determine if they can be reduced.
Most importantly, first check that the standards relating to the product have been interpreted correctly. For example, has the environment been correctly interpreted and the test requirements properly defined? Also, do some research in the short-term to make long-term savings by reviewing what testing has already been performed, look for similarity with other products and consider those that already exist in field performance evidence. If you do have to test, you can save time and money by looking at how you might combine tests and reduce test durations where possible.
By spending some effort in the short term at the outset of a testing relationship and considering some of the advice outlined above, you could find that you can make significant long-term costs savings, as well as release your product onto the market quicker to stay one step head of the competition.
Figure 1 Electrical testing
Figure 2 HALT testing in action
Contact Details and Archive...