How Many Products To Test

The number of products to test depends on many things. These include the risk involved, the cost of testing, and how many items you make. For many simple items, testing a small batch is enough.

For very important items, like those used in medicine, you test a lot more.

Understanding Product Testing Numbers

Deciding how many items to test isn’t a simple guess. It’s a smart choice based on facts. We need to think about what could go wrong.

We also need to think about how much it costs to find out. Every business is a little different. What works for one might not work for another.

But there are solid ways to figure this out.

The main goal is to be confident. You want to be sure the products you send out are good. You want them to work as they should.

You also want them to be safe for folks to use. Finding the right number means you don’t waste resources. You also don’t take on too much risk.

It’s about smart sampling.

My Own Testing Wake-Up Call

I remember when I first started my small craft business. I made these hand-poured soy candles. They smelled amazing, and people loved them.

I was so excited. For my first big batch, I made about 100 candles. I packed one box to test shipping and sent it to a friend.

It arrived fine. Then, I just boxed up the rest and sent them out.

A few weeks later, I got an email. A customer said her candle was tunneling. That means the wax burns down the middle, leaving wax on the sides.

It wastes the candle. I felt a sinking feeling. I’d never even thought about checking each candle for that.

I realized I had sent out many others that might have the same issue. I was lucky it wasn’t something more serious. That day, I learned that just assuming things are okay is not enough.

You need a plan for testing.

What Is a “Test Lot”?

When we talk about testing, we often think about a “lot” or a “batch.” This is a group of products. They are made under the same conditions. They are supposed to be all the same.

Think of it like a single baking day. All the cookies made that day are one lot. If you test one cookie from that batch, you hope it tells you about all the others.

The number you choose to test from this lot is your “sample size.” This sample size is key. It’s the heart of making a good decision. A bigger sample size gives you more confidence.

But it also costs more time and money. A smaller sample size is cheaper. But you might miss a problem.

Factors That Shape Your Testing Number

So, how do you pick that sample size? Several things come into play. Think of them as puzzle pieces.

You need to fit them together.

Risk Level of the Product

This is a big one. How risky is it if your product fails? If you sell t-shirts, a defect might mean a customer is unhappy.

They might return it. But if you make car brakes, a defect could be very dangerous. It could cause a crash.

Products that can cause harm or major loss have a higher risk. For these, you must test more items. You want to be very sure there are no defects.

Agencies like the FDA for food and drugs or the NHTSA for car safety have strict rules. These rules often dictate how much testing is needed.

Production Volume

How many products do you make in total? If you make only 50 items, you might test 5 or 10. If you make 50,000 items, testing 10 isn’t enough.

You can’t test every single one. That would be too costly and slow.

For large volumes, you rely more on statistical sampling. This means using math to pick a sample. This sample is meant to represent the whole group fairly.

The bigger the total number of items made, the more complex your sampling plan needs to be. But usually, the percentage you test goes down as the total number goes up.

Cost of Testing

Testing costs money and time. Some tests are quick and cheap. Others need special equipment or experts.

They can take days or weeks. You have to balance the cost of testing against the cost of a failure.

What’s the cost if a product fails in the customer’s hands? It might be a refund. It could be a damaged reputation.

It might even be a lawsuit. Compare that to the cost of running the tests. You want to find the sweet spot.

Acceptable Quality Level (AQL)

This is a fancy term for “how many defects are we okay with?” For most consumer goods, you aim for a very low AQL. This means you want almost zero defects. For some less critical items, you might accept a tiny percentage of defects.

An AQL is often set by industry standards or by the buyer. For example, an AQL of 1% means you are okay if up to 1 out of every 100 items has a minor flaw. It doesn’t mean you aim to have flaws.

It means if a few slip by, it’s within an agreed limit.

Common Sampling Methods

There are several ways to choose your sample size. Here are a few common ones.

Simple Random Sampling

This is like drawing names from a hat. Every product in the lot has an equal chance of being chosen. This is fair.

It helps make sure your sample truly represents the whole batch.

How to do it: Assign a number to each product. Then, use a random number generator to pick numbers. The products with those numbers are your sample.

This works well when your products are all mixed up.

Systematic Sampling

Here, you pick items at regular intervals. For example, if you make 1000 items and want a sample of 100, you might pick every 10th item. So, item #10, #20, #30, all the way to #1000.

This is easier than random sampling. But you must be sure there’s no pattern in your production line. If every 10th item is made slightly differently, this method could give you a bad sample.

Stratified Sampling

This is used when your product lot has subgroups. Imagine you make red and blue shirts. You want to make sure you test both colors fairly.

You would divide your lot into strata (groups). Then, you take a random sample from each group.

For example, if 70% of your shirts are red and 30% are blue, you’d take 70% of your sample from the red shirts. You’d take 30% from the blue shirts. This ensures each important subgroup is tested.

Using Standards: MIL-STD-105E and ISO 2859

For many industries, there are established standards. These help guide sample size. The most famous ones were MIL-STD-105D (and its updates like MIL-STD-105E).

Now, the ISO 2859 standards are widely used. These are international versions.

These standards provide tables. You look up your lot size. You also pick an AQL.

The table then tells you the sample size. It also tells you how many defects are acceptable before you reject the whole lot. These standards are based on statistics.

They help ensure consistent quality across many businesses.

For example, if you make 5000 items and set an AQL of 2.5% for critical defects, the standard might tell you to inspect 200 items. It might say that if you find 10 or more defects in those 200, the lot is rejected. This takes the guesswork out.

How Many Products to Test: The Numbers Game

Let’s get to the heart of it. How many products do you actually test? There’s no single magic number.

But we can use guides and common sense. Here’s a breakdown for different scenarios.

For Small Batches (Under 100 items)

If you make a very small number of items, say for a limited run or a craft fair, you might be able to test a good portion. For a batch of 20 items, testing 5-10 might be reasonable. This gives you high confidence.

It’s not too expensive.

Experience: When I make a small batch of custom-painted art pieces, I often inspect every single one. It’s faster than picking a few. It ensures each unique piece is perfect before I deliver it.

For Medium Batches (100-1000 items)

Here’s where statistical sampling really becomes useful. You can’t test them all. Using a table like ISO 2859 is common.

For a lot size of 500, and a general AQL of 1.5%, you might test around 80 items.

You’d look up your lot size in the table. Then choose your inspection level (normal, tightened, reduced). You’d also pick your AQL.

The table gives you a sample size code letter. Then another table tells you the specific number of items to test.

For Large Batches (Over 1000 items)

As the lot size grows, the sample size usually grows too, but at a slower rate. For 10,000 items with an AQL of 1%, you might test around 315 items. The statistical methods become very important here.

Expertise: Relying on established standards like ISO 2859 is crucial for large-scale production. They are designed by experts. They use proven mathematical models to give you the best chance of catching defects without testing every item.

Different Types of Testing Affect Numbers

The kind of test you do also changes things. Some tests are destructive. This means the product is ruined during the test.

You can’t test many of these. Others are non-destructive. You can test them and then sell them.

Destructive Testing

This is common for checking strength, durability, or material limits. For example, you might bend a metal rod until it breaks. Or you might run an electronic device until its battery dies completely.

Because the product is destroyed, you can’t test many. Your sample size will be small. You rely heavily on the accuracy of these few tests.

You also need to be sure the manufacturing process is very consistent.

Non-Destructive Testing

This includes visual inspection, electrical checks, or performance tests that don’t damage the item. You can test more items this way. This allows for larger sample sizes.

Real-World Context: In a clothing factory, they do non-destructive tests. They check stitching, color, and measurements on many garments. They might do a destructive test on one sample garment per batch to check fabric strength.

This balances resources.

What to Look For in Your Sample

What makes a product “fail” your test? This comes down to your quality standards. These should be clear before you start.

Critical Defects

These are defects that could cause harm. They could make the product unsafe. Or they could lead to major failure.

Example: A crack in a baby bottle. A faulty brake line in a car. A short circuit in an electronic device that could cause a fire.

If you find critical defects, you often stop the entire production line. You reject the whole lot immediately.

Major Defects

These defects significantly reduce the product’s usability or value. They don’t necessarily cause harm but make the product not work as intended.

Example: A smartphone screen that doesn’t light up properly. A piece of clothing with a ripped seam. A tool that doesn’t perform its main function.

Minor Defects

These are small flaws that don’t affect the product’s function or safety. They are mostly cosmetic.

Example: A small scratch on the back of a phone. A slightly uneven stitch in an inconspicuous area. A bit of dried glue on a shoe that can be easily removed.

When Is It Normal to Test Fewer Products?

There are times when testing a smaller number makes sense.

Very Low-Risk Products

If you sell something with almost zero risk, like simple paper decorations or decorative items that don’t move or change, you might test less. But you still need some checks.

Mature and Stable Processes

If you have been making the same product for years. Your factory has a very reliable process. You have a long history of very few defects.

You might be able to reduce your sample size. This is often called “reduced inspection” in the standards.

Experience: A well-established bakery that has made the same bread recipe for decades likely doesn’t test every loaf for taste. They have great confidence in their process. But they still do quick visual checks and bake a few loaves to ensure consistency.

When Quality Control is Built-In

If your manufacturing process has many checks built in at each step, you might need fewer final tests. For example, if sensors automatically check the size of every part as it’s made, your final inspection might be lighter.

When Do You Need to Test More Products?

On the flip side, there are clear reasons to increase your testing.

High-Risk Products

As mentioned before, anything that could harm someone requires more scrutiny. This includes medical devices, children’s toys, and automotive parts.

New or Changing Processes

If you’ve just started making a product. Or if you’ve changed a key part of your manufacturing. You don’t know how stable the process is yet.

Test more to catch any early problems.

Expertise: When a new material is introduced or a new machine is installed, quality engineers will typically increase the sample size. They watch the data closely. They look for unexpected issues.

This is a standard practice to manage new risks.

Issues from Previous Batches

If a past batch had problems, you might need to test more for a while. This is often called “tightened inspection.” You keep testing more until you are confident the problem is fixed.

Specific Customer Requirements

Sometimes, your customer, the company buying your products, will have specific testing demands. They might require a certain sample size or a specific AQL. You must meet these requirements.

Practical Steps for Your Business

So, how do you put this into action for your own products?

1. Understand Your Product’s Risk

On a scale of 1 to 10, how dangerous is it if your product fails? Be honest. Is it an inconvenience or a hazard?

2. Know Your Production Volume

How many units do you make per batch or per day/week?

3. Define Your Quality Standards

What is an acceptable defect? What is not? Create a simple list or visual guide.

4. Look Up Relevant Standards (If Applicable)

For many industries, standards like ISO 2859 exist. Consult them if they fit your product type.

5. Choose a Sampling Method

Simple random, systematic, or stratified? Keep it practical for your setup.

6. Determine Your Sample Size

Use tables, online calculators, or consult with a quality expert if needed. Start with general inspection levels.

7. Set Your AQL

What percentage of defects is acceptable for minor issues? For major ones?

8. Document Everything

Keep records of your testing, your sample sizes, and the results. This builds your history.

Trustworthiness: It’s vital to document your testing process. This shows you are serious about quality. If a problem does occur, your records can help you understand what happened and how to prevent it in the future.

Can I Just Test 5% of Everything?

Many people wonder about a simple rule like “test 5%.” While 5% might be reasonable for some products and lot sizes, it’s not a universal rule. For a lot of 20 items, 5% is only 1 item. That’s probably too low for confidence.

For a lot of 1000 items, 5% is 50 items. This might be a good sample size, depending on the risk and AQL. The percentage you test often goes down as the total number of items goes up.

This is why using tables and statistical methods is better than just picking a percentage.

What About Testing “One of Each Kind”?

If you make many different versions of a product (like different colors or sizes), testing just one of each might not be enough. Let’s say you make a chair. You offer it in red, blue, and green.

If the red chair has a weak leg, testing only one blue and one green chair won’t catch it.

You need to consider the number of items in your lot. If you make 100 red chairs and 100 blue chairs in the same batch, your total lot size is 200. You would use the tables for a lot size of 200.

You might then ensure your sample includes some of each color. This is where stratified sampling helps.

The Role of Suppliers

If you buy components or finished goods from suppliers, you’ll need to consider their testing. Do they provide test reports? Do you perform incoming inspections on their products?

Authority: Reputable suppliers often have their own quality control. They might follow ISO 9001 standards. However, it’s still wise for you to do some level of incoming inspection, especially for critical parts.

This verifies their quality claims. Agencies like the Consumer Product Safety Commission (CPSC) provide guidelines for product safety testing, which can influence what you require from suppliers.

For example, if you use a supplier for electronic chips, you might ask for their detailed test results. You might also take a sample of the chips you receive and test them yourself to confirm they meet specifications. The number you test would depend on the risk of the chip failing.

Your Questions Answered

Can I test every single product I make?

You can, but it’s usually not practical or cost-effective for most businesses. Testing every item is called “100% inspection.” It’s only done for very high-risk products like life-support equipment or when defects are extremely easy and quick to spot.

What is a good sample size for a small batch?

For small batches under 100 items, a sample size of 5-20% is often a good starting point. For example, if you have 30 items, testing 3 to 6 of them gives you decent confidence.

How does AQL affect the number of products to test?

A lower AQL (meaning you want fewer defects) generally requires a larger sample size. A higher AQL (meaning you can tolerate more defects) might allow for a smaller sample size.

What if I find defects in my sample?

If you find critical or too many major defects in your sample, you typically reject the entire production lot. You then need to investigate the cause, fix the problem, and potentially re-inspect or remake the lot.

Do I need special software to calculate sample sizes?

Not always. For many standard situations, you can use tables from standards like ISO 2859 or online sample size calculators. For very complex needs, specialized software might be helpful.

Is testing the same for physical and digital products?

No. Physical products involve manufacturing defects. Digital products (like software) involve bugs and usability issues.

Software testing uses different methods, like unit testing, integration testing, and user acceptance testing, with varying numbers of test cases rather than physical product samples.

Conclusion: Making the Right Choice

Figuring out how many products to test is a balance. It’s about protecting your customers. It’s also about protecting your business.

Use the risk, volume, and cost as your guide. Don’t be afraid to use standard tables and methods. They are there to help you make smart, confident decisions about your product quality.