How to Compare Prototype Revisions with an A/B Build

Revision B of a prototype cover closes with less force than revision A. The design team shortened the latch, but the new cover was printed in a different orientation, measured with a new fixture and assembled to a housing from another batch. The team can say that one assembly closed more easily. It still has to find out whether shortening the latch caused the change.

An A/B prototype build puts a baseline and a candidate into a comparison planned around one engineering decision. It can show whether a geometry change improves fit, whether a material change reduces deflection or whether a new manufacturing route produces an assembly worth developing. This guide shows how to scope the two builds, control what changes, measure the result and decide what evidence is needed before moving to production.

Decide what the comparison needs to prove

Write down the decision before ordering parts. “Compare revision A and revision B” identifies two configurations. “Select a latch geometry that reduces peak closing force while preserving retention” identifies a decision. For that decision, closing force is the main response and retention is a separate condition for approval.

Define both the size and direction of the change that would matter. If an assembly station can reliably handle a peak force of 30 N, lowering the average from 35 N to 34 N may show a trend while leaving the manufacturing problem unresolved. If the original version occasionally spikes to 45 N, a candidate with a similar average and fewer spikes may still be valuable. Record which observation will drive the choice before seeing the measurements.

Write the criteria with units, test conditions and a reason for each limit. For example, a target for force at the handle needs a specified point of application. A limit for assembled gap needs a measurement location and assembly condition. A retention requirement needs a load direction and a definition of release or failure. The right values come from the product and its assembly process.

Choose the baseline that represents the current problem

A prior physical prototype is convenient, but it may be worn, aged or made to an earlier CAD revision than its label suggests. Check the baseline file, actual geometry, material, finish and build history. If the old cover has gone through dozens of closure cycles, use it to reproduce the complaint and consider making fresh A parts for the formal comparison.

For a geometry-only change, a fresh baseline built alongside the candidate often makes the comparison easier to interpret. Match the process, material lot and finishing route where practical. If the baseline must come from an existing lot, record how that lot differs from the new build and state which conclusion the comparison can support. A comparison of old and new packages can still guide a product decision even when it cannot isolate a single geometry change.

Freeze the source files before quoting. Include the matching mating geometry, BOM revision and change summary. Two CAD files with the same filename can be difficult to distinguish once they are exported or copied into a manufacturing system. Use versioned filenames and request the revision on the build paperwork and parts or packaging.

Decide what to change and what to hold fixed

When the question concerns one design change, keep the other influences as similar as practical. A longer latch arm, a lower hook and a smoother surface can each affect closing force. Change all three together and the test can evaluate the package. Separate changes are needed if the team wants to attribute the result to one feature.

When two variables may interact, a comparison of only the original and combined changes can hide the interaction. For example, a larger root radius may help one resin grade while another grade shows little benefit. If both factors matter and the schedule allows, build each resin with each radius. A four-condition matrix can reveal whether the geometry effect depends on the material. Use the simplest experiment that can answer the question. NIST engineering statistics handbook on selecting an experimental design

Control the build before the parts reach the fixture

Ask the supplier what can be matched across variants and what must differ. For a CNC comparison, machining the same resin stock, using a common setup strategy and documenting deburring can reduce unexplained variation. For additive prototypes, orientation, location in the build and post-processing can affect an interface. For cast prototypes, identify the mold, cavity, pour and cure batch.

Manufacturing both variants together can help, but physical constraints may still separate them. A wide revision B might only fit at the edge of a print platform. A different mold is often necessary for a geometry change in vacuum casting. Record such differences and decide whether another build or an alternative manufacturing route is needed to separate the effects.

Inspect the actual changed features before testing. If both latch designs arrive with nearly the same effective undercut after finishing, the functional test no longer compares the intended contrast. Record the measured undercut, arm thickness and mating-edge position for each sample. Preserve this information even if the drawing allows both parts to pass.

Avoid changing the baseline and candidate in different ways during finishing. If a technician hand-polishes a tight interface on B and leaves A as delivered, that intervention becomes part of the comparison. Record the work or test the original and finished states separately. A trial that includes hand fitting can be informative when hand fitting is the process being evaluated.

Pair samples where it helps and balance the test order

Look for conditions that can move the measurement even when the design stays the same. Common examples include variation between mating housings, fixture warm-up, operator technique or differences between build batches. A paired comparison places A and B under a shared condition so the design difference can be examined within that pair. For example, evaluate one A and one B cover against a matched housing condition in each group.

Use a shared physical mating part only if the test leaves it in a comparable state. Repeated assembly can polish a catch, compress a seal or loosen a threaded connection. If that happens, use separate matched mating parts or define a replacement schedule. Balance which version goes first so wear or warm-up does not always favor the same variant.

For trials across two builds or two operators, include both revisions in each group when practical. Testing all A parts with operator one and all B parts with operator two ties the operator effect to the revision. Randomizing the run order within comparable groups helps spread time-related effects. When the order cannot be randomized, record the reason and the likely consequence for interpretation. NIST guidance on randomized block designs

Count independent parts separately from repeated measurements

Three force readings on the same latch tell you about that latch and the way it was tested. They do not give the same evidence as readings on three independently manufactured latches. Keep part count and cycles per part separate in the report. If the first closure is the production event of interest, use that reading as the main result and analyze later cycles separately.

For a feasibility check, a small group may be enough to show a gross interference or a consistently lower force in one configuration. When the decision depends on a narrow difference, rare failures or future batch variability, plan broader sampling. Include independent builds or lots if the conclusion is meant to cover them. Choose the scope using the expected variation, smallest important difference and cost of a wrong decision.

Reserve units for destructive tests before assigning the rest. Using a failed pull test to supply a later dimensional check can confuse permanent deformation with the original geometry. Identify untouched retain samples and specify what happens when a run stops early or a fixture fault invalidates a reading.

Check whether the measurement can see the difference

Use the same calibrated equipment, fixture configuration and operating method for both revisions. A force gauge records the load at its sensor, so fixture alignment, contact location and friction can change the reading. A simple check with a retained reference assembly at the start and end of the session can reveal drift or setup changes.

For manual measurements, have a second operator repeat selected checks when a subjective technique could affect the result. If the expected difference is around 1 N and repeated measurements on the same assembly vary by 2 N, improve the method or collect better evidence before making a close call. When feasible, give the operator coded specimens so expectations about the new revision do not influence borderline visual judgments.

Save raw curves or images when they explain the outcome. Two samples can show the same peak force while one sticks early and the other peaks during normal engagement. A force-displacement trace can separate initial contact, latch movement and final seating. Link each file to the exact specimen and test run.

Worked example: compare two CNC-machined prototype covers

Suppose an engineering team is selecting a latch arm length for an enclosure. The team orders five A covers and five B covers from the same grade and stock lot, using the same CNC route and finish. Revision B extends the arm. The undercut and mating housing geometry are held to the same drawing. The numbers below are illustrative and apply only to this example.

The team checks the actual arm dimensions and hook engagement before assembly. It then forms five comparison groups, each with one A and one B cover and matched housing conditions. Each cover receives one first-closure test. The order alternates across groups. Both versions must seat fully, and every tested B assembly must retain at least 65 N in the specified pull direction. The candidate also needs a median first-closure force at or below 30 N.

B reaches a median closure peak of 28 N and improves closure force in all five groups. One B unit misses the retention limit. The team can retain the longer arm as a promising direction while holding release. Inspect the low-retention unit for actual hook overlap, housing variation and damage. If changing overlap becomes necessary, make that a controlled follow-up. Dropping the 64 N result would remove the condition the trial was designed to catch.

The result is evidence about CNC prototypes of the chosen stock. If the intended product is injection molded, confirm the decision with parts made in the production-intent resin and route. Molding can change geometry and material behavior at the latch. Retain the same functional requirements and update the test conditions wherever the final assembly differs.

Read the results at the level of the decision

Look at per-part values and, when the test is paired, the within-pair differences. Record the number of parts tested, number of failures, range, central result and relevant anomalies. With a small trial, raw values often communicate the evidence more clearly than a percentage improvement alone. Report whether a change clears the functional threshold as well as whether it moves in the desired direction.

Distinguish a clear directional result from proof of a stable production process. A candidate may advance to a more representative prototype even though its qualification remains open. If A and B overlap heavily and both pass, decide whether the remaining difference matters enough to fund another build. If both fail, the next experiment should address the likely failure mechanism or reconsider the assembly architecture.

Investigate missing runs and outliers. A fixture slip documented at the time of the run supports repeating a measurement under the agreed rules. A part that behaves unexpectedly with a functioning fixture is an engineering result. Keep the original record and make the reason for any exclusion visible.

Put the comparison in the RFQ

Provide the prototype supplier with both revision-controlled models and drawings, the list of intended differences, quantity per variant, material and finish, and any process conditions that need to be matched. Include the mating assembly and identify critical dimensions to report before test. Request part-level identification and any records needed to link parts to stock, build, mold or finishing batch.

A short comparison brief can travel with the RFQ:

• Decision. Select a latch geometry that reaches the specified closure force and retention targets.

• Configurations. List the baseline and candidate revision IDs and the one geometry change under review.

• Build plan. State quantity by revision, shared stock or process requirements, and conditions the supplier should record.

• Inspection. Identify mating dimensions, datums, finish condition and any witness samples.

• Functional test. Define fixture, order, sample allocation, conditioning and pass criteria.

• Deliverables. Request labeled parts, dimensional results by specimen, test records and a list of process deviations.

If the supplier proposes a faster build method, ask which comparisons it can preserve. A substitute can work well for clearance studies. A change in material or processing can require an additional build for force, fatigue or long-term performance. Send the comparison brief and CAD revisions to AluPrototype for a quotation covering both variants and their inspection requirements.

Frequently asked questions

Can two prototype revisions use different manufacturing processes?

Yes, when the decision is which complete manufacturing route to develop. If the question is whether a geometry change caused the result, match the process where practical or use a build matrix that separates geometry from process.

Should I test A and B on the same mating component?

A shared mating component can reduce between-part variation when it remains stable. If insertion wears the catch, compresses a seal or changes thread friction, use a matched component allocation or a planned replacement schedule. Record which component each specimen used.

How many prototypes do I need?

Base quantity on the decision, expected spread and consequence of a wrong choice. A few parts can expose major interference. Close performance differences or claims about repeat production require broader evidence, often across multiple builds. Count independent parts and repeat measurements separately.

What if revision B improves one metric and fails another?

Keep both findings in the decision record. If the failed metric is a mandatory release criterion, resolve it before approval. Use the passing result to inform the next controlled change.

What should a comparison report include?

Include the decision, exact revisions, measured build differences, test method, raw results by part ID, exceptions and the disposition. State which conditions were represented and what the next build still needs to verify.