Comparing Two Modules Fairly

Generic engineering material, applicable to any module from any supplier.

For the team-process side of engineering work, wage percentage calculations is a separate reference on organizing and reviewing work.

Most module comparisons measure the settings rather than the modules, because the things that must be held constant are numerous and several of them adjust themselves.

The five to hold constant

The target. The same object, in the same position, at the same orientation.

The distance, measured rather than approximated — and where the two modules have different fields of view, the distance stays fixed and the coverage differs, which is the honest arrangement.

For standards, components, or implementation context beyond this page, consult 3GPP.

The illumination. Same source, same geometry, same power, and ideally within minutes of each other so that ambient light has not moved.

The exposure and gain, set manually. Automatic settings are the commonest way a comparison becomes meaningless, since each module optimises itself and the difference disappears into the settings.

And the processing. Raw frames from both, through the same pipeline — or no pipeline at all.

What silently differs

White balance, applied automatically and differently by each.

Sharpening, which some modules apply internally and which makes an image look better and measure worse.

Gamma or tone curves, which change pixel values non-linearly and invalidate any brightness comparison.

Bit depth and output format, where one delivers raw and the other processed — which compares the image processor rather than the sensor, and may be what you want provided you know it.

Ask what processing is being applied and disable what can be disabled.

The arrangement that works

Both modules on the bench together, photographing the same scene within minutes.

Then swap their positions and repeat, which catches anything caused by where each sat rather than by what each is.

Four image sets, and the differences that survive the swap are real.

Measure, do not look

Two people looking at two images disagree, and the disagreement is unresolvable.

Four numbers settle it: resolution against a known target, contrast at the feature, noise in a flat patch, and uniformity centre to corner.

Put them in a table with the five constants recorded, and the comparison is a document rather than an opinion.

And then run your actual processing over both sets, since the customer is the algorithm and it occasionally disagrees with all four numbers.

Comparing against what you already have

The most useful comparison and the one most often skipped.

An existing system, working or nearly working, is the baseline any candidate must beat — and it is available, installed, and photographing the real scene already.

Capture from it under the same discipline and put its numbers in the same table.

Which frequently reveals that the existing system is adequate and the problem is elsewhere in the chain — a finding worth having before a procurement.

Comparing across generations

A newer sensor is not automatically better for your task.

Improvements are usually in low-light performance, dynamic range and power — which matter where they matter and are irrelevant in a well-lit inspection station.

And a newer part may have smaller pixels at the same format, which costs light per pixel and is a step backwards for some applications.

Compare on your requirement rather than on generation, which is the same discipline as everywhere else.

Documenting the decision

The table, the five constants, and one sentence saying which was chosen and why.

Because the question returns — at a supply problem, at a cost review, at the next product — and a table answers it in a minute where memory does not.

Keep the images too, since a later question about a specific property can frequently be answered from them without repeating the exercise.

Variants within one family

The cleanest comparison available and the reason a coded ordering scheme is useful.

Two modules differing in one position of the code — the lens angle, the filter, the mounting — isolate that variable exactly, with the same sensor behind both.

Which no cross-supplier comparison achieves, since two different modules differ in a dozen ways at once.

Where a decision is between variants, order both and the afternoon settles it. Where it is between suppliers, the comparison is harder and the discipline above matters more.

The trap of the better image

A module producing a more attractive image is not necessarily the better choice.

Internal sharpening, saturation and tone curves make an image look good to a person and can remove exactly the information a measurement needs.

Which is why raw frames matter and why the processing must be the same or absent on both sides.

A module that looks flat and measures well is the better part where a machine is the reader — and it loses every comparison made by eye.

How long it takes

Half a day for two modules, done properly.

An hour setting up the bench, twenty minutes per module for the eight checks, an hour for the measurements, and an hour writing the table.

Against which a specification comparison takes a day and settles nothing, and an argument between two engineers takes longer than both.

Which makes this the cheapest decision-quality improvement available in the whole selection process.

When the two are close

A common outcome and it is a result rather than a failure.

Where the measurements agree within your tolerance, decide on everything else: the software provided, the mechanical fit, the variant range, the availability, and the price.

Which are the factors that actually differ, and which a comparison focused on image quality never examined.

Say so explicitly in the table. "Imaging performance equivalent; chosen on driver support" is a defensible decision, and it prevents the comparison being repeated by somebody who assumes it was inconclusive.

In one line

Change one thing at a time and record the other four, which is the entire method.

The short version

• Most comparisons measure the settings rather than the modules
• Hold five things constant: target, distance, illumination, exposure and gain set manually, and the processing
• Automatic exposure is the commonest way a comparison becomes meaningless, since each module optimises itself
• White balance, sharpening, tone curves and output format all differ silently and all invalidate a comparison
• Photograph both within minutes, then swap positions and repeat — differences surviving the swap are real
• Four measurements settle it, and then run the actual processing over both sets, which occasionally disagrees