Measuring Instead of Looking

Generic engineering material, applicable to any module from any supplier.

For the non-technical side of coordinating engineering work, see remote companies.

Two engineers comparing two images will disagree, and neither will be wrong, because "sharper" and "better" are not properties an image has.

Four measurements replace the argument, and none needs specialist equipment.

Resolution, against a known target

Photograph a target with features of known size and count pixels across them.

For broader technical context, see IPC.

Which converts "it looks sharp" into millimetres per pixel at the actual working distance — the number the whole specification rests on.

A printed test chart works. So does a steel rule, a printed grid, or anything whose dimensions you can verify with callipers.

And it detects the error the calculation misses: a working distance measured optimistically, a lens whose actual focal length differs from its nominal, or a sensor smaller than assumed.

Contrast at the feature

The thing the processing actually consumes.

Take the pixel values on the feature and on its immediate background and compute the difference relative to their sum.

A number between zero and one, comparable between images, and it captures what "the feature stands out" means.

It is the measurement that settles lighting arguments, since a geometry change frequently doubles it where a sensor change does not.

Noise, in a flat region

Photograph something uniform — a grey card, a blank wall, a sheet of paper — and take the standard deviation of pixel values across a patch.

Repeat at the exposure and temperature you will actually use, since both change the answer substantially.

And repeat with the lens capped for the dark case, which isolates sensor noise from anything in the scene.

Comparing two modules requires the same exposure and the same illumination, or the comparison measures the settings rather than the sensors.

Uniformity across the frame

Photograph a uniformly lit flat surface and compare the corners to the centre.

Falling illumination toward the edges is inherent to lenses and more pronounced on wide angles — and how much is acceptable depends on where in the frame your feature will appear.

The same image shows corner softness, which is where a lens is worst and where sample images published by anybody rarely look.

The discipline that makes it work

Same distance, same lighting, same exposure, same target, one variable changed.

Record all five with every image.

An afternoon of measurements with one variable moving produces a table. The same afternoon adjusting several produces impressions, which is what the exercise was meant to replace.

What the numbers do not capture

Worth saying, because this page could read as dismissing judgement.

Artefacts. A pattern, a banding, a colour fringe at high-contrast edges — visible immediately and not represented in any of the four measurements above.

Behaviour over time. A module that produces one good image and drifts is measured identically to one that does not, on a single frame.

And how the processing responds. The customer is the algorithm rather than the eye, and the decisive test is frequently running your actual processing over both sets of images.

Measure, then look, then run the pipeline — in that order, since the first prevents the second becoming the whole basis.

Tools

Nothing specialised is required.

Any image analysis package, a spreadsheet, or a few lines of script will take a standard deviation over a region and a mean over two patches.

What matters is that the numbers come from raw frames rather than from a preview, and that the regions are the same between images — which means recording pixel coordinates alongside everything else.

Recording the result

A table, one row per image.

Variant, distance, lighting, exposure, temperature, and the four numbers.

Which is the document that survives the project and answers the question somebody asks a year later about why this module was chosen — where an unlabelled folder of images does not.

The measurement that decides most projects

Pixels across the feature, at the working distance, on the real part.

One number, compared against the requirement written down at the start.

It either meets it or it does not, and no amount of image quality elsewhere compensates for falling short — which makes it the first measurement to take and the one to take before any other work proceeds.

Comparing against a competitor's module

The same discipline, and one addition.

Confirm both are set to the same output format and the same processing. A module producing a processed image and one producing raw data are not comparable until the raw one has been through equivalent processing.

Otherwise the comparison measures the image signal processor rather than the sensor and lens — which may be what you want to know, provided you know that is what you measured.

When the numbers say the wrong thing

Occasionally a module measures better and performs worse in the application.

Which is information rather than a failure of measurement: something the four numbers do not capture is mattering, and finding out what is the useful next question.

Usually it is an artefact, a temporal behaviour, or an interaction with the processing — all three listed above as known blind spots.

Run the actual pipeline over both image sets and the disagreement usually explains itself.

In one line

Four numbers and one variable at a time, which converts an argument into a table.

The short version

• Two people comparing images disagree because "sharper" is not a property an image has
• Resolution: photograph a target of known size and count pixels across it, which gives millimetres per pixel at the real distance
• Contrast: pixel values on the feature against its background, which is what the processing consumes and what settles lighting arguments
• Noise: standard deviation across a uniform patch, at the real exposure and temperature, and again with the lens capped
• Uniformity: a flat lit surface, comparing corners to centre, which also reveals corner softness
• Same distance, lighting, exposure and target, with one variable changed and all five recorded per image