This is agreed with the controls engineer at design time, and it is the question that separates a commissioned system from a pilot.
Options are: reject the part by default, pass the part by default, stop the line, or fall through to manual inspection. Which is correct depends entirely on the consequence of an escaped defect against the cost of a stoppage, and it is a business decision informed by engineering rather than the reverse.
The system also has to distinguish between fault types. A camera that has failed, illumination that has degraded past a threshold, and inference that has exceeded its time budget are three different conditions that may warrant three different responses.
Degradation is the harder case. A hard failure is obvious. Illumination that has dimmed by fifteen per cent over a year is not, which is why a reference target imaged on a schedule belongs in the design rather than in a maintenance wish list.