MIPI-CSI-2 capture, sensor correction, demosaic, colour and tone, then JPEG — an entire image-signal pipeline realised as RTL, where every algorithm choice is also an FPGA-area and image-quality decision.
A camera does not produce an image. A sensor produces a stream of raw, mosaiced, noisy samples, and everything a viewer would call a photograph happens afterwards — in the image-signal pipeline that corrects, reconstructs, colours and compresses it. Moving that pipeline into an FPGA means every stage stops being a library call and becomes an area, timing and memory decision.
The reference for each stage is a floating-point model. The hardware is not. So the real work is two problems at once: choosing algorithms that give the image quality the application needs, and realising them in fixed-point RTL that fits the fabric and closes timing at the sensor's pixel rate. The two problems are coupled — the cheaper algorithm is usually the lower-quality one — and the coupling is present at every stage from black-level correction to the JPEG entropy coder.
Stated before they had answers. The order of the pipeline is fixed by physics; what is negotiable is how much quality each stage buys and how much fabric it costs.
The same demosaic or sharpening block can be excellent or cheap, rarely both. Made stage by stage without a model, those choices only reveal their cost after place-and-route.
Each stage has to become fixed point without visible artefacts. Word lengths are a design variable, not a default, and the wrong one shows up as banding, colour shift or lost detail.
Demosaic, noise reduction, sharpening and the DCT all need neighbours or whole blocks. Line and block buffers, not the arithmetic, decide whether the pipeline fits on the device.
Before any processing there is MIPI-CSI-2 or parallel capture, sensor configuration over I²C, and formatting the raw stream — the part that must be right before a single correction can run.
Three streaming stages on one device: capture and sensor preprocessing, colour/tone correction and enhancement, then JPEG encoding. The stages that decide perceived quality — demosaic, the colour-correction matrix, sharpening and the DCT — are the ones where the area and fixed-point decisions bite hardest. Underneath all of them sit the four things that never appear as a block but consume the schedule: performance/area modelling, fixed-point conversion, image-buffering optimisation, and verification against a golden reference.
The specific scope, rather than a capability list.
Rarely the stage that sounds difficult. Written out because a reader facing the same programme gets more from this than from a summary of what went well.
Every stage carries quantisation error, and the errors accumulate down the pipeline. Choosing word lengths so the compressed output still matches the reference — and knowing which stages are sensitive — took more care than the arithmetic itself.
Line and block buffers for demosaic, filtering and the DCT dominate on-chip memory. The buffering scheme was designed early, because it decides whether everything else fits.
These two stages set most of the perceived quality and cost. A cheaper interpolation looks fine on a test chart and fails on real edges, so the evaluation used representative images, not synthetic ones.
The JPEG quantiser trades file size against quality continuously. A single quality setting describes one operating point; the design had to behave across the range the application actually uses.
Correctness is perceptual as well as numeric, so verification compared each stage's fixed-point output to the golden model and exercised the whole chain across images, rather than checking a handful of vectors.
Capture, sensor correction, demosaic, colour and tone correction and JPEG compression integrated as one streaming design and verified against the reference.
Image-quality against FPGA-area was decided per stage from models, so resource use was a design output rather than a place-and-route surprise.
Word lengths were chosen so the hardware output tracks the floating-point model without visible artefacts across representative images.
The stages are parameterised and reusable across sensors and formats, rather than tuned to a single camera.
Customer projects are presented at property, capability, outcome and integration level. Customer names, internal architecture, register maps and confidential deliverables are not disclosed. Where a figure would identify a customer or a design, it is omitted rather than approximated. More detail is available under a non-disclosure agreement, within the limits each customer has agreed.
Every item links to its own page.
Machine vision and edge inference.
CAPABILITYThe imaging chain, end to end.
SERVICEArchitecture, HDL, timing closure.
CAPABILITYFixed-point datapaths and RTL.
SOLUTIONThe wider body of imaging work.
CASE STUDYThe pipeline in reverse.
The chain that turns raw sensor output into a usable image: sensor capture and correction, demosaicing, colour and tone correction, and encoding or compression. In hardware, each stage is both an algorithm choice and an FPGA-area and image-quality decision.
An FPGA implementation cannot carry the floating-point precision of a reference model economically. The pipeline is converted to fixed point stage by stage, with word lengths chosen so the hardware output matches the reference without visible artefacts.
Several stages need neighbouring pixels or whole blocks — demosaic, noise reduction, sharpening, DCT — so line and block buffers consume most of the on-chip memory. The buffering scheme often decides whether the pipeline fits at all.
The same stage — demosaic, sharpening, chroma handling — can be excellent or cheap, rarely both. Each choice is modelled for image quality and for FPGA area so the trade-off is made deliberately rather than discovered after place-and-route.
Colour-space conversion to YCbCr, chroma subsampling and block partitioning, a 2D discrete cosine transform, quantisation, zig-zag reordering, entropy coding (DPCM, run-length and Huffman) and a JPEG formatter — all as streaming RTL.
Against a golden reference, stage by stage: each block's fixed-point output is compared to the model, and the full pipeline is exercised across representative images so quality regressions are caught before silicon or deployment.
Tell us the sensor, the quality target and the device. Image pipelines that have to close in fixed point on real fabric are where Faststream is strongest.