When power, cost or latency will not close on any available part, the accelerator becomes a design problem rather than a purchasing decision. Faststream works on AI accelerator architecture, datapath and memory hierarchy design, quantisation strategy at the hardware level, RTL implementation and SoC integration — the same silicon capability applied to a workload that has a shape.
An accelerator is worth designing when three things are true at once: the workload is stable enough that fixing it in hardware is not a bet, the volume is high enough to amortise the design and mask cost, and no available part closes the power, cost or latency budget.
If any of those is false, an off-the-shelf NPU or a well-optimised software implementation on an existing SoC is the better engineering answer, and that is what will be recommended. Most edge AI problems are solved properly at the edge AI layer rather than in new silicon.
Moving weights and activations costs far more energy than multiplying them. Architecture that ignores this optimises the wrong thing.
| Operation class | Relative energy cost | Design implication |
|---|---|---|
| On-chip multiply-accumulate | Lowest | Adding arithmetic units is cheap; keeping them fed is not |
| On-chip buffer access | Low to moderate | Tiling and reuse strategy determine how often this happens |
| External memory access | Highest by a wide margin | Minimising off-chip traffic is the primary architectural objective |
Relative ordering holds broadly across process nodes and memory technologies; specific ratios depend on the node and memory choice.
When the workload is stable, the volume amortises design and mask cost, and no available part meets the power, cost or latency budget. If any of those does not hold, an off-the-shelf NPU with well-optimised software is the better answer.
Memory bandwidth, not arithmetic. External memory access costs far more energy than a multiply-accumulate operation, so tiling, buffer sizing and data reuse determine both performance and power more than the size of the arithmetic array does.
Yes. Integrating an existing accelerator into an SoC — interfaces, memory subsystem, power domains, toolchain bring-up — is often the right path and is standard work within the Silicon platform.
Yes. An accelerator without a mapping path from a trained model is not usable, so compiler and runtime work is part of the scope rather than a later problem.
Tell us the specification, the constraint and the deadline. Programmes that cross silicon, radio, embedded and AI are where Faststream is strongest.