HERALD: Optimizing Heterogeneous DNN Accelerators for Edge Devices
share
Recent advances in deep neural networks (DNNs) have made DNNs the backbone of many applications on edge devices such as face recognition, object detection, and so on. To deal with massive computation requirements of DNN inferences within stringent energy and latency constraints, DNN accelerator (i.e., hardware specialized forDNN inferences), have emerged as a promising solution. Such advancement of hardware supporting DNNs has led to multiple DNN-based applications running at the same time on edge devices. They often run in parallel as background processes or as sub-tasks of a complex application. Thus, DNN workloads on a DNN accelerator now include a variety of layer operations and sizes from DNN models for diverse applications making them heterogeneous in layer granularity. Such heterogeneous workloads introduce a new major challenge for monolithic DNN accelerators because the efficiency of DNN accelerators relies on its dataflow, and different DNN layer types and shapes prefer different dataflows. In this work, we propose to tackle this challenge by designing heterogeneous DNN accelerators (HDAs) that deploy multiple DNN accelerators each optimized for different layer shapes and operations. To enable this approach, we propose HERALD, an optimization framework that explores the design space an HDA and layer schedules. Design time-optimized HDAs with the best energy-delay-product (EDP) HERALD identified provided 24.93 average across workloads and accelerators we evaluate compared to the best case of monolithic accelerators for each evaluation setting by deploying two complementary-style DNN accelerators.HERALD's scheduler employs heuristics that exploit the characteristics of DNN workloads, which provided 6.4 average compared to a baseline scheduler.
READ FULL TEXT