Deci Boosts Computer Vision & NLP Models’ Performance at MLPerf

Deci, the deep learning company harnessing Artificial Intelligence (AI) to build AI, announced its results for both Computer Vision (CV) and Natural Language Processing (NLP) inference models that were submitted to the MLPerf v2.0 Datacenter Open division. These submissions demonstrated the power of Deci’s Automated Neural Architecture Construction (AutoNAC) technology, which automatically generated models dubbed DeciNets and DeciBERT, thus delivering breakthrough accuracy and throughput performance on Intel’s CPUs.

“We are excited to showcase another significant milestone in our journey to enable efficient deep learning inference on any hardware including resource constrained devices such as CPUs and other edge devices” said Yonatan Geifman, CEO and co-founder of Deci. “This major increase both in accuracy and throughput means that resource-intensive tasks that previously could not be carried out on CPUs are now possible, and in fact, will see a marked performance improvement. Hardware availability or compute power should never be a limiting factor for enterprises looking to employ the latest developments in deep learning.”

For their CV submission, Deci submitted three of its DeciNets models in the ResNet50 category under the offline scenario in the open division. Deci made submissions on two different hardware platforms: a 12-core Intel Cascade Lake CPU and two different Intel Ice Lake CPUs with 4 and 32-cores. Models were optimized on a batch size of 32 and quantized to INT8 using OpenVINO. Compared to the 8-bit ResNet50 model, Deci delivered +1.74% improvement in accuracy and 4x improvement in throughput.

Deci’s CV submission this year demonstrated a 37% improvement in throughput performance, as well as a notable improvement in accuracy, compared to their previous submission in 2020.

For NLP, Deci submitted its optimized DeciBERT models that produced outstanding results: accelerating question-answering tasks’ throughput performance on various Intel CPUs by 5x (depending on the hardware type and quantization level) while also improving the accuracy by +1.03%.

MLPerf gathers expert deep learning leaders to build fair and useful benchmarks for measuring training and inference performance of ML hardware, software, and services. The models submitted were optimized using Deci’s AutoNAC technology and quantized with Intel’s OpenVINO to 8-bit precision.

Deci’s AI-based AutoNAC technology automatically generates and optimizes deep learning architecture for any given data set and hardware to maximize its accuracy and inference performance. Deci’s AutoNAC technology, as well as its auto-generated DeciNets & DeciBERT, are ready for deployment and commercial use and can be easily integrated to support any CV or NLP task on a wide range of hardware types.

Deci’s Computer Vision Submission Results:

Computer Vision Submission:

Accuracy and Throughput [samples/sec] Results on Different Hardware Types