English | 简体中文

Tengine

Introduction

Tengine is developed by OPEN AI LAB. This project meet the demand of fast and efficient deployment of deep learning neural network models on embedded devices. In order to achieve cross-platform deployment in many AIoT applications, this project is based on the original Tengine project using C language for reconstruction, and deep frame tailoring for the characteristics of limited embedded device resources. Also, it adopts a completely separated front-end/back-end design, which makes it possible to be transplanted and deployed onto CPU, GPU, NPU and other heterogeneous computing units rapidly, conveniently. At the same time, it is compatible with the original API and model format tmfile of Tengine, which reduces the cost of evaluation and migration.

The core code of Tengine Lite consists of 4 modules:

• device: NN Operators back-end module, currently provides CPU code, and gradually open source GPU and NPU reference code;
• scheduler: core components of the framework, including NNIR, Computational Graphs, Hardware Resources, and the scheduling and execution modules of model serializer;
• operator: NN Operators front-end module, which realizes registration and initialization of NN Operators;
• serializer: Model decoder, which decodes binary tmfile format into serialized model parameter.

Architecture

How to use

Compile

• Quick Compilation Simple cross-platform compilation based on cmake.

Example

• examples provides basic classification and detection algorithm use cases, which are continuously updated according to the needs of issues.

Model Zoo

• Baidu Netdisk (password: 7ke5)

• Google Drive

Model Convert tool

• Pre-compiled version: Pre-compiled model convert tool is provided on Linux system;
• Online Convert tool: Based on WebAssembly (the models are converted locally by browsers, no private data will be uploaded);
• Source Compilation: Refer to Tengine-Convert-Tools project, convert tool could be built by users.

Speed assessment

• Benchmark Basic network speed assessment tool, any pull request is welcomed.

NPU Plugin

• TIM-VX VeriSilicon NPU user manual.

AutoKernel Plugin

• AutoKernel.

Container

• SuperEdge How to use Tengine with SuperEdge.

Roadmap

• Road map

Acknowledgement

Tengine Lite got ideas and developed based on these projects：

• Caffe
• Tensorflow
• MegEngine
• ONNX
• ncnn
• FeatherCNN
• MNN
• Paddle Lite
• ACL
• stb
• convertmodel
• TIM-VX
• SuperEdge

License

• Apache 2.0

FAQ

• FAQ common questions

Tech Forum

• Github issues
• QQ groupchat: 829565581
• Email: [email protected]
• Tengine Community: http://www.tengine.org.cn