first commit
Some checks are pending
Build/Publish Develop Docs / deploy (push) Waiting to run

This commit is contained in:
2025-07-02 08:57:16 +03:00
commit 56532cc9a9
1901 changed files with 457695 additions and 0 deletions

View File

@@ -0,0 +1,114 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# Android Demo
## 1. Introduction
This document describes the [Android Demo](https://github.com/PaddlePaddle/PaddleOCR/tree/main/deploy/android_demo) for PaddleOCR. The demo currently supports text detection, text direction classification, and text recognition models. It is developed using [PaddleLite v2.10](https://github.com/PaddlePaddle/Paddle-Lite/tree/release/v2.10).
## 2. Recent Updates
- **February 27, 2022**
- Updated the prediction library to PaddleLite v2.10.
- Added support for six operation modes:
- Detection + Classification + Recognition
- Detection + Recognition
- Classification + Recognition
- Detection
- Recognition
- Classification
## 3. Quick Start
### 3.1 Environment Setup
1. **Install Android Studio:**
Download and install Android Studio on your development machine. For detailed instructions, please refer to the [Android Studio official website](https://developer.android.com/studio).
2. **Prepare Your Android Device:**
Ensure that you have an Android phone and enable USB debugging. To do so, navigate to:
**Settings → Developer Options → Enable Developer Options and USB Debugging.**
**Note:** If your Android Studio installation does not have the NDK configured, please follow the [Install and Configure NDK and CMake](https://developer.android.com/studio/projects/install-ndk) guide. You may choose the latest NDK version or use the same version as the PaddleLite prediction library.
### 3.2 Importing the Project
1. In Android Studio, select **File → New → Import Project…** and follow the wizard to import the project.
2. Upon successful import, your project interface should resemble the following:
[](./images/import_demo.jpg)
### 3.3 Running the Demo
1. Connect your Android device to your computer.
2. Click the **Run** button in the Android Studio toolbar.
- Your device may prompt you with "Allow installation of software from USB." Click **Allow**.
3. Once the installation is complete, the app icon will appear on the last page of your home screen.
4. Tap the app icon to launch the application. The home page will display as shown below:
![App Home Screen](./images/68747470733a.png)
On the home screen, you will find the following elements:
- **Run Model:** Executes the model(s) using the selected operation mode.
- **Take Photo for Recognition:** Activates the device camera to capture an image. After taking a photo, tap **Run Model** to perform recognition.
- **Select Image:** Opens the photo gallery to select an image. After selection, tap **Run Model** to perform recognition.
- **Clear Drawing:** Clears any drawn text boxes from the currently displayed image.
- **Dropdown List:** Allows you to choose one of the six supported operation modes. The default mode is **Detection + Classification + Recognition**.
- **Menu Button:** Opens the settings interface for configurations related to the models and built-in images.
When you tap **Run Model**, the demo executes the corresponding model(s) in your selected mode. For example, in **Detection + Classification + Recognition** mode, the output appears as follows:
<img src="./images/run_det_cls_rec.jpg" width="400">
The status display area shows the current model status (e.g., `run model succeeded`), indicating that the model ran successfully. The recognition results are formatted as follows:
```text
Serial Number: Det: (x1,y1)(x2,y2)(x3,y3)(x4,y4) Rec: Recognized Text, Confidence Score Cls: Classification Label, Classification Score
```
### 3.4 Operation Modes
The PaddleOCR demo supports six operation modes, shown in the illustration below:
![Operation Modes Overview](./images/68747470733adfd.jpeg)
The expected output for each mode is as follows:
| **Detection + Classification + Recognition** | **Detection + Recognition** | **Classification + Recognition** |
|----------------------------------------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------|
| <img src="./images/run_det_cls_rec-20240806100407184.jpg" width="400"> | <img src="./images/run_det_rec.jpg" width="400"> | <img src="./images/run_cls_rec.jpg" width="400"> |
| **Detection** | **Recognition** | **Classification** |
|--------------------------------------------------------------------------------|--------------------------------------------------------|--------------------------------------------------------|
| <img src="./images/run_det.jpg" width="400"> | <img src="./images/run_rec.jpg" width="400"> | <img src="./images/run_cls.jpg" width="400"> |
### 3.5 Settings
The settings interface allows you to customize the following parameters:
1. **General Settings**
- **Enable Custom Settings:** Check this box to allow modifications.
- **Model Path:** The file path to the model (use the default value unless necessary).
- **Label Path:** The dictionary file for the text recognition model.
- **Image Path:** The filename of the built-in image used for recognition.
2. **Model Runtime Settings:** (Changes here trigger an automatic model reload when returning to the main interface.)
- **CPU Thread Number:** The number of CPU cores allocated for model inference.
- **CPU Power Mode:** Defines the models execution strategy, corresponding to different core configurations.
3. **Input Settings**
- **det long size:** The target value for the longer side of the image during DB model preprocessing. If the image's long side exceeds this value, it will be resized proportionally; if it is shorter, no resizing is applied.
4. **Output Settings**
- **Score Threshold:** The minimum score threshold for detection boxes in DB model post-processing. Boxes scoring below this threshold are filtered out and not displayed.
## 4. Further Support
For additional support and updates:
1. Refer to the [Paddle-Lite Demo for OCR](https://github.com/PaddlePaddle/Paddle-Lite-Demo/tree/develop/ocr/android/app/cxx/ppocr_demo) for real-time recognition and prediction library updates.
2. Visit the [PaddleLite repository](https://github.com/PaddlePaddle/Paddle-Lite) for more development support and community discussions.

View File

@@ -0,0 +1,114 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# Android Demo
## 1. 简介
此为PaddleOCR的[Android Demo](https://github.com/PaddlePaddle/PaddleOCR/tree/main/deploy/android_demo),目前支持文本检测,文本方向分类器和文本识别模型的使用。使用 [PaddleLite v2.10](https://github.com/PaddlePaddle/Paddle-Lite/tree/release/v2.10) 进行开发。
## 2. 近期更新
- 2022.02.27
- 预测库更新到PaddleLite v2.10
- 支持6种运行模式
- 检测+分类+识别
- 检测+识别
- 分类+识别
- 检测
- 识别
- 分类
## 3. 快速使用
### 3.1 环境准备
1. 在本地环境安装好 Android Studio 工具,详细安装方法请见[Android Studio 官网](https://developer.android.com/studio)。
2. 准备一部 Android 手机,并开启 USB 调试模式。开启方法: `手机设置 -> 查找开发者选项 -> 打开开发者选项和 USB 调试模式`
**注意**:如果您的 Android Studio 尚未配置 NDK ,请根据 Android Studio 用户指南中的[安装及配置 NDK 和 CMake](https://developer.android.com/studio/projects/install-ndk)内容,预先配置好 NDK 。您可以选择最新的 NDK 版本,或者使用 Paddle Lite 预测库版本一样的 NDK
### 3.2 导入项目
点击 File->New->Import Project... 然后跟着Android Studio的引导导入
导入完成后呈现如下界面
![](./images/import_demo.jpg)
### 3.3 运行demo
将手机连接上电脑后点击Android Studio工具栏中的运行按钮即可运行demo。在此过程中手机会弹出"允许从 USB 安装软件权限"的弹窗,点击允许即可。
软件安转到手机上后会在手机主屏最后一页看到如下app
![img](./images/68747470733a.jpeg)
点击app图标即可启动app启动后app主页如下
![img](./images/68747470733a.png)
app主页中有四个按钮一个下拉列表和一个菜单按钮他们的功能分别为
- 运行模型:按照已选择的模式,运行对应的模型组合
- 拍照识别:唤起手机相机拍照并获取拍照的图像,拍照完成后需要点击运行模型进行识别
- 选取图片:唤起手机相册拍照选择图像,选择完成后需要点击运行模型进行识别
- 清空绘图:清空当前显示图像上绘制的文本框,以便进行下一次识别(每次识别使用的图像都是当前显示的图像)
- 下拉列表进行运行模式的选择目前包含6种运行模式默认模式为**检测+分类+识别**详细说明见下一节。
- 菜单按钮:点击后会进入菜单界面,进行模型和内置图像有关设置
点击运行模型后,会按照所选择的模式运行对应的模型,**检测+分类+识别**模式下运行的模型结果如下所示:
<img src="./images/run_det_cls_rec.jpg" width="400">
模型运行完成后,模型和运行状态显示区`STATUS`字段显示了当前模型的运行状态,这里显示为`run model succeeded`表明模型运行成功。
模型的运行结果显示在运行结果显示区,显示格式为
```text
序号Det(x1,y1)(x2,y2)(x3,y3)(x4,y4) Rec: 识别文本,识别置信度 Cls分类类别,分类分时
```
### 3.4 运行模式
PaddleOCR demo共提供了6种运行模式如下图
![img](./images/68747470733adfd.jpeg)
每种模式的运行结果如下表所示
| 检测+分类+识别 | 检测+识别 | 分类+识别 |
| ------------------------------------------------------------ | ------------------------------------------------ | ------------------------------------------------ |
| <img src="./images/run_det_cls_rec-20240806100407184.jpg" width="400"> | <img src="./images/run_det_rec.jpg" width="400"> | <img src="./images/run_cls_rec.jpg" width="400"> |
| 检测 | 识别 | 分类 |
| -------------------------------------------- | -------------------------------------------- | -------------------------------------------- |
| <img src="./images/run_det.jpg" width="400"> | <img src="./images/run_rec.jpg" width="400"> | <img src="./images/run_cls.jpg" width="400"> |
### 3.5 设置
设置界面如下
![img](./images/68747470733a2f2f7061.jpeg)
在设置界面可以进行如下几项设定:
1. 普通设置
- Enable custom settings: 选中状态下才能更改设置
- Model Path: 所运行的模型地址,使用默认值就好
- Label Path: 识别模型的字典
- Image Path: 进行识别的内置图像名
2. 模型运行态设置,此项设置更改后返回主界面时,会自动重新加载模型
- CPU Thread Num: 模型运行使用的CPU核心数量
- CPU Power Mode: 模型运行模式,大小核设定
3. 输入设置
- det long size: DB模型预处理时图像的长边长度超过此长度resize到该值短边进行等比例缩放小于此长度不进行处理。
4. 输出设置
- Score Threshold: DB模型后处理box的阈值低于此阈值的box进行过滤不显示。
## 4 更多支持
1. 实时识别,更新预测库可参考 <https://github.com/PaddlePaddle/Paddle-Lite-Demo/tree/develop/ocr/android/app/cxx/ppocr_demo>
2. 更多Paddle-Lite相关问题可前往[Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) ,获得更多开发支持

View File

@@ -0,0 +1,43 @@
---
comments: true
hide:
- toc
---
# Benchmark
This document gives the performance of the series models for Chinese and English recognition.
## Test Data
We collected 300 images for different real application scenarios to evaluate the overall OCR system, including contract samples, license plates, nameplates, train tickets, test sheets, forms, certificates, street view images, business cards, digital meter, etc. The following figure shows some images of the test set.
![img](./images/doc.jpg)
## Measurement
Explanation:
- The long size of the input for the text detector is 960.
- The evaluation time-consuming stage is the complete stage from image input to result output, including image pre-processing and post-processing.
- `Intel Xeon 6148` is the server-side CPU model. Intel MKL-DNN is used in the test to accelerate the CPU prediction speed.
- `Snapdragon 855` is a mobile processing platform model.
Compares the model size and F-score:
| Model Name | Model Size <br> of the <br> Whole System\(M\) | Model Size <br>of the Text <br> Detector\(M\) | Model Size <br> of the Direction <br> Classifier\(M\) | Model Size<br>of the Text <br> Recognizer \(M\) | F\-score |
|:-:|:-:|:-:|:-:|:-:|:-:|
| PP-OCRv2 | 11\.6 | 3\.0 | 0\.9 | 8\.6 | 0\.5224 |
| PP-OCR mobile | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.503 |
| PP-OCR server | 155\.1 | 47\.2 | 0\.9 | 107 | 0\.570 |
Compares the time-consuming on CPU and T4 GPU (ms):
| Model Name | CPU | T4 GPU |
|:-:|:-:|:-:|
| PP-OCRv2 | 330 | 111 |
| PP-OCR mobile | 356 | 116|
| PP-OCR server | 1056 | 200 |

View File

@@ -0,0 +1,41 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# Benchmark
本文给出了中英文OCR系列模型精度指标和在各平台预测耗时的benchmark。
## 测试数据
针对OCR实际应用场景包括合同车牌铭牌火车票化验单表格证书街景文字名片数码显示屏等收集的300张图像每张图平均有17个文本框下图给出了一些图像示例。
![img](./images/doc.jpg)
## 评估指标
说明:
- 检测输入图像的长边尺寸是960。
- 评估耗时阶段为图像预测耗时,不包括图像的预处理和后处理。
- `Intel至强6148`为服务器端CPU型号测试中使用Intel MKL-DNN 加速。
- `骁龙855`为移动端处理平台型号。
预测模型大小和整体识别精度对比
| 模型名称 | 整体模型<br>大小\(M\) | 检测模型<br>大小\(M\) | 方向分类器<br>模型大小\(M\) | 识别模型<br>大小\(M\) | 整体识别<br>F\-score |
| :-----------: | :-------------------: | :-------------------: | :-------------------------: | :-------------------: | :------------------: |
| PP-OCRv2 | 11\.6 | 3\.0 | 0\.9 | 8\.6 | 0\.5224 |
| PP-OCR mobile | 8\.1 | 2\.6 | 0\.9 | 4\.6 | 0\.503 |
| PP-OCR server | 155\.1 | 47\.2 | 0\.9 | 107 | 0\.570 |
预测模型在CPU和GPU上的速度对比单位ms
| 模型名称 | CPU | T4 GPU |
| :-----------: | :---: | :----: |
| PP-OCRv2 | 330 | 111 |
| PP-OCR mobile | 356 | 11 6 |
| PP-OCR server | 1056 | 200 |

View File

@@ -0,0 +1,435 @@
---
comments: true
hide:
- toc
---
# Server-side C++ Inference
This chapter introduces the C++ deployment steps of the PaddleOCR model. C++ is better than Python in terms of performance. Therefore, in CPU and GPU deployment scenarios, C++ deployment is mostly used.
This section will introduce how to configure the C++ environment and deploy PaddleOCR in Linux (CPU\GPU) environment. For Windows deployment please refer to [Windows](./windows_vs2019_build.en.md) compilation guidelines.
## 1. Prepare the Environment
### 1.1 Environment
- Linux, docker is recommended.
- Windows.
### 1.2 Compile OpenCV
- First of all, you need to download the source code compiled package in the Linux environment from the OpenCV official website. Taking OpenCV 3.4.7 as an example, the download command is as follows.
```bash linenums="1"
cd deploy/cpp_infer
wget https://paddleocr.bj.bcebos.com/libs/opencv/opencv-3.4.7.tar.gz
tar -xf opencv-3.4.7.tar.gz
```
Finally, you will see the folder of `opencv-3.4.7/` in the current directory.
- Compile OpenCV, the OpenCV source path (`root_path`) and installation path (`install_path`) should be set by yourself. Enter the OpenCV source code path and compile it in the following way.
```bash linenums="1"
root_path=your_opencv_root_path
install_path=${root_path}/opencv3
rm -rf build
mkdir build
cd build
cmake .. \
-DCMAKE_INSTALL_PREFIX=${install_path} \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=OFF \
-DWITH_IPP=OFF \
-DBUILD_IPP_IW=OFF \
-DWITH_LAPACK=OFF \
-DWITH_EIGEN=OFF \
-DCMAKE_INSTALL_LIBDIR=lib64 \
-DWITH_ZLIB=ON \
-DBUILD_ZLIB=ON \
-DWITH_JPEG=ON \
-DBUILD_JPEG=ON \
-DWITH_PNG=ON \
-DBUILD_PNG=ON \
-DWITH_TIFF=ON \
-DBUILD_TIFF=ON
make -j
make install
```
In the above commands, `root_path` is the downloaded OpenCV source code path, and `install_path` is the installation path of OpenCV. After `make install` is completed, the OpenCV header file and library file will be generated in this folder for later OCR source code compilation.
The final file structure under the OpenCV installation path is as follows.
```text linenums="1"
opencv3/
|-- bin
|-- include
|-- lib
|-- lib64
|-- share
```
### 1.3 Compile or Download or the Paddle Inference Library
- There are 2 ways to obtain the Paddle inference library, described in detail below.
#### 1.3.1 Direct download and installation
[Paddle inference library official website](https://www.paddlepaddle.org.cn/inference/master/guides/install/download_lib.html#linux). You can review and select the appropriate version of the inference library on the official website.
- After downloading, use the following command to extract files.
```bash linenums="1"
tar -xf paddle_inference.tgz
```
Finally you will see the folder of `paddle_inference/` in the current path.
#### 1.3.2 Compile the inference source code
- If you want to get the latest Paddle inference library features, you can download the latest code from Paddle GitHub repository and compile the inference library from the source code. It is recommended to download the inference library with paddle version greater than or equal to 2.0.1.
- You can refer to [Paddle inference library](https://www.paddlepaddle.org.cn/documentation/docs/en/advanced_guide/inference_deployment/inference/build_and_install_lib_en.html) to get the Paddle source code from GitHub, and then compile To generate the latest inference library. The method of using git to access the code is as follows.
```bash linenums="1"
git clone https://github.com/PaddlePaddle/Paddle.git
git checkout develop
```
- Enter the Paddle directory and run the following commands to compile the paddle inference library.
```bash linenums="1"
rm -rf build
mkdir build
cd build
cmake .. \
-DWITH_CONTRIB=OFF \
-DWITH_MKL=ON \
-DWITH_MKLDNN=ON \
-DWITH_TESTING=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DWITH_INFERENCE_API_TEST=OFF \
-DON_INFER=ON \
-DWITH_PYTHON=ON
make -j
make inference_lib_dist
```
For more compilation parameter options, please refer to the [document](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0/guides/05_inference_deployment/inference/build_and_install_lib_cn.html#congyuanmabianyi).
- After the compilation process, you can see the following files in the folder of `build/paddle_inference_install_dir/`.
```text linenums="1"
build/paddle_inference_install_dir/
|-- CMakeCache.txt
|-- paddle
|-- third_party
|-- version.txt
```
`paddle` is the Paddle library required for C++ prediction later, and `version.txt` contains the version information of the current inference library.
## 2. Compile and Run the Demo
### 2.1 Export the inference model
- You can refer to [Model inference](./python_infer.en.md) and export the inference model. After the model is exported, assuming it is placed in the `inference` directory, the directory structure is as follows.
```text linenums="1"
inference/
|-- det_db
| |--inference.pdiparams
| |--inference.pdmodel
|-- rec_rcnn
| |--inference.pdiparams
| |--inference.pdmodel
|-- cls
| |--inference.pdiparams
| |--inference.pdmodel
|-- table
| |--inference.pdiparams
| |--inference.pdmodel
|-- layout
| |--inference.pdiparams
| |--inference.pdmodel
```
### 2.2 Compile PaddleOCR C++ inference demo
- The compilation commands are as follows. The addresses of Paddle C++ inference library, opencv and other Dependencies need to be replaced with the actual addresses on your own machines.
```bash linenums="1"
sh tools/build.sh
```
Specifically, you should modify the paths in `tools/build.sh`. The related content is as follows.
```bash linenums="1"
OPENCV_DIR=your_opencv_dir
LIB_DIR=your_paddle_inference_dir
CUDA_LIB_DIR=your_cuda_lib_dir
CUDNN_LIB_DIR=your_cudnn_lib_dir
```
`OPENCV_DIR` is the OpenCV installation path; `LIB_DIR` is the download (`paddle_inference` folder)
or the generated Paddle inference library path (`build/paddle_inference_install_dir` folder);
`CUDA_LIB_DIR` is the CUDA library file path, in docker; it is `/usr/local/cuda/lib64`; `CUDNN_LIB_DIR` is the cuDNN library file path, in docker it is `/usr/lib/x86_64-linux-gnu/`.
- After the compilation is completed, an executable file named `ppocr` will be generated in the `build` folder.
### 2.3 Run the demo
Execute the built executable file:
```bash linenums="1"
./build/ppocr [--param1] [--param2] [...]
```
**Note**:ppocr uses the `PP-OCRv3` model by default, and the input shape used by the recognition model is `3, 48, 320`, if you want to use the old version model, you should add the parameter `--rec_img_h=32`.
Specifically,
#### 1. det+cls+rec
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--rec_model_dir=inference/rec_rcnn \
--cls_model_dir=inference/cls \
--image_dir=../../doc/imgs/12.jpg \
--use_angle_cls=true \
--det=true \
--rec=true \
--cls=true \
```
#### 2. det+rec
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--rec_model_dir=inference/rec_rcnn \
--image_dir=../../doc/imgs/12.jpg \
--use_angle_cls=false \
--det=true \
--rec=true \
--cls=false \
```
#### 3. det
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--image_dir=../../doc/imgs/12.jpg \
--det=true \
--rec=false
```
#### 4. cls+rec
```bash linenums="1"
./build/ppocr --rec_model_dir=inference/rec_rcnn \
--cls_model_dir=inference/cls \
--image_dir=../../doc/imgs_words/ch/word_1.jpg \
--use_angle_cls=true \
--det=false \
--rec=true \
--cls=true \
```
#### 5. rec
```bash linenums="1"
./build/ppocr --rec_model_dir=inference/rec_rcnn \
--image_dir=../../doc/imgs_words/ch/word_1.jpg \
--use_angle_cls=false \
--det=false \
--rec=true \
--cls=false \
```
#### 6. cls
```bash linenums="1"
./build/ppocr --cls_model_dir=inference/cls \
--cls_model_dir=inference/cls \
--image_dir=../../doc/imgs_words/ch/word_1.jpg \
--use_angle_cls=true \
--det=false \
--rec=false \
--cls=true \
```
#### 7. layout+table
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--rec_model_dir=inference/rec_rcnn \
--table_model_dir=inference/table \
--image_dir=../../ppstructure/docs/table/table.jpg \
--layout_model_dir=inference/layout \
--type=structure \
--table=true \
--layout=true
```
#### 8. layout
```bash linenums="1"
./build/ppocr --layout_model_dir=inference/layout \
--image_dir=../../ppstructure/docs/table/1.png \
--type=structure \
--table=false \
--layout=true \
--det=false \
--rec=false
```
#### 9. table
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--rec_model_dir=inference/rec_rcnn \
--table_model_dir=inference/table \
--image_dir=../../ppstructure/docs/table/table.jpg \
--type=structure \
--table=true
```
More parameters are as follows,
Common parameters
|parameter|data type|default|meaning|
| --- | --- | --- | --- |
|use_gpu|bool|false|Whether to use GPU|
|gpu_id|int|0|GPU id when use_gpu is true|
|gpu_mem|int|4000|GPU memory requested|
|cpu_math_library_num_threads|int|10|Number of threads when using CPU inference. When machine cores is enough, the large the value, the faster the inference speed|
|enable_mkldnn|bool|true|Whether to use mkdlnn library|
|output|str|./output|Path where visualization results are saved|
forward
|parameter|data type|default|meaning|
| :---: | :---: | :---: | :---: |
|det|bool|true|Whether to perform text detection in the forward direction|
|rec|bool|true|Whether to perform text recognition in the forward direction|
|cls|bool|false|Whether to perform text direction classification in the forward direction|
Detection related parameters
|parameter|data type|default|meaning|
| --- | --- | --- | --- |
|det_model_dir|string|-|Address of detection inference model|
|max_side_len|int|960|Limit the maximum image height and width to 960|
|det_db_thresh|float|0.3|Used to filter the binarized image of DB prediction, setting 0.-0.3 has no obvious effect on the result|
|det_db_box_thresh|float|0.5|DB post-processing filter box threshold, if there is a missing box detected, it can be reduced as appropriate|
|det_db_unclip_ratio|float|1.6|Indicates the compactness of the text box, the smaller the value, the closer the text box to the text|
|det_db_score_mode|string|slow| slow: use polygon box to calculate bbox score, fast: use rectangle box to calculate. Use rectangular box to calculate faster, and polygonal box more accurate for curved text area.|
|visualize|bool|true|Whether to visualize the resultswhen it is set as true, the prediction results will be saved in the folder specified by the `output` field on an image with the same name as the input image.|
Classifier related parameters
|parameter|data type|default|meaning|
| --- | --- | --- | --- |
|use_angle_cls|bool|false|Whether to use the direction classifier|
|cls_model_dir|string|-|Address of direction classifier inference model|
|cls_thresh|float|0.9|Score threshold of the direction classifier|
|cls_batch_num|int|1|batch size of classifier|
Recognition related parameters
|parameter|data type|default|meaning|
| --- | --- | --- | --- |
|rec_model_dir|string|-|Address of recognition inference model|
|rec_char_dict_path|string|../../ppocr/utils/ppocr_keys_v1.txt|dictionary file|
|rec_batch_num|int|6|batch size of recognition|
|rec_img_h|int|48|image height of recognition|
|rec_img_w|int|320|image width of recognition|
Layout related parameters
|parameter|data type|default|meaning|
| :---: | :---: | :---: | :---: |
|layout_model_dir|string|-| Address of layout inference model|
|layout_dict_path|string|../../ppocr/utils/dict/layout_dict/layout_publaynet_dict.txt|dictionary file|
|layout_score_threshold|float|0.5|Threshold of score.|
|layout_nms_threshold|float|0.5|Threshold of nms.|
Table recognition related parameters
|parameter|data type|default|meaning|
| :---: | :---: | :---: | :---: |
|table_model_dir|string|-|Address of table recognition inference model|
|table_char_dict_path|string|../../ppocr/utils/dict/table_structure_dict.txt|dictionary file|
|table_max_len|int|488|The size of the long side of the input image of the table recognition model, the final input image size of the network istable_max_lentable_max_len|
|merge_no_span_structure|bool|true|Whether to merge <td> and </td> to <td></td|
Multi-language inference is also supported in PaddleOCR, you can refer to [recognition tutorial](../model_train/recognition.en.md) for more supported languages and models in PaddleOCR. Specifically, if you want to infer using multi-language models, you just need to modify values of `rec_char_dict_path` and `rec_model_dir`.
The detection results will be shown on the screen, which is as follows.
```bash linenums="1"
predict img: ../../doc/imgs/12.jpg
../../doc/imgs/12.jpg
0 det boxes: [[74,553],[427,542],[428,571],[75,582]] rec text: 打浦路252935号 rec score: 0.947724
1 det boxes: [[23,507],[513,488],[515,529],[24,548]] rec text: 绿洲仕格维花园公寓 rec score: 0.993728
2 det boxes: [[187,456],[399,448],[400,480],[188,488]] rec text: 打浦路15号 rec score: 0.964994
3 det boxes: [[42,413],[483,391],[484,428],[43,450]] rec text: 上海斯格威铂尔大酒店 rec score: 0.980086
The detection visualized image saved in ./output//12.jpg
```
- layout+table
```bash linenums="1"
predict img: ../../ppstructure/docs/table/1.png
0 type: text, region: [12,729,410,848], score: 0.781044, res: count of ocr result is : 7
********** print ocr result **********
0 det boxes: [[4,1],[79,1],[79,12],[4,12]] rec text: CTW1500. rec score: 0.769472
...
6 det boxes: [[4,99],[391,99],[391,112],[4,112]] rec text: sate-of-the-artmethods[12.34.36l.ourapproachachieves rec score: 0.90414
********** end print ocr result **********
1 type: text, region: [69,342,342,359], score: 0.703666, res: count of ocr result is : 1
********** print ocr result **********
0 det boxes: [[8,2],[269,2],[269,13],[8,13]] rec text: Table6.Experimentalresults on CTW-1500 rec score: 0.890454
********** end print ocr result **********
2 type: text, region: [70,316,706,332], score: 0.659738, res: count of ocr result is : 2
********** print ocr result **********
0 det boxes: [[373,2],[630,2],[630,11],[373,11]] rec text: oroposals.andthegreencontoursarefinal rec score: 0.919729
1 det boxes: [[8,3],[357,3],[357,11],[8,11]] rec text: Visualexperimentalresultshebluecontoursareboundar rec score: 0.915963
********** end print ocr result **********
3 type: text, region: [489,342,789,359], score: 0.630538, res: count of ocr result is : 1
********** print ocr result **********
0 det boxes: [[8,2],[294,2],[294,14],[8,14]] rec text: Table7.Experimentalresults onMSRA-TD500 rec score: 0.942251
********** end print ocr result **********
4 type: text, region: [444,751,841,848], score: 0.607345, res: count of ocr result is : 5
********** print ocr result **********
0 det boxes: [[19,3],[389,3],[389,17],[19,17]] rec text: Inthispaper,weproposeanovel adaptivebound rec score: 0.941031
1 det boxes: [[4,22],[390,22],[390,36],[4,36]] rec text: aryproposalnetworkforarbitraryshapetextdetection rec score: 0.960172
2 det boxes: [[4,42],[392,42],[392,56],[4,56]] rec text: whichadoptanboundaryproposalmodeltogeneratecoarse rec score: 0.934647
3 det boxes: [[4,61],[389,61],[389,75],[4,75]] rec text: ooundaryproposals,andthenadoptanadaptiveboundary rec score: 0.946296
4 det boxes: [[5,80],[387,80],[387,93],[5,93]] rec text: leformationmodelcombinedwithGCNandRNNtoper rec score: 0.952401
********** end print ocr result **********
5 type: title, region: [444,705,564,724], score: 0.785429, res: count of ocr result is : 1
********** print ocr result **********
0 det boxes: [[6,2],[113,2],[113,14],[6,14]] rec text: 5.Conclusion rec score: 0.856903
********** end print ocr result **********
6 type: table, region: [14,360,402,711], score: 0.963643, res: <html><body><table><thead><tr><td>Methods</td><td>Ext</td><td>R</td><td>P</td><td>F</td><td>FPS</td></tr></thead><tbody><tr><td>TextSnake [18]</td><td>Syn</td><td>85.3</td><td>67.9</td><td>75.6</td><td></td></tr><tr><td>CSE [17]</td><td>MiLT</td><td>76.1</td><td>78.7</td><td>77.4</td><td>0.38</td></tr><tr><td>LOMO[40]</td><td>Syn</td><td>76.5</td><td>85.7</td><td>80.8</td><td>4.4</td></tr><tr><td>ATRR[35]</td><td>Sy-</td><td>80.2</td><td>80.1</td><td>80.1</td><td>-</td></tr><tr><td>SegLink++ [28]</td><td>Syn</td><td>79.8</td><td>82.8</td><td>81.3</td><td>-</td></tr><tr><td>TextField [37]</td><td>Syn</td><td>79.8</td><td>83.0</td><td>81.4</td><td>6.0</td></tr><tr><td>MSR[38]</td><td>Syn</td><td>79.0</td><td>84.1</td><td>81.5</td><td>4.3</td></tr><tr><td>PSENet-1s [33]</td><td>MLT</td><td>79.7</td><td>84.8</td><td>82.2</td><td>3.9</td></tr><tr><td>DB [12]</td><td>Syn</td><td>80.2</td><td>86.9</td><td>83.4</td><td>22.0</td></tr><tr><td>CRAFT [2]</td><td>Syn</td><td>81.1</td><td>86.0</td><td>83.5</td><td>-</td></tr><tr><td>TextDragon [5]</td><td>MLT+</td><td>82.8</td><td>84.5</td><td>83.6</td><td></td></tr><tr><td>PAN [34]</td><td>Syn</td><td>81.2</td><td>86.4</td><td>83.7</td><td>39.8</td></tr><tr><td>ContourNet [36]</td><td></td><td>84.1</td><td>83.7</td><td>83.9</td><td>4.5</td></tr><tr><td>DRRG [41]</td><td>MLT</td><td>83.02</td><td>85.93</td><td>84.45</td><td>-</td></tr><tr><td>TextPerception[23]</td><td>Syn</td><td>81.9</td><td>87.5</td><td>84.6</td><td></td></tr><tr><td>Ours</td><td> Syn</td><td>80.57</td><td>87.66</td><td>83.97</td><td>12.08</td></tr><tr><td>Ours</td><td></td><td>81.45</td><td>87.81</td><td>84.51</td><td>12.15</td></tr><tr><td>Ours</td><td>MLT</td><td>83.60</td><td>86.45</td><td>85.00</td><td>12.21</td></tr></tbody></table></body></html>
The table visualized image saved in ./output//6_1.png
7 type: table, region: [462,359,820,657], score: 0.953917, res: <html><body><table><thead><tr><td>Methods</td><td>R</td><td>P</td><td>F</td><td>FPS</td></tr></thead><tbody><tr><td>SegLink [26]</td><td>70.0</td><td>86.0</td><td>77.0</td><td>8.9</td></tr><tr><td>PixelLink [4]</td><td>73.2</td><td>83.0</td><td>77.8</td><td>-</td></tr><tr><td>TextSnake [18]</td><td>73.9</td><td>83.2</td><td>78.3</td><td>1.1</td></tr><tr><td>TextField [37]</td><td>75.9</td><td>87.4</td><td>81.3</td><td>5.2 </td></tr><tr><td>MSR[38]</td><td>76.7</td><td>87.4</td><td>81.7</td><td>-</td></tr><tr><td>FTSN[3]</td><td>77.1</td><td>87.6</td><td>82.0</td><td>:</td></tr><tr><td>LSE[30]</td><td>81.7</td><td>84.2</td><td>82.9</td><td></td></tr><tr><td>CRAFT [2]</td><td>78.2</td><td>88.2</td><td>82.9</td><td>8.6</td></tr><tr><td>MCN [16]</td><td>79</td><td>88</td><td>83</td><td>-</td></tr><tr><td>ATRR[35]</td><td>82.1</td><td>85.2</td><td>83.6</td><td>-</td></tr><tr><td>PAN [34]</td><td>83.8</td><td>84.4</td><td>84.1</td><td>30.2</td></tr><tr><td>DB[12]</td><td>79.2</td><td>91.5</td><td>84.9</td><td>32.0</td></tr><tr><td>DRRG [41]</td><td>82.30</td><td>88.05</td><td>85.08</td><td>-</td></tr><tr><td>Ours (SynText)</td><td>80.68</td><td>85.40</td><td>82.97</td><td>12.68</td></tr><tr><td>Ours (MLT-17)</td><td>84.54</td><td>86.62</td><td>85.57</td><td>12.31</td></tr></tbody></table></body></html>
The table visualized image saved in ./output//7_1.png
8 type: figure, region: [14,3,836,310], score: 0.969443, res: count of ocr result is : 26
********** print ocr result **********
0 det boxes: [[506,14],[539,15],[539,22],[506,21]] rec text: E rec score: 0.318073
...
25 det boxes: [[680,290],[759,288],[759,303],[680,305]] rec text: (d) CTW1500 rec score: 0.95911
********** end print ocr result **********
```
## 3. FAQ
1. Encountered the error `unable to access 'https://github.com/LDOUBLEV/AutoLog.git/': gnutls_handshake() failed: The TLS connection was non-properly terminated.`, change the github address in `deploy/cpp_infer/external-cmake/auto-log.cmake` to the <https://gitee.com/Double_V/AutoLog> address.

View File

@@ -0,0 +1,445 @@
---
comments: true
hide:
- toc
---
# 服务器端C++预测
本章节介绍PaddleOCR 模型的C++部署方法。C++在性能计算上优于Python因此在大多数CPU、GPU部署场景多采用C++的部署方式本节将介绍如何在Linux\Windows (CPU\GPU)环境下配置C++环境并完成PaddleOCR模型部署。
## 1. 准备环境
### 1.1 运行准备
- Linux环境推荐使用docker。
- Windows环境。
- 该文档主要介绍基于Linux环境的PaddleOCR C++预测流程如果需要在Windows下基于预测库进行C++预测,具体编译方法请参考[Windows下编译教程](./windows_vs2019_build.md)
### 1.2 编译opencv库
- 首先需要从opencv官网上下载在Linux环境下源码编译的包以opencv3.4.7为例,下载命令如下:
```bash linenums="1"
cd deploy/cpp_infer
wget https://paddleocr.bj.bcebos.com/libs/opencv/opencv-3.4.7.tar.gz
tar -xf opencv-3.4.7.tar.gz
```
最终可以在当前目录下看到`opencv-3.4.7/`的文件夹。
- 编译opencv设置opencv源码路径(`root_path`)以及安装路径(`install_path`)。进入opencv源码路径下按照下面的方式进行编译。
```bash linenums="1"
root_path="your_opencv_root_path"
install_path=${root_path}/opencv3
build_dir=${root_path}/build
rm -rf ${build_dir}
mkdir ${build_dir}
cd ${build_dir}
cmake .. \
-DCMAKE_INSTALL_PREFIX=${install_path} \
-DCMAKE_BUILD_TYPE=Release \
-DBUILD_SHARED_LIBS=OFF \
-DWITH_IPP=OFF \
-DBUILD_IPP_IW=OFF \
-DWITH_LAPACK=OFF \
-DWITH_EIGEN=OFF \
-DCMAKE_INSTALL_LIBDIR=lib64 \
-DWITH_ZLIB=ON \
-DBUILD_ZLIB=ON \
-DWITH_JPEG=ON \
-DBUILD_JPEG=ON \
-DWITH_PNG=ON \
-DBUILD_PNG=ON \
-DWITH_TIFF=ON \
-DBUILD_TIFF=ON
make -j
make install
```
也可以直接修改`tools/build_opencv.sh`的内容,然后直接运行下面的命令进行编译。
```bash linenums="1"
sh tools/build_opencv.sh
```
其中`root_path`为下载的opencv源码路径`install_path`为opencv的安装路径`make install`完成之后会在该文件夹下生成opencv头文件和库文件用于后面的OCR代码编译。
最终在安装路径下的文件结构如下所示。
```
opencv3/
|-- bin
|-- include
|-- lib
|-- lib64
|-- share
```
### 1.3 下载或者编译Paddle预测库
可以选择直接下载安装或者从源码编译,下文分别进行具体说明。
#### 1.3.1 直接下载安装
[Paddle预测库官网](https://www.paddlepaddle.org.cn/inference/master/guides/install/download_lib.html#linux) 上提供了不同cuda版本的Linux预测库可以在官网查看并选择合适的预测库版本*建议选择paddle版本>=2.0.1版本的预测库* )。
下载之后解压:
```bash linenums="1"
tar -xf paddle_inference.tgz
```
最终会在当前的文件夹中生成`paddle_inference/`的子文件夹。
#### 1.3.2 预测库源码编译
如果希望获取最新预测库特性可以从github上克隆最新Paddle代码进行编译生成最新的预测库。
使用git获取代码:
```bash linenums="1"
git clone https://github.com/PaddlePaddle/Paddle.git
git checkout develop
```
进入Paddle目录进行编译:
```bash linenums="1"
rm -rf build
mkdir build
cd build
cmake .. \
-DWITH_CONTRIB=OFF \
-DWITH_MKL=ON \
-DWITH_MKLDNN=ON \
-DWITH_TESTING=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DWITH_INFERENCE_API_TEST=OFF \
-DON_INFER=ON \
-DWITH_PYTHON=ON
make -j
make inference_lib_dist
```
更多编译参数选项介绍可以参考[Paddle预测库编译文档](https://www.paddlepaddle.org.cn/documentation/docs/zh/2.0/guides/05_inference_deployment/inference/build_and_install_lib_cn.html#congyuanmabianyi)。
编译完成之后,可以在`build/paddle_inference_install_dir/`文件下看到生成了以下文件及文件夹。
```
build/paddle_inference_install_dir/
|-- CMakeCache.txt
|-- paddle
|-- third_party
|-- version.txt
```
其中`paddle`就是C++预测所需的Paddle库`version.txt`中包含当前预测库的版本信息。
## 2. 开始运行
### 2.1 准备模型
直接下载PaddleOCR提供的推理模型或者参考[模型预测章节](./python_infer.md),将训练好的模型导出为推理模型。模型导出之后,假设放在`inference`目录下,则目录结构如下:
```text linenums="1"
inference/
|-- det_db
| |--inference.pdiparams
| |--inference.pdmodel
|-- rec_rcnn
| |--inference.pdiparams
| |--inference.pdmodel
|-- cls
| |--inference.pdiparams
| |--inference.pdmodel
|-- table
| |--inference.pdiparams
| |--inference.pdmodel
|-- layout
| |--inference.pdiparams
| |--inference.pdmodel
```
### 2.2 编译PaddleOCR C++预测demo
编译命令如下其中Paddle C++预测库、opencv等其他依赖库的地址需要换成自己机器上的实际地址。
```bash linenums="1"
sh tools/build.sh
```
具体的,需要修改`tools/build.sh`中环境路径,相关内容如下:
```bash linenums="1"
OPENCV_DIR=your_opencv_dir
LIB_DIR=your_paddle_inference_dir
CUDA_LIB_DIR=your_cuda_lib_dir
CUDNN_LIB_DIR=/your_cudnn_lib_dir
```
其中,`OPENCV_DIR`为opencv编译安装的地址`LIB_DIR`为下载(`paddle_inference`文件夹)或者编译生成的Paddle预测库地址(`build/paddle_inference_install_dir`文件夹)`CUDA_LIB_DIR`为cuda库文件地址在docker中为`/usr/local/cuda/lib64``CUDNN_LIB_DIR`为cudnn库文件地址在docker中为`/usr/lib/x86_64-linux-gnu/`。**注意:以上路径都写绝对路径,不要写相对路径。**
编译完成之后,会在`build`文件夹下生成一个名为`ppocr`的可执行文件。
### 2.3 运行demo
本demo支持系统串联调用也支持单个功能的调用只使用检测或识别功能。
**注意** ppocr默认使用`PP-OCRv3`模型识别模型使用的输入shape为`3,48,320`, 如需使用旧版本的PP-OCR模型则需要设置参数`--rec_img_h=32`。
运行方式:
```bash linenums="1"
./build/ppocr [--param1] [--param2] [...]
```
具体命令如下:
##### 1. 检测+分类+识别
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--rec_model_dir=inference/rec_rcnn \
--cls_model_dir=inference/cls \
--image_dir=../../doc/imgs/12.jpg \
--use_angle_cls=true \
--det=true \
--rec=true \
--cls=true \
```
##### 2. 检测+识别
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--rec_model_dir=inference/rec_rcnn \
--image_dir=../../doc/imgs/12.jpg \
--use_angle_cls=false \
--det=true \
--rec=true \
--cls=false \
```
##### 3. 检测
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--image_dir=../../doc/imgs/12.jpg \
--det=true \
--rec=false
```
##### 4. 分类+识别
```bash linenums="1"
./build/ppocr --rec_model_dir=inference/rec_rcnn \
--cls_model_dir=inference/cls \
--image_dir=../../doc/imgs_words/ch/word_1.jpg \
--use_angle_cls=true \
--det=false \
--rec=true \
--cls=true \
```
##### 5. 识别
```bash linenums="1"
./build/ppocr --rec_model_dir=inference/rec_rcnn \
--image_dir=../../doc/imgs_words/ch/word_1.jpg \
--use_angle_cls=false \
--det=false \
--rec=true \
--cls=false \
```
##### 6. 分类
```bash linenums="1"
./build/ppocr --cls_model_dir=inference/cls \
--cls_model_dir=inference/cls \
--image_dir=../../doc/imgs_words/ch/word_1.jpg \
--use_angle_cls=true \
--det=false \
--rec=false \
--cls=true \
```
##### 7. 版面分析+表格识别
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--rec_model_dir=inference/rec_rcnn \
--table_model_dir=inference/table \
--image_dir=../../ppstructure/docs/table/table.jpg \
--layout_model_dir=inference/layout \
--type=structure \
--table=true \
--layout=true
```
##### 8. 版面分析
```bash linenums="1"
./build/ppocr --layout_model_dir=inference/layout \
--image_dir=../../ppstructure/docs/table/1.png \
--type=structure \
--table=false \
--layout=true \
--det=false \
--rec=false
```
##### 9. 表格识别
```bash linenums="1"
./build/ppocr --det_model_dir=inference/det_db \
--rec_model_dir=inference/rec_rcnn \
--table_model_dir=inference/table \
--image_dir=../../ppstructure/docs/table/table.jpg \
--type=structure \
--table=true
```
更多支持的可调节参数解释如下:
通用参数
| 参数名称 | 类型 | 默认参数 | 意义 |
| :--------------------------: | :---: | :------: | :---------------------------------------------------------------: |
| use_gpu | bool | false | 是否使用GPU |
| gpu_id | int | 0 | GPU id使用GPU时有效 |
| gpu_mem | int | 4000 | 申请的GPU内存 |
| cpu_math_library_num_threads | int | 10 | CPU预测时的线程数在机器核数充足的情况下该值越大预测速度越快 |
| enable_mkldnn | bool | true | 是否使用mkldnn库 |
| output | str | ./output | 可视化结果保存的路径 |
前向相关
| 参数名称 | 类型 | 默认参数 | 意义 |
| :------: | :---: | :------: | :----------------------: |
| det | bool | true | 前向是否执行文字检测 |
| rec | bool | true | 前向是否执行文字识别 |
| cls | bool | false | 前向是否执行文字方向分类 |
检测模型相关
| 参数名称 | 类型 | 默认参数 | 意义 |
| :-----------------: | :----: | :------: | :----------------------------------------------------------------------------------------------------------: |
| det_model_dir | string | - | 检测模型inference model地址 |
| max_side_len | int | 960 | 输入图像长宽大于960时等比例缩放图像使得图像最长边为960 |
| det_db_thresh | float | 0.3 | 用于过滤DB预测的二值化图像设置为0.-0.3对结果影响不明显 |
| det_db_box_thresh | float | 0.5 | DB后处理过滤box的阈值如果检测存在漏框情况可酌情减小 |
| det_db_unclip_ratio | float | 1.6 | 表示文本框的紧致程度,越小则文本框更靠近文本 |
| det_db_score_mode | string | slow | slow:使用多边形框计算bbox scorefast:使用矩形框计算。矩形框计算速度更快,多边形框对弯曲文本区域计算更准确。 |
| visualize | bool | true | 是否对结果进行可视化为1时预测结果会保存在`output`字段指定的文件夹下和输入图像同名的图像上。 |
方向分类器相关
| 参数名称 | 类型 | 默认参数 | 意义 |
| :-----------: | :----: | :------: | :---------------------------: |
| use_angle_cls | bool | false | 是否使用方向分类器 |
| cls_model_dir | string | - | 方向分类器inference model地址 |
| cls_thresh | float | 0.9 | 方向分类器的得分阈值 |
| cls_batch_num | int | 1 | 方向分类器batchsize |
文字识别模型相关
| 参数名称 | 类型 | 默认参数 | 意义 |
| :----------------: | :----: | :---------------------------------: | :-----------------------------: |
| rec_model_dir | string | - | 文字识别模型inference model地址 |
| rec_char_dict_path | string | ../../ppocr/utils/ppocr_keys_v1.txt | 字典文件 |
| rec_batch_num | int | 6 | 文字识别模型batchsize |
| rec_img_h | int | 48 | 文字识别模型输入图像高度 |
| rec_img_w | int | 320 | 文字识别模型输入图像宽度 |
版面分析模型相关
| 参数名称 | 类型 | 默认参数 | 意义 |
| :--------------------: | :----: | :----------------------------------------------------------: | :-----------------------------: |
| layout_model_dir | string | - | 版面分析模型inference model地址 |
| layout_dict_path | string | ../../ppocr/utils/dict/layout_dict/layout_publaynet_dict.txt | 字典文件 |
| layout_score_threshold | float | 0.5 | 检测框的分数阈值 |
| layout_nms_threshold | float | 0.5 | nms的阈值 |
表格识别模型相关
| 参数名称 | 类型 | 默认参数 | 意义 |
| :---------------------: | :----: | :------------------------------------------------: | :----------------------------------------------------------------------------------: |
| table_model_dir | string | - | 表格识别模型inference model地址 |
| table_char_dict_path | string | ../../ppocr/utils/dict/table_structure_dict_ch.txt | 字典文件 |
| table_max_len | int | 488 | 表格识别模型输入图像长边大小最终网络输入图像大小为table_max_lentable_max_len |
| merge_no_span_structure | bool | true | 是否合并<td> 和 </td> 为<td></td> |
- PaddleOCR也支持多语言的预测更多支持的语言和模型可以参考[识别文档](../model_train/recognition.md)中的多语言字典与模型部分,如果希望进行多语言预测,只需将修改`rec_char_dict_path`(字典文件路径)以及`rec_model_dir`inference模型路径字段即可。
最终屏幕上会输出检测结果如下:
- ocr
```bash linenums="1"
predict img: ../../doc/imgs/12.jpg
../../doc/imgs/12.jpg
0 det boxes: [[74,553],[427,542],[428,571],[75,582]] rec text: 打浦路252935号 rec score: 0.947724
1 det boxes: [[23,507],[513,488],[515,529],[24,548]] rec text: 绿洲仕格维花园公寓 rec score: 0.993728
2 det boxes: [[187,456],[399,448],[400,480],[188,488]] rec text: 打浦路15号 rec score: 0.964994
3 det boxes: [[42,413],[483,391],[484,428],[43,450]] rec text: 上海斯格威铂尔大酒店 rec score: 0.980086
The detection visualized image saved in ./output//12.jpg
```
- layout+table
```bash linenums="1"
predict img: ../../ppstructure/docs/table/1.png
0 type: text, region: [12,729,410,848], score: 0.781044, res: count of ocr result is : 7
********** print ocr result **********
0 det boxes: [[4,1],[79,1],[79,12],[4,12]] rec text: CTW1500. rec score: 0.769472
...
6 det boxes: [[4,99],[391,99],[391,112],[4,112]] rec text: sate-of-the-artmethods[12.34.36l.ourapproachachieves rec score: 0.90414
********** end print ocr result **********
1 type: text, region: [69,342,342,359], score: 0.703666, res: count of ocr result is : 1
********** print ocr result **********
0 det boxes: [[8,2],[269,2],[269,13],[8,13]] rec text: Table6.Experimentalresults on CTW-1500 rec score: 0.890454
********** end print ocr result **********
2 type: text, region: [70,316,706,332], score: 0.659738, res: count of ocr result is : 2
********** print ocr result **********
0 det boxes: [[373,2],[630,2],[630,11],[373,11]] rec text: oroposals.andthegreencontoursarefinal rec score: 0.919729
1 det boxes: [[8,3],[357,3],[357,11],[8,11]] rec text: Visualexperimentalresultshebluecontoursareboundar rec score: 0.915963
********** end print ocr result **********
3 type: text, region: [489,342,789,359], score: 0.630538, res: count of ocr result is : 1
********** print ocr result **********
0 det boxes: [[8,2],[294,2],[294,14],[8,14]] rec text: Table7.Experimentalresults onMSRA-TD500 rec score: 0.942251
********** end print ocr result **********
4 type: text, region: [444,751,841,848], score: 0.607345, res: count of ocr result is : 5
********** print ocr result **********
0 det boxes: [[19,3],[389,3],[389,17],[19,17]] rec text: Inthispaper,weproposeanovel adaptivebound rec score: 0.941031
1 det boxes: [[4,22],[390,22],[390,36],[4,36]] rec text: aryproposalnetworkforarbitraryshapetextdetection rec score: 0.960172
2 det boxes: [[4,42],[392,42],[392,56],[4,56]] rec text: whichadoptanboundaryproposalmodeltogeneratecoarse rec score: 0.934647
3 det boxes: [[4,61],[389,61],[389,75],[4,75]] rec text: ooundaryproposals,andthenadoptanadaptiveboundary rec score: 0.946296
4 det boxes: [[5,80],[387,80],[387,93],[5,93]] rec text: leformationmodelcombinedwithGCNandRNNtoper rec score: 0.952401
********** end print ocr result **********
5 type: title, region: [444,705,564,724], score: 0.785429, res: count of ocr result is : 1
********** print ocr result **********
0 det boxes: [[6,2],[113,2],[113,14],[6,14]] rec text: 5.Conclusion rec score: 0.856903
********** end print ocr result **********
6 type: table, region: [14,360,402,711], score: 0.963643, res: <html><body><table><thead><tr><td>Methods</td><td>Ext</td><td>R</td><td>P</td><td>F</td><td>FPS</td></tr></thead><tbody><tr><td>TextSnake [18]</td><td>Syn</td><td>85.3</td><td>67.9</td><td>75.6</td><td></td></tr><tr><td>CSE [17]</td><td>MiLT</td><td>76.1</td><td>78.7</td><td>77.4</td><td>0.38</td></tr><tr><td>LOMO[40]</td><td>Syn</td><td>76.5</td><td>85.7</td><td>80.8</td><td>4.4</td></tr><tr><td>ATRR[35]</td><td>Sy-</td><td>80.2</td><td>80.1</td><td>80.1</td><td>-</td></tr><tr><td>SegLink++ [28]</td><td>Syn</td><td>79.8</td><td>82.8</td><td>81.3</td><td>-</td></tr><tr><td>TextField [37]</td><td>Syn</td><td>79.8</td><td>83.0</td><td>81.4</td><td>6.0</td></tr><tr><td>MSR[38]</td><td>Syn</td><td>79.0</td><td>84.1</td><td>81.5</td><td>4.3</td></tr><tr><td>PSENet-1s [33]</td><td>MLT</td><td>79.7</td><td>84.8</td><td>82.2</td><td>3.9</td></tr><tr><td>DB [12]</td><td>Syn</td><td>80.2</td><td>86.9</td><td>83.4</td><td>22.0</td></tr><tr><td>CRAFT [2]</td><td>Syn</td><td>81.1</td><td>86.0</td><td>83.5</td><td>-</td></tr><tr><td>TextDragon [5]</td><td>MLT+</td><td>82.8</td><td>84.5</td><td>83.6</td><td></td></tr><tr><td>PAN [34]</td><td>Syn</td><td>81.2</td><td>86.4</td><td>83.7</td><td>39.8</td></tr><tr><td>ContourNet [36]</td><td></td><td>84.1</td><td>83.7</td><td>83.9</td><td>4.5</td></tr><tr><td>DRRG [41]</td><td>MLT</td><td>83.02</td><td>85.93</td><td>84.45</td><td>-</td></tr><tr><td>TextPerception[23]</td><td>Syn</td><td>81.9</td><td>87.5</td><td>84.6</td><td></td></tr><tr><td>Ours</td><td> Syn</td><td>80.57</td><td>87.66</td><td>83.97</td><td>12.08</td></tr><tr><td>Ours</td><td></td><td>81.45</td><td>87.81</td><td>84.51</td><td>12.15</td></tr><tr><td>Ours</td><td>MLT</td><td>83.60</td><td>86.45</td><td>85.00</td><td>12.21</td></tr></tbody></table></body></html>
The table visualized image saved in ./output//6_1.png
7 type: table, region: [462,359,820,657], score: 0.953917, res: <html><body><table><thead><tr><td>Methods</td><td>R</td><td>P</td><td>F</td><td>FPS</td></tr></thead><tbody><tr><td>SegLink [26]</td><td>70.0</td><td>86.0</td><td>77.0</td><td>8.9</td></tr><tr><td>PixelLink [4]</td><td>73.2</td><td>83.0</td><td>77.8</td><td>-</td></tr><tr><td>TextSnake [18]</td><td>73.9</td><td>83.2</td><td>78.3</td><td>1.1</td></tr><tr><td>TextField [37]</td><td>75.9</td><td>87.4</td><td>81.3</td><td>5.2 </td></tr><tr><td>MSR[38]</td><td>76.7</td><td>87.4</td><td>81.7</td><td>-</td></tr><tr><td>FTSN[3]</td><td>77.1</td><td>87.6</td><td>82.0</td><td>:</td></tr><tr><td>LSE[30]</td><td>81.7</td><td>84.2</td><td>82.9</td><td></td></tr><tr><td>CRAFT [2]</td><td>78.2</td><td>88.2</td><td>82.9</td><td>8.6</td></tr><tr><td>MCN [16]</td><td>79</td><td>88</td><td>83</td><td>-</td></tr><tr><td>ATRR[35]</td><td>82.1</td><td>85.2</td><td>83.6</td><td>-</td></tr><tr><td>PAN [34]</td><td>83.8</td><td>84.4</td><td>84.1</td><td>30.2</td></tr><tr><td>DB[12]</td><td>79.2</td><td>91.5</td><td>84.9</td><td>32.0</td></tr><tr><td>DRRG [41]</td><td>82.30</td><td>88.05</td><td>85.08</td><td>-</td></tr><tr><td>Ours (SynText)</td><td>80.68</td><td>85.40</td><td>82.97</td><td>12.68</td></tr><tr><td>Ours (MLT-17)</td><td>84.54</td><td>86.62</td><td>85.57</td><td>12.31</td></tr></tbody></table></body></html>
The table visualized image saved in ./output//7_1.png
8 type: figure, region: [14,3,836,310], score: 0.969443, res: count of ocr result is : 26
********** print ocr result **********
0 det boxes: [[506,14],[539,15],[539,22],[506,21]] rec text: E rec score: 0.318073
...
25 det boxes: [[680,290],[759,288],[759,303],[680,305]] rec text: (d) CTW1500 rec score: 0.95911
********** end print ocr result **********
```
## 3. FAQ
1. 遇到报错 `unable to access 'https://github.com/LDOUBLEV/AutoLog.git/': gnutls_handshake() failed: The TLS connection was non-properly terminated.` 将 `deploy/cpp_infer/external-cmake/auto-log.cmake` 中的github地址改为 <https://gitee.com/Double_V/AutoLog> 地址即可。

Binary file not shown.

After

Width:  |  Height:  |  Size: 286 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 372 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 554 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 195 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 112 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 173 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 510 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 258 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 332 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 476 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 530 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 325 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 218 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 125 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 331 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 332 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 332 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 404 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 352 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 722 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 561 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 140 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 206 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 206 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 493 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 655 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 119 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 108 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 116 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 197 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 213 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 213 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 207 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 108 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 195 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 418 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 KiB

View File

@@ -0,0 +1,30 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# Legacy Features
## Overview
This section introduces the features and models related to the PaddleOCR 2.x branch. Due to the upgrades in the 3.x branch, some models and features are no longer compatible with the older branch. Therefore, users who need to use or refer to the features of the older branch can refer to this part of the documentation.
## Models Supported by PaddleOCR 2.x Branch:
* [Model List](model_list.md)
## Features Supported by PaddleOCR 2.x Branch:
* [Inference with Python Prediction Engine](python_infer.en.md)
* [Inference with C++ Prediction Engine](cpp_infer.en.md)
* [Compilation Guide for Visual Studio 2019 Community CMake](windows_vs2019_build.en.md)
* [Service-Oriented Deployment](paddle_server.en.md)
* [Android Deployment](android_demo.en.md)
* [Jetson Deployment](Jetson_infer.en.md)
* [Edge Device Deployment](lite.en.md)
* [Web Frontend Deployment](paddle_js.en.md)
* [Paddle2ONNX Model Conversion and Prediction](paddle2onnx.en.md)
* [PaddlePaddle Cloud Deployment Tool](paddle_cloud.en.md)
* [Benchmark](benchmark.en.md)

View File

@@ -0,0 +1,30 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# 历史遗留功能
## 概述
本节介绍了 PaddleOCR 2.x 分支的相关功能和模型。由于 3.x 分支的升级,部分模型和功能与旧分支不再兼容。因此,需要使用或参考旧分支特性的用户可以参考这部分文档。
## PaddleOCR 2.x 分支支持的模型:
* [模型列表](model_list.md)
## PaddleOCR 2.x 分支支持的功能:
* [基于Python预测引擎推理](python_infer.md)
* [基于C++预测引擎推理](cpp_infer.md)
* [Visual Studio 2019 Community CMake 编译指南](windows_vs2019_build.md)
* [服务化部署](paddle_server.md)
* [Android 部署](android_demo.md)
* [Jetson 部署](Jetson_infer.md)
* [端侧部署](lite.md)
* [网页前端部署](paddle_js.md)
* [Paddle2ONNX模型转化与预测](paddle2onnx.md)
* [云上飞桨部署工具](paddle_cloud.md)
* [Benchmark](benchmark.md)

View File

@@ -0,0 +1,309 @@
---
comments: true
hide:
- toc
---
# Mobile deployment based on Paddle-Lite
This tutorial will introduce how to use [Paddle-Lite](https://github.com/PaddlePaddle/Paddle-Lite) to deploy PaddleOCR ultra-lightweight Chinese and English detection models on mobile phones.
Paddle-Lite is a lightweight inference engine for PaddlePaddle. It provides efficient inference capabilities for mobile phones and IoT, and extensively integrates cross-platform hardware to provide lightweight deployment solutions for end-side deployment issues.
## 1. Preparation
### Preparation environment
- Computer (for Compiling Paddle Lite)
- Mobile phone (arm7 or arm8)
### 1.1 Prepare the cross-compilation environment
The cross-compilation environment is used to compile C++ demos of Paddle Lite and PaddleOCR.
Supports multiple development environments.
For the compilation process of different development environments, please refer to the corresponding documents.
1. [Docker](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#docker)
2. [Linux](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#linux)
3. [MAC OS](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#mac-os)
### 1.2 Prepare Paddle-Lite library
There are two ways to obtain the Paddle-Lite library:
1. [Recommended] Download directly, the download link of the Paddle-Lite library is as follows:
| Platform | Paddle-Lite library download link |
| --- | --- |
|Android|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.android.armv7.gcc.c++_shared.with_extra.with_cv.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.android.armv8.gcc.c++_shared.with_extra.with_cv.tar.gz)|
|IOS|[arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.ios.armv7.with_cv.with_extra.with_log.tiny_publish.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.ios.armv8.with_cv.with_extra.with_log.tiny_publish.tar.gz)|
Note: 1. The above Paddle-Lite library is compiled from the Paddle-Lite 2.10 branch. For more information about Paddle-Lite 2.10, please refer to [link](https://github.com/PaddlePaddle/Paddle-Lite/releases/tag/v2.10).
**Note: It is recommended to use paddlelite>=2.10 version of the prediction library, other prediction library versions [download link](https://github.com/PaddlePaddle/Paddle-Lite/tags)**
2. Compile Paddle-Lite to get the prediction library. The compilation method of Paddle-Lite is as follows:
```bash linenums="1"
git clone https://github.com/PaddlePaddle/Paddle-Lite.git
cd Paddle-Lite
# Switch to Paddle-Lite release/v2.10 stable branch
git checkout release/v2.10
./lite/tools/build_android.sh --arch=armv8 --with_cv=ON --with_extra=ON
```
Note: When compiling Paddle-Lite to obtain the Paddle-Lite library, you need to turn on the two options `--with_cv=ON --with_extra=ON`, `--arch` means the `arm` version, here is designated as armv8,
More compilation commands refer to the introduction [link](https://paddle-lite.readthedocs.io/zh/release-v2.10_a/source_compile/linux_x86_compile_android.html) 。
After directly downloading the Paddle-Lite library and decompressing it, you can get the `inference_lite_lib.android.armv8/` folder, and the Paddle-Lite library obtained by compiling Paddle-Lite is located
`Paddle-Lite/build.lite.android.armv8.gcc/inference_lite_lib.android.armv8/` folder.
The structure of the prediction library is as follows:
```text linenums="1"
inference_lite_lib.android.armv8/
|-- cxx C++ prebuild library
| |-- include C++
| | |-- paddle_api.h
| | |-- paddle_image_preprocess.h
| | |-- paddle_lite_factory_helper.h
| | |-- paddle_place.h
| | |-- paddle_use_kernels.h
| | |-- paddle_use_ops.h
| | `-- paddle_use_passes.h
| `-- lib C++ library
| |-- libpaddle_api_light_bundled.a C++ static library
| `-- libpaddle_light_api_shared.so C++ dynamic library
|-- java Java library
| |-- jar
| | `-- PaddlePredictor.jar
| |-- so
| | `-- libpaddle_lite_jni.so
| `-- src
|-- demo C++ and Java demo
| |-- cxx C++ demo
| `-- java Java demo
```
## 2 Run
### 2.1 Inference Model Optimization
Paddle Lite provides a variety of strategies to automatically optimize the original training model, including quantization, sub-graph fusion, hybrid scheduling, Kernel optimization and so on. In order to make the optimization process more convenient and easy to use, Paddle Lite provide opt tools to automatically complete the optimization steps and output a lightweight, optimal executable model.
If you have prepared the model file ending in .nb, you can skip this step.
The following table also provides a series of models that can be deployed on mobile phones to recognize Chinese. You can directly download the optimized model.
|Version|Introduction|Model size|Detection model|Text Direction model|Recognition model|Paddle-Lite branch|
|---|---|---|---|---|---|---|
|PP-OCRv3|extra-lightweight chinese OCR optimized model|16.2M|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/PP-OCRv3_mobile_det_infer.nb)|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_ppocr_mobile_v2.0_cls_infer_opt.nb)|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/PP-OCRv3_mobile_rec_infer.nb)|v2.10|
|PP-OCRv3(slim)|extra-lightweight chinese OCR optimized model|5.9M|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_slim_infer.nb)|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_ppocr_mobile_v2.0_cls_slim_opt.nb)|[download link](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_slim_infer.nb)|v2.10|
If you directly use the model in the above table for deployment, you can skip the following steps and directly read [Section 2.2](#2.2-Run-optimized-model-on-Phone).
If the model to be deployed is not in the above table, you need to follow the steps below to obtain the optimized model.
Step 1: Refer to [document](https://www.paddlepaddle.org.cn/lite/v2.10/user_guides/opt/opt_python.html) to install paddlelite, which is used to convert paddle inference model to paddlelite required for running nb model
```bash linenums="1"
pip install paddlelite==2.10 # The paddlelite version should be the same as the prediction library version
```
After installation, the following commands can view the help information
```bash linenums="1"
paddle_lite_opt
```
Introduction to paddle_lite_opt parameters:
|Options|Description|
|---|---|
|--model_dir|The path of the PaddlePaddle model to be optimized (non-combined form)|
|--model_file|The network structure file path of the PaddlePaddle model (combined form) to be optimized|
|--param_file|The weight file path of the PaddlePaddle model (combined form) to be optimized|
|--optimize_out_type|Output model type, currently supports two types: protobuf and naive_buffer, among which naive_buffer is a more lightweight serialization/deserialization implementation. If you need to perform model prediction on the mobile side, please set this option to naive_buffer. The default is protobuf|
|--optimize_out|The output path of the optimized model|
|--valid_targets|The executable backend of the model, the default is arm. Currently it supports x86, arm, opencl, npu, xpu, multiple backends can be specified at the same time (separated by spaces), and Model Optimize Tool will automatically select the best method. If you need to support Huawei NPU (DaVinci architecture NPU equipped with Kirin 810/990 Soc), it should be set to npu, arm|
|--record_tailoring_info|When using the function of cutting library files according to the model, set this option to true to record the kernel and OP information contained in the optimized model. The default is false|
`--model_dir` is suitable for the non-combined mode of the model to be optimized, and the inference model of PaddleOCR is the combined mode, that is, the model structure and model parameters are stored in a single file.
Step 2: Use paddle_lite_opt to convert the inference model to the mobile model format.
The following takes the ultra-lightweight Chinese model of PaddleOCR as an example to introduce the use of the compiled opt file to complete the conversion of the inference model to the Paddle-Lite optimized model
```bash linenums="1"
# 【[Recommendation] Download the Chinese and English inference model of PP-OCRv3
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_slim_infer.tar && tar xf ch_PP-OCRv3_det_slim_infer.tar
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_slim_infer.tar && tar xf ch_PP-OCRv2_rec_slim_quant_infer.tar
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/slim/ch_ppocr_mobile_v2.0_cls_slim_infer.tar && tar xf ch_ppocr_mobile_v2.0_cls_slim_infer.tar
# Convert detection model
paddle_lite_opt --model_file=./ch_PP-OCRv3_det_slim_infer/inference.pdmodel --param_file=./ch_PP-OCRv3_det_slim_infer/inference.pdiparams --optimize_out=./ch_PP-OCRv3_det_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer
# Convert recognition model
paddle_lite_opt --model_file=./ch_PP-OCRv3_rec_slim_infer/inference.pdmodel --param_file=./ch_PP-OCRv3_rec_slim_infer/inference.pdiparams --optimize_out=./ch_PP-OCRv3_rec_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer
# Convert angle classifier model
paddle_lite_opt --model_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdmodel --param_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdiparams --optimize_out=./ch_ppocr_mobile_v2.0_cls_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer
```
After the conversion is successful, there will be more files ending with `.nb` in the inference model directory, which is the successfully converted model file.
### 2.2 Run optimized model on Phone
Some preparatory work is required first.
1. Prepare an Android phone with arm8. If the compiled prediction library and opt file are armv7, you need an arm7 phone and modify ARM_ABI = arm7 in the Makefile.
2. Make sure the phone is connected to the computer, open the USB debugging option of the phone, and select the file transfer mode.
3. Install the adb tool on the computer.
3.1. Install ADB for MAC:
```bash linenums="1"
brew cask install android-platform-tools
```
3.2. Install ADB for Linux
```bash linenums="1"
sudo apt update
sudo apt install -y wget adb
```
3.3. Install ADB for windows
To install on win, you need to go to Google's Android platform to download the adb package for installation:[link](https://developer.android.com/studio)
Verify whether adb is installed successfully
```bash linenums="1"
adb devices
```
If there is device output, it means the installation is successful。
```bash linenums="1"
List of devices attached
744be294 device
```
4. Prepare optimized models, prediction library files, test images and dictionary files used.
```bash linenums="1"
git clone https://github.com/PaddlePaddle/PaddleOCR.git
cd PaddleOCR/deploy/lite/
# run prepare.sh
sh prepare.sh /{lite prediction library path}/inference_lite_lib.android.armv8
#
cd /{lite prediction library path}/inference_lite_lib.android.armv8/
cd demo/cxx/ocr/
# copy paddle-lite C++ .so file to debug/ directory
cp ../../../cxx/lib/libpaddle_light_api_shared.so ./debug/
cd inference_lite_lib.android.armv8/demo/cxx/ocr/
cp ../../../cxx/lib/libpaddle_light_api_shared.so ./debug/
```
Prepare the test image, taking PaddleOCR/doc/imgs/11.jpg as an example, copy the image file to the demo/cxx/ocr/debug/ folder. Prepare the model files optimized by the lite opt tool, ch_PP-OCRv3_det_slim_opt.nb , ch_PP-OCRv3_rec_slim_opt.nb , and place them under the demo/cxx/ocr/debug/ folder.
The structure of the OCR demo is as follows after the above command is executed:
```text linenums="1"
demo/cxx/ocr/
|-- debug/
| |--ch_PP-OCRv3_det_slim_opt.nb Detection model
| |--ch_PP-OCRv3_rec_slim_opt.nb Recognition model
| |--ch_ppocr_mobile_v2.0_cls_slim_opt.nb Text direction classification model
| |--11.jpg Image for OCR
| |--ppocr_keys_v1.txt Dictionary file
| |--libpaddle_light_api_shared.so C++ .so file
| |--config.txt Config file
|-- config.txt Config file
|-- cls_process.cc Pre-processing and post-processing files for the angle classifier
|-- cls_process.h
|-- crnn_process.cc Pre-processing and post-processing files for the CRNN model
|-- crnn_process.h
|-- db_post_process.cc Pre-processing and post-processing files for the DB model
|-- db_post_process.h
|-- Makefile
|-- ocr_db_crnn.cc C++ main code
```
**Note**:
1. `ppocr_keys_v1.txt` is a Chinese dictionary file. If the nb model is used for English recognition or other language recognition, dictionary file should be replaced with a dictionary of the corresponding language. PaddleOCR provides a variety of dictionaries under ppocr/utils/, including:
```python linenums="1"
dict/french_dict.txt # french
dict/german_dict.txt # german
ic15_dict.txt # english
dict/japan_dict.txt # japan
dict/korean_dict.txt # korean
ppocr_keys_v1.txt # chinese
```
2. `config.txt` of the detector and classifier, as shown below:
```python linenums="1"
max_side_len 960 # Limit the maximum image height and width to 960
det_db_thresh 0.3 # Used to filter the binarized image of DB prediction, setting 0.-0.3 has no obvious effect on the result
det_db_box_thresh 0.5 # DDB post-processing filter box threshold, if there is a missing box detected, it can be reduced as appropriate
det_db_unclip_ratio 1.6 # Indicates the compactness of the text box, the smaller the value, the closer the text box to the text
use_direction_classify 0 # Whether to use the direction classifier, 0 means not to use, 1 means to use
rec_image_height 48 # The height of the input image of the recognition model, the PP-OCRv3 model needs to be set to 48, and the PP-OCRv2 model needs to be set to 32
```
3. Run Model on phone
After the above steps are completed, you can use adb to push the file to the phone to run, the steps are as follows:
```bash linenums="1"
# Execute the compilation and get the executable file ocr_db_crnn
# The first execution of this command will download dependent libraries such as opencv. After the download is complete, you need to execute it again
make -j
# Move the compiled executable file to the debug folder
mv ocr_db_crnn ./debug/
# Push the debug folder to the phone
adb push debug /data/local/tmp/
adb shell
cd /data/local/tmp/debug
export LD_LIBRARY_PATH=${PWD}:$LD_LIBRARY_PATH
# The use of ocr_db_crnn is:
# ./ocr_db_crnn Mode Detection model file Orientation classifier model file Recognition model file Hardware Precision Threads Batchsize Test image path Dictionary file path
./ocr_db_crnn system ch_PP-OCRv3_det_slim_opt.nb ch_PP-OCRv3_rec_slim_opt.nb ch_ppocr_mobile_v2.0_cls_slim_opt.nb arm8 INT8 10 1 ./11.jpg config.txt ppocr_keys_v1.txt True
# precision can be INT8 for quantitative model or FP32 for normal model.
# Only using detection model
./ocr_db_crnn det ch_PP-OCRv3_det_slim_opt.nb arm8 INT8 10 1 ./11.jpg config.txt
# Only using recognition model
./ocr_db_crnn rec ch_PP-OCRv3_rec_slim_opt.nb arm8 INT8 10 1 word_1.jpg ppocr_keys_v1.txt config.txt
```
If you modify the code, you need to recompile and push to the phone.
The outputs are as follows:
![img](./images/lite_demo.png)
## FAQ
Q1: What if I want to change the model, do I need to run it again according to the process?
A1: If you have performed the above steps, you only need to replace the .nb model file to complete the model replacement.
Q2: How to test with another picture?
A2: Replace the .jpg test image under ./debug with the image you want to test, and run adb push to push new image to the phone.
Q3: How to package it into the mobile APP?
A3: This demo aims to provide the core algorithm part that can run OCR on mobile phones. Further, PaddleOCR/deploy/android_demo is an example of encapsulating this demo into a mobile app for reference.
Q4: When running the demo, an error is reported `Error: This model is not supported, because kernel for 'io_copy' is not supported by Paddle-Lite.`
A4: The problem is that the installed paddlelite version does not match the downloaded prediction library version. Make sure that the paddleliteopt tool matches your prediction library version, and try to switch to the nb model again.

View File

@@ -0,0 +1,309 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# 端侧部署
本教程将介绍基于[Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite) 在移动端部署PaddleOCR超轻量中文检测、识别模型的详细步骤。
Paddle Lite是飞桨轻量化推理引擎为手机、IOT端提供高效推理能力并广泛整合跨平台硬件为端侧部署及应用落地问题提供轻量化的部署方案。
## 1. 准备环境
### 运行准备
- 电脑编译Paddle Lite
- 安卓手机armv7或armv8
### 1.1 准备交叉编译环境
交叉编译环境用于编译 Paddle Lite 和 PaddleOCR 的C++ demo。
支持多种开发环境,不同开发环境的编译流程请参考对应文档。
1. [Docker](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#docker)
2. [Linux](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#linux)
3. [MAC OS](https://paddle-lite.readthedocs.io/zh/latest/source_compile/compile_env.html#mac-os)
### 1.2 准备预测库
预测库有两种获取方式:
1. [推荐]直接下载,预测库下载链接如下:
| 平台 | 预测库下载链接 |
| ---- | ---- |
| Android | [arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.android.armv7.gcc.c++_shared.with_extra.with_cv.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.android.armv8.gcc.c++_shared.with_extra.with_cv.tar.gz) |
| IOS | [arm7](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.ios.armv7.with_cv.with_extra.with_log.tiny_publish.tar.gz) / [arm8](https://github.com/PaddlePaddle/Paddle-Lite/releases/download/v2.10/inference_lite_lib.ios.armv8.with_cv.with_extra.with_log.tiny_publish.tar.gz) |
1. 上述预测库为PaddleLite 2.10分支编译得到有关PaddleLite 2.10 详细信息可参考 [链接](https://github.com/PaddlePaddle/Paddle-Lite/releases/tag/v2.10) 。
**注建议使用paddlelite>=2.10版本的预测库,其他预测库版本[下载链接](https://github.com/PaddlePaddle/Paddle-Lite/tags)**
2. 编译Paddle-Lite得到预测库Paddle-Lite的编译方式如下
```bash linenums="1"
git clone https://github.com/PaddlePaddle/Paddle-Lite.git
cd Paddle-Lite
# 切换到Paddle-Lite release/v2.10 稳定分支
git checkout release/v2.10
./lite/tools/build_android.sh --arch=armv8 --with_cv=ON --with_extra=ON
```
注意编译Paddle-Lite获得预测库时需要打开`--with_cv=ON --with_extra=ON`两个选项,`--arch`表示`arm`版本这里指定为armv8
更多编译命令
介绍请参考 [链接](https://paddle-lite.readthedocs.io/zh/release-v2.10_a/source_compile/linux_x86_compile_android.html) 。
直接下载预测库并解压后,可以得到`inference_lite_lib.android.armv8/`文件夹通过编译Paddle-Lite得到的预测库位于
`Paddle-Lite/build.lite.android.armv8.gcc/inference_lite_lib.android.armv8/`文件夹下。
预测库的文件目录如下:
```text linenums="1"
inference_lite_lib.android.armv8/
|-- cxx C++ 预测库和头文件
| |-- include C++ 头文件
| | |-- paddle_api.h
| | |-- paddle_image_preprocess.h
| | |-- paddle_lite_factory_helper.h
| | |-- paddle_place.h
| | |-- paddle_use_kernels.h
| | |-- paddle_use_ops.h
| | `-- paddle_use_passes.h
| `-- lib C++预测库
| |-- libpaddle_api_light_bundled.a C++静态库
| `-- libpaddle_light_api_shared.so C++动态库
|-- java Java预测库
| |-- jar
| | `-- PaddlePredictor.jar
| |-- so
| | `-- libpaddle_lite_jni.so
| `-- src
|-- demo C++和Java示例代码
| |-- cxx C++ 预测库demo
| `-- java Java 预测库demo
```
## 2 开始运行
### 2.1 模型优化
Paddle-Lite 提供了多种策略来自动优化原始的模型其中包括量化、子图融合、混合调度、Kernel优选等方法使用Paddle-lite的opt工具可以自动
对inference模型进行优化优化后的模型更轻量模型运行速度更快。
如果已经准备好了 `.nb` 结尾的模型文件,可以跳过此步骤。
下述表格中也提供了一系列中文移动端模型:
| 模型版本 | 模型简介 | 模型大小 | 检测模型 | 文本方向分类模型 | 识别模型 | Paddle-Lite版本 |
| -------------- | ----------------------------- | -------- | ------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | --------------- |
| PP-OCRv3 | 蒸馏版超轻量中文OCR移动端模型 | 16.2M | [下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/PP-OCRv3_mobile_det_infer.nb) | [下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_ppocr_mobile_v2.0_cls_infer_opt.nb) | [下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_infer.nb) | v2.10 |
| PP-OCRv3(slim) | 蒸馏版超轻量中文OCR移动端模型 | 5.9M | [下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_slim_infer.nb) | [下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv2/lite/ch_ppocr_mobile_v2.0_cls_slim_opt.nb) | [下载地址](https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_slim_infer.nb) | v2.10 |
如果直接使用上述表格中的模型进行部署,可略过下述步骤,直接阅读 [2.2节](#2.2与手机联调)。
如果要部署的模型不在上述表格中,则需要按照如下步骤获得优化后的模型。
步骤1参考[文档](https://www.paddlepaddle.org.cn/lite/v2.10/user_guides/opt/opt_python.html)安装paddlelite用于转换paddle inference model为paddlelite运行所需的nb模型
```bash linenums="1"
pip install paddlelite==2.10 # paddlelite版本要与预测库版本一致
```
安装完后,如下指令可以查看帮助信息
```bash linenums="1"
paddle_lite_opt
```
paddle_lite_opt 参数介绍:
| 选项 | 说明 |
| ---| --- |
| --model_dir | 待优化的PaddlePaddle模型非combined形式的路径 |
| --model_file | 待优化的PaddlePaddle模型combined形式的网络结构文件路径 |
| --param_file | 待优化的PaddlePaddle模型combined形式的权重文件路径 |
| --optimize_out_type | 输出模型类型目前支持两种类型protobuf和naive_buffer其中naive_buffer是一种更轻量级的序列化/反序列化实现。若您需要在mobile端执行模型预测请将此选项设置为naive_buffer。默认为protobuf |
| --optimize_out | 优化模型的输出路径 |
| --valid_targets | 指定模型可执行的backend默认为arm。目前可支持x86、arm、opencl、npu、xpu可以同时指定多个backend(以空格分隔)Model Optimize Tool将会自动选择最佳方式。如果需要支持华为NPUKirin 810/990 Soc搭载的达芬奇架构NPU应当设置为npu, arm |
| --record_tailoring_info | 当使用 根据模型裁剪库文件 功能时则设置该选项为true以记录优化后模型含有的kernel和OP信息默认为false |
`--model_dir`适用于待优化的模型是非combined方式PaddleOCR的inference模型是combined方式即模型结构和模型参数使用单独一个文件存储。
步骤2使用paddle_lite_opt将inference模型转换成移动端模型格式。
下面以PaddleOCR的超轻量中文模型为例介绍使用编译好的opt文件完成inference模型到Paddle-Lite优化模型的转换。
```bash linenums="1"
# 【推荐】 下载 PP-OCRv3版本的中英文 inference模型
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_slim_infer.tar && tar xf ch_PP-OCRv3_det_slim_infer.tar
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_rec_slim_infer.tar && tar xf ch_PP-OCRv2_rec_slim_quant_infer.tar
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/slim/ch_ppocr_mobile_v2.0_cls_slim_infer.tar && tar xf ch_ppocr_mobile_v2.0_cls_slim_infer.tar
# 转换检测模型
paddle_lite_opt --model_file=./ch_PP-OCRv3_det_slim_infer/inference.pdmodel --param_file=./ch_PP-OCRv3_det_slim_infer/inference.pdiparams --optimize_out=./ch_PP-OCRv3_det_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer
# 转换识别模型
paddle_lite_opt --model_file=./ch_PP-OCRv3_rec_slim_infer/inference.pdmodel --param_file=./ch_PP-OCRv3_rec_slim_infer/inference.pdiparams --optimize_out=./ch_PP-OCRv3_rec_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer
# 转换方向分类器模型
paddle_lite_opt --model_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdmodel --param_file=./ch_ppocr_mobile_v2.0_cls_slim_infer/inference.pdiparams --optimize_out=./ch_ppocr_mobile_v2.0_cls_slim_opt --valid_targets=arm --optimize_out_type=naive_buffer
```
转换成功后inference模型目录下会多出`.nb`结尾的文件,即是转换成功的模型文件。
注意使用paddle-lite部署时需要使用opt工具优化后的模型。 opt工具的输入模型是paddle保存的inference模型
### 2.2 与手机联调
首先需要进行一些准备工作:
1. 准备一台arm8的安卓手机如果编译的预测库和opt文件是armv7则需要arm7的手机并修改Makefile中`ARM_ABI = arm7`。
2. 打开手机的USB调试选项选择文件传输模式连接电脑。
3. 电脑上安装adb工具用于调试。 adb安装方式如下
3.1. MAC电脑安装ADB:
```bash linenums="1"
brew cask install android-platform-tools
```
3.2. Linux安装ADB
```bash linenums="1"
sudo apt update
sudo apt install -y wget adb
```
3.3. Window安装ADB
win上安装需要去谷歌的安卓平台下载adb软件包进行安装[链接](https://developer.android.com/studio)
打开终端,手机连接电脑,在终端中输入
```bash linenums="1"
adb devices
```
如果有device输出则表示安装成功。
```bash linenums="1"
List of devices attached
744be294 device
```
4. 准备优化后的模型、预测库文件、测试图像和使用的字典文件。
```bash linenums="1"
git clone https://github.com/PaddlePaddle/PaddleOCR.git
cd PaddleOCR/deploy/lite/
# 运行prepare.sh准备预测库文件、测试图像和使用的字典文件并放置在预测库中的demo/cxx/ocr文件夹下
sh prepare.sh /{lite prediction library path}/inference_lite_lib.android.armv8
# 进入OCR demo的工作目录
cd /{lite prediction library path}/inference_lite_lib.android.armv8/
cd demo/cxx/ocr/
# 将C++预测动态库so文件复制到debug文件夹中
cp ../../../cxx/lib/libpaddle_light_api_shared.so ./debug/
```
准备测试图像,以`PaddleOCR/doc/imgs/11.jpg`为例,将测试的图像复制到`demo/cxx/ocr/debug/`文件夹下。
准备lite opt工具优化后的模型文件比如使用`ch_PP-OCRv3_det_slim_opt.ch_PP-OCRv3_rec_slim_rec.nb, ch_ppocr_mobile_v2.0_cls_slim_opt.nb`,模型文件放置在`demo/cxx/ocr/debug/`文件夹下。
执行完成后ocr文件夹下将有如下文件格式
```text linenums="1"
demo/cxx/ocr/
|-- debug/
| |--ch_PP-OCRv3_det_slim_opt.nb 优化后的检测模型文件
| |--ch_PP-OCRv3_rec_slim_opt.nb 优化后的识别模型文件
| |--ch_ppocr_mobile_v2.0_cls_slim_opt.nb 优化后的文字方向分类器模型文件
| |--11.jpg 待测试图像
| |--ppocr_keys_v1.txt 中文字典文件
| |--libpaddle_light_api_shared.so C++预测库文件
| |--config.txt 超参数配置
|-- config.txt 超参数配置
|-- cls_process.cc 方向分类器的预处理和后处理文件
|-- cls_process.h
|-- crnn_process.cc 识别模型CRNN的预处理和后处理文件
|-- crnn_process.h
|-- db_post_process.cc 检测模型DB的后处理文件
|-- db_post_process.h
|-- Makefile 编译文件
|-- ocr_db_crnn.cc C++预测源文件
```
#### 注意
1. ppocr_keys_v1.txt是中文字典文件如果使用的 nb 模型是英文数字或其他语言的模型需要更换为对应语言的字典。PaddleOCR 在ppocr/utils/下存放了多种字典,包括:
```text linenums="1"
dict/french_dict.txt # 法语字典
dict/german_dict.txt # 德语字典
ic15_dict.txt # 英文字典
dict/japan_dict.txt # 日语字典
dict/korean_dict.txt # 韩语字典
ppocr_keys_v1.txt # 中文字典
...
```
2. `config.txt` 包含了检测器、分类器、识别器的超参数,如下:
```python linenums="1"
max_side_len 960 # 输入图像长宽大于960时等比例缩放图像使得图像最长边 为960
det_db_thresh 0.3 # 用于过滤DB预测的二值化图像设置为0.-0.3对结果影响不 明显
det_db_box_thresh 0.5 # 检测器后处理过滤box的阈值如果检测存在漏框情况可酌 情减小
det_db_unclip_ratio 1.6 # 表示文本框的紧致程度,越小则文本框更靠近文本
use_direction_classify 0 # 是否使用方向分类器0表示不使用1表示使用
rec_image_height 48 # 识别模型输入图像的高度PP-OCRv3模型设置为48 PP-OCRv2模型需要设置为32
```
3. 启动调试
上述步骤完成后就可以使用adb将文件push到手机上运行步骤如下
```bash linenums="1"
# 执行编译得到可执行文件ocr_db_crnn, 第一次执行此命令会下载opencv等依赖库下载完成后需要再执行一次
make -j
# 将编译的可执行文件移动到debug文件夹中
mv ocr_db_crnn ./debug/
# 将debug文件夹push到手机上
adb push debug /data/local/tmp/
adb shell
cd /data/local/tmp/debug
export LD_LIBRARY_PATH=${PWD}:$LD_LIBRARY_PATH
# 开始使用ocr_db_crnn可执行文件的使用方式为:
# ./ocr_db_crnn 预测模式 检测模型文件 方向分类器模型文件 识别模型文件 运行硬件 运行精度 线程数 batchsize 测试图像路径 参数配置路径 字典文件路径 是否使用benchmark参数
./ocr_db_crnn system ch_PP-OCRv3_det_slim_opt.nb ch_PP-OCRv3_rec_slim_opt.nb ch_ppocr_mobile_v2.0_cls_slim_opt.nb arm8 INT8 10 1 ./11.jpg config.txt ppocr_keys_v1.txt True
# 仅使用文本检测模型,使用方式如下:
./ocr_db_crnn det ch_PP-OCRv3_det_slim_opt.nb arm8 INT8 10 1 ./11.jpg config.txt
# 仅使用文本识别模型,使用方式如下:
./ocr_db_crnn rec ch_PP-OCRv3_rec_slim_opt.nb arm8 INT8 10 1 word_1.jpg ppocr_keys_v1.txt config.txt
```
如果对代码做了修改则需要重新编译并push到手机上。
运行效果如下:
![img](./images/lite_demo.png)
## FAQ
Q1如果想更换模型怎么办需要重新按照流程走一遍吗
A1如果已经走通了上述步骤更换模型只需要替换 .nb 模型文件即可,同时要注意更新字典
Q2换一个图测试怎么做
A2替换debug下的.jpg测试图像为你想要测试的图像adb push 到手机上即可
Q3如何封装到手机APP中
A3此demo旨在提供能在手机上运行OCR的核心算法部分PaddleOCR/deploy/android_demo是将这个demo封装到手机app的示例供参考
Q4运行demo时遇到报错`Error: This model is not supported, because kernel for 'io_copy' is not supported by Paddle-Lite.`
A4问题是安装的paddlelite版本和下载的预测库版本不匹配确保paddleliteopt工具和你的预测库版本匹配重新转nb模型试试。

View File

@@ -0,0 +1,106 @@
# Models Supported by PaddleOCR 2.x and Earlier Versions
Due to differences in the concatenation logic and configurations used during model training and inference, the PP-OCRv4 and PP-OCRv3 series models from the PaddleOCR 2.x branch cannot be used interchangeably with those from the PaddleOCR 3.0 and later branches.
## Detection Models
### Chinese Detection Models
* ch_PP-OCRv4_det
* ch_PP-OCRv4_server_det
* ch_PP-OCRv3_det_slim
* ch_PP-OCRv3_det
* ch_PP-OCRv2_det_slim
* ch_PP-OCRv2_det
* ch_ppocr_mobile_slim_v2.0_det
* ch_ppocr_mobile_v2.0_det
* ch_ppocr_server_v2.0_det
### English Detection Models
* en_PP-OCRv3_det_slim
* en_PP-OCRv3_det
### Multilingual Detection Models
* ml_PP-OCRv3_det_slim
* ml_PP-OCRv3_det
## Recognition Models
### Chinese Recognition Models
* ch_PP-OCRv4_rec
* ch_PP-OCRv4_server_rec
* ch_PP-OCRv4_server_rec_doc
* ch_PP-OCRv3_rec_slim
* ch_PP-OCRv3_rec
* ch_PP-OCRv2_rec_slim
* ch_PP-OCRv2_rec
* ch_ppocr_mobile_slim_v2.0_rec
* ch_ppocr_mobile_v2.0_rec
* ch_ppocr_server_v2.0_rec
* SVTRv2(Rec Sever)
* RepSVTR(Mobile)
### English Recognition Models
* en_PP-OCRv4_rec
* en_PP-OCRv3_rec_slim
* en_PP-OCRv3_rec
* en_number_mobile_slim_v2.0_rec
* en_number_mobile_v2.0_rec
### Multilingual Recognition Models
* korean_PP-OCRv3_rec
* japan_PP-OCRv3_rec
* chinese_cht_PP-OCRv3_rec
* te_PP-OCRv3_rec
* ka_PP-OCRv3_rec
* ta_PP-OCRv3_rec
* latin_PP-OCRv3_rec
* arabic_PP-OCRv3_rec
* cyrillic_PP-OCRv3_rec
* devanagari_PP-OCRv3_rec
## End-to-End OCR Models
* PGNet
## Text Direction Classification Models
* ch_ppocr_mobile_slim_v2.0_cls
* ch_ppocr_mobile_v2.0_cls
## Formula Recognition Models
* CAN
* UniMERNet
* LaTeX-OCR
* PP-FormulaNet-S
* PP-FormulaNet-L
## Table Recognition Models
* TableMaster
* SLANet
* SLANeXt_wired
* SLANeXt_wireless
* en_ppocr_mobile_v2.0_table_structure
* en_ppstructure_mobile_v2.0_SLANet
* ch_ppstructure_mobile_v2.0_SLANet
## Table OCR Models
* en_ppocr_mobile_v2.0_table_det
* en_ppocr_mobile_v2.0_table_rec
## Layout Detection Models
* picodet_lcnet_x1_0_fgd_layout
* ppyolov2_r50vd_dcn_365e_publaynet
* picodet_lcnet_x1_0_fgd_layout_cdla
* picodet_lcnet_x1_0_fgd_layout_table
* ppyolov2_r50vd_dcn_365e_tableBank_word
* ppyolov2_r50vd_dcn_365e_tableBank_latex

View File

@@ -0,0 +1,106 @@
# PaddleOCR 2.x及更低版本支持的模型
由于串联逻辑和模型训练、推理时用到的配置不同PaddleOCR 2.x 分支的 PP-OCRv4、PP-OCRv3 系列模型与 PaddleOCR 3.0 及以上分支的 PP-OCRv4、PP-OCRv3 系列模型不能互相使用。
## 检测模型
### 中文检测模型
* ch_PP-OCRv4_det
* ch_PP-OCRv4_server_det
* ch_PP-OCRv3_det_slim
* ch_PP-OCRv3_det
* ch_PP-OCRv2_det_slim
* ch_PP-OCRv2_det
* ch_ppocr_mobile_slim_v2.0_det
* ch_ppocr_mobile_v2.0_det
* ch_ppocr_server_v2.0_det
### 英文检测模型
* en_PP-OCRv3_det_slim
* en_PP-OCRv3_det
### 多语言检测模型
* ml_PP-OCRv3_det_slim
* ml_PP-OCRv3_det
## 识别模型
### 中文识别模型
* ch_PP-OCRv4_rec
* ch_PP-OCRv4_server_rec
* ch_PP-OCRv4_server_rec_doc
* ch_PP-OCRv3_rec_slim
* ch_PP-OCRv3_rec
* ch_PP-OCRv2_rec_slim
* ch_PP-OCRv2_rec
* ch_ppocr_mobile_slim_v2.0_rec
* ch_ppocr_mobile_v2.0_rec
* ch_ppocr_server_v2.0_rec
* SVTRv2(Rec Sever)
* RepSVTR(Mobile)
### 英文识别模型
* en_PP-OCRv4_rec
* en_PP-OCRv3_rec_slim
* en_PP-OCRv3_rec
* en_number_mobile_slim_v2.0_rec
* en_number_mobile_v2.0_rec
### 多语言识别模型
* korean_PP-OCRv3_rec
* japan_PP-OCRv3_rec
* chinese_cht_PP-OCRv3_rec
* te_PP-OCRv3_rec
* ka_PP-OCRv3_rec
* ta_PP-OCRv3_rec
* latin_PP-OCRv3_rec
* arabic_PP-OCRv3_rec
* cyrillic_PP-OCRv3_rec
* devanagari_PP-OCRv3_rec
## 端到端OCR模型
* PGNet
## 文本方向分类模型
* ch_ppocr_mobile_slim_v2.0_cls
* ch_ppocr_mobile_v2.0_cls
## 公式识别模型
* CAN
* UniMERNet
* LaTeX-OCR
* PP-FormulaNet-S
* PP-FormulaNet-L
## 表格识别模型
* TableMaster
* SLANet
* SLANeXt_wired
* SLANeXt_wireless
* en_ppocr_mobile_v2.0_table_structure
* en_ppstructure_mobile_v2.0_SLANet
* ch_ppstructure_mobile_v2.0_SLANet
## 表格OCR模型
* en_ppocr_mobile_v2.0_table_det
* en_ppocr_mobile_v2.0_table_rec
## 版面检测模型
* picodet_lcnet_x1_0_fgd_layout
* ppyolov2_r50vd_dcn_365e_publaynet
* picodet_lcnet_x1_0_fgd_layout_cdla
* picodet_lcnet_x1_0_fgd_layout_table
* ppyolov2_r50vd_dcn_365e_tableBank_word
* ppyolov2_r50vd_dcn_365e_tableBank_latex

View File

@@ -0,0 +1,267 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# Paddle2ONNX Model Conversion and Prediction
This chapter introduces how to convert PaddleOCR models to ONNX models and perform predictions based on the ONNXRuntime engine. We also prepared an [online AI Studio project](https://aistudio.baidu.com/projectdetail/8808858) for your convenience.
## 1. Environment Setup
You need to prepare the environments for PaddleOCR, Paddle2ONNX model conversion, and ONNXRuntime prediction.
### PaddleOCR
Clone the PaddleOCR repository, use the main branch, and install it. Since the PaddleOCR repository is relatively large and cloning via `git clone` can be slow, this tutorial has already downloaded it.
```bash linenums="1"
git clone -b main https://github.com/PaddlePaddle/PaddleOCR.git
cd PaddleOCR && python3 -m pip install -e .
```
### Paddle2ONNX
Paddle2ONNX supports converting models in the PaddlePaddle format to the ONNX format. Operators currently stably support exporting ONNX Opset versions 7~19, and some Paddle operators support conversion to lower ONNX Opsets. For more details, please refer to [Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX.git).
- Install Paddle2ONNX
```bash linenums="1"
python3 -m pip install paddle2onnx
```
- Install ONNXRuntime
```bash linenums="1"
python3 -m pip install onnxruntime
```
## 2. Model Conversion
### Get Paddle Static Graph Models
There are two ways to obtain Paddle static graph models: download the prediction models provided by PaddleOCR in the [model list](../ppocr/model_list.en.md); or refer to the [Model Export Instructions](./python_infer.en.md#inference) to convert trained weights into inference models.
Using the PP-OCR series English detection, recognition, and classification models as examples:
#### Download Provided Inference Models
=== "PP-OCRv3"
```bash linenums="1"
wget -nc -P ./inference https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_det_infer.tar
cd ./inference && tar xf en_PP-OCRv3_det_infer.tar && cd ..
wget -nc -P ./inference https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/en_PP-OCRv3_mobile_rec_infer.tar
cd ./inference && tar xf en_PP-OCRv3_mobile_rec_infer.tar && cd ..
wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
cd ./inference && tar xf ch_ppocr_mobile_v2.0_cls_infer.tar && cd ..
```
=== "PP-OCRv4"
```bash linenums="1"
wget -nc -P ./inference https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_det_infer.tar
cd ./inference && tar xf en_PP-OCRv3_det_infer.tar && cd ..
wget -nc -P ./inference https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/en_PP-OCRv4_mobile_rec_infer.tar
cd ./inference && tar xf en_PP-OCRv4_mobile_rec_infer.tar && cd ..
wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
cd ./inference && tar xf ch_ppocr_mobile_v2.0_cls_infer.tar && cd ..
```
#### Convert Dynamic Graph to Static Graph (Optional)
Download dynamic graph models:
```bash linenums="1"
wget -nc -P pretrained https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_det_pretrained.pdparams
wget -nc -P pretrained https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_rec_train.tar
cd pretrained && tar xf ch_PP-OCRv4_rec_train.tar && cd ..
wget -nc -P pretrained https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar
cd pretrained && tar xf ch_ppocr_mobile_v2.0_cls_train.tar && cd ..
```
Convert to static graph models:
```bash linenums="1"
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_mobile_det.yml \
-o Global.pretrained_model=./pretrained/PP-OCRv4_mobile_det_pretrained \
Global.save_inference_dir=./inference/PP-OCRv4_mobile_det_infer/
python3 tools/export_model.py -c configs/rec/PP-OCRv4/PP-OCRv4_mobile_rec.yml \
-o Global.pretrained_model=./pretrained/ch_PP-OCRv4_rec_train/student \
Global.save_inference_dir=./inference/PP-OCRv4_mobile_rec_infer/
python3 tools/export_model.py -c configs/cls/cls_mv3.yml \
-o Global.pretrained_model=./pretrained/ch_ppocr_mobile_v2.0_cls_train/best_accuracy \
Global.save_inference_dir=./inference/ch_ppocr_mobile_v2.0_cls_infer/
```
### Model Conversion
Use Paddle2ONNX to convert Paddle static graph models to the ONNX model format:
=== "PP-OCRv3"
```bash linenums="1"
paddle2onnx --model_dir ./inference/en_PP-OCRv3_det_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/det_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
paddle2onnx --model_dir ./inference/en_PP-OCRv3_mobile_rec_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/rec_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
paddle2onnx --model_dir ./inference/ch_ppocr_mobile_v2.0_cls_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/cls_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
```
=== "PP-OCRv4"
```bash linenums="1"
paddle2onnx --model_dir ./inference/en_PP-OCRv3_det_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/det_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
paddle2onnx --model_dir ./inference/en_PP-OCRv4_mobile_rec_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/rec_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
paddle2onnx --model_dir ./inference/ch_ppocr_mobile_v2.0_cls_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/cls_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
```
After execution, the ONNX models will be saved respectively under `./inference/det_onnx/`, `./inference/rec_onnx/`, and `./inference/cls_onnx/`.
- **Note**: For OCR models, dynamic shapes must be used during conversion; otherwise, the prediction results may slightly differ from directly using Paddle for prediction. Additionally, the following models currently do not support conversion to ONNX models: NRTR, SAR, RARE, SRN.
- **Note**: After [Paddle2ONNX version v1.2.3](https://github.com/PaddlePaddle/Paddle2ONNX/releases/tag/v1.2.3), dynamic shapes are supported by default, i.e., `float32[p2o.DynamicDimension.0,3,p2o.DynamicDimension.1,p2o.DynamicDimension.2]`. The option `--input_shape_dict` has been deprecated. If you need to adjust shapes, you can use the following command to adjust the input shape of the Paddle model.
```bash linenums="1"
python3 -m paddle2onnx.optimize --input_model inference/det_onnx/model.onnx \
--output_model inference/det_onnx/model.onnx \
--input_shape_dict "{'x': [-1,3,-1,-1]}"
```
If you have optimization requirements for the exported ONNX model, it is recommended to use [OnnxSlim](https://github.com/inisis/OnnxSlim) to optimize the model:
```bash linenums="1"
pip install onnxslim
onnxslim model.onnx slim.onnx
```
## 3. Inference and Prediction
Using the Chinese OCR model as an example, you can perform prediction using ONNXRuntime by executing the following command:
```bash linenums="1"
python3 tools/infer/predict_system.py --use_gpu=False --use_onnx=True \
--det_model_dir=./inference/det_onnx/model.onnx \
--rec_model_dir=./inference/rec_onnx/model.onnx \
--cls_model_dir=./inference/cls_onnx/model.onnx \
--image_dir=./docs/infer_deploy/images/img_12.jpg \
--rec_char_dict_path=ppocr/utils/en_dict.txt
```
Taking the English OCR model as an example, you can perform prediction using Paddle Inference by executing the following command:
=== "PP-OCRv3"
```bash linenums="1"
python3 tools/infer/predict_system.py --use_gpu=False \
--cls_model_dir=./inference/ch_ppocr_mobile_v2.0_cls_infer \
--rec_model_dir=./inference/en_PP-OCRv3_mobile_rec_infer \
--det_model_dir=./inference/en_PP-OCRv3_det_infer \
--image_dir=./docs/infer_deploy/images/img_12.jpg\
--rec_char_dict_path=ppocr/utils/en_dict.txt
```
=== "PP-OCRv4"
```bash linenums="1"
python3 tools/infer/predict_system.py --use_gpu=False \
--cls_model_dir=./inference/ch_ppocr_mobile_v2.0_cls_infer \
--rec_model_dir=./inference/en_PP-OCRv4_mobile_rec_infer \
--det_model_dir=./inference/en_PP-OCRv3_det_infer \
--image_dir=./docs/infer_deploy/images/img_12.jpg \
--rec_char_dict_path=ppocr/utils/en_dict.txt
```
After executing the command, the terminal will print out the predicted recognition information, and the visualization results will be saved under `./inference_results/`.
**ONNXRuntime Execution Result:**
![](./images/img_12_result.jpg)
**Paddle Inference Execution Result:**
![](./images/img_12_result.jpg)
Using ONNXRuntime for prediction, terminal output:
```bash linenums="1"
[2022/10/10 12:06:28] ppocr DEBUG: dt_boxes num : 11, elapse : 0.3568880558013916
[2022/10/10 12:06:31] ppocr DEBUG: rec_res num : 11, elapse : 2.6445000171661377
[2022/10/10 12:06:31] ppocr DEBUG: 0 Predict time of doc/imgs_en/img_12.jpg: 3.021s
[2022/10/10 12:06:31] ppocr DEBUG: ACKNOWLEDGEMENTS, 0.997
[2022/10/10 12:06:31] ppocr DEBUG: We would like to thank all the designers and, 0.976
[2022/10/10 12:06:31] ppocr DEBUG: contributors who have been involved in the, 0.979
[2022/10/10 12:06:31] ppocr DEBUG: production of this book; their contributions, 0.989
[2022/10/10 12:06:31] ppocr DEBUG: have been indispensable to its creation. We, 0.956
[2022/10/10 12:06:31] ppocr DEBUG: would also like to express our gratitude to all, 0.991
[2022/10/10 12:06:31] ppocr DEBUG: the producers for their invaluable opinions, 0.978
[2022/10/10 12:06:31] ppocr DEBUG: and assistance throughout this project. And to, 0.988
[2022/10/10 12:06:31] ppocr DEBUG: the many others whose names are not credited, 0.958
[2022/10/10 12:06:31] ppocr DEBUG: but have made specific input in this book, we, 0.970
[2022/10/10 12:06:31] ppocr DEBUG: thank you for your continuous support., 0.998
[2022/10/10 12:06:31] ppocr DEBUG: The visualized image saved in ./inference_results/img_12.jpg
[2022/10/10 12:06:31] ppocr INFO: The predict total time is 3.2482550144195557
```
Using Paddle Inference for prediction, terminal output:
```bash linenums="1"
[2022/10/10 12:06:28] ppocr DEBUG: dt_boxes num : 11, elapse : 0.3568880558013916
[2022/10/10 12:06:31] ppocr DEBUG: rec_res num : 11, elapse : 2.6445000171661377
[2022/10/10 12:06:31] ppocr DEBUG: 0 Predict time of doc/imgs_en/img_12.jpg: 3.021s
[2022/10/10 12:06:31] ppocr DEBUG: ACKNOWLEDGEMENTS, 0.997
[2022/10/10 12:06:31] ppocr DEBUG: We would like to thank all the designers and, 0.976
[2022/10/10 12:06:31] ppocr DEBUG: contributors who have been involved in the, 0.979
[2022/10/10 12:06:31] ppocr DEBUG: production of this book; their contributions, 0.989
[2022/10/10 12:06:31] ppocr DEBUG: have been indispensable to its creation. We, 0.956
[2022/10/10 12:06:31] ppocr DEBUG: would also like to express our gratitude to all, 0.991
[2022/10/10 12:06:31] ppocr DEBUG: the producers for their invaluable opinions, 0.978
[2022/10/10 12:06:31] ppocr DEBUG: and assistance throughout this project. And to, 0.988
[2022/10/10 12:06:31] ppocr DEBUG: the many others whose names are not credited, 0.958
[2022/10/10 12:06:31] ppocr DEBUG: but have made specific input in this book, we, 0.970
[2022/10/10 12:06:31] ppocr DEBUG: thank you for your continuous support., 0.998
[2022/10/10 12:06:31] ppocr DEBUG: The visualized image saved in ./inference_results/img_12.jpg
[2022/10/10 12:06:31] ppocr INFO: The predict total time is 3.2482550144195557
```

View File

@@ -0,0 +1,315 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# Paddle2ONNX模型转化与预测
本章节介绍 PaddleOCR 模型如何转化为 ONNX 模型,并基于 ONNXRuntime 引擎预测。同时我们准备了一个[在线 AI Studio 项目](https://aistudio.baidu.com/projectdetail/8808858),可以方便用户进行测试。
## 1. 环境准备
需要准备 PaddleOCR、Paddle2ONNX 模型转化环境,和 ONNXRuntime 预测环境。
### PaddleOCR
克隆PaddleOCR的仓库使用 main 分支,并进行安装,由于 PaddleOCR 仓库比较大git clone 速度比较慢,所以本教程已下载。
```bash linenums="1"
git clone -b main https://github.com/PaddlePaddle/PaddleOCR.git
cd PaddleOCR && python3 -m pip install -e .
```
### Paddle2ONNX
Paddle2ONNX 支持将 PaddlePaddle 模型格式转化到 ONNX 模型格式,算子目前稳定支持导出 ONNX Opset 7~19部分Paddle算子支持更低的ONNX Opset转换。
更多细节可参考 [Paddle2ONNX](https://github.com/PaddlePaddle/Paddle2ONNX.git)
- 安装 Paddle2ONNX
```bash linenums="1"
python3 -m pip install paddle2onnx
```
- 安装 ONNXRuntime
```bash linenums="1"
python3 -m pip install onnxruntime
```
## 2. 模型转换
### 获取 Paddle 静态图模型
有两种方式获取Paddle静态图模型在 [model_list](../ppocr/model_list.md) 中下载PaddleOCR提供的预测模型参考[模型导出说明](./python_infer.md#inference)把训练好的权重转为推理模型。
#### 下载准备好的静态图模型
以 PP-OCR 系列中文检测、识别、分类模型为例:
=== "PP-OCRv3"
```bash linenums="1"
wget -nc -P ./inference https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv3_mobile_det_infer.tar
cd ./inference && tar xf PP-OCRv3_mobile_det_infer.tar && cd ..
wget -nc -P ./inference https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv3_mobile_rec_infer.tar
cd ./inference && tar xf PP-OCRv3_mobile_rec_infer.tar && cd ..
wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
cd ./inference && tar xf ch_ppocr_mobile_v2.0_cls_infer.tar && cd ..
```
=== "PP-OCRv4"
```bash linenums="1"
wget -nc -P ./inference https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_det_infer.tar
cd ./inference && tar xf PP-OCRv4_mobile_det_infer.tar && cd ..
wget -nc -P ./inference https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar
cd ./inference && tar xf PP-OCRv4_mobile_rec_infer.tar && cd ..
wget -nc -P ./inference https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
cd ./inference && tar xf ch_ppocr_mobile_v2.0_cls_infer.tar && cd ..
```
#### 将动态图导出为静态图模型 (可选)
下载动态图模型:
```bash linenums="1"
wget -nc -P pretrained https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_det_pretrained.pdparams
wget -nc -P pretrained https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_rec_train.tar
cd pretrained && tar xf ch_PP-OCRv4_rec_train.tar && cd ..
wget -nc -P pretrained https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar
cd pretrained && tar xf ch_ppocr_mobile_v2.0_cls_train.tar && cd ..
```
转换为静态图模型:
```bash linenums="1"
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_mobile_det.yml \
-o Global.pretrained_model=./pretrained/PP-OCRv4_mobile_det_pretrained \
Global.save_inference_dir=./inference/PP-OCRv4_mobile_det_infer/
python3 tools/export_model.py -c configs/rec/PP-OCRv4/PP-OCRv4_mobile_rec.yml \
-o Global.pretrained_model=./pretrained/ch_PP-OCRv4_rec_train/student \
Global.save_inference_dir=./inference/PP-OCRv4_mobile_rec_infer/
python3 tools/export_model.py -c configs/cls/cls_mv3.yml \
-o Global.pretrained_model=./pretrained/ch_ppocr_mobile_v2.0_cls_train/best_accuracy \
Global.save_inference_dir=./inference/ch_ppocr_mobile_v2.0_cls_infer/
```
### 模型转换
使用 Paddle2ONNX 将Paddle静态图模型转换为ONNX模型格式
=== "PP-OCRv3"
```bash linenums="1"
paddle2onnx --model_dir ./inference/PP-OCRv3_mobile_det_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/det_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
paddle2onnx --model_dir ./inference/PP-OCRv3_mobile_rec_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/rec_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
paddle2onnx --model_dir ./inference/ch_ppocr_mobile_v2.0_cls_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/cls_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
```
=== "PP-OCRv4"
```bash linenums="1"
paddle2onnx --model_dir ./inference/PP-OCRv4_mobile_det_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/det_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
paddle2onnx --model_dir ./inference/PP-OCRv4_mobile_rec_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/rec_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
paddle2onnx --model_dir ./inference/ch_ppocr_mobile_v2.0_cls_infer \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--save_file ./inference/cls_onnx/model.onnx \
--opset_version 11 \
--enable_onnx_checker True
```
执行完毕后ONNX 模型会被分别保存在 `./inference/det_onnx/``./inference/rec_onnx/``./inference/cls_onnx/`路径下
- 注意对于OCR模型转化过程中必须采用动态shape的形式否则预测结果可能与直接使用Paddle预测有细微不同。
另外,以下几个模型暂不支持转换为 ONNX 模型:
NRTR、SAR、RARE、SRN
- 注意:[Paddle2ONNX 版本 v1.2.3](https://github.com/PaddlePaddle/Paddle2ONNX/releases/tag/v1.2.3)后已默认支持动态shape即 `float32[p2o.DynamicDimension.0,3,p2o.DynamicDimension.1,p2o.DynamicDimension.2]`,选项 `--input_shape_dict` 已废弃。如果有shape调整需求可使用如下命令进行Paddle模型输入shape调整。
```bash linenums="1"
python3 -m paddle2onnx.optimize --input_model inference/det_onnx/model.onnx \
--output_model inference/det_onnx/model.onnx \
--input_shape_dict "{'x': [-1,3,-1,-1]}"
```
如你对导出的 ONNX 模型有优化的需求,推荐使用 [onnxslim](https://github.com/inisis/OnnxSlim) 对模型进行优化:
```bash linenums="1"
pip install onnxslim
onnxslim model.onnx slim.onnx
```
## 3. 推理预测
以中文OCR模型为例使用 ONNXRuntime 预测可执行如下命令:
```bash linenums="1"
python3 tools/infer/predict_system.py --use_gpu=False --use_onnx=True \
--det_model_dir=./inference/det_onnx/model.onnx \
--rec_model_dir=./inference/rec_onnx/model.onnx \
--cls_model_dir=./inference/cls_onnx/model.onnx \
--image_dir=./docs/infer_deploy/images/lite_demo.png
```
以中文OCR模型为例使用 Paddle Inference 预测可执行如下命令:
=== "PP-OCRv3"
```bash linenums="1"
python3 tools/infer/predict_system.py --use_gpu=False \
--cls_model_dir=./inference/ch_ppocr_mobile_v2.0_cls_infer \
--rec_model_dir=./inference/PP-OCRv3_mobile_rec_infer \
--det_model_dir=./inference/PP-OCRv3_mobile_det_infer \
--image_dir=./docs/infer_deploy/images/lite_demo.png
```
=== "PP-OCRv4"
```bash linenums="1"
python3 tools/infer/predict_system.py --use_gpu=False \
--cls_model_dir=./inference/ch_ppocr_mobile_v2.0_cls_infer \
--rec_model_dir=./inference/PP-OCRv4_mobile_rec_infer \
--det_model_dir=./inference/PP-OCRv4_mobile_det_infer \
--image_dir=./docs/infer_deploy/images/lite_demo.png
```
执行命令后在终端会打印出预测的识别信息,并在 `./inference_results/` 下保存可视化结果。
ONNXRuntime 执行效果:
![](./images/lite_demo_onnx.png)
Paddle Inference 执行效果:
![](./images/lite_demo_paddle.png)
使用 ONNXRuntime 预测,终端输出:
```bash linenums="1"
[2022/02/22 17:48:27] root DEBUG: dt_boxes num : 38, elapse : 0.043187856674194336
[2022/02/22 17:48:27] root DEBUG: rec_res num : 38, elapse : 0.592170000076294
[2022/02/22 17:48:27] root DEBUG: 0 Predict time of ./deploy/lite/imgs/lite_demo.png: 0.642s
[2022/02/22 17:48:27] root DEBUG: The, 0.984
[2022/02/22 17:48:27] root DEBUG: visualized, 0.882
[2022/02/22 17:48:27] root DEBUG: etect18片, 0.720
[2022/02/22 17:48:27] root DEBUG: image saved in./vis.jpg, 0.947
[2022/02/22 17:48:27] root DEBUG: 纯臻营养护发素0.993604, 0.996
[2022/02/22 17:48:27] root DEBUG: 产品信息/参数, 0.922
[2022/02/22 17:48:27] root DEBUG: 0.992728, 0.914
[2022/02/22 17:48:27] root DEBUG: 45元每公斤100公斤起订, 0.926
[2022/02/22 17:48:27] root DEBUG: 0.97417, 0.977
[2022/02/22 17:48:27] root DEBUG: 每瓶22元1000瓶起订0.993976, 0.962
[2022/02/22 17:48:27] root DEBUG: 【品牌】:代加工方式/0EMODM, 0.945
[2022/02/22 17:48:27] root DEBUG: 0.985133, 0.980
[2022/02/22 17:48:27] root DEBUG: 【品名】:纯臻营养护发素, 0.921
[2022/02/22 17:48:27] root DEBUG: 0.995007, 0.883
[2022/02/22 17:48:27] root DEBUG: 【产品编号】YM-X-30110.96899, 0.955
[2022/02/22 17:48:27] root DEBUG: 【净含量】220ml, 0.943
[2022/02/22 17:48:27] root DEBUG: Q.996577, 0.932
[2022/02/22 17:48:27] root DEBUG: 【适用人群】:适合所有肤质, 0.913
[2022/02/22 17:48:27] root DEBUG: 0.995842, 0.969
[2022/02/22 17:48:27] root DEBUG: 【主要成分】鲸蜡硬脂醇、燕麦B-葡聚, 0.883
[2022/02/22 17:48:27] root DEBUG: 0.961928, 0.964
[2022/02/22 17:48:27] root DEBUG: 10, 0.812
[2022/02/22 17:48:27] root DEBUG: 糖、椰油酰胺丙基甜菜碱、泛醒, 0.866
[2022/02/22 17:48:27] root DEBUG: 0.925898, 0.943
[2022/02/22 17:48:27] root DEBUG: (成品包材), 0.974
[2022/02/22 17:48:27] root DEBUG: 0.972573, 0.961
[2022/02/22 17:48:27] root DEBUG: 【主要功能】:可紧致头发磷层,从而达到, 0.936
[2022/02/22 17:48:27] root DEBUG: 0.994448, 0.952
[2022/02/22 17:48:27] root DEBUG: 13, 0.998
[2022/02/22 17:48:27] root DEBUG: 即时持久改善头发光泽的效果,给干燥的头, 0.994
[2022/02/22 17:48:27] root DEBUG: 0.990198, 0.975
[2022/02/22 17:48:27] root DEBUG: 14, 0.977
[2022/02/22 17:48:27] root DEBUG: 发足够的滋养, 0.991
[2022/02/22 17:48:27] root DEBUG: 0.997668, 0.918
[2022/02/22 17:48:27] root DEBUG: 花费了0.457335秒, 0.901
[2022/02/22 17:48:27] root DEBUG: The visualized image saved in ./inference_results/lite_demo.png
[2022/02/22 17:48:27] root INFO: The predict total time is 0.7003889083862305
```
使用 Paddle Inference 预测,终端输出:
```bash linenums="1"
[2022/02/22 17:47:25] root DEBUG: dt_boxes num : 38, elapse : 0.11791276931762695
[2022/02/22 17:47:27] root DEBUG: rec_res num : 38, elapse : 2.6206860542297363
[2022/02/22 17:47:27] root DEBUG: 0 Predict time of ./deploy/lite/imgs/lite_demo.png: 2.746s
[2022/02/22 17:47:27] root DEBUG: The, 0.984
[2022/02/22 17:47:27] root DEBUG: visualized, 0.882
[2022/02/22 17:47:27] root DEBUG: etect18片, 0.720
[2022/02/22 17:47:27] root DEBUG: image saved in./vis.jpg, 0.947
[2022/02/22 17:47:27] root DEBUG: 纯臻营养护发素0.993604, 0.996
[2022/02/22 17:47:27] root DEBUG: 产品信息/参数, 0.922
[2022/02/22 17:47:27] root DEBUG: 0.992728, 0.914
[2022/02/22 17:47:27] root DEBUG: 45元每公斤100公斤起订, 0.926
[2022/02/22 17:47:27] root DEBUG: 0.97417, 0.977
[2022/02/22 17:47:27] root DEBUG: 每瓶22元1000瓶起订0.993976, 0.962
[2022/02/22 17:47:27] root DEBUG: 【品牌】:代加工方式/0EMODM, 0.945
[2022/02/22 17:47:27] root DEBUG: 0.985133, 0.980
[2022/02/22 17:47:27] root DEBUG: 【品名】:纯臻营养护发素, 0.921
[2022/02/22 17:47:27] root DEBUG: 0.995007, 0.883
[2022/02/22 17:47:27] root DEBUG: 【产品编号】YM-X-30110.96899, 0.955
[2022/02/22 17:47:27] root DEBUG: 【净含量】220ml, 0.943
[2022/02/22 17:47:27] root DEBUG: Q.996577, 0.932
[2022/02/22 17:47:27] root DEBUG: 【适用人群】:适合所有肤质, 0.913
[2022/02/22 17:47:27] root DEBUG: 0.995842, 0.969
[2022/02/22 17:47:27] root DEBUG: 【主要成分】鲸蜡硬脂醇、燕麦B-葡聚, 0.883
[2022/02/22 17:47:27] root DEBUG: 0.961928, 0.964
[2022/02/22 17:47:27] root DEBUG: 10, 0.812
[2022/02/22 17:47:27] root DEBUG: 糖、椰油酰胺丙基甜菜碱、泛醒, 0.866
[2022/02/22 17:47:27] root DEBUG: 0.925898, 0.943
[2022/02/22 17:47:27] root DEBUG: (成品包材), 0.974
[2022/02/22 17:47:27] root DEBUG: 0.972573, 0.961
[2022/02/22 17:47:27] root DEBUG: 【主要功能】:可紧致头发磷层,从而达到, 0.936
[2022/02/22 17:47:27] root DEBUG: 0.994448, 0.952
[2022/02/22 17:47:27] root DEBUG: 13, 0.998
[2022/02/22 17:47:27] root DEBUG: 即时持久改善头发光泽的效果,给干燥的头, 0.994
[2022/02/22 17:47:27] root DEBUG: 0.990198, 0.975
[2022/02/22 17:47:27] root DEBUG: 14, 0.977
[2022/02/22 17:47:27] root DEBUG: 发足够的滋养, 0.991
[2022/02/22 17:47:27] root DEBUG: 0.997668, 0.918
[2022/02/22 17:47:27] root DEBUG: 花费了0.457335秒, 0.901
[2022/02/22 17:47:27] root DEBUG: The visualized image saved in ./inference_results/lite_demo.png
[2022/02/22 17:47:27] root INFO: The predict total time is 2.8338775634765625
```

View File

@@ -0,0 +1,343 @@
---
comments: true
hide:
- toc
---
# 云上飞桨部署工具
[云上飞桨PaddleCloud](https://github.com/PaddlePaddle/PaddleCloud) 是面向飞桨框架及其模型套件的部署工具,
为用户提供了模型套件Docker化部署和Kubernetes集群部署两种方式可以满足不同场景与环境的部署需求。
本章节我们将使用PaddleCloud提供的OCR标准镜像以及云原生组件来训练和部署PP-OCRv3识别模型。
## 云上飞桨部署工具的优势
<div align="center">
<img src="./images/paddlecloud.png" title="architecture" width="80%" height="80%" alt="">
</div>
- **模型套件Docker镜像大礼包。**
PaddleCloud为用户提供了飞桨模型套件Docker镜像大礼包这些镜像中包含运行模型套件案例的所有依赖并能持续更新支持异构硬件环境和常见CUDA版本、开箱即用。
- **具有丰富的云上飞桨组件。**
云上飞桨具有丰富的云原生功能组件包括样本数据缓存组件、分布式训练组件、推理服务组件等使用这些组件用户可以快速地在Kubernetes集群上进行训练和部署工作。
- **功能强大的自运维能力。**
云上飞桨组件基于Kubernetes的Operator机制提供了功能强大的自运维能力如训练组件支持多种架构模式并具有分布式容错与弹性训练的能力推理服务组件支持自动扩缩容与蓝绿发版等。
- **针对飞桨框架的定制优化。**
除了部署便捷与自运维的优势PaddleCloud还针对飞桨框架进行了正对性优化如通过缓存样本数据来加速云上飞桨分布式训练作业、基于飞桨框架和调度器的协同设计来优化集群GPU利用率等。
## 1. PP-OCRv3 Docker化部署
PaddleCloud基于 [Tekton](https://github.com/tektoncd/pipeline) 为OCR模型套件提供了镜像持续构建的能力并支持CPU、GPU以及常见CUDA版本的镜像。
您可以查看 [PaddleOCR 镜像仓库](https://hub.docker.com/repository/docker/paddlecloud/paddleocr) 来获取所有的镜像列表。
同时我们也将PP-OCRv3识别模型的训练与推理实战案例放置到了AI Studio平台上您可以点击 [PP-OCRv3识别训推一体项目实战](https://aistudio.baidu.com/aistudio/projectdetail/3916206?channelType=0&channel=0) 在平台上快速体验。
> **适用场景**:本地测试开发环境、单机部署环境。
### 1.1 安装Docker
如果您所使用的机器上还没有安装 Docker您可以参考 [Docker 官方文档](https://docs.docker.com/get-docker/) 来进行安装。
如果您需要使用支持 GPU 版本的镜像则还需安装好NVIDIA相关驱动和 [nvidia-docker](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#docker) 。
**注意**如果您使用的是Windows系统需要开启 [WSL2Linux子系统功能功能](https://docs.microsoft.com/en-us/windows/wsl/install)。
### 1.2 启动容器
**使用CPU版本的Docker镜像**
```bash linenums="1"
# 这是加上参数 --shm-size=32g 是为了防止容器里内存不足
docker run --name ppocr -v $PWD:/mnt -p 8888:8888 -it --shm-size=32g paddlecloud/paddleocr:2.5-cpu-efbb0a /bin/bash
```
**使用GPU版本的Docker镜像**
```bash linenums="1"
docker run --name ppocr --runtime=nvidia -v $PWD:/mnt -p 8888:8888 -it --shm-size=32g paddlecloud/paddleocr:2.5-gpu-cuda10.2-cudnn7-efbb0a /bin/bash
```
进入容器内,则可进行 PP-OCRv3 模型的训练和部署工作。
### 1.3 准备训练数据
本教程以HierText数据集为例HierText是第一个具有自然场景和文档中文本分层注释的数据集。
该数据集包含从 Open Images 数据集中选择的 11639 张图像,提供高质量的单词 (~1.2M)、行和段落级别的注释。
我们已经将数据集上传到百度云对象存储BOS您可以通过运行如下指令完成数据集的下载和解压操作
```bash linenums="1"
# 下载数据集
$ wget -P /mnt https://paddleflow-public.hkg.bcebos.com/ppocr/hiertext1.tar
# 解压数据集
$ tar xf /mnt/hiertext1.tar -C /mnt && mv /mnt/hiertext1 /mnt/hiertext
```
运行上述命令后,在 `/mnt` 目录下包含以下文件:
```
/mnt/hiertext
└─ train/ HierText训练集数据
└─ validation/ HierText验证集数据
└─ label_hiertext_train.txt HierText训练集的行标注
└─ label_hiertext_val.txt HierText验证集的行标注
```
### 1.4 修改配置文件
PP-OCRv3模型配置文件位于`/home/PaddleOCR/configs/det/PP-OCRv3/PP-OCRv3_det_cml.yml`,需要修改的配置如下:
- 修改训练数据配置:
```yaml linenums="1"
Train:
dataset:
name: SimpleDataSet
data_dir: ./train_data/icdar2015/text_localization/
label_file_list:
- ./train_data/icdar2015/text_localization/train_icdar2015_label.txt
```
修改为:
```yaml linenums="1"
Train:
dataset:
name: SimpleDataSet
data_dir: /mnt/
label_file_list:
- /mnt/hiertext/label_hiertext_train.txt
```
- 修改验证数据配置:
```yaml linenums="1"
Eval:
dataset:
name: SimpleDataSet
data_dir: ./train_data/icdar2015/text_localization/
label_file_list:
- ./train_data/icdar2015/text_localization/test_icdar2015_label.txt
```
修改为:
```yaml linenums="1"
Eval:
dataset:
name: SimpleDataSet
data_dir: /mnt/
label_file_list:
- /mnt/hiertext/label_hiertext_val.txt
```
### 1.5 启动训练
下载PP-OCRv3的蒸馏预训练模型并进行训练的方式如下
```bash linenums="1"
# 下载预训练模型到/home/PaddleOCR/pre_train文件夹下
$ mkdir /home/PaddleOCR/pre_train
$ wget -P /home/PaddleOCR/pre_train https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv3_mobile_det_pretrained.pdparams
```
启动训练,训练模型默认保存在`output`目录下加载PP-OCRv3检测预训练模型。
```bash linenums="1"
# 这里以 GPU 训练为例,使用 CPU 进行训练的话,需要指定参数 Global.use_gpu=false
python3 tools/train.py -c configs/det/PP-OCRv3/PP-OCRv3_det_cml.yml -o Global.save_model_dir=./output/ Global.pretrained_model=./pre_train/ch_PP-OCRv3_det_distill_train/best_accuracy
```
如果要使用多GPU分布式训练请使用如下命令
```bash linenums="1"
# 启动训练训练模型默认保存在output目录下--gpus '0,1,2,3'表示使用0123号GPU训练
python3 -m paddle.distributed.launch --log_dir=./debug/ --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv3/PP-OCRv3_det_cml.yml -o Global.save_model_dir=./output/ Global.pretrained_model=./pre_train/ch_PP-OCRv3_det_distill_train/best_accuracy
```
### 1.6 模型评估
训练过程中保存的模型在output目录下包含以下文件
```
best_accuracy.states
best_accuracy.pdparams # 默认保存最优精度的模型参数
best_accuracy.pdopt # 默认保存最优精度的优化器相关参数
latest.states
latest.pdparams # 默认保存的最新模型参数
latest.pdopt # 默认保存的最新模型的优化器相关参数
```
其中best_accuracy是保存的最优模型可以直接使用该模型评估
```bash linenums="1"
# 进行模型评估
cd /home/PaddleOCR/
python3 tools/eval.py -c configs/det/PP-OCRv3/PP-OCRv3_det_cml.yml -o Global.checkpoints=./output/best_accuracy
```
## 2. PP-OCRv3云端部署
PaddleCloud基于Kubernetes的Operator机制为您提供了多个功能强大的云原生组件如样本数据缓存组件、分布式训练组件、 以及模型推理服务组件,
使用这些组件您可以快速地在云上进行分布式训练和模型服务化部署。更多关于PaddleCloud云原生组件的内容请参考文档 [PaddleCloud架构概览](https://github.com/PaddlePaddle/PaddleCloud/blob/main/docs/zh_CN/paddlecloud-overview.md) 。
> **适用场景**基于Kubernetes的多机部署环境。
### 2.1 安装云上飞桨组件
#### 环境要求
- [Kubernetes v1.16+](https://kubernetes.io/zh/)
- [kubectl](https://kubernetes.io/docs/tasks/tools/)
- [Helm](https://helm.sh/zh/docs/intro/install/)
如果您没有Kubernetes环境可以使用MicroK8S在本地搭建环境更多详情请参考 [MicroK8S官方文档](https://microk8s.io/docs/getting-started)。
使用Helm一键安装所有组件和所有依赖
```bash linenums="1"
# 添加PaddleCloud Chart仓库
$ helm repo add paddlecloud https://paddleflow-public.hkg.bcebos.com/charts
$ helm repo update
# 安装云上飞桨组件
$ helm install pdc paddlecloud/paddlecloud --set tags.all-dep=true --namespace paddlecloud --create-namespace
# 检查所有云上飞桨组件是否成功启动命名空间下的所有Pod都为Runing状态则安装成功。
$ kubectl get pods -n paddlecloud
NAME READY STATUS RESTARTS AGE
pdc-hostpath-5b6bd6787d-bxvxg 1/1 Running 0 10h
juicefs-csi-node-pkldt 3/3 Running 0 10h
juicefs-csi-controller-0 3/3 Running 0 10h
pdc-paddlecloud-sampleset-767bdf6947-pb6zm 1/1 Running 0 10h
pdc-paddlecloud-paddlejob-7cc8b7bfc6-7gqnh 1/1 Running 0 10h
pdc-minio-7cc967669d-824q5 1/1 Running 0 10h
pdc-redis-master-0 1/1 Running 0 10h
```
更多安装参数请参考[PaddleCloud安装指南](https://github.com/PaddlePaddle/PaddleCloud/blob/main/docs/zh_CN/installation.md)
### 2.2 云原生组件介绍
<div align="center">
<img src="./images/architecture.jpeg" title="architecture" width="60%" height="60%" alt="">
</div>
- **数据缓存组件。** 数据缓存组件使用JuiceFS作为缓存引擎能够将远程样本数据缓存到训练集群本地大幅加速云上飞桨分布式训练作业。
- **分布式训练组件。** 分布式训练组件支持参数服务器PS与集合通信Collective两种架构模式方便用户在云上快速运行飞桨分布式训练作业。
以下内容我们将使用这两个云原生组件来在Kubernetes集群中部署PP-OCRv3识别模型的训练作业。
### 2.3 准备hiertext数据集
使用数据缓存组件来准备数据集编写SampleSet Yaml文件如下
```yaml linenums="1"
# hiertext.yaml
apiVersion: batch.paddlepaddle.org/v1alpha1
kind: SampleSet
metadata:
name: hiertext
namespace: paddlecloud
spec:
partitions: 1
source:
uri: bos://paddleflow-public.hkg.bcebos.com/ppocr/hiertext
secretRef:
name: none
secretRef:
name: data-center
```
然后在命令行中使用kubectl执行如下命令。
```bash linenums="1"
# 创建hiertext数据集
$ kubectl apply -f hiertext.yaml
sampleset.batch.paddlepaddle.org/hiertext created
# 查看数据集的状态
$ kubectl get sampleset hiertext -n paddlecloud
NAME TOTAL SIZE CACHED SIZE AVAIL SPACE RUNTIME PHASE AGE
hiertext 3.3 GiB 3.2 GiB 12 GiB 1/1 Ready 11m
```
### 2.4 训练PP-OCRv3模型
使用训练组件在Kubernetes集群上训练PP-OCRv3模型编写PaddleJob Yaml文件如下
```yaml linenums="1"
# ppocrv3.yaml
apiVersion: batch.paddlepaddle.org/v1
kind: PaddleJob
metadata:
name: ppocrv3
namespace: paddlecloud
spec:
cleanPodPolicy: OnCompletion
sampleSetRef:
name: hiertext
namespace: paddlecloud
mountPath: /mnt/hiertext
worker:
replicas: 1
template:
spec:
containers:
- name: ppocrv3
image: paddlecloud/paddleocr:2.5-gpu-cuda10.2-cudnn7-efbb0a
command:
- /bin/bash
args:
- "-c"
- >
mkdir /home/PaddleOCR/pre_train &&
wget -P ./pre_train https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_distill_train.tar &&
tar xf ./pre_train/ch_PP-OCRv3_det_distill_train.tar -C ./pre_train/ &&
python tools/train.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml -o
Train.dataset.data_dir=/mnt/
Train.dataset.label_file_list=[\"/mnt/hiertext/label_hiertext_train.txt\"]
Eval.dataset.data_dir=/mnt/
Eval.dataset.label_file_list=[\"/mnt/hiertext/label_hiertext_val.txt\"]
Global.save_model_dir=./output/
Global.pretrained_model=./pre_train/ch_PP-OCRv3_det_distill_train/best_accuracy
resources:
limits:
nvidia.com/gpu: 1
volumeMounts: # 添加 shared memory 挂载以防止缓存出错
- mountPath: /dev/shm
name: dshm
volumes:
- name: dshm
emptyDir:
medium: Memory
```
本案例采用GPU进行训练如果您只有CPU机器则可以将镜像替换成CPU版本 `paddlecloud/paddleocr:2.5-cpu-efbb0a`并在args中加上参数`Global.use_gpu=false`。
```bash linenums="1"
# 创建PaddleJob训练模型
$ kubectl apply -f ppocrv3.yaml
paddlejob.batch.paddlepaddle.org/ppocrv3 created
# 查看PaddleJob状态
$ kubectl get pods -n paddlecloud -l paddle-res-name=ppocrv3-worker-0
NAME READY STATUS RESTARTS AGE
ppocrv3-worker-0 1/1 Running 0 4s
# 查看训练日志
$ kubectl logs -f ppocrv3-worker-0 -n paddlecloud
```
## 更多资源
欢迎关注[云上飞桨项目PaddleCloud](https://github.com/PaddlePaddle/PaddleCloud),我们为您提供了飞桨模型套件标准镜像以及全栈的云原生模型套件部署组件,如您有任何关于飞桨模型套件的部署问题,请联系我们。
如果你发现任何PaddleCloud存在的问题或者是建议, 欢迎通过[GitHub Issues](https://github.com/PaddlePaddle/PaddleCloud/issues)给我们提issues。

View File

@@ -0,0 +1,288 @@
---
comments: true
hide:
- toc
---
## OCR Pipeline WebService
PaddleOCR provides two service deployment methods:
- Based on **PaddleHub Serving**: Code path is "`./deploy/hubserving`". Please refer to the [tutorial](https://github.com/PaddlePaddle/PaddleOCR/blob/db0ad17cf631fafc01650c177e00ce76413af97f/deploy/hubserving/readme_en.md)
- Based on **PaddleServing**: Code path is "`./deploy/pdserving`". Please follow this tutorial.
### Service deployment based on PaddleServing
This document will introduce how to use the [PaddleServing](https://github.com/PaddlePaddle/Serving/blob/develop/README.md) to deploy the PPOCR dynamic graph model as a pipeline online service.
Some Key Features of Paddle Serving:
- Integrate with Paddle training pipeline seamlessly, most paddle models can be deployed with one line command.
- Industrial serving features supported, such as models management, online loading, online A/B testing etc.
- Highly concurrent and efficient communication between clients and servers supported.
PaddleServing supports deployment in multiple languages. In this example, two deployment methods, python pipeline and C++, are provided. The comparison between the two is as follows:
| Language | Speed | Secondary development | Do you need to compile |
|-----|-----|---------|------------|
| C++ | fast | Slightly difficult | Single model prediction does not need to be compiled, multi-model concatenation needs to be compiled |
| python | general | easy | single-model/multi-model no compilation required |
The introduction and tutorial of Paddle Serving service deployment framework reference [document](https://github.com/PaddlePaddle/Serving/blob/develop/README.md).
### Environmental preparation
PaddleOCR operating environment and Paddle Serving operating environment are needed.
1. Please prepare PaddleOCR operating environment reference [link](../environment.en.md).
Download the corresponding paddlepaddle whl package according to the environment, it is recommended to install version 2.2.2.
2. The steps of PaddleServing operating environment prepare are as follows:
```bash linenums="1"
# Install serving which used to start the service
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.8.3.post102-py3-none-any.whl
pip3 install paddle_serving_server_gpu-0.8.3.post102-py3-none-any.whl
# Install paddle-serving-server for cuda10.1
# wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.8.3.post101-py3-none-any.whl
# pip3 install paddle_serving_server_gpu-0.8.3.post101-py3-none-any.whl
# Install serving which used to start the service
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_client-0.8.3-cp37-none-any.whl
pip3 install paddle_serving_client-0.8.3-cp37-none-any.whl
# Install serving-app
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_app-0.8.3-py3-none-any.whl
pip3 install paddle_serving_app-0.8.3-py3-none-any.whl
```
**note:** If you want to install the latest version of PaddleServing, refer to [link](https://github.com/PaddlePaddle/Serving/blob/v0.8.3/doc/Latest_Packages_CN.md).
### Model conversion
When using PaddleServing for service deployment, you need to convert the saved inference model into a serving model that is easy to deploy.
Firstly, download the [inference model](../model_list.en.md) of PPOCR
```bash linenums="1"
# Download and unzip the OCR text detection model
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv3_mobile_det_infer.tar -O PP-OCRv3_mobile_det_infer.tar && tar -xf PP-OCRv3_mobile_det_infer.tar
# Download and unzip the OCR text recognition model
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv3_mobile_rec_infer.tar -O PP-OCRv3_mobile_rec_infer.tar && tar -xf PP-OCRv3_mobile_rec_infer.tar
```
Then, you can use installed paddle_serving_client tool to convert inference model to mobile model.
```bash linenums="1"
# Detection model conversion
python3 -m paddle_serving_client.convert --dirname ./PP-OCRv3_mobile_det_infer/ \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--serving_server ./ppocr_det_v3_serving/ \
--serving_client ./ppocr_det_v3_client/
# Recognition model conversion
python3 -m paddle_serving_client.convert --dirname ./PP-OCRv3_mobile_rec_infer/ \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--serving_server ./ppocr_rec_v3_serving/ \
--serving_client ./ppocr_rec_v3_client/
```
After the detection model is converted, there will be additional folders of `ppocr_det_v3_serving` and `ppocr_det_v3_client` in the current folder, with the following format:
```text linenums="1"
|- ppocr_det_v3_serving/
|- __model__
|- __params__
|- serving_server_conf.prototxt
|- serving_server_conf.stream.prototxt
|- ppocr_det_v3_client
|- serving_client_conf.prototxt
|- serving_client_conf.stream.prototxt
```
The recognition model is the same.
### Paddle Serving pipeline deployment
1. Download the PaddleOCR code, if you have already downloaded it, you can skip this step.
```bash linenums="1"
git clone https://github.com/PaddlePaddle/PaddleOCR
# Enter the working directory
cd PaddleOCR/deploy/pdserving/
```
The pdserver directory contains the code to start the pipeline service and send prediction requests, including:
```bash linenums="1"
__init__.py
config.yml # Start the service configuration file
ocr_reader.py # OCR model pre-processing and post-processing code implementation
pipeline_http_client.py # Script to send pipeline prediction request
web_service.py # Start the script of the pipeline server
```
2. Run the following command to start the service.
```bash linenums="1"
# Start the service and save the running log in log.txt
python3 web_service.py --config=config.yml &>log.txt &
```
After the service is successfully started, a log similar to the following will be printed in log.txt
![](./images/start_server.png)
3. Send service request
```bash linenums="1"
python3 pipeline_http_client.py
```
After successfully running, the predicted result of the model will be printed in the cmd window. An example of the result is:
![](./images/results.png)
Adjust the number of concurrency in config.yml to get the largest QPS. Generally, the number of concurrent detection and recognition is 2:1
```yaml linenums="1"
det:
concurrency: 8
...
rec:
concurrency: 4
...
```
Multiple service requests can be sent at the same time if necessary.
The predicted performance data will be automatically written into the `PipelineServingLogs/pipeline.tracer` file.
Tested on 200 real pictures, and limited the detection long side to 960. The average QPS on T4 GPU can reach around 23:
```bash linenums="1"
2021-05-13 03:42:36,895 ==================== TRACER ======================
2021-05-13 03:42:36,975 Op(rec):
2021-05-13 03:42:36,976 in[14.472382882882883 ms]
2021-05-13 03:42:36,976 prep[9.556855855855856 ms]
2021-05-13 03:42:36,976 midp[59.921905405405404 ms]
2021-05-13 03:42:36,976 postp[15.345945945945946 ms]
2021-05-13 03:42:36,976 out[1.9921216216216215 ms]
2021-05-13 03:42:36,976 idle[0.16254943864471572]
2021-05-13 03:42:36,976 Op(det):
2021-05-13 03:42:36,976 in[315.4468035714286 ms]
2021-05-13 03:42:36,976 prep[69.5980625 ms]
2021-05-13 03:42:36,976 midp[18.989535714285715 ms]
2021-05-13 03:42:36,976 postp[18.857803571428573 ms]
2021-05-13 03:42:36,977 out[3.1337544642857145 ms]
2021-05-13 03:42:36,977 idle[0.7477961159203756]
2021-05-13 03:42:36,977 DAGExecutor:
2021-05-13 03:42:36,977 Query count[224]
2021-05-13 03:42:36,977 QPS[22.4 q/s]
2021-05-13 03:42:36,977 Succ[0.9910714285714286]
2021-05-13 03:42:36,977 Error req[169, 170]
2021-05-13 03:42:36,977 Latency:
2021-05-13 03:42:36,977 ave[535.1678348214285 ms]
2021-05-13 03:42:36,977 .50[172.651 ms]
2021-05-13 03:42:36,977 .60[187.904 ms]
2021-05-13 03:42:36,977 .70[245.675 ms]
2021-05-13 03:42:36,977 .80[526.684 ms]
2021-05-13 03:42:36,977 .90[854.596 ms]
2021-05-13 03:42:36,977 .95[1722.728 ms]
2021-05-13 03:42:36,977 .99[3990.292 ms]
2021-05-13 03:42:36,978 Channel (server worker num[10]):
2021-05-13 03:42:36,978 chl0(In: ['@DAGExecutor'], Out: ['det']) size[0/0]
2021-05-13 03:42:36,979 chl1(In: ['det'], Out: ['rec']) size[6/0]
2021-05-13 03:42:36,979 chl2(In: ['rec'], Out: ['@DAGExecutor']) size[0/0]
```
### C++ Serving
Service deployment based on python obviously has the advantage of convenient secondary development. However, the real application often needs to pursue better performance. PaddleServing also provides a more performant C++ deployment version.
The C++ service deployment is the same as python in the environment setup and data preparation stages, the difference is when the service is started and the client sends requests.
1. Compile Serving
To improve predictive performance, C++ services also provide multiple model concatenation services. Unlike Python Pipeline services, multiple model concatenation requires the pre - and post-model processing code to be written on the server side, so local recompilation is required to generate serving. Specific may refer to the official document: [how to compile Serving](https://github.com/PaddlePaddle/Serving/blob/v0.8.3/doc/Compile_EN.md)
2. Run the following command to start the service.
```bash linenums="1"
# Start the service and save the running log in log.txt
python3 -m paddle_serving_server.serve --model ppocr_det_v3_serving ppocr_rec_v3_serving --op GeneralDetectionOp GeneralInferOp --port 8181 &>log.txt &
```
After the service is successfully started, a log similar to the following will be printed in log.txt
![](./images/start_server.png)
3. Send service request
Due to the need for pre and post-processing in the C++Server part, in order to speed up the input to the C++Server is only the base64 encoded string of the picture, it needs to be manually modified
Change the feed_type field and shape field in ppocr_det_v3_client/serving_client_conf.prototxt to the following:
```bash linenums="1"
feed_var {
name: "x"
alias_name: "x"
is_lod_tensor: false
feed_type: 20
shape: 1
}
```
start the client:
```bash linenums="1"
python3 ocr_cpp_client.py ppocr_det_v3_client ppocr_rec_v3_client
```
After successfully running, the predicted result of the model will be printed in the cmd window. An example of the result is:
![](./images/results.png)
### WINDOWS Users
Windows does not support Pipeline Serving, if we want to launch paddle serving on Windows, we should use Web Service, for more information please refer to [Paddle Serving for Windows Users](https://github.com/PaddlePaddle/Serving/blob/develop/doc/Windows_Tutorial_EN.md)
**WINDOWS user can only use version 0.5.0 CPU Mode**
**Prepare Stage:**
```bash linenums="1"
pip3 install paddle-serving-server==0.5.0
pip3 install paddle-serving-app==0.3.1
```
1. Start Server
```bash linenums="1"
cd win
python3 ocr_web_server.py gpu(for gpu user)
or
python3 ocr_web_server.py cpu(for cpu user)
```
2. Client Send Requests
```bash linenums="1"
python3 ocr_web_client.py
```
### FAQ
**Q1**: No result return after sending the request.
**A1**: Do not set the proxy when starting the service and sending the request. You can close the proxy before starting the service and before sending the request. The command to close the proxy is:
```bash linenums="1"
unset https_proxy
unset http_proxy
```

View File

@@ -0,0 +1,307 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
## PPOCR 服务化部署
PaddleOCR提供2种服务部署方式
- 基于PaddleHub Serving的部署代码路径为"`./deploy/hubserving`",使用方法参考[文档](https://github.com/PaddlePaddle/PaddleOCR/blob/db0ad17cf631fafc01650c177e00ce76413af97f/deploy/hubserving/readme.md)
- 基于PaddleServing的部署代码路径为"`./deploy/pdserving`",按照本教程使用。
### 基于PaddleServing的服务部署
本文档将介绍如何使用[PaddleServing](https://github.com/PaddlePaddle/Serving/blob/develop/README_CN.md) 工具部署PP-OCR动态图模型的pipeline在线服务。
相比较于hubserving部署PaddleServing具备以下优点
- 支持客户端和服务端之间高并发和高效通信
- 支持 工业级的服务能力 例如模型管理在线加载在线A/B测试等
- 支持 多种编程语言 开发客户端例如C++, Python和Java
PaddleServing 支持多种语言部署本例中提供了python pipeline 和 C++ 两种部署方式,两者的对比如下:
| 语言 | 速度 | 二次开发 | 是否需要编译 |
| ------ | ---- | -------- | -------------------------------------- |
| C++ | 很快 | 略有难度 | 单模型预测无需编译,多模型串联需要编译 |
| python | 一般 | 容易 | 单模型/多模型 均无需编译 |
更多有关PaddleServing服务化部署框架介绍和使用教程参考[文档](https://github.com/PaddlePaddle/Serving/blob/develop/README_CN.md)。
AIStudio演示案例可参考 [基于PaddleServing的OCR服务化部署实战](https://aistudio.baidu.com/aistudio/projectdetail/3630726)。
#### 环境准备
需要准备PaddleOCR的运行环境和Paddle Serving的运行环境。
- 准备PaddleOCR的运行环境[链接](../environment.md)
```bash linenums="1"
git clone https://github.com/PaddlePaddle/PaddleOCR
# 进入到工作目录
cd PaddleOCR/deploy/pdserving/
```
- 准备PaddleServing的运行环境步骤如下
```bash linenums="1"
# 安装serving用于启动服务
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.8.3.post102-py3-none-any.whl
pip3 install paddle_serving_server_gpu-0.8.3.post102-py3-none-any.whl
# 如果是cuda10.1环境可以使用下面的命令安装paddle-serving-server
# wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_server_gpu-0.8.3.post101-py3-none-any.whl
# pip3 install paddle_serving_server_gpu-0.8.3.post101-py3-none-any.whl
# 安装client用于向服务发送请求
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_client-0.8.3-cp37-none-any.whl
pip3 install paddle_serving_client-0.8.3-cp37-none-any.whl
# 安装serving-app
wget https://paddle-serving.bj.bcebos.com/test-dev/whl/paddle_serving_app-0.8.3-py3-none-any.whl
pip3 install paddle_serving_app-0.8.3-py3-none-any.whl
```
**Note:** 如果要安装最新版本的PaddleServing参考[链接](https://github.com/PaddlePaddle/Serving/blob/v0.8.3/doc/Latest_Packages_CN.md)。
### 模型转换
使用PaddleServing做服务化部署时需要将保存的inference模型转换为serving易于部署的模型。
首先下载PP-OCR的[inference模型](https://github.com/PaddlePaddle/PaddleOCR#pp-ocr-series-model-listupdate-on-september-8th)
```bash linenums="1"
# 下载并解压 OCR 文本检测模型
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv3_mobile_det_infer.tar -O PP-OCRv3_mobile_det_infer.tar && tar -xf PP-OCRv3_mobile_det_infer.tar
# 下载并解压 OCR 文本识别模型
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar -O PP-OCRv3_mobile_rec_infer.tar && tar -xf PP-OCRv3_mobile_rec_infer.tar
```
接下来用安装的paddle_serving_client把下载的inference模型转换成易于server部署的模型格式。
```bash linenums="1"
# 转换检测模型
python3 -m paddle_serving_client.convert --dirname ./PP-OCRv3_mobile_det_infer/ \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--serving_server ./ppocr_det_v3_serving/ \
--serving_client ./ppocr_det_v3_client/
# 转换识别模型
python3 -m paddle_serving_client.convert --dirname ./PP-OCRv3_mobile_rec_infer/ \
--model_filename inference.pdmodel \
--params_filename inference.pdiparams \
--serving_server ./ppocr_rec_v3_serving/ \
--serving_client ./ppocr_rec_v3_client/
```
检测模型转换完成后,会在当前文件夹多出`ppocr_det_v3_serving` 和`ppocr_det_v3_client`的文件夹,具备如下格式:
```text linenums="1"
|- ppocr_det_v3_serving/
|- __model__
|- __params__
|- serving_server_conf.prototxt
|- serving_server_conf.stream.prototxt
|- ppocr_det_v3_client
|- serving_client_conf.prototxt
|- serving_client_conf.stream.prototxt
```
识别模型同理。
### Paddle Serving pipeline部署
#### 1. 确认工作目录下文件结构
pdserver目录包含启动pipeline服务和发送预测请求的代码包括
```bash linenums="1"
__init__.py
config.yml # 启动服务的配置文件
ocr_reader.py # OCR模型预处理和后处理的代码实现
pipeline_http_client.py # 发送pipeline预测请求的脚本
web_service.py # 启动pipeline服务端的脚本
```
#### 2. 启动服务
```bash linenums="1"
# 启动服务运行日志保存在log.txt
python3 web_service.py --config=config.yml &>log.txt &
```
成功启动服务后log.txt中会打印类似如下日志
![img](./images/start_server.png)
#### 3. 发送服务请求
```bash linenums="1"
python3 pipeline_http_client.py
```
成功运行后模型预测的结果会打印在cmd窗口中结果示例为
![img](./images/pipeline_result.png)
调整 config.yml 中的并发个数获得最大的QPS, 一般检测和识别的并发数为21
```yaml linenums="1"
det:
#并发数is_thread_op=True时为线程并发否则为进程并发
concurrency: 8
...
rec:
#并发数is_thread_op=True时为线程并发否则为进程并发
concurrency: 4
...
```
有需要的话可以同时发送多个服务请求
预测性能数据会被自动写入 `PipelineServingLogs/pipeline.tracer` 文件中。
在200张真实图片上测试把检测长边限制为960。T4 GPU 上 QPS 均值可达到23左右
```bash linenums="1"
2021-05-13 03:42:36,895 ==================== TRACER ======================
2021-05-13 03:42:36,975 Op(rec):
2021-05-13 03:42:36,976 in[14.472382882882883 ms]
2021-05-13 03:42:36,976 prep[9.556855855855856 ms]
2021-05-13 03:42:36,976 midp[59.921905405405404 ms]
2021-05-13 03:42:36,976 postp[15.345945945945946 ms]
2021-05-13 03:42:36,976 out[1.9921216216216215 ms]
2021-05-13 03:42:36,976 idle[0.16254943864471572]
2021-05-13 03:42:36,976 Op(det):
2021-05-13 03:42:36,976 in[315.4468035714286 ms]
2021-05-13 03:42:36,976 prep[69.5980625 ms]
2021-05-13 03:42:36,976 midp[18.989535714285715 ms]
2021-05-13 03:42:36,976 postp[18.857803571428573 ms]
2021-05-13 03:42:36,977 out[3.1337544642857145 ms]
2021-05-13 03:42:36,977 idle[0.7477961159203756]
2021-05-13 03:42:36,977 DAGExecutor:
2021-05-13 03:42:36,977 Query count[224]
2021-05-13 03:42:36,977 QPS[22.4 q/s]
2021-05-13 03:42:36,977 Succ[0.9910714285714286]
2021-05-13 03:42:36,977 Error req[169, 170]
2021-05-13 03:42:36,977 Latency:
2021-05-13 03:42:36,977 ave[535.1678348214285 ms]
2021-05-13 03:42:36,977 .50[172.651 ms]
2021-05-13 03:42:36,977 .60[187.904 ms]
2021-05-13 03:42:36,977 .70[245.675 ms]
2021-05-13 03:42:36,977 .80[526.684 ms]
2021-05-13 03:42:36,977 .90[854.596 ms]
2021-05-13 03:42:36,977 .95[1722.728 ms]
2021-05-13 03:42:36,977 .99[3990.292 ms]
2021-05-13 03:42:36,978 Channel (server worker num[10]):
2021-05-13 03:42:36,978 chl0(In: ['@DAGExecutor'], Out: ['det']) size[0/0]
2021-05-13 03:42:36,979 chl1(In: ['det'], Out: ['rec']) size[6/0]
2021-05-13 03:42:36,979 chl2(In: ['rec'], Out: ['@DAGExecutor']) size[0/0]
```
### Paddle Serving C++ 部署
基于python的服务部署显然具有二次开发便捷的优势然而真正落地应用往往需要追求更优的性能。PaddleServing 也提供了性能更优的C++部署版本。
C++ 服务部署在环境搭建和数据准备阶段与 python 相同,区别在于启动服务和客户端发送请求时不同。
#### 1. 准备 Serving 环境
为了提高预测性能C++ 服务同样提供了多模型串联服务。与python pipeline服务不同多模型串联的过程中需要将模型前后处理代码写在服务端因此需要在本地重新编译生成serving。
首先需要下载Serving代码库, 把OCR文本检测预处理相关代码替换到Serving库中
```bash linenums="1"
git clone https://github.com/PaddlePaddle/Serving
cp -rf general_detection_op.cpp Serving/core/general-server/op
```
具体可参考官方文档:[如何编译Serving](https://github.com/PaddlePaddle/Serving/blob/v0.8.3/doc/Compile_CN.md),注意需要开启 WITH_OPENCV 选项。
完成编译后注意要安装编译出的三个whl包并设置SERVING_BIN环境变量。
#### 2. 启动服务
一个服务启动两个模型串联,只需要在--model后依次按顺序传入模型文件夹的相对路径且需要在--op后依次传入自定义C++OP类名称
```bash linenums="1"
# 启动服务运行日志保存在log.txt
python3 -m paddle_serving_server.serve --model ppocr_det_v3_serving ppocr_rec_v3_serving --op GeneralDetectionOp GeneralInferOp --port 8181 &>log.txt &
```
成功启动服务后log.txt中会打印类似如下日志
![](./imgs/start_server.png)
#### 3. 发送服务请求
由于需要在C++Server部分进行前后处理为了加速传入C++Server的仅仅是图片的base64编码的字符串故需要手动修改
ppocr_det_v3_client/serving_client_conf.prototxt 中 feed_type 字段 和 shape 字段,修改成如下内容:
```bash linenums="1"
feed_var {
name: "x"
alias_name: "x"
is_lod_tensor: false
feed_type: 20
shape: 1
}
```
启动客户端
```bash linenums="1"
python3 ocr_cpp_client.py ppocr_det_v3_client ppocr_rec_v3_client
```
成功运行后模型预测的结果会打印在cmd窗口中结果示例为
![img](./images/results.png)
在浏览器中输入服务器 ip:端口号可以看到当前服务的实时QPS。(端口号范围需要是8000-9000)
在200张真实图片上测试把检测长边限制为960。T4 GPU 上 QPS 峰值可达到51左右,约为pipeline的 2.12 倍。
### Windows用户
Windows用户不能使用上述的启动方式需要使用Web Service详情参见[Windows平台使用Paddle Serving指导](https://github.com/PaddlePaddle/Serving/blob/develop/doc/Windows_Tutorial_CN.md)
**WINDOWS只能使用0.5.0版本的CPU模式**
准备阶段:
```bash linenums="1"
pip3 install paddle-serving-server==0.5.0
pip3 install paddle-serving-app==0.3.1
```
#### 1. 启动服务端程序
```bash linenums="1"
cd win
python3 ocr_web_server.py gpu(使用gpu方式)
或者
python3 ocr_web_server.py cpu(使用cpu方式)
```
#### 2. 发送服务请求
```bash linenums="1"
python3 ocr_web_client.py
```
### FAQ
**Q1** 发送请求后没有结果返回或者提示输出解码报错
**A1** 启动服务和发送请求时不要设置代理,可以在启动服务前和发送请求前关闭代理,关闭代理的命令是:
```bash linenums="1"
unset https_proxy
unset http_proxy
```

View File

@@ -0,0 +1,178 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# Python Inference for PP-OCR Model Zoo
This article introduces the use of the Python inference engine for the PP-OCR model library. The content is in order of text detection, text recognition, direction classifier and the prediction method of the three in series on the CPU and GPU.
## Text Detection Model Inference
The default configuration is based on the inference setting of the DB text detection model. For lightweight Chinese detection model inference, you can execute the following commands:
```bash linenums="1"
# download DB text detection inference model
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv3_mobile_det_infer.tar
tar xf PP-OCRv3_mobile_det_infer.tar
# run inference
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./PP-OCRv3_mobile_det_infer/"
```
The visual text detection results are saved to the ./inference_results folder by default, and the name of the result file is prefixed with 'det_res'. Examples of results are as follows:
![img](./images/det_res_00018069.jpg)
You can use the parameters `limit_type` and `det_limit_side_len` to limit the size of the input image,
The optional parameters of `limit_type` are [`max`, `min`], and
`det_limit_size_len` is a positive integer, generally set to a multiple of 32, such as 960.
The default setting of the parameters is `limit_type='max', det_limit_side_len=960`. Indicates that the longest side of the network input image cannot exceed 960,
If this value is exceeded, the image will be resized with the same width ratio to ensure that the longest side is `det_limit_side_len`.
Set as `limit_type='min', det_limit_side_len=960`, it means that the shortest side of the image is limited to 960.
If the resolution of the input picture is relatively large and you want to use a larger resolution prediction, you can set det_limit_side_len to the desired value, such as 1216:
```bash linenums="1"
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/1.jpg" --det_model_dir="./PP-OCRv3_mobile_det_infer/" --det_limit_type=max --det_limit_side_len=1216
```
If you want to use the CPU for prediction, execute the command as follows
```bash linenums="1"
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/1.jpg" --det_model_dir="./PP-OCRv3_mobile_det_infer/" --use_gpu=False
```
## Text Recognition Model Inference
### 1. Lightweight Chinese Recognition Model Inference
**Note**: The input shape used by the recognition model of `PP-OCRv3` is `3, 48, 320`. If you use other recognition models, you need to set the parameter `--rec_image_shape` according to the model. In addition, the `rec_algorithm` used by the recognition model of `PP-OCRv3` is `SVTR_LCNet` by default. Note the difference from the original `SVTR`.
For lightweight Chinese recognition model inference, you can execute the following commands:
```bash linenums="1"
# download CRNN text recognition inference model
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar
tar xf PP-OCRv3_mobile_rec_infer.tar
# run inference
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_10.png" --rec_model_dir="./PP-OCRv3_mobile_rec_infer/" --rec_image_shape=3,48,320
```
![img](./images/word_10.png)
After executing the command, the prediction results (recognized text and score) of the above image will be printed on the screen.
```bash linenums="1"
Predicts of ./doc/imgs_words_en/word_10.png:('PAIN', 0.988671)
```
### 2. English Recognition Model Inference
For English recognition model inference, you can execute the following commands,you need to specify the dictionary path used by `--rec_char_dict_path`:
```bash linenums="1"
# download en model
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/en_PP-OCRv3_mobile_rec_infer.tar
tar xf en_PP-OCRv3_mobile_rec_infer.tar
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words/en/word_1.png" --rec_model_dir="./en_PP-OCRv3_mobile_rec_infer/" --rec_char_dict_path="ppocr/utils/en_dict.txt"
```
![img](./images/word_1.png)
After executing the command, the prediction result of the above figure is:
```bash linenums="1"
Predicts of ./doc/imgs_words/en/word_1.png: ('JOINT', 0.998160719871521)
```
### 3. Multilingual Model Inference
If you need to predict [other language models](../model_list.en.md), when using inference model prediction, you need to specify the dictionary path used by `--rec_char_dict_path`. At the same time, in order to get the correct visualization results,
You need to specify the visual font path through `--vis_font_path`. There are small language fonts provided by default under the `doc/fonts` path, such as Korean recognition:
```bash linenums="1"
wget wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/multilingual/korean_mobile_v2.0_rec_infer.tar
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words/korean/1.jpg" --rec_model_dir="./your inference model" --rec_char_dict_path="ppocr/utils/dict/korean_dict.txt" --vis_font_path="doc/fonts/korean.ttf"
```
![img](./images/1.jpg)
After executing the command, the prediction result of the above figure is:
```text linenums="1"
Predicts of ./doc/imgs_words/korean/1.jpg:('바탕으로', 0.9948904)
```
## Angle Classification Model Inference
For angle classification model inference, you can execute the following commands:
```bash linenums="1"
# download text angle class inference model
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
tar xf ch_ppocr_mobile_v2.0_cls_infer.tar
python3 tools/infer/predict_cls.py --image_dir="./doc/imgs_words_en/word_10.png" --cls_model_dir="ch_ppocr_mobile_v2.0_cls_infer"
```
![img](./images/word_10.png)
After executing the command, the prediction results (classification angle and score) of the above image will be printed on the screen.
```text linenums="1"
Predicts of ./doc/imgs_words_en/word_10.png:['0', 0.9999995]
```
## Text Detection Angle Classification and Recognition Inference Concatenation
**Note**: The input shape used by the recognition model of `PP-OCRv3` is `3, 48, 320`. If you use other recognition models, you need to set the parameter `--rec_image_shape` according to the model. In addition, the `rec_algorithm` used by the recognition model of `PP-OCRv3` is `SVTR_LCNet` by default. Note the difference from the original `SVTR`.
When performing prediction, you need to specify the path of a single image or a folder of images through the parameter `image_dir`, pdf file is also supported, the parameter `det_model_dir` specifies the path to detect the inference model, the parameter `cls_model_dir` specifies the path to angle classification inference model and the parameter `rec_model_dir` specifies the path to identify the inference model. The parameter `use_angle_cls` is used to control whether to enable the angle classification model. The parameter `use_mp` specifies whether to use multi-process to infer `total_process_num` specifies process number when using multi-process. The parameter . The visualized recognition results are saved to the `./inference_results` folder by default.
```bash linenums="1"
# use direction classifier
python3 tools/infer/predict_system.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./PP-OCRv3_mobile_det_infer/" --cls_model_dir="./cls/" --rec_model_dir="./PP-OCRv3_mobile_rec_infer/" --use_angle_cls=true
# not use use direction classifier
python3 tools/infer/predict_system.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./PP-OCRv3_mobile_det_infer/" --rec_model_dir="./PP-OCRv3_mobile_rec_infer/" --use_angle_cls=false
# use multi-process
python3 tools/infer/predict_system.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./PP-OCRv3_mobile_det_infer/" --rec_model_dir="./PP-OCRv3_mobile_rec_infer/" --use_angle_cls=false --use_mp=True --total_process_num=6
# use PDF files, you can infer the first few pages by using the `page_num` parameter, the default is 0, which means infer all pages
python3 tools/infer/predict_system.py --image_dir="./xxx.pdf" --det_model_dir="./PP-OCRv3_mobile_det_infer/" --cls_model_dir="./cls/" --rec_model_dir="./PP-OCRv3_mobile_rec_infer/" --use_angle_cls=true --page_num=2
```
After executing the command, the recognition result image is as follows:
![](./images/system_res_00018069_v3.jpg)
For more configuration and explanation of inference parameters, please refer to[Model Inference Parameters Explained Tutorial](../blog/inference_args.en.md)。
## TensorRT Inference
Paddle Inference ensembles TensorRT using subgraph mode. For GPU deployment scenarios, TensorRT can optimize some subgraphs, including horizontal and vertical integration of OPs, filter redundant OPs, and automatically select the optimal OP kernels for to speed up inference.
You need to do the following 2 steps for inference using TRT.
* (1) Collect the dynamic shape information of the model about a specific dataset and store it in a file.
* (2) Load the dynamic shape information file for TRT inference.
Taking the text detection model as an example. Firstly, you can use the following command to generate a dynamic shape file, which will eventually be named as `det_trt_dynamic_shape.txt` and stored in the `PP-OCRv3_mobile_det_infer` folder.
```bash linenums="1"
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/1.jpg" --det_model_dir="./PP-OCRv3_mobile_det_infer/" --use_tensorrt=True
```
The above command is only used to collect dynamic shape information, and TRT is not used during inference.
Then, you can use the following command to perform TRT inference.
```bash linenums="1"
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/1.jpg" --det_model_dir="./PP-OCRv3_mobile_det_infer/" --use_tensorrt=True
```
**Note:**
* In the first step, if the dynamic shape information file already exists, it does not need to be collected again. If you want to regenerate the dynamic shape information file, you need to delete the dynamic shape information file in the model folder firstly, and then regenerate it.
* In general, dynamic shape information file only needs to be generated once. In the actual deployment process, it is recommended that the dynamic shape information file can be generated on offline validation set or test set, and then the file can be directly loaded for online TRT inference.

View File

@@ -0,0 +1,504 @@
---
typora-copy-images-to: images
comments: true
hide:
- toc
---
# 基于Python预测引擎推理
inference 模型(`paddle.jit.save`保存的模型)
一般是模型训练,把模型结构和模型参数保存在文件中的固化模型,多用于预测部署场景。
训练过程中保存的模型是checkpoints模型保存的只有模型的参数多用于恢复训练等。
与checkpoints模型相比inference 模型会额外保存模型的结构信息,在预测部署、加速推理上性能优越,灵活方便,适合于实际系统集成。
本文介绍针对PP-OCR模型库的Python推理引擎使用方法内容依次为文本检测、文本识别、方向分类器以及三者串联在CPU、GPU上的预测方法。
## 一、训练模型转inference模型
### 检测模型转inference模型
下载超轻量级中文检测模型:
```bash linenums="1"
wget -P ./ch_lite/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_train.tar && tar xf ./ch_lite/ch_ppocr_mobile_v2.0_det_train.tar -C ./ch_lite/
```
上述模型是以MobileNetV3为backbone训练的DB算法将训练好的模型转换成inference模型只需要运行如下命令
```bash linenums="1"
# -c 后面设置训练算法的yml配置文件
# -o 配置可选参数
# Global.pretrained_model 参数设置待转换的训练模型地址,不用添加文件后缀 .pdmodel.pdopt或.pdparams。
# Global.save_inference_dir参数设置转换的模型将保存的地址。
python3 tools/export_model.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o Global.pretrained_model=./ch_lite/ch_ppocr_mobile_v2.0_det_train/best_accuracy Global.save_inference_dir=./inference/det_db/
```
转inference模型时使用的配置文件和训练时使用的配置文件相同。另外还需要设置配置文件中的`Global.pretrained_model`参数,其指向训练中保存的模型参数文件。
转换成功后,在模型保存目录下有三个文件:
```text linenums="1"
inference/det_db/
├── inference.pdiparams # 检测inference模型的参数文件
├── inference.pdiparams.info # 检测inference模型的参数信息可忽略
└── inference.pdmodel # 检测inference模型的program文件
```
### 识别模型转inference模型
下载超轻量中文识别模型:
```bash linenums="1"
wget -P ./ch_lite/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_train.tar && tar xf ./ch_lite/ch_ppocr_mobile_v2.0_rec_train.tar -C ./ch_lite/
```
识别模型转inference模型与检测的方式相同如下
```bash linenums="1"
# -c 后面设置训练算法的yml配置文件
# -o 配置可选参数
# Global.pretrained_model 参数设置待转换的训练模型地址,不用添加文件后缀 .pdmodel.pdopt或.pdparams。
# Global.save_inference_dir参数设置转换的模型将保存的地址。
python3 tools/export_model.py -c configs/rec/ch_ppocr_v2.0/rec_chinese_lite_train_v2.0.yml -o Global.pretrained_model=./ch_lite/ch_ppocr_mobile_v2.0_rec_train/best_accuracy Global.save_inference_dir=./inference/rec_crnn/
```
**注意:** 如果您是在自己的数据集上训练的模型,并且调整了中文字符的字典文件,请注意修改配置文件中的`character_dict_path`是否是所需要的字典文件。
转换成功后,在目录下有三个文件:
```bash linenums="1"
/inference/rec_crnn/
├── inference.pdiparams # 识别inference模型的参数文件
├── inference.pdiparams.info # 识别inference模型的参数信息可忽略
└── inference.pdmodel # 识别inference模型的program文件
```
### 方向分类模型转inference模型
下载方向分类模型:
```bash linenums="1"
wget -P ./ch_lite/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_train.tar && tar xf ./ch_lite/ch_ppocr_mobile_v2.0_cls_train.tar -C ./ch_lite/
```
方向分类模型转inference模型与检测的方式相同如下
```bash linenums="1"
# -c 后面设置训练算法的yml配置文件
# -o 配置可选参数
# Global.pretrained_model 参数设置待转换的训练模型地址,不用添加文件后缀 .pdmodel.pdopt或.pdparams。
# Global.save_inference_dir参数设置转换的模型将保存的地址。
python3 tools/export_model.py -c configs/cls/cls_mv3.yml -o Global.pretrained_model=./ch_lite/ch_ppocr_mobile_v2.0_cls_train/best_accuracy Global.save_inference_dir=./inference/cls/
```
转换成功后,在目录下有三个文件:
```text linenums="1"
/inference/cls/
├── inference.pdiparams # 分类inference模型的参数文件
├── inference.pdiparams.info # 分类inference模型的参数信息可忽略
└── inference.pdmodel # 分类inference模型的program文件
```
## 二、文本检测模型推理
文本检测模型推理默认使用DB模型的配置参数。当不使用DB模型时在推理时需要通过传入相应的参数进行算法适配细节参考下文。
### 1. 超轻量中文检测模型推理
超轻量中文检测模型推理,可以执行如下命令:
```bash linenums="1"
# 下载超轻量中文检测模型:
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_det_infer.tar
tar xf ch_ppocr_mobile_v2.0_det_infer.tar
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./ch_ppocr_mobile_v2.0_det_infer/"
```
可视化文本检测结果默认保存到`./inference_results`文件夹里面,结果文件的名称前缀为`det_res`。结果示例如下:
![](./images/det_res_00018069.jpg)
通过参数`limit_type`和`det_limit_side_len`来对图片的尺寸进行限制,
`limit_type`可选参数为[`max`, `min`]
`det_limit_size_len` 为正整数一般设置为32 的倍数比如960。
参数默认设置为`limit_type='max', det_limit_side_len=960`。表示网络输入图像的最长边不能超过960
如果超过这个值会对图像做等宽比的resize操作确保最长边为`det_limit_side_len`。
设置为`limit_type='min', det_limit_side_len=960` 则表示限制图像的最短边为960。
如果输入图片的分辨率比较大而且想使用更大的分辨率预测可以设置det_limit_side_len 为想要的值比如1216
```bash linenums="1"
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/1.jpg" --det_model_dir="./inference/det_db/" --det_limit_type=max --det_limit_side_len=1216
```
如果想使用CPU进行预测执行命令如下
```bash linenums="1"
python3 tools/infer/predict_det.py --image_dir="./doc/imgs/1.jpg" --det_model_dir="./inference/det_db/" --use_gpu=False
```
### 2. DB文本检测模型推理
首先将DB文本检测训练过程中保存的模型转换成inference model。以基于Resnet50_vd骨干网络在ICDAR2015英文数据集训练的模型为例 [模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_db_v2.0_train.tar) ),可以使用如下命令进行转换:
```bash linenums="1"
python3 tools/export_model.py -c configs/det/det_r50_vd_db.yml -o Global.pretrained_model=./det_r50_vd_db_v2.0_train/best_accuracy Global.save_inference_dir=./inference/det_db
```
DB文本检测模型推理可以执行如下命令
```bash linenums="1"
python3 tools/infer/predict_det.py --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_db/"
```
可视化文本检测结果默认保存到`./inference_results`文件夹里面,结果文件的名称前缀为`det_res`。结果示例如下:
![img](./images/det_res_img_10_db.jpg)
**注意**由于ICDAR2015数据集只有1000张训练图像且主要针对英文场景所以上述模型对中文文本图像检测效果会比较差。
### 3. EAST文本检测模型推理
首先将EAST文本检测训练过程中保存的模型转换成inference model。以基于Resnet50_vd骨干网络在ICDAR2015英文数据集训练的模型为例 [模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_east_v2.0_train.tar) ),可以使用如下命令进行转换:
```bash linenums="1"
python3 tools/export_model.py -c configs/det/det_r50_vd_east.yml -o Global.pretrained_model=./det_r50_vd_east_v2.0_train/best_accuracy Global.save_inference_dir=./inference/det_east
```
**EAST文本检测模型推理需要设置参数`--det_algorithm="EAST"`**,可以执行如下命令:
```bash linenums="1"
python3 tools/infer/predict_det.py --det_algorithm="EAST" --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_east/"
```
可视化文本检测结果默认保存到`./inference_results`文件夹里面,结果文件的名称前缀为`det_res`。结果示例如下:
![img](./images/det_res_img_10_east.jpg)
**注意**本代码库中EAST后处理Locality-Aware NMS有python和c++两种版本c++版速度明显快于python版。由于c++版本nms编译版本问题只有python3.5环境下会调用c++版nms其他情况将调用python版nms。
### 4. SAST文本检测模型推理
#### (1). 四边形文本检测模型ICDAR2015
首先将SAST文本检测训练过程中保存的模型转换成inference model。以基于Resnet50_vd骨干网络在ICDAR2015英文数据集训练的模型为例([模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_sast_icdar15_v2.0_train.tar)),可以使用如下命令进行转换:
```bash linenums="1"
python3 tools/export_model.py -c configs/det/det_r50_vd_sast_icdar15.yml -o Global.pretrained_model=./det_r50_vd_sast_icdar15_v2.0_train/best_accuracy Global.save_inference_dir=./inference/det_sast_ic15
```
**SAST文本检测模型推理需要设置参数`--det_algorithm="SAST"`**,可以执行如下命令:
```bash linenums="1"
python3 tools/infer/predict_det.py --det_algorithm="SAST" --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_sast_ic15/"
```
可视化文本检测结果默认保存到`./inference_results`文件夹里面,结果文件的名称前缀为`det_res`。结果示例如下:
![](./images/det_res_img_10_sast.jpg)
#### (2). 弯曲文本检测模型Total-Text
首先将SAST文本检测训练过程中保存的模型转换成inference model。以基于Resnet50_vd骨干网络在Total-Text英文数据集训练的模型为例[模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/det_r50_vd_sast_totaltext_v2.0_train.tar)),可以使用如下命令进行转换:
```bash linenums="1"
python3 tools/export_model.py -c configs/det/det_r50_vd_sast_totaltext.yml -o Global.pretrained_model=./det_r50_vd_sast_totaltext_v2.0_train/best_accuracy Global.save_inference_dir=./inference/det_sast_tt
```
SAST文本检测模型推理需要设置参数`--det_algorithm="SAST"`,同时,还需要增加参数`--det_sast_polygon=True`,可以执行如下命令:
```bash linenums="1"
python3 tools/infer/predict_det.py --det_algorithm="SAST" --image_dir="./doc/imgs_en/img623.jpg" --det_model_dir="./inference/det_sast_tt/" --det_sast_polygon=True
```
可视化文本检测结果默认保存到`./inference_results`文件夹里面,结果文件的名称前缀为`det_res`。结果示例如下:
![img](./images/det_res_img623_sast.jpg)
**注意**本代码库中SAST后处理Locality-Aware NMS有python和c++两种版本c++版速度明显快于python版。由于c++版本nms编译版本问题只有python3.5环境下会调用c++版nms其他情况将调用python版nms。
## 三、文本识别模型推理
下面将介绍超轻量中文识别模型推理、基于CTC损失的识别模型推理和基于Attention损失的识别模型推理。对于中文文本识别建议优先选择基于CTC损失的识别模型实践中也发现基于Attention损失的效果不如基于CTC损失的识别模型。此外如果训练时修改了文本的字典请参考下面的自定义文本识别字典的推理。
### 1. 超轻量中文识别模型推理
超轻量中文识别模型推理,可以执行如下命令:
```bash linenums="1"
# 下载超轻量中文识别模型:
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_rec_infer.tar
tar xf ch_ppocr_mobile_v2.0_rec_infer.tar
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words/ch/word_4.jpg" --rec_model_dir="ch_ppocr_mobile_v2.0_rec_infer"
```
![img](./images/word_4.jpg)
执行命令后,上面图像的预测结果(识别的文本和得分)会打印到屏幕上,示例如下:
```bash linenums="1"
Predicts of ./doc/imgs_words/ch/word_4.jpg:('实力活力', 0.98458153)
```
### 2. 基于CTC损失的识别模型推理
我们以 CRNN 为例介绍基于CTC损失的识别模型推理。 Rosetta 使用方式类似不用设置识别算法参数rec_algorithm。
首先将 CRNN 文本识别训练过程中保存的模型转换成inference model。以基于Resnet34_vd骨干网络使用MJSynth和SynthText两个英文文本识别合成数据集训练
的模型为例( [模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/en/rec_r34_vd_none_bilstm_ctc_v2.0_train.tar) ),可以使用如下命令进行转换:
```bash linenums="1"
python3 tools/export_model.py -c configs/rec/rec_r34_vd_none_bilstm_ctc.yml -o Global.pretrained_model=./rec_r34_vd_none_bilstm_ctc_v2.0_train/best_accuracy Global.save_inference_dir=./inference/rec_crnn
```
CRNN 文本识别模型推理,可以执行如下命令:
```bash linenums="1"
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_336.png" --rec_model_dir="./inference/rec_crnn/" --rec_image_shape="3, 32, 100" --rec_char_dict_path="./ppocr/utils/ic15_dict.txt"
```
![img](./images/word_336.png)
执行命令后,上面图像的识别结果如下:
```bash linenums="1"
Predicts of ./doc/imgs_words_en/word_336.png:('super', 0.9999073)
```
**注意**:由于上述模型是参考[DTRB](https://arxiv.org/abs/1904.01906)文本识别训练和评估流程,与超轻量级中文识别模型训练有两方面不同:
- 训练时采用的图像分辨率不同,训练上述模型采用的图像分辨率是[332100],而中文模型训练时,为了保证长文本的识别效果,训练时采用的图像分辨率是[3, 32, 320]。预测推理程序默认的形状参数是训练中文采用的图像分辨率,即[3, 32, 320]。因此这里推理上述英文模型时需要通过参数rec_image_shape设置识别图像的形状。
- 字符列表DTRB论文中实验只是针对26个小写英文本母和10个数字进行实验总共36个字符。所有大小字符都转成了小写字符不在上面列表的字符都忽略认为是空格。因此这里没有输入字符字典而是通过如下命令生成字典.因此在推理时需要设置参数rec_char_dict_path指定为英文字典"./ppocr/utils/ic15_dict.txt"。
```python linenums="1"
self.character_str = "0123456789abcdefghijklmnopqrstuvwxyz"
dict_character = list(self.character_str)
```
### 3. 基于SRN损失的识别模型推理
基于SRN损失的识别模型需要额外设置识别算法参数 --rec_algorithm="SRN"。
同时需要保证预测shape与训练时一致 --rec_image_shape="1, 64, 256"
```bash linenums="1"
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_336.png" \
--rec_model_dir="./inference/srn/" \
--rec_image_shape="1, 64, 256" \
--rec_char_dict_path="./ppocr/utils/ic15_dict.txt" \
--rec_algorithm="SRN"
```
### 4. 自定义文本识别字典的推理
如果训练时修改了文本的字典在使用inference模型预测时需要通过`--rec_char_dict_path`指定使用的字典路径
```bash linenums="1"
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words_en/word_336.png" --rec_model_dir="./your inference model" --rec_image_shape="3, 32, 100" --rec_char_dict_path="your text dict path"
```
### 5. 多语言模型的推理
如果您需要预测的是其他语言模型在使用inference模型预测时需要通过`--rec_char_dict_path`指定使用的字典路径, 同时为了得到正确的可视化结果,
需要通过 `--vis_font_path` 指定可视化的字体路径,`doc/fonts/` 路径下有默认提供的小语种字体,例如韩文识别:
```bash linenums="1"
python3 tools/infer/predict_rec.py --image_dir="./doc/imgs_words/korean/1.jpg" --rec_model_dir="./your inference model" --rec_char_dict_path="ppocr/utils/dict/korean_dict.txt" --vis_font_path="doc/fonts/korean.ttf"
```
![img](./images/1.jpg)
执行命令后,上图的预测结果为:
``` text
Predicts of ./doc/imgs_words/korean/1.jpg:('바탕으로', 0.9948904)
```
## 四、方向分类模型推理
下面将介绍方向分类模型推理。
### 1. 方向分类模型推理
方向分类模型推理,可以执行如下命令:
```bash linenums="1"
# 下载超轻量中文方向分类器模型:
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/ch/ch_ppocr_mobile_v2.0_cls_infer.tar
tar xf ch_ppocr_mobile_v2.0_cls_infer.tar
python3 tools/infer/predict_cls.py --image_dir="./doc/imgs_words/ch/word_4.jpg" --cls_model_dir="ch_ppocr_mobile_v2.0_cls_infer"
```
![img](./images/word_1.jpg)
执行命令后,上面图像的预测结果(分类的方向和得分)会打印到屏幕上,示例如下:
```bash linenums="1"
Predicts of ./doc/imgs_words/ch/word_4.jpg:['0', 0.9999982]
```
## 五、文本检测、方向分类和文字识别串联推理
### 1. 超轻量中文OCR模型推理
在执行预测时,需要通过参数`image_dir`指定单张图像或者图像集合的路径、参数`det_model_dir`,`cls_model_dir`和`rec_model_dir`分别指定检测方向分类和识别的inference模型路径。参数`use_angle_cls`用于控制是否启用方向分类模型。`use_mp`表示是否使用多进程。`total_process_num`表示在使用多进程时的进程数。可视化识别结果默认保存到 ./inference_results 文件夹里面。
```bash linenums="1"
# 使用方向分类器
python3 tools/infer/predict_system.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./inference/det_db/" --cls_model_dir="./inference/cls/" --rec_model_dir="./inference/rec_crnn/" --use_angle_cls=true
# 不使用方向分类器
python3 tools/infer/predict_system.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./inference/det_db/" --rec_model_dir="./inference/rec_crnn/" --use_angle_cls=false
# 使用多进程
python3 tools/infer/predict_system.py --image_dir="./doc/imgs/00018069.jpg" --det_model_dir="./inference/det_db/" --rec_model_dir="./inference/rec_crnn/" --use_angle_cls=false --use_mp=True --total_process_num=6
```
执行命令后,识别结果图像如下:
![](./images/system_res_00018069_v3.jpg)
### 2. 其他模型推理
如果想尝试使用其他检测算法或者识别算法,请参考上述文本检测模型推理和文本识别模型推理,更新相应配置和模型。
**注意由于检测框矫正逻辑的局限性暂不支持使用SAST弯曲文本检测模型使用参数`--det_sast_polygon=True`时)进行模型串联。**
下面给出基于EAST文本检测和STAR-Net文本识别执行命令
```bash linenums="1"
python3 tools/infer/predict_system.py --image_dir="./doc/imgs_en/img_10.jpg" --det_model_dir="./inference/det_east/" --det_algorithm="EAST" --rec_model_dir="./inference/starnet/" --rec_image_shape="3, 32, 100" --rec_char_dict_path="./ppocr/utils/ic15_dict.txt"
```
执行命令后,识别结果图像如下:
![img](./images/img_10_east_starnet.jpg)
## 六、参数解释
更多关于预测过程的参数解释如下所示。
- 全局信息
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| image_dir | str | 无,必须显式指定 | 图像或者文件夹路径 |
| vis_font_path | str | "./doc/fonts/simfang.ttf" | 用于可视化的字体路径 |
| drop_score | float | 0.5 | 识别得分小于该值的结果会被丢弃,不会作为返回结果 |
| use_pdserving | bool | False | 是否使用Paddle Serving进行预测 |
| warmup | bool | False | 是否开启warmup在统计预测耗时的时候可以使用这种方法 |
| draw_img_save_dir | str | "./inference_results" | 系统串联预测OCR结果的保存文件夹 |
| save_crop_res | bool | False | 是否保存OCR的识别文本图像 |
| crop_res_save_dir | str | "./output" | 保存OCR识别出来的文本图像路径 |
| use_mp | bool | False | 是否开启多进程预测 |
| total_process_num | int | 6 | 开启的进城数,`use_mp`为`True`时生效 |
| process_id | int | 0 | 当前进程的id号无需自己修改 |
| benchmark | bool | False | 是否开启benchmark对预测速度、显存占用等进行统计 |
| save_log_path | str | "./log_output/" | 开启`benchmark`时,日志结果的保存文件夹 |
| show_log | bool | True | 是否显示预测中的日志信息 |
| use_onnx | bool | False | 是否开启onnx预测 |
- 预测引擎相关
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| use_gpu | bool | True | 是否使用GPU进行预测 |
| ir_optim | bool | True | 是否对计算图进行分析与优化,开启后可以加速预测过程 |
| use_tensorrt | bool | False | 是否开启tensorrt |
| min_subgraph_size | int | 15 | tensorrt中最小子图size当子图的size大于该值时才会尝试对该子图使用trt engine计算 |
| precision | str | fp32 | 预测的精度,支持`fp32`, `fp16`, `int8` 3种输入 |
| enable_mkldnn | bool | True | 是否开启mkldnn |
| cpu_threads | int | 10 | 开启mkldnn时cpu预测的线程数 |
- 文本检测模型相关
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| det_algorithm | str | "DB" | 文本检测算法名称,目前支持`DB`, `EAST`, `SAST`, `PSE` |
| det_model_dir | str | xx | 检测inference模型路径 |
| det_limit_side_len | int | 960 | 检测的图像边长限制 |
| det_limit_type | str | "max" | 检测的变成限制类型,目前支持`min`, `max``min`表示保证图像最短边不小于`det_limit_side_len``max`表示保证图像最长边不大于`det_limit_side_len` |
其中DB算法相关参数如下
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| det_db_thresh | float | 0.3 | DB输出的概率图中得分大于该阈值的像素点才会被认为是文字像素点 |
| det_db_box_thresh | float | 0.6 | 检测结果边框内,所有像素点的平均得分大于该阈值时,该结果会被认为是文字区域 |
| det_db_unclip_ratio | float | 1.5 | `Vatti clipping`算法的扩张系数,使用该方法对文字区域进行扩张 |
| max_batch_size | int | 10 | 预测的batch size |
| use_dilation | bool | False | 是否对分割结果进行膨胀以获取更优检测效果 |
| det_db_score_mode | str | "fast" | DB的检测结果得分计算方法支持`fast`和`slow``fast`是根据polygon的外接矩形边框内的所有像素计算平均得分`slow`是根据原始polygon内的所有像素计算平均得分计算速度相对较慢一些但是更加准确一些。 |
EAST算法相关参数如下
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| det_east_score_thresh | float | 0.8 | EAST后处理中score map的阈值 |
| det_east_cover_thresh | float | 0.1 | EAST后处理中文本框的平均得分阈值 |
| det_east_nms_thresh | float | 0.2 | EAST后处理中nms的阈值 |
SAST算法相关参数如下
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| det_sast_score_thresh | float | 0.5 | SAST后处理中的得分阈值 |
| det_sast_nms_thresh | float | 0.5 | SAST后处理中nms的阈值 |
| det_sast_polygon | bool | False | 是否多边形检测弯曲文本场景如Total-Text设置为True |
PSE算法相关参数如下
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| det_pse_thresh | float | 0.0 | 对输出图做二值化的阈值 |
| det_pse_box_thresh | float | 0.85 | 对box进行过滤的阈值低于此阈值的丢弃 |
| det_pse_min_area | float | 16 | box的最小面积低于此阈值的丢弃 |
| det_pse_box_type | str | "box" | 返回框的类型box:四点坐标poly: 弯曲文本的所有点坐标 |
| det_pse_scale | int | 1 | 输入图像相对于进后处理的图的比例,如`640*640`的图像,网络输出为`160*160`scale为2的情况下进后处理的图片shape为`320*320`。这个值调大可以加快后处理速度,但是会带来精度的下降 |
- 文本识别模型相关
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| rec_algorithm | str | "CRNN" | 文本识别算法名称,目前支持`CRNN`, `SRN`, `RARE`, `NETR`, `SAR` |
| rec_model_dir | str | 无,如果使用识别模型,该项是必填项 | 识别inference模型路径 |
| rec_image_shape | list | [3, 32, 320] | 识别时的图像尺寸, |
| rec_batch_num | int | 6 | 识别的batch size |
| max_text_length | int | 25 | 识别结果最大长度,在`SRN`中有效 |
| rec_char_dict_path | str | "./ppocr/utils/ppocr_keys_v1.txt" | 识别的字符字典文件 |
| use_space_char | bool | True | 是否包含空格,如果为`True`,则会在最后字符字典中补充`空格`字符 |
- 端到端文本检测与识别模型相关
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| e2e_algorithm | str | "PGNet" | 端到端算法名称,目前支持`PGNet` |
| e2e_model_dir | str | 无,如果使用端到端模型,该项是必填项 | 端到端模型inference模型路径 |
| e2e_limit_side_len | int | 768 | 端到端的输入图像边长限制 |
| e2e_limit_type | str | "max" | 端到端的边长限制类型,目前支持`min`, `max``min`表示保证图像最短边不小于`e2e_limit_side_len``max`表示保证图像最长边不大于`e2e_limit_side_len` |
| e2e_pgnet_score_thresh | float | 0.5 | 端到端得分阈值,小于该阈值的结果会被丢弃 |
| e2e_char_dict_path | str | "./ppocr/utils/ic15_dict.txt" | 识别的字典文件路径 |
| e2e_pgnet_valid_set | str | "totaltext" | 验证集名称,目前支持`totaltext`, `partvgg`,不同数据集对应的后处理方式不同,与训练过程保持一致即可 |
| e2e_pgnet_mode | str | "fast" | PGNet的检测结果得分计算方法支持`fast`和`slow``fast`是根据polygon的外接矩形边框内的所有像素计算平均得分`slow`是根据原始polygon内的所有像素计算平均得分计算速度相对较慢一些但是更加准确一些。 |
- 方向分类器模型相关
| 参数名称 | 类型 | 默认值 | 含义 |
| :--: | :--: | :--: | :--: |
| use_angle_cls | bool | False | 是否使用方向分类器 |
| cls_model_dir | str | 无,如果需要使用,则必须显式指定路径 | 方向分类器inference模型路径 |
| cls_image_shape | list | [3, 48, 192] | 预测尺度 |
| label_list | list | ['0', '180'] | class id对应的角度值 |
| cls_batch_num | int | 6 | 方向分类器预测的batch size |
| cls_thresh | float | 0.9 | 预测阈值模型预测结果为180度且得分大于该阈值时认为最终预测结果为180度需要翻转 |
## 七、FAQ
- 如果是使用paddle2.0之前版本的代码导出的`inference模型`,则其文件名为`model`与`params`分别对应paddle2.0或者之后版本导出的`inference.pdmodel`与`inference.pdiparams`不过目前PaddleOCR的release分支已经不支持paddle2.0之前版本导出的inference 模型如果希望使用需要使用develop分支静态图分支的代码与文档。

View File

@@ -0,0 +1,147 @@
---
comments: true
hide:
- toc
---
# Visual Studio 2022 Community CMake Compilation Guide
PaddleOCR has been tested on Windows using `Visual Studio 2022 Community`. Microsoft started supporting direct `CMake` project management from `Visual Studio 2017`, but it wasn't fully stable and reliable until `2019`. If you want to use CMake for project management and compilation, we recommend using `Visual Studio 2022`.
**All examples below assume the working directory is **`D:\projects\cpp`**.**
## 1. Environment Preparation
### 1.1 Install Required Dependencies
- Visual Studio 2019 or newer
- CUDA 10.2, cuDNN 7+ (only required for the GPU version of the prediction library). Additionally, the NVIDIA Computing Toolkit must be installed, and the NVIDIA cuDNN library must be downloaded.
- CMake 3.22+
Ensure that the above dependencies are installed before proceeding. In this tutorial the Community Edition of `VS2022` was used.
### 1.2 Download PaddlePaddle C++ Prediction Library and OpenCV
#### 1.2.1 Download PaddlePaddle C++ Prediction Library
PaddlePaddle C++ prediction libraries offer different precompiled versions for various `CPU` and `CUDA` configurations. Download the appropriate version from: [C++ Prediction Library Download List](https://www.paddlepaddle.org.cn/inference/master/guides/install/download_lib.html#windows)
After extraction, the `D:\projects\paddle_inference` directory should contain:
```
paddle_inference
├── paddle # Core Paddle library and header files
|
├── third_party # Third-party dependencies and headers
|
└── version.txt # Version and compilation information
```
#### 1.2.2 Install and Configure OpenCV
1. Download OpenCV for Windows from the [official release page](https://github.com/opencv/opencv/releases).
2. Run the downloaded executable and extract OpenCV to a specified directory, e.g., `D:\projects\cpp\opencv`.
#### 1.2.3 Download PaddleOCR Code
```bash
git clone https://github.com/PaddlePaddle/Paddle.git
git checkout develop
```
## 2. Running the Project
### Step 1: Create a Visual Studio Project
Once CMake is installed, open the `cmake-gui` application. Specify the source code directory in the first input box and the build output directory in the second input box.
![step1](./images/cmake_step1.png)
### Step 2: Run CMake Configuration
Click the `Configure` button at the bottom of the interface. The first time you run it, a prompt will appear asking for the Visual Studio configuration. Select your `Visual Studio` version and set the target platform to `x64`. Click `Finish` to start the configuration process.
![step2](./images/cmake_step2.jpg)
The first run will result in errors, which is expected. You now need to configure OpenCV and the prediction library.
- **For CPU version**, configure the following variables:
- `OPENCV_DIR`: Path to the OpenCV `lib` folder
- `OpenCV_DIR`: Same as `OPENCV_DIR`
- `PADDLE_LIB`: Path to the `paddle_inference` folder
- **For GPU version**, configure additional variables:
- `CUDA_LIB`: CUDA path, e.g., `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\lib\x64`
- `CUDNN_LIB`: Path to extracted CuDNN library, e.g., `D:\CuDNN-8.9.7.29`
- `TENSORRT_DIR`: Path to extracted TensorRT, e.g., `D:\TensorRT-8.0.1.6`
- `WITH_GPU`: Check this option
- `WITH_TENSORRT`: Check this option
Example configuration:
![step3](./images/cmake_step3.jpg)
Once configured, click `Configure` again.
**Note:**
1. If using `openblas`, uncheck `WITH_MKL`.
2. If you encounter the error `unable to access 'https://github.com/LDOUBLEV/AutoLog.git/': gnutls_handshake() failed`, update `deploy/cpp_infer/external-cmake/auto-log.cmake` to use `https://gitee.com/Double_V/AutoLog`.
### Step 3: Generate Visual Studio Project
Click `Generate` to create the `.sln` file for the Visual Studio project.
![step4](./images/cmake_step4.jpg)
Click `Open Project` to launch the project in Visual Studio. The interface should look like this:
![step5](./images/vs_step1.jpg)
Before building the solution, perform the following steps:
1. Change `Debug` to `Release` mode.
2. Download [dirent.h](https://paddleocr.bj.bcebos.com/deploy/cpp_infer/cpp_files/dirent.h) and copy it to the Visual Studio include directory, e.g., `C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\include`.
Click `Build -> Build Solution`. Once completed, the `ppocr.exe` file should appear in the `build/Release/` folder.
Before running, copy the following files to `build/Release/`:
1. `paddle_inference/paddle/lib/paddle_inference.dll`
2. `paddle_inference/paddle/lib/common.dll`
3. `paddle_inference/third_party/install/mklml/lib/mklml.dll`
4. `paddle_inference/third_party/install/mklml/lib/libiomp5md.dll`
5. `paddle_inference/third_party/install/onednn/lib/mkldnn.dll`
6. `opencv/build/x64/vc15/bin/opencv_world455.dll`
7. If using the `openblas` version, also copy `paddle_inference/third_party/install/openblas/lib/openblas.dll`.
### Step 4: Run the Prediction
The compiled executable is located in the `build/Release/` directory. Open `cmd` and navigate to `D:\projects\cpp\PaddleOCR\deploy\cpp_infer\`:
```
cd /d D:\projects\cpp\PaddleOCR\deploy\cpp_infer
```
Run the prediction using `ppocr.exe`. For more usage details, refer to the to the [Instructions](./cpp_infer.en.md) section of running the demo.
```shell
# Switch terminal encoding to UTF-8
CHCP 65001
# If using PowerShell, run this command before execution to fix character encoding issues:
$OutputEncoding = [console]::InputEncoding = [console]::OutputEncoding = New-Object System.Text.UTF8Encoding
# Execute prediction
.\build\Release\ppocr.exe system --det_model_dir=D:\projects\cpp\ch_PP-OCRv2_det_slim_quant_infer --rec_model_dir=D:\projects\cpp\ch_PP-OCRv2_rec_slim_quant_infer --image_dir=D:\projects\cpp\PaddleOCR\doc\imgs\11.jpg
```
<br>
## Sample result:
![result](./images/result.jpg)
## FAQ
- **Issue:** Application fails to start with error `(0xc0000142)` and `cmd` output shows `You are using Paddle compiled with TensorRT, but TensorRT dynamic library is not found.`
- **Solution:** Copy all `.dll` files from the `TensorRT` directory's `lib` folder into the `release` directory and try running it again.

View File

@@ -0,0 +1,139 @@
---
comments: true
hide:
- toc
---
# Visual Studio 2019 Community CMake 编译指南
PaddleOCR在Windows 平台下基于`Visual Studio 2019 Community` 进行了测试。微软从`Visual Studio 2017`开始即支持直接管理`CMake`跨平台编译项目,但是直到`2019`才提供了稳定和完全的支持所以如果你想使用CMake管理项目编译构建我们推荐你使用`Visual Studio 2019`环境下构建。
**下面所有示例以工作目录为 `D:\projects\cpp`演示**
## 1. 环境准备
### 1.1 安装必须环境
- Visual Studio 2019
- CUDA 10.2cudnn 7+ 仅在使用GPU版本的预测库时需要
- CMake 3.22+
请确保系统已经安装好上述基本软件,我们使用的是`VS2019`的社区版。
### 1.2 下载 PaddlePaddle C++ 预测库和 Opencv
#### 1.2.1 下载 PaddlePaddle C++ 预测库
PaddlePaddle C++ 预测库针对不同的`CPU``CUDA`版本提供了不同的预编译版本,请根据实际情况下载: [C++预测库下载列表](https://www.paddlepaddle.org.cn/inference/master/guides/install/download_lib.html#windows)
解压后`D:\projects\paddle_inference`目录包含内容为:
```
paddle_inference
├── paddle # paddle核心库和头文件
|
├── third_party # 第三方依赖库和头文件
|
└── version.txt # 版本和编译信息
```
#### 1.2.2 安装配置OpenCV
1. 在OpenCV官网下载适用于Windows平台的Opencv [下载地址](https://github.com/opencv/opencv/releases)
2. 运行下载的可执行文件将OpenCV解压至指定目录`D:\projects\cpp\opencv`
#### 1.2.3 下载PaddleOCR代码
```bash linenums="1"
git clone -b dygraph https://github.com/PaddlePaddle/PaddleOCR
```
## 2. 开始运行
### Step1: 构建Visual Studio项目
cmake安装完后后系统里会有一个cmake-gui程序打开cmake-gui在第一个输入框处填写源代码路径第二个输入框处填写编译输出路径
![step1](./images/cmake_step1.png)
### Step2: 执行cmake配置
点击界面下方的`Configure`按钮第一次点击会弹出提示框进行Visual Studio配置如下图选择你的Visual Studio版本即可目标平台选择x64。然后点击`finish`按钮即开始自动执行配置。
![step2](./images/cmake_step2.jpg)
第一次执行会报错这是正常现象接下来进行Opencv和预测库的配置
- cpu版本仅需考虑OPENCV_DIR、OpenCV_DIR、PADDLE_LIB三个参数
- OPENCV_DIR填写opencv lib文件夹所在位置
- OpenCV_DIR同填写opencv lib文件夹所在位置
- PADDLE_LIBpaddle_inference文件夹所在位置
- GPU版本在cpu版本的基础上还需填写以下变量
CUDA_LIB、CUDNN_LIB、TENSORRT_DIR、WITH_GPU、WITH_TENSORRT
- CUDA_LIB: CUDA地址如 `C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.2\lib\x64`
- CUDNN_LIB: 和CUDA_LIB一致
- TENSORRT_DIRTRT下载后解压缩的位置如 `D:\TensorRT-8.0.1.6`
- WITH_GPU: 打钩
- WITH_TENSORRT打勾
配置好的截图如下
![step3](./images/cmake_step3.jpg)
配置完成后,再次点击`Configure`按钮。
**注意:**
1. 如果使用的是`openblas`版本,请把`WITH_MKL`勾去掉
2. 遇到报错 `unable to access 'https://github.com/LDOUBLEV/AutoLog.git/': gnutls_handshake() failed: The TLS connection was non-properly terminated.` 将 `deploy/cpp_infer/external-cmake/auto-log.cmake` 中的github地址改为 <https://gitee.com/Double_V/AutoLog> 地址即可。
### Step3: 生成Visual Studio 项目
点击`Generate`按钮即可生成Visual Studio 项目的sln文件。
![step4](./images/cmake_step4.jpg)
点击`Open Project`按钮即可在Visual Studio 中打开项目。打开后截图如下
![step5](./images/vs_step1.jpg)
在开始生成解决方案之前,执行下面步骤:
1. 将`Debug`改为`Release`
2. 下载[dirent.h](https://paddleocr.bj.bcebos.com/deploy/cpp_infer/cpp_files/dirent.h),并拷贝到 Visual Studio 的 include 文件夹下,如`C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\VS\include`。
点击`生成->生成解决方案`,即可在`build/Release/`文件夹下看见`ppocr.exe`文件。
运行之前,将下面文件拷贝到`build/Release/`文件夹下
1. `paddle_inference/paddle/lib/paddle_inference.dll`
2. `paddle_inference/third_party/install/onnxruntime/lib/onnxruntime.dll`
3. `paddle_inference/third_party/install/paddle2onnx/lib/paddle2onnx.dll`
4. `opencv/build/x64/vc15/bin/opencv_world455.dll`
5. 如果使用openblas版本的预测库还需要拷贝 `paddle_inference/third_party/install/openblas/lib/openblas.dll`
### Step4: 预测
上述`Visual Studio 2019`编译产出的可执行文件在`build/Release/`目录下,打开`cmd`,并切换到`D:\projects\cpp\PaddleOCR\deploy\cpp_infer\`
```bash linenums="1"
cd /d D:\projects\cpp\PaddleOCR\deploy\cpp_infer
```
可执行文件`ppocr.exe`即为样例的预测程序,其主要使用方法如下,更多使用方法可以参考[说明文档](./cpp_infer.md)`运行demo`部分。
```bash linenums="1"
# 切换终端编码为utf8
CHCP 65001
# 执行预测
.\build\Release\ppocr.exe system --det_model_dir=D:\projects\cpp\ch_PP-OCRv2_det_slim_quant_infer --rec_model_dir=D:\projects\cpp\ch_PP-OCRv2_rec_slim_quant_infer --image_dir=D:\projects\cpp\PaddleOCR\doc\imgs\11.jpg
```
识别结果如下
![result](./images/result.jpg)
## FAQ
- 运行时,弹窗报错提示`应用程序无法正常启动(0xc0000142)`,并且`cmd`窗口内提示`You are using Paddle compiled with TensorRT, but TensorRT dynamic library is not found.`把tensort目录下的lib里面的所有dll文件复制到release目录下再次运行即可。