This commit is contained in:
303
deploy/slim/auto_compression/README.md
Normal file
303
deploy/slim/auto_compression/README.md
Normal file
@@ -0,0 +1,303 @@
|
||||
# OCR模型自动压缩示例
|
||||
|
||||
目录：
|
||||
- [OCR模型自动压缩示例](#ocr模型自动压缩示例)
|
||||
- [1. 简介](#1-简介)
|
||||
- [2. Benchmark](#2-benchmark)
|
||||
- [PPOCRV4\_det](#ppocrv4_det)
|
||||
- [PPOCRV4\_rec](#ppocrv4_rec)
|
||||
- [3. 自动压缩流程](#3-自动压缩流程)
|
||||
- [3.1 准备环境](#31-准备环境)
|
||||
- [3.2 准备数据集](#32-准备数据集)
|
||||
- [3.2.1 PPOCRV4\_det\_server数据集预处理](#321-ppocrv4_det_server数据集预处理)
|
||||
- [3.3 准备预测模型](#33-准备预测模型)
|
||||
- [4.预测部署](#4预测部署)
|
||||
- [4.1 Paddle Inference 验证性能](#41-paddle-inference-验证性能)
|
||||
- [4.1.1 使用测试脚本进行批量测试：](#411-使用测试脚本进行批量测试)
|
||||
- [4.1.2 基于压缩模型进行基于GPU的批量测试：](#412-基于压缩模型进行基于gpu的批量测试)
|
||||
- [4.1.3 基于压缩前模型进行基于GPU的批量测试：](#413-基于压缩前模型进行基于gpu的批量测试)
|
||||
- [4.1.4 基于压缩模型进行基于CPU的批量测试：](#414-基于压缩模型进行基于cpu的批量测试)
|
||||
- [4.2 PaddleLite端侧部署](#42-paddlelite端侧部署)
|
||||
- [5.FAQ](#5faq)
|
||||
- [5.1 报错找不到模型文件或者数据集文件](#51-报错找不到模型文件或者数据集文件)
|
||||
- [5.2 软件环境一致，硬件不同导致精度差异很大？](#52-软件环境一致硬件不同导致精度差异很大)
|
||||
|
||||
|
||||
## 1. 简介
|
||||
本示例将以图像分类模型PPOCRV3为例，介绍如何使用PaddleOCR中Inference部署模型进行自动压缩。本示例使用的自动压缩策略为量化训练和蒸馏。
|
||||
|
||||
## 2. Benchmark
|
||||
|
||||
### PPOCRV4_det
|
||||
| 模型 | 策略 | Metric(hmean) | GPU 耗时(ms) | ARM CPU 耗时(ms) | 配置文件 | Inference模型 |
|
||||
|:------:|:------:|:------:|:------:|:------:|:------:|:------:|
|
||||
| PP-OCRv4_mobile_det | Baseline | 72.71 | 5.7 | 92.0 | - | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/det/ch_PP-OCRv4_det_infer.tar) |
|
||||
| PP-OCRv4_mobile_det | 量化+蒸馏 | 71.10 | 2.3 | 94.1 | [Config](./configs/ppocrv4/ppocrv4_det_qat_dist.yaml) | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/det/det_mobile_qat_3090.zip) |
|
||||
| PP-OCRv4_server_det | Baseline | 79.82 | 32.6 | 844.7 | - | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/det/ch_PP-OCRv4_det_server_infer.tar) |
|
||||
| PP-OCRv4_server_det | 量化+蒸馏 | 79.27 | 12.3 | 635.0 | [Config](./configs/ppocrv4/ppocrv4_rec_server_qat_dist.yaml) | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/det/det_server_qat_3090.zip) |
|
||||
> - GPU测试环境：RTX 3090, cuda11.7+tensorrt8.4.2.4+paddle2.5
|
||||
> - CPU测试环境：Intel(R) Xeon(R) Gold 6226R，使用12线程测试
|
||||
> - PP-OCRv4_server_det在不完整的数据集上测试，数据处理流程参考[ppocrv4_det_server数据集预处理](#321-ppocrv4_det_server数据集预处理)，仅为了展示自动压缩效果，指标并不具有参考性，模型真实表现请参考[PPOCRV4介绍](../../../doc/doc_ch/PP-OCRv4_introduction.md)
|
||||
|
||||
| 模型 | 策略 | Metric(hmean) | GPU 耗时(ms) | ARM CPU 耗时(ms) | 配置文件 | Inference模型 |
|
||||
|:------:|:------:|:------:|:------:|:------:|:------:|:------:|
|
||||
| PP-OCRv4_mobile_det | Baseline | 72.71 | 4.7 | 198.4 | - | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/det/ch_PP-OCRv4_det_infer.tar) |
|
||||
| PP-OCRv4_mobile_det | 量化+蒸馏 | 71.38 | 3.3 | 205.2 | [Config](./configs/ppocrv4/ppocrv4_det_qat_dist.yaml) | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/det/det_server_qat_v100.zip) |
|
||||
| PP-OCRv4_server_det | Baseline | 79.77 | 50.0 | 2159.4 | - | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/det/ch_PP-OCRv4_det_server_infer.tar) |
|
||||
| PP-OCRv4_server_det | 量化+蒸馏 | 79.81 | 42.4 | 1834.8 | [Config](./configs/ppocrv4/ppocrv4_rec_server_qat_dist.yaml) | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/det/det_server_qat_v100.zip) |
|
||||
> - GPU测试环境：Tesla V100, cuda11.7+tensorrt8.4.2.4+paddle2.5.2
|
||||
> - CPU测试环境：Intel(R) Xeon(R) Gold 6271C，使用12线程测试
|
||||
> - PP-OCRv4_server_det在不完整的数据集上测试，数据处理流程参考[ppocrv4_det_server数据集预处理](#321-ppocrv4_det_server数据集预处理)，仅为了展示自动压缩效果，指标并不具有参考性，模型真实表现请参考[PPOCRV4介绍](../../../doc/doc_ch/PP-OCRv4_introduction.md)
|
||||
|
||||
### PPOCRV4_rec
|
||||
| 模型 | 策略 | Metric(accuracy) | GPU 耗时(ms) | ARM CPU 耗时(ms) | 配置文件 | Inference模型 |
|
||||
|:------:|:------:|:------:|:------:|:------:|:------:|:------:|
|
||||
| 中文PPOCRV4-rec_mobile | Baseline | 78.92 | 1.7 | 33.3 | - | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/rec/ch_PP-OCRv4_rec_infer.tar.gz) |
|
||||
| 中文PPOCRV4-rec_mobile | 量化+蒸馏 | 78.41 | 1.4 | 34.0 | [Config](./configs/ppocrv4/ppocrv4_rec_qat_dist.yaml) | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/rec/rec_mobile_qat.tar.gz) |
|
||||
| 中文PPOCRV4-rec_server | Baseline | 81.62 | 4.0 | 62.5 | - | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/rec/ch_PP-OCRv4_rec_server_infer.tar.gz) |
|
||||
| 中文PPOCRV4-rec_server | 量化+蒸馏 | 81.03 | 2.0 | 64.4 | [Config](./configs/ppocrv4/ppocrv4_rec_server_qat_dist.yaml) | [Model](https://paddle-ocr-models.bj.bcebos.com/ppocrv4_qat/rec/rec_server_qat.tar.gz) |
|
||||
> - GPU测试环境：Tesla V100, cuda11.2+tensorrt8.0.3.4+paddle2.5
|
||||
> - CPU测试环境：Intel(R) Xeon(R) Gold 6271C，使用12线程测试
|
||||
|
||||
|
||||
## 3. 自动压缩流程
|
||||
|
||||
### 3.1 准备环境
|
||||
|
||||
- PaddlePaddle == 2.5 （可从[Paddle官网](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/install/pip/linux-pip.html)下载安装）
|
||||
- PaddleSlim == 2.5
|
||||
- PaddleOCR == develop
|
||||
|
||||
安装paddlepaddle：
|
||||
```shell
|
||||
# CPU
|
||||
python -m pip install paddlepaddle==2.5.1 -i https://pypi.tuna.tsinghua.edu.cn/simple
|
||||
# GPU 以Ubuntu、CUDA 10.2为例
|
||||
python -m pip install paddlepaddle-gpu==2.5.1.post102 -f https://www.paddlepaddle.org.cn/whl/linux/mkl/avx/stable.html
|
||||
```
|
||||
|
||||
安装paddleslim 2.5：
|
||||
```shell
|
||||
pip install paddleslim@git+https://gitee.com/paddlepaddle/PaddleSlim.git@release/2.5
|
||||
```
|
||||
|
||||
安装其他依赖：
|
||||
```shell
|
||||
pip install scikit-image imgaug
|
||||
```
|
||||
|
||||
|
||||
下载PaddleOCR:
|
||||
```shell
|
||||
git clone -b release/2.7 https://github.com/PaddlePaddle/PaddleOCR.git
|
||||
cd PaddleOCR/
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 3.2 准备数据集
|
||||
公开数据集可参考[OCR数据集](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.7/doc/doc_ch/dataset/ocr_datasets.md)，然后根据程序运行过程中提示放置到对应位置。
|
||||
|
||||
#### 3.2.1 PPOCRV4_det_server数据集预处理
|
||||
PPOCRV4_det_server在使用原始数据集推理时，默认将输入图像的最小边缩放到736，然而原始数据集中存在一些长宽比很大的图像，比如13:1，此时再进行缩放就会导致长边的尺寸非常大，在实验过程中发现最大的长边尺寸有10000+，这导致在构建TensorRT子图的时候显存不足。
|
||||
|
||||
为了能顺利跑通自动压缩的流程，展示自动压缩的效果，因此需要对原始数据集进行预处理，将长宽比过大的图像进行剔除，处理脚本可见[ppocrv4_det_server_dataset_process.py](./ppocrv4_det_server_dataset_process.py)。
|
||||
|
||||
|
||||
> 注意：使用不同的数据集需要修改配置文件中`dataset`中数据路径和数据处理部分。
|
||||
|
||||
### 3.3 准备预测模型
|
||||
预测模型的格式为：`model.pdmodel` 和 `model.pdiparams`两个，带`pdmodel`的是模型文件，带`pdiparams`后缀的是权重文件。
|
||||
|
||||
> 注：其他像`__model__`和`__params__`分别对应`model.pdmodel` 和 `model.pdiparams`文件。
|
||||
|
||||
可在[PaddleOCR模型库](https://github.com/PaddlePaddle/PaddleOCR/blob/release/2.7/doc/doc_ch/models_list.md)中直接获取Inference模型，具体可参考下方获取中文PPOCRV4模型示例：
|
||||
|
||||
```shell
|
||||
https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_rec_infer.tar
|
||||
tar -xf ch_PP-OCRv4_rec_infer.tar
|
||||
```
|
||||
|
||||
```shell
|
||||
wget https://paddleocr.bj.bcebos.com/PP-OCRv4/chinese/ch_PP-OCRv4_det_infer.tar
|
||||
tar -xf ch_PP-OCRv4_det_infer.tar
|
||||
```
|
||||
|
||||
蒸馏量化自动压缩示例通过run.py脚本启动，会使用接口 ```paddleslim.auto_compression.AutoCompression``` 对模型进行量化训练和蒸馏。配置config文件中模型路径、数据集路径、蒸馏、量化和训练等部分的参数，配置完成后便可开始自动压缩。
|
||||
|
||||
**单卡启动**
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0
|
||||
python run.py --save_dir='./save_quant_ppocrv4_det/' --config_path='./configs/ppocrv4/ppocrv4_det_qat_dist.yaml'
|
||||
```
|
||||
|
||||
**多卡启动**
|
||||
|
||||
若训练任务中包含大量训练数据，如果使用单卡训练，会非常耗时，使用分布式训练可以达到几乎线性的加速比。
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0,1,2,3
|
||||
python -m paddle.distributed.launch run.py --save_dir='./save_quant_ppocrv4_det/' --config_path='./configs/ppocrv4/ppocrv4_det_qat_dist.yaml'
|
||||
```
|
||||
多卡训练指的是将训练任务按照一定方法拆分到多个训练节点完成数据读取、前向计算、反向梯度计算等过程，并将计算出的梯度上传至服务节点。服务节点在收到所有训练节点传来的梯度后，会将梯度聚合并更新参数。最后将参数发送给训练节点，开始新一轮的训练。多卡训练一轮训练能训练```batch size * num gpus```的数据，比如单卡的```batch size```为32，单轮训练的数据量即32，而四卡训练的```batch size```为32，单轮训练的数据量为128。
|
||||
|
||||
注意 ```learning rate``` 与 ```batch size``` 呈线性关系，这里单卡 ```batch size``` 8，对应的 ```learning rate``` 为0.00005，那么如果 ```batch size``` 增大4倍改为32，```learning rate``` 也需乘以4；多卡时 ```batch size``` 为8，```learning rate``` 需乘上卡数。所以改变 ```batch size``` 或改变训练卡数都需要对应修改 ```learning rate```。
|
||||
|
||||
|
||||
**验证精度**
|
||||
|
||||
根据训练log可以看到模型验证的精度，若需再次验证精度，修改配置文件```./configs/ppocrv3_det_qat_dist.yaml```中所需验证模型的文件夹路径及模型和参数名称```model_dir, model_filename, params_filename```，然后使用以下命令进行验证：
|
||||
|
||||
```shell
|
||||
export CUDA_VISIBLE_DEVICES=0
|
||||
python eval.py --config_path='./configs/ppocrv3_det_qat_dist.yaml'
|
||||
```
|
||||
|
||||
## 4.预测部署
|
||||
|
||||
#### 4.1 Paddle Inference 验证性能
|
||||
|
||||
输出的量化模型也是静态图模型，静态图模型在GPU上可以使用TensorRT进行加速，在CPU上可以使用MKLDNN进行加速。
|
||||
|
||||
TensorRT预测环境配置：
|
||||
1. 如果使用 TesorRT 预测引擎，需安装 ```WITH_TRT=ON``` 的Paddle，上述paddle下载的2.5满足打开TensorRT编译的要求。
|
||||
2. 使用TensorRT预测需要进一步安装TensorRT，安装TensorRT的方式参考[TensorRT安装说明](../../../docs/deployment/installtrt.md)。
|
||||
|
||||
以下字段用于配置预测参数：
|
||||
|
||||
| 参数名 | 含义 |
|
||||
|:------:|:------:|
|
||||
| model_path | inference 模型文件所在目录，该目录下需要有文件 .pdmodel 和 .pdiparams 两个文件 |
|
||||
| model_filename | inference_model_dir文件夹下的模型文件名称 |
|
||||
| params_filename | inference_model_dir文件夹下的参数文件名称 |
|
||||
| dataset_config | 数据集配置的config |
|
||||
| image_file | 待测试单张图片的路径，如果设置image_file，则dataset_config将无效。 |
|
||||
| device | 预测时的设备，可选：`CPU`, `GPU`。 |
|
||||
| use_trt | 是否使用 TesorRT 预测引擎，在device为```GPU```时生效。 |
|
||||
| use_mkldnn | 是否启用```MKL-DNN```加速库，注意```use_mkldnn```，在device为```CPU```时生效。 |
|
||||
| cpu_threads | CPU预测时，使用CPU线程数量，默认10 |
|
||||
| precision | 预测时精度，可选：`fp32`, `fp16`, `int8`。 |
|
||||
|
||||
|
||||
准备好预测模型，并且修改dataset_config中数据集路径为正确的路径后，启动测试：
|
||||
|
||||
##### 4.1.1 使用测试脚本进行批量测试：
|
||||
|
||||
我们提供两个脚本文件用于测试模型自动化压缩的效果，分别是[test_ocr_det.sh](./test_ocr_det.sh)和[test_ocr_rec.sh](./test_ocr_rec.sh)，这两个脚本都接收一个`model_type`参数，用于区分是测试mobile模型还是server模型，可选参数为`mobile`和`server`，使用示例：
|
||||
|
||||
```shell
|
||||
# 测试mobile模型
|
||||
bash test_ocr_det.sh mobile
|
||||
bash test_ocr_rec.sh mobile
|
||||
# 测试server模型
|
||||
bash test_ocr_det.sh server
|
||||
bash test_ocr_rec.sh server
|
||||
```
|
||||
|
||||
##### 4.1.2 基于压缩模型进行基于GPU的批量测试：
|
||||
|
||||
```shell
|
||||
cd deploy/slim/auto_compression
|
||||
python test_ocr.py \
|
||||
--model_path save_quant_ppocrv4_det \
|
||||
--config_path configs/ppocrv4/ppocrv4_det_qat_dist.yaml \
|
||||
--device GPU \
|
||||
--use_trt True \
|
||||
--precision int8
|
||||
```
|
||||
|
||||
|
||||
##### 4.1.3 基于压缩前模型进行基于GPU的批量测试：
|
||||
|
||||
```shell
|
||||
cd deploy/slim/auto_compression
|
||||
python test_ocr.py \
|
||||
--model_path ch_PP-OCRv4_det_infer \
|
||||
--config_path configs/ppocrv4/ppocrv4_rec_det_dist.yaml \
|
||||
--device GPU \
|
||||
--use_trt True \
|
||||
--precision int8
|
||||
```
|
||||
|
||||
|
||||
##### 4.1.4 基于压缩模型进行基于CPU的批量测试：
|
||||
|
||||
- MKLDNN预测：
|
||||
|
||||
```shell
|
||||
cd deploy/slim/auto_compression
|
||||
python test_ocr.py \
|
||||
--model_path save_quant_ppocrv4_det \
|
||||
--config_path configs/ppocrv4/ppocrv4_det_qat_dist.yaml \
|
||||
--device GPU \
|
||||
--use_trt True \
|
||||
--use_mkldnn=True \
|
||||
--precision=int8 \
|
||||
--cpu_threads=10
|
||||
```
|
||||
|
||||
### 4.2 PaddleLite端侧部署
|
||||
PaddleLite端侧部署可参考：
|
||||
- [Paddle Lite部署](https://github.com/PaddlePaddle/PaddleOCR/tree/9cdab61d909eb595af849db885c257ca8c74cb57/deploy/lite)
|
||||
|
||||
## 5.FAQ
|
||||
|
||||
### 5.1 报错找不到模型文件或者数据集文件
|
||||
|
||||
如果在推理或者跑ACT时报错找不到模型文件或者数据集文件，可以检查一下配置文件中的路径是否正确，以det_mobile为例，配置文件中的指定模型路径的配置信息如下：
|
||||
|
||||
```yaml
|
||||
Global:
|
||||
model_dir: ./models/ch_PP-OCRv4_det_infer
|
||||
model_filename: inference.pdmodel
|
||||
params_filename: inference.pdiparams
|
||||
```
|
||||
指定训练集验证集路径的配置信息如下：
|
||||
|
||||
```yaml
|
||||
Train:
|
||||
dataset:
|
||||
name: SimpleDataSet
|
||||
data_dir: datasets/chinese
|
||||
label_file_list:
|
||||
- datasets/chinese/zhongce_training_fix_1.6k.txt
|
||||
- datasets/chinese/label_train_all_f4_part2.txt
|
||||
- datasets/chinese/label_train_all_f4_part3.txt
|
||||
- datasets/chinese/label_train_all_f4_part4.txt
|
||||
- datasets/chinese/label_train_all_f4_part5.txt
|
||||
- datasets/chinese/synth_en_my_clip.txt
|
||||
- datasets/chinese/synth_ch_my_clip.txt
|
||||
- datasets/chinese/synth_en_my_largeword_clip.txt
|
||||
Eval:
|
||||
dataset:
|
||||
name: SimpleDataSet
|
||||
data_dir: datasets/v4_4_test_dataset
|
||||
label_file_list:
|
||||
- datasets/v4_4_test_dataset/label.txt
|
||||
```
|
||||
|
||||
### 5.2 软件环境一致，硬件不同导致精度差异很大？
|
||||
|
||||
这种情况是正常的，TensorRT针对不同的硬件设备有着不同的优化方法，同一种优化策略在不同硬件上可能有着截然不同的表现，以本实验的ppocrv4_det_server为举例。截取[test_ocr.py](./test_ocr.py)中的一部分代码如下所示：
|
||||
```python
|
||||
if args.precision == 'int8' and "ppocrv4_det_server_qat_dist.yaml" in args.config_path:
|
||||
# Use the following settings only when the hardware is a Tesla V100. If you are using
|
||||
# a RTX 3090, use the settings in the else branch.
|
||||
pred_cfg.enable_tensorrt_engine(
|
||||
workspace_size=1 << 30,
|
||||
max_batch_size=1,
|
||||
min_subgraph_size=30,
|
||||
precision_mode=precision_map[args.precision],
|
||||
use_static=True,
|
||||
use_calib_mode=False, )
|
||||
pred_cfg.exp_disable_tensorrt_ops(["elementwise_add"])
|
||||
else:
|
||||
pred_cfg.enable_tensorrt_engine(
|
||||
workspace_size=1 << 30,
|
||||
max_batch_size=1,
|
||||
min_subgraph_size=4,
|
||||
precision_mode=precision_map[args.precision],
|
||||
use_static=True,
|
||||
use_calib_mode=False, )
|
||||
```
|
||||
当硬件为RTX 3090的时候，使用else分支中的策略即可得到正常的结果，但是当硬件是Tesla V100的时候，必须使用if分支中的策略才能保证量化后精度不下降，具体结果参考[benchmark](#2-benchmark)。
|
||||
@@ -0,0 +1,163 @@
|
||||
Global:
|
||||
model_type: det
|
||||
model_dir: ./models/ch_PP-OCRv4_det_infer
|
||||
model_filename: inference.pdmodel
|
||||
params_filename: inference.pdiparams
|
||||
algorithm: DB
|
||||
|
||||
Distillation:
|
||||
alpha: 1.0
|
||||
loss: l2
|
||||
|
||||
QuantAware:
|
||||
use_pact: false
|
||||
activation_bits: 8
|
||||
is_full_quantize: false
|
||||
onnx_format: false
|
||||
activation_quantize_type: moving_average_abs_max
|
||||
weight_quantize_type: channel_wise_abs_max
|
||||
not_quant_pattern:
|
||||
- skip_quant
|
||||
quantize_op_types:
|
||||
- conv2d
|
||||
weight_bits: 8
|
||||
|
||||
TrainConfig:
|
||||
epochs: 2
|
||||
eval_iter: 200
|
||||
learning_rate:
|
||||
type: CosineAnnealingDecay
|
||||
learning_rate: 0.000005
|
||||
optimizer_builder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
weight_decay: 5.0e-05
|
||||
|
||||
PostProcess:
|
||||
name: DBPostProcess
|
||||
thresh: 0.3
|
||||
box_thresh: 0.6
|
||||
max_candidates: 1000
|
||||
unclip_ratio: 1.5
|
||||
|
||||
Metric:
|
||||
name: DetMetric
|
||||
main_indicator: hmean
|
||||
|
||||
Train:
|
||||
dataset:
|
||||
name: SimpleDataSet
|
||||
data_dir: datasets/chinese
|
||||
label_file_list:
|
||||
- datasets/chinese/zhongce_training_fix_1.6k.txt
|
||||
- datasets/chinese/label_train_all_f4_part2.txt
|
||||
- datasets/chinese/label_train_all_f4_part3.txt
|
||||
- datasets/chinese/label_train_all_f4_part4.txt
|
||||
- datasets/chinese/label_train_all_f4_part5.txt
|
||||
- datasets/chinese/synth_en_my_clip.txt
|
||||
- datasets/chinese/synth_ch_my_clip.txt
|
||||
- datasets/chinese/synth_en_my_largeword_clip.txt
|
||||
ratio_list:
|
||||
- 0.3
|
||||
- 0.2
|
||||
- 0.1
|
||||
- 0.2
|
||||
- 0.2
|
||||
- 0.1
|
||||
- 0.2
|
||||
- 0.2
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
img_mode: BGR
|
||||
channel_first: false
|
||||
- DetLabelEncode: null
|
||||
- IaaAugment:
|
||||
augmenter_args:
|
||||
- type: Fliplr
|
||||
args:
|
||||
p: 0.5
|
||||
- type: Affine
|
||||
args:
|
||||
rotate:
|
||||
- -10
|
||||
- 10
|
||||
- type: Resize
|
||||
args:
|
||||
size:
|
||||
- 0.5
|
||||
- 3
|
||||
- EastRandomCropData:
|
||||
size:
|
||||
- 960
|
||||
- 960
|
||||
max_tries: 50
|
||||
keep_ratio: true
|
||||
- MakeBorderMap:
|
||||
shrink_ratio: 0.4
|
||||
thresh_min: 0.3
|
||||
thresh_max: 0.7
|
||||
- MakeShrinkMap:
|
||||
shrink_ratio: 0.4
|
||||
min_text_size: 8
|
||||
- NormalizeImage:
|
||||
scale: 1./255.
|
||||
mean:
|
||||
- 0.485
|
||||
- 0.456
|
||||
- 0.406
|
||||
std:
|
||||
- 0.229
|
||||
- 0.224
|
||||
- 0.225
|
||||
order: hwc
|
||||
- ToCHWImage: null
|
||||
- KeepKeys:
|
||||
keep_keys:
|
||||
- image
|
||||
- threshold_map
|
||||
- threshold_mask
|
||||
- shrink_map
|
||||
- shrink_mask
|
||||
loader:
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
batch_size_per_card: 4
|
||||
num_workers: 4
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: SimpleDataSet
|
||||
data_dir: datasets/v4_4_test_dataset
|
||||
label_file_list:
|
||||
- datasets/v4_4_test_dataset/label.txt
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
img_mode: BGR
|
||||
channel_first: false
|
||||
- DetLabelEncode: null
|
||||
- DetResizeForTest:
|
||||
limit_side_len: 960
|
||||
limit_type: max
|
||||
- NormalizeImage:
|
||||
scale: 1./255.
|
||||
mean:
|
||||
- 0.485
|
||||
- 0.456
|
||||
- 0.406
|
||||
std:
|
||||
- 0.229
|
||||
- 0.224
|
||||
- 0.225
|
||||
order: hwc
|
||||
- ToCHWImage: null
|
||||
- KeepKeys:
|
||||
keep_keys:
|
||||
- image
|
||||
- shape
|
||||
- polys
|
||||
- ignore_tags
|
||||
loader:
|
||||
shuffle: false
|
||||
drop_last: false
|
||||
batch_size_per_card: 1
|
||||
num_workers: 10
|
||||
@@ -0,0 +1,161 @@
|
||||
Global:
|
||||
model_type: det
|
||||
model_dir: ./models/ch_PP-OCRv4_det_server_infer
|
||||
model_filename: inference.pdmodel
|
||||
params_filename: inference.pdiparams
|
||||
algorithm: DB
|
||||
|
||||
Distillation:
|
||||
alpha: 1.0
|
||||
loss: l2
|
||||
|
||||
QuantAware:
|
||||
use_pact: false
|
||||
activation_bits: 8
|
||||
is_full_quantize: false
|
||||
onnx_format: false
|
||||
activation_quantize_type: moving_average_abs_max
|
||||
weight_quantize_type: channel_wise_abs_max
|
||||
not_quant_pattern:
|
||||
- skip_quant
|
||||
quantize_op_types:
|
||||
- conv2d
|
||||
weight_bits: 8
|
||||
|
||||
TrainConfig:
|
||||
epochs: 1
|
||||
eval_iter: 200
|
||||
learning_rate:
|
||||
type: CosineAnnealingDecay
|
||||
learning_rate: 0.000005
|
||||
optimizer_builder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
weight_decay: 5.0e-05
|
||||
|
||||
PostProcess:
|
||||
name: DBPostProcess
|
||||
thresh: 0.3
|
||||
box_thresh: 0.6
|
||||
max_candidates: 1000
|
||||
unclip_ratio: 1.5
|
||||
|
||||
Metric:
|
||||
name: DetMetric
|
||||
main_indicator: hmean
|
||||
|
||||
Train:
|
||||
dataset:
|
||||
name: SimpleDataSet
|
||||
data_dir: datasets/chinese
|
||||
label_file_list:
|
||||
- datasets/chinese/zhongce_training_fix_1.6k.txt
|
||||
- datasets/chinese/label_train_all_f4_part2.txt
|
||||
- datasets/chinese/label_train_all_f4_part3.txt
|
||||
- datasets/chinese/label_train_all_f4_part4.txt
|
||||
- datasets/chinese/label_train_all_f4_part5.txt
|
||||
- datasets/chinese/synth_en_my_clip.txt
|
||||
- datasets/chinese/synth_ch_my_clip.txt
|
||||
- datasets/chinese/synth_en_my_largeword_clip.txt
|
||||
ratio_list:
|
||||
- 0.3
|
||||
- 0.2
|
||||
- 0.1
|
||||
- 0.2
|
||||
- 0.2
|
||||
- 0.1
|
||||
- 0.2
|
||||
- 0.2
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
img_mode: BGR
|
||||
channel_first: false
|
||||
- DetLabelEncode: null
|
||||
- IaaAugment:
|
||||
augmenter_args:
|
||||
- type: Fliplr
|
||||
args:
|
||||
p: 0.5
|
||||
- type: Affine
|
||||
args:
|
||||
rotate:
|
||||
- -10
|
||||
- 10
|
||||
- type: Resize
|
||||
args:
|
||||
size:
|
||||
- 0.5
|
||||
- 3
|
||||
- EastRandomCropData:
|
||||
size:
|
||||
- 960
|
||||
- 960
|
||||
max_tries: 50
|
||||
keep_ratio: true
|
||||
- MakeBorderMap:
|
||||
shrink_ratio: 0.4
|
||||
thresh_min: 0.3
|
||||
thresh_max: 0.7
|
||||
- MakeShrinkMap:
|
||||
shrink_ratio: 0.4
|
||||
min_text_size: 8
|
||||
- NormalizeImage:
|
||||
scale: 1./255.
|
||||
mean:
|
||||
- 0.485
|
||||
- 0.456
|
||||
- 0.406
|
||||
std:
|
||||
- 0.229
|
||||
- 0.224
|
||||
- 0.225
|
||||
order: hwc
|
||||
- ToCHWImage: null
|
||||
- KeepKeys:
|
||||
keep_keys:
|
||||
- image
|
||||
- threshold_map
|
||||
- threshold_mask
|
||||
- shrink_map
|
||||
- shrink_mask
|
||||
loader:
|
||||
shuffle: true
|
||||
drop_last: false
|
||||
batch_size_per_card: 2
|
||||
num_workers: 8
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: SimpleDataSet
|
||||
data_dir: datasets/v4_4_test_dataset_small
|
||||
label_file_list:
|
||||
- datasets/v4_4_test_dataset_small/label.txt
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
img_mode: BGR
|
||||
channel_first: false
|
||||
- DetLabelEncode: null
|
||||
- DetResizeForTest: null
|
||||
- NormalizeImage:
|
||||
scale: 1./255.
|
||||
mean:
|
||||
- 0.485
|
||||
- 0.456
|
||||
- 0.406
|
||||
std:
|
||||
- 0.229
|
||||
- 0.224
|
||||
- 0.225
|
||||
order: hwc
|
||||
- ToCHWImage: null
|
||||
- KeepKeys:
|
||||
keep_keys:
|
||||
- image
|
||||
- shape
|
||||
- polys
|
||||
- ignore_tags
|
||||
loader:
|
||||
shuffle: false
|
||||
drop_last: false
|
||||
batch_size_per_card: 1
|
||||
num_workers: 2
|
||||
@@ -0,0 +1,115 @@
|
||||
Global:
|
||||
model_dir: ./models/ch_PP-OCRv4_rec_infer
|
||||
model_filename: inference.pdmodel
|
||||
params_filename: inference.pdiparams
|
||||
model_type: rec
|
||||
algorithm: SVTR
|
||||
character_dict_path: ./ppocr_keys_v1.txt
|
||||
max_text_length: &max_text_length 25
|
||||
use_space_char: true
|
||||
|
||||
Distillation:
|
||||
alpha: [1.0, 1.0]
|
||||
loss: ['skd', 'l2']
|
||||
node:
|
||||
- ['softmax_11.tmp_0']
|
||||
- ['linear_170.tmp_1']
|
||||
|
||||
QuantAware:
|
||||
use_pact: false
|
||||
activation_bits: 8
|
||||
is_full_quantize: false
|
||||
onnx_format: false
|
||||
activation_quantize_type: moving_average_abs_max
|
||||
weight_quantize_type: channel_wise_abs_max
|
||||
not_quant_pattern:
|
||||
- skip_quant
|
||||
quantize_op_types:
|
||||
- conv2d
|
||||
weight_bits: 8
|
||||
|
||||
TrainConfig:
|
||||
epochs: 1
|
||||
eval_iter: 1000
|
||||
logging_iter: 100
|
||||
learning_rate:
|
||||
type: CosineAnnealingDecay
|
||||
learning_rate: 0.00001
|
||||
optimizer_builder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
weight_decay: 5.0e-05
|
||||
|
||||
PostProcess:
|
||||
name: CTCLabelDecode
|
||||
|
||||
Metric:
|
||||
name: RecMetric
|
||||
main_indicator: acc
|
||||
ignore_space: False
|
||||
|
||||
Train:
|
||||
dataset:
|
||||
name: MultiScaleDataSet
|
||||
ds_width: false
|
||||
data_dir: datasets/real_data/
|
||||
label_file_list:
|
||||
- datasets/real_data/train_list.txt
|
||||
ext_op_transform_idx: 1
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
img_mode: BGR
|
||||
channel_first: false
|
||||
- RecConAug:
|
||||
prob: 0.5
|
||||
ext_data_num: 2
|
||||
image_shape: [48, 320, 3]
|
||||
max_text_length: *max_text_length
|
||||
- RecAug:
|
||||
- MultiLabelEncode:
|
||||
gtc_encode: NRTRLabelEncode
|
||||
- KeepKeys:
|
||||
keep_keys:
|
||||
- image
|
||||
- label_ctc
|
||||
- label_gtc
|
||||
- length
|
||||
- valid_ratio
|
||||
sampler:
|
||||
name: MultiScaleSampler
|
||||
scales: [[320, 32], [320, 48], [320, 64]]
|
||||
first_bs: &bs 64
|
||||
fix_bs: false
|
||||
divided_factor: [8, 16] # w, h
|
||||
is_training: True
|
||||
loader:
|
||||
shuffle: true
|
||||
batch_size_per_card: *bs
|
||||
drop_last: true
|
||||
num_workers: 8
|
||||
Eval:
|
||||
dataset:
|
||||
name: SimpleDataSet
|
||||
data_dir: datasets/real_data/
|
||||
label_file_list:
|
||||
- datasets/real_data/val_list.txt
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
img_mode: BGR
|
||||
channel_first: false
|
||||
- MultiLabelEncode:
|
||||
gtc_encode: NRTRLabelEncode
|
||||
- RecResizeImg:
|
||||
image_shape: [3, 48, 320]
|
||||
- KeepKeys:
|
||||
keep_keys:
|
||||
- image
|
||||
- label_ctc
|
||||
- label_gtc
|
||||
- length
|
||||
- valid_ratio
|
||||
loader:
|
||||
shuffle: false
|
||||
drop_last: false
|
||||
batch_size_per_card: 1
|
||||
num_workers: 4
|
||||
@@ -0,0 +1,113 @@
|
||||
Global:
|
||||
model_dir: ./models/ch_PP-OCRv4_rec_server_infer
|
||||
model_filename: inference.pdmodel
|
||||
params_filename: inference.pdiparams
|
||||
model_type: rec
|
||||
algorithm: SVTR
|
||||
character_dict_path: ./ppocr_keys_v1.txt
|
||||
max_text_length: &max_text_length 25
|
||||
use_space_char: true
|
||||
|
||||
Distillation:
|
||||
alpha: 1.0
|
||||
loss: 'l2'
|
||||
|
||||
QuantAware:
|
||||
use_pact: false
|
||||
activation_bits: 8
|
||||
is_full_quantize: false
|
||||
onnx_format: false
|
||||
activation_quantize_type: moving_average_abs_max
|
||||
weight_quantize_type: channel_wise_abs_max
|
||||
not_quant_pattern:
|
||||
- skip_quant
|
||||
quantize_op_types:
|
||||
- conv2d
|
||||
weight_bits: 8
|
||||
|
||||
TrainConfig:
|
||||
epochs: 1
|
||||
eval_iter: 1000
|
||||
logging_iter: 100
|
||||
learning_rate:
|
||||
type: CosineAnnealingDecay
|
||||
learning_rate: 0.00001
|
||||
optimizer_builder:
|
||||
optimizer:
|
||||
type: Adam
|
||||
weight_decay: 5.0e-05
|
||||
|
||||
PostProcess:
|
||||
name: CTCLabelDecode
|
||||
|
||||
Metric:
|
||||
name: RecMetric
|
||||
main_indicator: acc
|
||||
ignore_space: False
|
||||
|
||||
Train:
|
||||
dataset:
|
||||
name: MultiScaleDataSet
|
||||
ds_width: false
|
||||
data_dir: datasets/real_data/
|
||||
ext_op_transform_idx: 1
|
||||
label_file_list:
|
||||
- datasets/real_data/train_list.txt
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
img_mode: BGR
|
||||
channel_first: false
|
||||
- RecConAug:
|
||||
prob: 0.5
|
||||
ext_data_num: 2
|
||||
image_shape: [48, 320, 3]
|
||||
max_text_length: *max_text_length
|
||||
- RecAug:
|
||||
- MultiLabelEncode:
|
||||
gtc_encode: NRTRLabelEncode
|
||||
- KeepKeys:
|
||||
keep_keys:
|
||||
- image
|
||||
- label_ctc
|
||||
- label_gtc
|
||||
- length
|
||||
- valid_ratio
|
||||
sampler:
|
||||
name: MultiScaleSampler
|
||||
scales: [[320, 32], [320, 48], [320, 64]]
|
||||
first_bs: &bs 64
|
||||
fix_bs: false
|
||||
divided_factor: [8, 16] # w, h
|
||||
is_training: True
|
||||
loader:
|
||||
shuffle: true
|
||||
batch_size_per_card: *bs
|
||||
drop_last: true
|
||||
num_workers: 8
|
||||
|
||||
Eval:
|
||||
dataset:
|
||||
name: SimpleDataSet
|
||||
data_dir: datasets/real_data/
|
||||
label_file_list:
|
||||
- datasets/real_data/val_list.txt
|
||||
transforms:
|
||||
- DecodeImage:
|
||||
img_mode: BGR
|
||||
channel_first: false
|
||||
- MultiLabelEncode:
|
||||
gtc_encode: NRTRLabelEncode
|
||||
- RecResizeImg:
|
||||
image_shape: [3, 48, 320]
|
||||
- KeepKeys:
|
||||
keep_keys:
|
||||
- image
|
||||
- label_ctc
|
||||
- label_gtc
|
||||
- length
|
||||
- valid_ratio
|
||||
loader:
|
||||
shuffle: false
|
||||
drop_last: false
|
||||
batch_size_per_card: 1
|
||||
num_workers: 4
|
||||
6623
deploy/slim/auto_compression/ppocr_keys_v1.txt
Normal file
6623
deploy/slim/auto_compression/ppocr_keys_v1.txt
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,33 @@
|
||||
import os
|
||||
import cv2
|
||||
|
||||
dataset_path = "datasets/v4_4_test_dataset"
|
||||
annotation_file = "datasets/v4_4_test_dataset/label.txt"
|
||||
|
||||
small_images_path = "datasets/v4_4_test_dataset_small"
|
||||
new_annotation_file = "datasets/v4_4_test_dataset_small/label.txt"
|
||||
|
||||
os.makedirs(small_images_path, exist_ok=True)
|
||||
|
||||
with open(annotation_file, "r") as f:
|
||||
lines = f.readlines()
|
||||
|
||||
for i, line in enumerate(lines):
|
||||
image_name = line.split(" ")[0]
|
||||
|
||||
image_path = os.path.join(dataset_path, image_name)
|
||||
|
||||
try:
|
||||
image = cv2.imread(image_path)
|
||||
height, width, _ = image.shape
|
||||
|
||||
# 如果图像的宽度和高度都小于2000而且长宽比小于2，将其复制到新的文件夹，并保存其标注信息
|
||||
if height < 2000 and width < 2000:
|
||||
if max(height, width) / min(height, width) < 2:
|
||||
print(i, height, width, image_path)
|
||||
small_image_path = os.path.join(small_images_path, image_name)
|
||||
cv2.imwrite(small_image_path, image)
|
||||
with open(new_annotation_file, "a") as f:
|
||||
f.write(f"{line}")
|
||||
except:
|
||||
continue
|
||||
171
deploy/slim/auto_compression/run.py
Normal file
171
deploy/slim/auto_compression/run.py
Normal file
@@ -0,0 +1,171 @@
|
||||
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import logging
|
||||
from tqdm import tqdm
|
||||
import numpy as np
|
||||
import argparse
|
||||
import paddle
|
||||
from paddleslim.common import load_config as load_slim_config
|
||||
from paddleslim.common import get_logger
|
||||
from paddleslim.auto_compression import AutoCompression
|
||||
from paddleslim.common.dataloader import get_feed_vars
|
||||
|
||||
import sys
|
||||
|
||||
sys.path.append("../../../")
|
||||
from ppocr.data import build_dataloader
|
||||
from ppocr.postprocess import build_post_process
|
||||
from ppocr.metrics import build_metric
|
||||
|
||||
logger = get_logger(__name__, level=logging.INFO)
|
||||
|
||||
|
||||
def argsparser():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument(
|
||||
"--config_path",
|
||||
type=str,
|
||||
default=None,
|
||||
help="path of compression strategy config.",
|
||||
required=True,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--save_dir",
|
||||
type=str,
|
||||
default="output",
|
||||
help="directory to save compressed model.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--devices", type=str, default="gpu", help="which device used to compress."
|
||||
)
|
||||
return parser
|
||||
|
||||
|
||||
def reader_wrapper(reader, input_name):
|
||||
if isinstance(input_name, list) and len(input_name) == 1:
|
||||
input_name = input_name[0]
|
||||
|
||||
def gen(): # 形成一个字典输入
|
||||
for i, batch in enumerate(reader()):
|
||||
yield {input_name: batch[0]}
|
||||
|
||||
return gen
|
||||
|
||||
|
||||
def eval_function(exe, compiled_test_program, test_feed_names, test_fetch_list):
|
||||
post_process_class = build_post_process(all_config["PostProcess"], global_config)
|
||||
eval_class = build_metric(all_config["Metric"])
|
||||
model_type = global_config["model_type"]
|
||||
|
||||
with tqdm(
|
||||
total=len(val_loader),
|
||||
bar_format="Evaluation stage, Run batch:|{bar}| {n_fmt}/{total_fmt}",
|
||||
ncols=80,
|
||||
) as t:
|
||||
for batch_id, batch in enumerate(val_loader):
|
||||
images = batch[0]
|
||||
|
||||
try:
|
||||
(preds,) = exe.run(
|
||||
compiled_test_program,
|
||||
feed={test_feed_names[0]: images},
|
||||
fetch_list=test_fetch_list,
|
||||
)
|
||||
except:
|
||||
preds, _ = exe.run(
|
||||
compiled_test_program,
|
||||
feed={test_feed_names[0]: images},
|
||||
fetch_list=test_fetch_list,
|
||||
)
|
||||
|
||||
batch_numpy = []
|
||||
for item in batch:
|
||||
batch_numpy.append(np.array(item))
|
||||
|
||||
if model_type == "det":
|
||||
preds_map = {"maps": preds}
|
||||
post_result = post_process_class(preds_map, batch_numpy[1])
|
||||
eval_class(post_result, batch_numpy)
|
||||
elif model_type == "rec":
|
||||
post_result = post_process_class(preds, batch_numpy[1])
|
||||
eval_class(post_result, batch_numpy)
|
||||
t.update()
|
||||
metric = eval_class.get_metric()
|
||||
logger.info("metric eval ***************")
|
||||
for k, v in metric.items():
|
||||
logger.info("{}:{}".format(k, v))
|
||||
|
||||
if model_type == "det":
|
||||
return metric["hmean"]
|
||||
elif model_type == "rec":
|
||||
return metric["acc"]
|
||||
return metric
|
||||
|
||||
|
||||
def main():
|
||||
rank_id = paddle.distributed.get_rank()
|
||||
if args.devices == "gpu":
|
||||
place = paddle.CUDAPlace(rank_id)
|
||||
paddle.set_device("gpu")
|
||||
else:
|
||||
place = paddle.CPUPlace()
|
||||
paddle.set_device("cpu")
|
||||
|
||||
global all_config, global_config
|
||||
all_config = load_slim_config(args.config_path)
|
||||
|
||||
if "Global" not in all_config:
|
||||
raise KeyError(f"Key 'Global' not found in config file. \n{all_config}")
|
||||
global_config = all_config["Global"]
|
||||
|
||||
gpu_num = paddle.distributed.get_world_size()
|
||||
|
||||
train_dataloader = build_dataloader(all_config, "Train", args.devices, logger)
|
||||
|
||||
global val_loader
|
||||
val_loader = build_dataloader(all_config, "Eval", args.devices, logger)
|
||||
|
||||
if (
|
||||
isinstance(all_config["TrainConfig"]["learning_rate"], dict)
|
||||
and all_config["TrainConfig"]["learning_rate"]["type"] == "CosineAnnealingDecay"
|
||||
):
|
||||
steps = len(train_dataloader) * all_config["TrainConfig"]["epochs"]
|
||||
all_config["TrainConfig"]["learning_rate"]["T_max"] = steps
|
||||
print("total training steps:", steps)
|
||||
|
||||
global_config["input_name"] = get_feed_vars(
|
||||
global_config["model_dir"],
|
||||
global_config["model_filename"],
|
||||
global_config["params_filename"],
|
||||
)
|
||||
|
||||
ac = AutoCompression(
|
||||
model_dir=global_config["model_dir"],
|
||||
model_filename=global_config["model_filename"],
|
||||
params_filename=global_config["params_filename"],
|
||||
save_dir=args.save_dir,
|
||||
config=all_config,
|
||||
train_dataloader=reader_wrapper(train_dataloader, global_config["input_name"]),
|
||||
eval_callback=eval_function if rank_id == 0 else None,
|
||||
eval_dataloader=reader_wrapper(val_loader, global_config["input_name"]),
|
||||
)
|
||||
ac.compress()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
paddle.enable_static()
|
||||
parser = argsparser()
|
||||
args = parser.parse_args()
|
||||
main()
|
||||
292
deploy/slim/auto_compression/test_ocr.py
Normal file
292
deploy/slim/auto_compression/test_ocr.py
Normal file
@@ -0,0 +1,292 @@
|
||||
# Copyright (c) 2023 PaddlePaddle Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import argparse
|
||||
import time
|
||||
import os
|
||||
import sys
|
||||
import cv2
|
||||
import numpy as np
|
||||
import paddle
|
||||
import logging
|
||||
import numpy as np
|
||||
import argparse
|
||||
from tqdm import tqdm
|
||||
import paddle
|
||||
from paddleslim.common import load_config as load_slim_config
|
||||
from paddleslim.common import get_logger
|
||||
|
||||
import sys
|
||||
|
||||
sys.path.append("../../../")
|
||||
from ppocr.data import build_dataloader
|
||||
from ppocr.postprocess import build_post_process
|
||||
from ppocr.metrics import build_metric
|
||||
|
||||
from paddle.inference import create_predictor, PrecisionType
|
||||
from paddle.inference import Config as PredictConfig
|
||||
|
||||
logger = get_logger(__name__, level=logging.INFO)
|
||||
|
||||
|
||||
def find_images_with_bounding_size(dataset: paddle.io.Dataset):
|
||||
max_length_index = -1
|
||||
max_width_index = -1
|
||||
min_length_index = -1
|
||||
min_width_index = -1
|
||||
|
||||
max_length = float("-inf")
|
||||
max_width = float("-inf")
|
||||
min_length = float("inf")
|
||||
min_width = float("inf")
|
||||
for idx, data in enumerate(dataset):
|
||||
image = np.array(data[0])
|
||||
h, w = image.shape[-2:]
|
||||
if h > max_length:
|
||||
max_length = h
|
||||
max_length_index = idx
|
||||
if w > max_width:
|
||||
max_width = w
|
||||
max_width_index = idx
|
||||
if h < min_length:
|
||||
min_length = h
|
||||
min_length_index = idx
|
||||
if w < min_width:
|
||||
min_width = w
|
||||
min_width_index = idx
|
||||
print(f"Found max image length: {max_length}, index: {max_length_index}")
|
||||
print(f"Found max image width: {max_width}, index: {max_width_index}")
|
||||
print(f"Found min image length: {min_length}, index: {min_length_index}")
|
||||
print(f"Found min image width: {min_width}, index: {min_width_index}")
|
||||
return paddle.io.Subset(
|
||||
dataset, [max_width_index, max_length_index, min_width_index, min_length_index]
|
||||
)
|
||||
|
||||
|
||||
def load_predictor(args):
|
||||
"""
|
||||
load predictor func
|
||||
"""
|
||||
rerun_flag = False
|
||||
model_file = os.path.join(args.model_path, args.model_filename)
|
||||
params_file = os.path.join(args.model_path, args.params_filename)
|
||||
pred_cfg = PredictConfig(model_file, params_file)
|
||||
pred_cfg.enable_memory_optim()
|
||||
pred_cfg.switch_ir_optim(True)
|
||||
if args.device == "GPU":
|
||||
pred_cfg.enable_use_gpu(100, 0)
|
||||
else:
|
||||
pred_cfg.disable_gpu()
|
||||
pred_cfg.set_cpu_math_library_num_threads(args.cpu_threads)
|
||||
if args.use_mkldnn:
|
||||
pred_cfg.enable_mkldnn()
|
||||
if args.precision == "int8":
|
||||
pred_cfg.enable_mkldnn_int8({"conv2d"})
|
||||
|
||||
if global_config["model_type"] == "rec":
|
||||
# delete pass which influence the accuracy, please refer to https://github.com/PaddlePaddle/Paddle/issues/55290
|
||||
pred_cfg.delete_pass("fc_mkldnn_pass")
|
||||
pred_cfg.delete_pass("fc_act_mkldnn_fuse_pass")
|
||||
|
||||
if args.use_trt:
|
||||
# To collect the dynamic shapes of inputs for TensorRT engine
|
||||
dynamic_shape_file = os.path.join(args.model_path, "dynamic_shape.txt")
|
||||
if os.path.exists(dynamic_shape_file):
|
||||
pred_cfg.enable_tuned_tensorrt_dynamic_shape(dynamic_shape_file, True)
|
||||
print("trt set dynamic shape done!")
|
||||
precision_map = {
|
||||
"fp16": PrecisionType.Half,
|
||||
"fp32": PrecisionType.Float32,
|
||||
"int8": PrecisionType.Int8,
|
||||
}
|
||||
if (
|
||||
args.precision == "int8"
|
||||
and "ppocrv4_det_server_qat_dist.yaml" in args.config_path
|
||||
):
|
||||
# Use the following settings only when the hardware is a Tesla V100. If you are using
|
||||
# a RTX 3090, use the settings in the else branch.
|
||||
pred_cfg.enable_tensorrt_engine(
|
||||
workspace_size=1 << 30,
|
||||
max_batch_size=1,
|
||||
min_subgraph_size=30,
|
||||
precision_mode=precision_map[args.precision],
|
||||
use_static=True,
|
||||
use_calib_mode=False,
|
||||
)
|
||||
pred_cfg.exp_disable_tensorrt_ops(["elementwise_add"])
|
||||
else:
|
||||
pred_cfg.enable_tensorrt_engine(
|
||||
workspace_size=1 << 30,
|
||||
max_batch_size=1,
|
||||
min_subgraph_size=4,
|
||||
precision_mode=precision_map[args.precision],
|
||||
use_static=True,
|
||||
use_calib_mode=False,
|
||||
)
|
||||
else:
|
||||
# pred_cfg.disable_gpu()
|
||||
# pred_cfg.set_cpu_math_library_num_threads(24)
|
||||
pred_cfg.collect_shape_range_info(dynamic_shape_file)
|
||||
print("Start collect dynamic shape...")
|
||||
rerun_flag = True
|
||||
|
||||
predictor = create_predictor(pred_cfg)
|
||||
return predictor, rerun_flag
|
||||
|
||||
|
||||
def eval(args):
|
||||
"""
|
||||
eval mIoU func
|
||||
"""
|
||||
# DataLoader need run on cpu
|
||||
paddle.set_device("cpu")
|
||||
devices = paddle.device.get_device().split(":")[0]
|
||||
|
||||
val_loader = build_dataloader(all_config, "Eval", devices, logger)
|
||||
post_process_class = build_post_process(all_config["PostProcess"], global_config)
|
||||
eval_class = build_metric(all_config["Metric"])
|
||||
model_type = global_config["model_type"]
|
||||
|
||||
predictor, rerun_flag = load_predictor(args)
|
||||
|
||||
if rerun_flag:
|
||||
eval_dataset = find_images_with_bounding_size(val_loader.dataset)
|
||||
batch_sampler = paddle.io.BatchSampler(
|
||||
eval_dataset, batch_size=1, shuffle=False, drop_last=False
|
||||
)
|
||||
val_loader = paddle.io.DataLoader(
|
||||
eval_dataset, batch_sampler=batch_sampler, num_workers=4, return_list=True
|
||||
)
|
||||
|
||||
input_names = predictor.get_input_names()
|
||||
input_handle = predictor.get_input_handle(input_names[0])
|
||||
output_names = predictor.get_output_names()
|
||||
output_handle = predictor.get_output_handle(output_names[0])
|
||||
sample_nums = len(val_loader)
|
||||
predict_time = 0.0
|
||||
time_min = float("inf")
|
||||
time_max = float("-inf")
|
||||
print("Start evaluating ( total_iters: {}).".format(sample_nums))
|
||||
|
||||
for batch_id, batch in enumerate(val_loader):
|
||||
images = np.array(batch[0])
|
||||
|
||||
batch_numpy = []
|
||||
for item in batch:
|
||||
batch_numpy.append(np.array(item))
|
||||
|
||||
# ori_shape = np.array(batch_numpy).shape[-2:]
|
||||
input_handle.reshape(images.shape)
|
||||
input_handle.copy_from_cpu(images)
|
||||
start_time = time.time()
|
||||
|
||||
predictor.run()
|
||||
preds = output_handle.copy_to_cpu()
|
||||
|
||||
end_time = time.time()
|
||||
timed = end_time - start_time
|
||||
time_min = min(time_min, timed)
|
||||
time_max = max(time_max, timed)
|
||||
predict_time += timed
|
||||
|
||||
if model_type == "det":
|
||||
preds_map = {"maps": preds}
|
||||
post_result = post_process_class(preds_map, batch_numpy[1])
|
||||
eval_class(post_result, batch_numpy)
|
||||
elif model_type == "rec":
|
||||
post_result = post_process_class(preds, batch_numpy[1])
|
||||
eval_class(post_result, batch_numpy)
|
||||
|
||||
if rerun_flag:
|
||||
if batch_id == 3:
|
||||
print(
|
||||
"***** Collect dynamic shape done, Please rerun the program to get correct results. *****"
|
||||
)
|
||||
return
|
||||
if batch_id % 100 == 0:
|
||||
print("Eval iter:", batch_id)
|
||||
sys.stdout.flush()
|
||||
|
||||
metric = eval_class.get_metric()
|
||||
|
||||
time_avg = predict_time / sample_nums
|
||||
print(
|
||||
"[Benchmark] Inference time(ms): min={}, max={}, avg={}".format(
|
||||
round(time_min * 1000, 2),
|
||||
round(time_max * 1000, 1),
|
||||
round(time_avg * 1000, 1),
|
||||
)
|
||||
)
|
||||
for k, v in metric.items():
|
||||
print("{}:{}".format(k, v))
|
||||
sys.stdout.flush()
|
||||
|
||||
|
||||
def main():
|
||||
global all_config, global_config
|
||||
all_config = load_slim_config(args.config_path)
|
||||
global_config = all_config["Global"]
|
||||
eval(args)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
paddle.enable_static()
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--model_path", type=str, help="inference model filepath")
|
||||
parser.add_argument(
|
||||
"--config_path",
|
||||
type=str,
|
||||
default="./configs/ppocrv3_det_qat_dist.yaml",
|
||||
help="path of compression strategy config.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--model_filename",
|
||||
type=str,
|
||||
default="inference.pdmodel",
|
||||
help="model file name",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--params_filename",
|
||||
type=str,
|
||||
default="inference.pdiparams",
|
||||
help="params file name",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--device",
|
||||
type=str,
|
||||
default="GPU",
|
||||
choices=["CPU", "GPU"],
|
||||
help="Choose the device you want to run, it can be: CPU/GPU, default is GPU",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--precision",
|
||||
type=str,
|
||||
default="fp32",
|
||||
choices=["fp32", "fp16", "int8"],
|
||||
help="The precision of inference. It can be 'fp32', 'fp16' or 'int8'. Default is 'fp16'.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--use_trt",
|
||||
type=bool,
|
||||
default=False,
|
||||
help="Whether to use tensorrt engine or not.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--use_mkldnn", type=bool, default=False, help="Whether use mkldnn or not."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cpu_threads", type=int, default=10, help="Num of cpu threads."
|
||||
)
|
||||
args = parser.parse_args()
|
||||
main()
|
||||
88
deploy/slim/auto_compression/test_ocr_det.sh
Normal file
88
deploy/slim/auto_compression/test_ocr_det.sh
Normal file
@@ -0,0 +1,88 @@
|
||||
#!/bin/bash
|
||||
|
||||
# 本脚本用于测试PPOCRV4_det系列模型的自动压缩功能
|
||||
## 运行脚本前，请确保处于以下环境：
|
||||
## CUDA11.7+TensorRT8.4.2.4+Paddle2.5.2
|
||||
|
||||
model_type="$1"
|
||||
|
||||
if [ "$model_type" = "mobile" ]; then
|
||||
echo "test ppocrv4_det_mobile model......"
|
||||
## 启动自动化压缩训练
|
||||
CUDA_VISIBLE_DEVICES=0 python run.py --save_dir ./models/det_mobile_qat --config_path configs/ppocrv4/ppocrv4_det_qat_dist.yaml
|
||||
|
||||
## GPU指标测试
|
||||
### 量化前，预期指标：hmean:72.71%;time:4.7ms
|
||||
python test_ocr.py --model_path ./models/ch_PP-OCRv4_det_infer --config ./configs/ppocrv4/ppocrv4_det_qat_dist.yaml --precision fp32 --use_trt True
|
||||
### 量化后，预期指标：hmean:71.38%;time:3.3ms
|
||||
python test_ocr.py --model_path ./models/det_mobile_qat --config ./configs/ppocrv4/ppocrv4_det_qat_dist.yaml --precision int8 --use_trt True
|
||||
|
||||
## CPU指标测试
|
||||
### 量化前，预期指标：hmean:72.71%;time:198.4ms
|
||||
python test_ocr.py --model_path ./models/ch_PP-OCRv4_det_infer --config ./configs/ppocrv4/ppocrv4_det_qat_dist.yaml --precision fp32 --use_mkldnn True --device CPU --cpu_threads 12
|
||||
### 量化后，预期指标：hmean:72.30%;time:205.2ms
|
||||
python test_ocr.py --model_path ./models/det_mobile_qat --config ./configs/ppocrv4/ppocrv4_det_qat_dist.yaml --precision int8 --use_mkldnn True --device CPU --cpu_threads 12
|
||||
|
||||
# 量化前模型推理
|
||||
# GPU
|
||||
python tools/infer/predict_det.py --det_model_dir deploy/slim/auto_compression/models/ch_PP-OCRv4_det_infer \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu True \
|
||||
--use_tensorrt True --warmup True --precision fp32
|
||||
|
||||
# CPU
|
||||
python tools/infer/predict_det.py --det_model_dir deploy/slim/auto_compression/models/ch_PP-OCRv4_det_infer \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu False \
|
||||
--enable_mkldnn True --warmup True --precision fp32
|
||||
|
||||
# 量化后模型推理
|
||||
# GPU
|
||||
python tools/infer/predict_det.py --det_model_dir deploy/slim/auto_compression/models/det_mobile_qat \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu True \
|
||||
--use_tensorrt True --warmup True --precision int8
|
||||
|
||||
# CPU
|
||||
python tools/infer/predict_det.py --det_model_dir deploy/slim/auto_compression/models/det_mobile_qat \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu False \
|
||||
--enable_mkldnn True --warmup True --precision int8
|
||||
|
||||
elif [ "$model_type" = "server" ]; then
|
||||
echo "test ppocrv4_det_server model......"
|
||||
## 启动自动化压缩训练
|
||||
CUDA_VISIBLE_DEVICES=0 python run.py --save_dir ./models/det_server_qat --config_path configs/ppocrv4/ppocrv4_det_server_qat_dist.yaml
|
||||
|
||||
## GPU指标测试
|
||||
### 量化前，预期指标：hmean:79.77%;time:50.0ms
|
||||
python test_ocr.py --model_path ./models/ch_PP-OCRv4_det_server_infer --config ./configs/ppocrv4/ppocrv4_det_server_qat_dist.yaml --precision fp32 --use_trt True
|
||||
### 量化后，预期指标：hmean:79.81%;time:42.4ms
|
||||
python test_ocr.py --model_path ./models/det_server_qat --config ./configs/ppocrv4/ppocrv4_det_server_qat_dist.yaml --precision int8 --use_trt True
|
||||
|
||||
## CPU指标测试
|
||||
### 量化前，预期指标：hmean:79.77%;time:2159.4ms
|
||||
python test_ocr.py --model_path ./models/ch_PP-OCRv4_det_server_infer --config ./configs/ppocrv4/ppocrv4_det_server_qat_dist.yaml --precision fp32 --use_mkldnn True --device CPU --cpu_threads 12
|
||||
### 量化后，预期指标：hmean:79.69%;time:1834.8ms
|
||||
python test_ocr.py --model_path ./models/det_server_qat --config ./configs/ppocrv4/ppocrv4_det_server_qat_dist.yaml --precision int8 --use_mkldnn True --device CPU --cpu_threads 12
|
||||
|
||||
## 量化前模型推理
|
||||
### GPU
|
||||
python tools/infer/predict_det.py --det_model_dir deploy/slim/auto_compression/models/ch_PP-OCRv4_det_server_infer \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu True \
|
||||
--use_tensorrt True --warmup True --precision fp32
|
||||
|
||||
### CPU
|
||||
python tools/infer/predict_det.py --det_model_dir deploy/slim/auto_compression/models/ch_PP-OCRv4_det_server_infer \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu False \
|
||||
--enable_mkldnn True --warmup True --precision fp32
|
||||
|
||||
## 量化后模型推理
|
||||
### GPU
|
||||
python tools/infer/predict_det.py --det_model_dir deploy/slim/auto_compression/models/det_server_qat \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu True \
|
||||
--use_tensorrt True --warmup True --precision int8
|
||||
|
||||
### CPU
|
||||
python tools/infer/predict_det.py --det_model_dir deploy/slim/auto_compression/models/det_server_qat \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu False \
|
||||
--enable_mkldnn True --warmup True --precision int8
|
||||
else
|
||||
echo "unrecgnized model_type"
|
||||
fi
|
||||
88
deploy/slim/auto_compression/test_ocr_rec.sh
Normal file
88
deploy/slim/auto_compression/test_ocr_rec.sh
Normal file
@@ -0,0 +1,88 @@
|
||||
#!/bin/bash
|
||||
|
||||
# 本脚本用于测试PPOCRV4_rec系列模型的自动压缩功能
|
||||
## 运行脚本前，请确保处于以下环境：
|
||||
## CUDA11.2+TensorRT8.0.3.4+Paddle2.5.2
|
||||
|
||||
model_type="$1"
|
||||
|
||||
if [ "$model_type" = "mobile" ]; then
|
||||
echo "test ppocrv4_rec_mobile model......"
|
||||
## 启动自动化压缩训练
|
||||
CUDA_VISIBLE_DEVICES=0 python run.py --save_dir ./models/rec_mobile_qat --config_path configs/ppocrv4/ppocrv4_rec_qat_dist.yaml
|
||||
|
||||
## GPU指标测试
|
||||
### 量化前，预期指标：accuracy:78.92%;time:1.7ms
|
||||
python test_ocr.py --model_path ./models/ch_PP-OCRv4_rec_infer --config ./configs/ppocrv4/ppocrv4_rec_qat_dist.yaml --precision fp32 --use_trt True
|
||||
### 量化后，预期指标：accuracy:78.41%;time:1.4ms
|
||||
python test_ocr.py --model_path ./models/rec_mobile_qat --config ./configs/ppocrv4/ppocrv4_rec_qat_dist.yaml --precision int8 --use_trt True
|
||||
|
||||
## CPU指标测试
|
||||
### 量化前，预期指标：accuracy:78.92%;time:33.3ms
|
||||
python test_ocr.py --model_path ./models/ch_PP-OCRv4_rec_infer --config ./configs/ppocrv4/ppocrv4_rec_qat_dist.yaml --precision fp32 --use_mkldnn True --device CPU --cpu_threads 12
|
||||
### 量化后，预期指标：accuracy:78.44%;time:34.0ms
|
||||
python test_ocr.py --model_path ./models/rec_mobile_qat --config ./configs/ppocrv4/ppocrv4_rec_qat_dist.yaml --precision int8 --use_mkldnn True --device CPU --cpu_threads 12
|
||||
|
||||
# 量化前模型推理
|
||||
# GPU
|
||||
python tools/infer/predict_det.py --rec_model_dir deploy/slim/auto_compression/models/ch_PP-OCRv4_rec_infer \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu True \
|
||||
--use_tensorrt True --warmup True --precision fp32
|
||||
|
||||
# CPU
|
||||
python tools/infer/predict_det.py --rec_model_dir deploy/slim/auto_compression/models/ch_PP-OCRv4_rec_infer \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu False \
|
||||
--enable_mkldnn True --warmup True --precision fp32
|
||||
|
||||
# 量化后模型推理
|
||||
# GPU
|
||||
python tools/infer/predict_det.py --rec_model_dir deploy/slim/auto_compression/models/rec_mobile_qat \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu True \
|
||||
--use_tensorrt True --warmup True --precision int8
|
||||
|
||||
# CPU
|
||||
python tools/infer/predict_det.py --rec_model_dir deploy/slim/auto_compression/models/rec_mobile_qat \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu False \
|
||||
--enable_mkldnn True --warmup True --precision int8
|
||||
|
||||
elif [ "$model_type" = "server" ]; then
|
||||
echo "test ppocrv4_rec_server model......"
|
||||
## 启动自动化压缩训练
|
||||
CUDA_VISIBLE_DEVICES=0 python run.py --save_dir ./models/rec_server_qat --config_path configs/ppocrv4/ppocrv4_rec_server_qat_dist.yaml
|
||||
|
||||
## GPU指标测试
|
||||
### 量化前，预期指标：accuracy:81.62%;time:4.0ms
|
||||
python test_ocr.py --model_path ./models/ch_PP-OCRv4_rec_server_infer --config ./configs/ppocrv4/ppocrv4_rec_server_qat_dist.yaml --precision fp32 --use_trt True
|
||||
### 量化后，预期指标：accuracy:81.03%;time:2.0ms
|
||||
python test_ocr.py --model_path ./models/rec_server_qat --config ./configs/ppocrv4/ppocrv4_rec_server_qat_dist.yaml --precision int8 --use_trt True
|
||||
|
||||
## CPU指标测试
|
||||
### 量化前，预期指标：accuracy:81.62%;time:62.5ms
|
||||
python test_ocr.py --model_path ./models/ch_PP-OCRv4_rec_server_infer --config ./configs/ppocrv4/ppocrv4_rec_server_qat_dist.yaml --precision fp32 --use_mkldnn True --device CPU --cpu_threads 12
|
||||
### 量化后，预期指标：accuracy:81.00%;time:64.4ms
|
||||
python test_ocr.py --model_path ./models/rec_server_qat --config ./configs/ppocrv4/ppocrv4_rec_server_qat_dist.yaml --precision int8 --use_mkldnn True --device CPU --cpu_threads 12
|
||||
|
||||
## 量化前模型推理
|
||||
### GPU
|
||||
python tools/infer/predict_det.py --rec_model_dir deploy/slim/auto_compression/models/ch_PP-OCRv4_rec_server_infer \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu True \
|
||||
--use_tensorrt True --warmup True --precision fp32
|
||||
|
||||
### CPU
|
||||
python tools/infer/predict_det.py --rec_model_dir deploy/slim/auto_compression/models/ch_PP-OCRv4_rec_server_infer \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu False \
|
||||
--enable_mkldnn True --warmup True --precision fp32
|
||||
|
||||
## 量化后模型推理
|
||||
### GPU
|
||||
python tools/infer/predict_det.py --rec_model_dir deploy/slim/auto_compression/models/rec_server_qat \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu True \
|
||||
--use_tensorrt True --warmup True --precision int8
|
||||
|
||||
### CPU
|
||||
python tools/infer/predict_det.py --rec_model_dir deploy/slim/auto_compression/models/rec_server_qat \
|
||||
--benchmark True --image_dir deploy/slim/auto_compression/datasets/v4_4_test_dataset --use_gpu False \
|
||||
--enable_mkldnn True --warmup True --precision int8
|
||||
else
|
||||
echo "unrecgnized model_type"
|
||||
fi
|
||||
67
deploy/slim/prune/README.md
Normal file
67
deploy/slim/prune/README.md
Normal file
@@ -0,0 +1,67 @@
|
||||
|
||||
# PP-OCR模型裁剪
|
||||
|
||||
复杂的模型有利于提高模型的性能，但也导致模型中存在一定冗余，模型裁剪通过移出网络模型中的子模型来减少这种冗余，达到减少模型计算复杂度，提高模型推理性能的目的。
|
||||
本教程将介绍如何使用飞桨模型压缩库PaddleSlim做PaddleOCR模型的压缩。
|
||||
[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim)集成了模型剪枝、量化（包括量化训练和离线量化）、蒸馏和神经网络搜索等多种业界常用且领先的模型压缩功能，如果您感兴趣，可以关注并了解。
|
||||
|
||||
|
||||
在开始本教程之前，建议先了解：
|
||||
1. [PaddleOCR模型的训练方法](../../../doc/doc_ch/training.md)
|
||||
2. [模型裁剪教程](https://github.com/PaddlePaddle/PaddleSlim/blob/release%2F2.0.0/docs/zh_cn/tutorials/pruning/dygraph/filter_pruning.md)
|
||||
|
||||
## 快速开始
|
||||
|
||||
模型裁剪主要包括四个步骤：
|
||||
|
||||
1. 安装 PaddleSlim
|
||||
2. 准备训练好的模型
|
||||
3. 敏感度分析、裁剪训练
|
||||
4. 导出模型、预测部署
|
||||
|
||||
### 1. 安装PaddleSlim
|
||||
|
||||
```bash
|
||||
git clone https://github.com/PaddlePaddle/PaddleSlim.git
|
||||
cd PaddleSlim
|
||||
git checkout develop
|
||||
python3 setup.py install
|
||||
```
|
||||
|
||||
### 2. 获取预训练模型
|
||||
模型裁剪需要加载事先训练好的模型，PaddleOCR也提供了一系列[模型](../../../doc/doc_ch/models_list.md)，开发者可根据需要自行选择模型或使用自己的模型。
|
||||
|
||||
### 3. 敏感度分析训练
|
||||
|
||||
加载预训练模型后，通过对现有模型的每个网络层进行敏感度分析，得到敏感度文件：sen.pickle，可以通过PaddleSlim提供的[接口](https://github.com/PaddlePaddle/PaddleSlim/blob/9b01b195f0c4bc34a1ab434751cb260e13d64d9e/paddleslim/dygraph/prune/filter_pruner.py#L75)加载文件，获得各网络层在不同裁剪比例下的精度损失。从而了解各网络层冗余度，决定每个网络层的裁剪比例。
|
||||
敏感度文件内容格式：
|
||||
```
|
||||
sen.pickle(Dict){
|
||||
'layer_weight_name_0': sens_of_each_ratio(Dict){'pruning_ratio_0': acc_loss, 'pruning_ratio_1': acc_loss}
|
||||
'layer_weight_name_1': sens_of_each_ratio(Dict){'pruning_ratio_0': acc_loss, 'pruning_ratio_1': acc_loss}
|
||||
}
|
||||
|
||||
例子：
|
||||
{
|
||||
'conv10_expand_weights': {0.1: 0.006509952684312718, 0.2: 0.01827734339798862, 0.3: 0.014528405644659832, 0.6: 0.06536008804270439, 0.8: 0.11798612250664964, 0.7: 0.12391408417493704, 0.4: 0.030615754498018757, 0.5: 0.047105205602406594}
|
||||
'conv10_linear_weights': {0.1: 0.05113190831455035, 0.2: 0.07705573833558801, 0.3: 0.12096721757739311, 0.6: 0.5135061352930738, 0.8: 0.7908166677143281, 0.7: 0.7272187676899062, 0.4: 0.1819252083008504, 0.5: 0.3728054727792405}
|
||||
}
|
||||
```
|
||||
|
||||
加载敏感度文件后会返回一个字典，字典中的keys为网络模型参数模型的名字，values为一个字典，里面保存了相应网络层的裁剪敏感度信息。例如在例子中，conv10_expand_weights所对应的网络层在裁掉10%的卷积核后模型性能相较原模型会下降0.65%，详细信息可见[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/zh_cn/algo/algo.md#2-%E5%8D%B7%E7%A7%AF%E6%A0%B8%E5%89%AA%E8%A3%81%E5%8E%9F%E7%90%86)
|
||||
|
||||
进入PaddleOCR根目录，通过以下命令对模型进行敏感度分析训练：
|
||||
```bash
|
||||
python3 deploy/slim/prune/sensitivity_anal.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o Global.pretrained_model="your trained model" Global.save_model_dir=./output/prune_model/
|
||||
```
|
||||
|
||||
### 4. 导出模型、预测部署
|
||||
|
||||
在得到裁剪训练保存的模型后，我们可以将其导出为inference_model：
|
||||
```bash
|
||||
pytho3 deploy/slim/prune/export_prune_model.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o Global.pretrained_model=./output/det_db/best_accuracy Global.save_inference_dir=./prune/prune_inference_model
|
||||
```
|
||||
|
||||
inference model的预测和部署参考：
|
||||
1. [inference model python端预测](../../../doc/doc_ch/inference.md)
|
||||
2. [inference model C++预测](../../cpp_infer/readme.md)
|
||||
73
deploy/slim/prune/README_en.md
Normal file
73
deploy/slim/prune/README_en.md
Normal file
@@ -0,0 +1,73 @@
|
||||
|
||||
# PP-OCR Models Pruning
|
||||
|
||||
Generally, a more complex model would achieve better performance in the task, but it also leads to some redundancy in the model. Model Pruning is a technique that reduces this redundancy by removing the sub-models in the neural network model, so as to reduce model calculation complexity and improve model inference performance.
|
||||
|
||||
This example uses PaddleSlim provided[APIs of Pruning](https://github.com/PaddlePaddle/PaddleSlim/tree/develop/docs/zh_cn/api_cn/dygraph/pruners) to compress the OCR model.
|
||||
[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim), an open source library which integrates model pruning, quantization (including quantization training and offline quantization), distillation, neural network architecture search, and many other commonly used and leading model compression technique in the industry.
|
||||
|
||||
It is recommended that you could understand following pages before reading this example：
|
||||
1. [PaddleOCR training methods](../../../doc/doc_ch/quickstart.md)
|
||||
2. [The demo of prune](https://github.com/PaddlePaddle/PaddleSlim/blob/release%2F2.0.0/docs/zh_cn/tutorials/pruning/dygraph/filter_pruning.md)
|
||||
|
||||
## Quick start
|
||||
|
||||
Five steps for OCR model prune:
|
||||
1. Install PaddleSlim
|
||||
2. Prepare the trained model
|
||||
3. Sensitivity analysis and tailoring training
|
||||
4. Export model, predict deployment
|
||||
|
||||
### 1. Install PaddleSlim
|
||||
|
||||
```bash
|
||||
git clone https://github.com/PaddlePaddle/PaddleSlim.git
|
||||
cd PaddleSlim
|
||||
git checkout develop
|
||||
python3 setup.py install
|
||||
```
|
||||
|
||||
|
||||
### 2. Download Pre-trained Model
|
||||
Model prune needs to load pre-trained models.
|
||||
PaddleOCR also provides a series of [models](../../../doc/doc_en/models_list_en.md). Developers can choose their own models or use their own models according to their needs.
|
||||
|
||||
|
||||
### 3. Pruning sensitivity analysis
|
||||
|
||||
After the pre-trained model is loaded, sensitivity analysis is performed on each network layer of the model to understand the redundancy of each network layer, and save a sensitivity file which named: sen.pickle. After that, user could load the sensitivity file via the [methods provided by PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/paddleslim/prune/sensitive.py#L221) and determining the pruning ratio of each network layer automatically. For specific details of sensitivity analysis, see：[Sensitivity analysis](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/en/tutorials/image_classification_sensitivity_analysis_tutorial_en.md)
|
||||
The data format of sensitivity file：
|
||||
|
||||
```
|
||||
sen.pickle(Dict){
|
||||
'layer_weight_name_0': sens_of_each_ratio(Dict){'pruning_ratio_0': acc_loss, 'pruning_ratio_1': acc_loss}
|
||||
'layer_weight_name_1': sens_of_each_ratio(Dict){'pruning_ratio_0': acc_loss, 'pruning_ratio_1': acc_loss}
|
||||
}
|
||||
example：
|
||||
{
|
||||
'conv10_expand_weights': {0.1: 0.006509952684312718, 0.2: 0.01827734339798862, 0.3: 0.014528405644659832, 0.6: 0.06536008804270439, 0.8: 0.11798612250664964, 0.7: 0.12391408417493704, 0.4: 0.030615754498018757, 0.5: 0.047105205602406594}
|
||||
'conv10_linear_weights': {0.1: 0.05113190831455035, 0.2: 0.07705573833558801, 0.3: 0.12096721757739311, 0.6: 0.5135061352930738, 0.8: 0.7908166677143281, 0.7: 0.7272187676899062, 0.4: 0.1819252083008504, 0.5: 0.3728054727792405}
|
||||
}
|
||||
The function would return a dict after loading the sensitivity file. The keys of the dict are name of parameters in each layer. And the value of key is the information about pruning sensitivity of corresponding layer. In example, pruning 10% filter of the layer corresponding to conv10_expand_weights would lead to 0.65% degradation of model performance. The details could be seen at: [Sensitivity analysis](https://github.com/PaddlePaddle/PaddleSlim/blob/release/2.0-alpha/docs/zh_cn/algo/algo.md)
|
||||
```
|
||||
|
||||
The function would return a dict after loading the sensitivity file. The keys of the dict are name of parameters in each layer. And the value of key is the information about pruning sensitivity of corresponding layer. In example, pruning 10% filter of the layer corresponding to conv10_expand_weights would lead to 0.65% degradation of model performance. The details could be seen at: [Sensitivity analysis](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/zh_cn/algo/algo.md#2-%E5%8D%B7%E7%A7%AF%E6%A0%B8%E5%89%AA%E8%A3%81%E5%8E%9F%E7%90%86)
|
||||
|
||||
|
||||
Enter the PaddleOCR root directory，perform sensitivity analysis on the model with the following command：
|
||||
|
||||
```bash
|
||||
python3 deploy/slim/prune/sensitivity_anal.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o Global.pretrained_model="your trained model" Global.save_model_dir=./output/prune_model/
|
||||
```
|
||||
|
||||
|
||||
### 5. Export inference model and deploy it
|
||||
|
||||
We can export the pruned model as inference_model for deployment:
|
||||
```bash
|
||||
python deploy/slim/prune/export_prune_model.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml -o Global.pretrained_model=./output/det_db/best_accuracy Global.save_inference_dir=./prune/prune_inference_model
|
||||
```
|
||||
|
||||
Reference for prediction and deployment of inference model:
|
||||
1. [inference model python prediction](../../../doc/doc_en/inference_en.md)
|
||||
2. [inference model C++ prediction](../../cpp_infer/readme_en.md)
|
||||
137
deploy/slim/prune/export_prune_model.py
Normal file
137
deploy/slim/prune/export_prune_model.py
Normal file
@@ -0,0 +1,137 @@
|
||||
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from __future__ import absolute_import
|
||||
from __future__ import division
|
||||
from __future__ import print_function
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
__dir__ = os.path.dirname(__file__)
|
||||
sys.path.append(__dir__)
|
||||
sys.path.append(os.path.join(__dir__, "..", "..", ".."))
|
||||
sys.path.append(os.path.join(__dir__, "..", "..", "..", "tools"))
|
||||
|
||||
import paddle
|
||||
from ppocr.data import build_dataloader, set_signal_handlers
|
||||
from ppocr.modeling.architectures import build_model
|
||||
|
||||
from ppocr.postprocess import build_post_process
|
||||
from ppocr.metrics import build_metric
|
||||
from ppocr.utils.save_load import load_model
|
||||
import tools.program as program
|
||||
|
||||
|
||||
def main(config, device, logger, vdl_writer):
|
||||
global_config = config["Global"]
|
||||
|
||||
# build dataloader
|
||||
set_signal_handlers()
|
||||
valid_dataloader = build_dataloader(config, "Eval", device, logger)
|
||||
|
||||
# build post process
|
||||
post_process_class = build_post_process(config["PostProcess"], global_config)
|
||||
|
||||
# build model
|
||||
# for rec algorithm
|
||||
if hasattr(post_process_class, "character"):
|
||||
char_num = len(getattr(post_process_class, "character"))
|
||||
config["Architecture"]["Head"]["out_channels"] = char_num
|
||||
model = build_model(config["Architecture"])
|
||||
|
||||
if config["Architecture"]["model_type"] == "det":
|
||||
input_shape = [1, 3, 640, 640]
|
||||
elif config["Architecture"]["model_type"] == "rec":
|
||||
input_shape = [1, 3, 32, 320]
|
||||
|
||||
flops = paddle.flops(model, input_shape)
|
||||
logger.info("FLOPs before pruning: {}".format(flops))
|
||||
|
||||
from paddleslim.dygraph import FPGMFilterPruner
|
||||
|
||||
model.train()
|
||||
pruner = FPGMFilterPruner(model, input_shape)
|
||||
|
||||
# build metric
|
||||
eval_class = build_metric(config["Metric"])
|
||||
|
||||
def eval_fn():
|
||||
metric = program.eval(model, valid_dataloader, post_process_class, eval_class)
|
||||
if config["Architecture"]["model_type"] == "det":
|
||||
main_indicator = "hmean"
|
||||
else:
|
||||
main_indicator = "acc"
|
||||
logger.info("metric[{}]: {}".format(main_indicator, metric[main_indicator]))
|
||||
return metric[main_indicator]
|
||||
|
||||
params_sensitive = pruner.sensitive(
|
||||
eval_func=eval_fn,
|
||||
sen_file="./sen.pickle",
|
||||
skip_vars=["conv2d_57.w_0", "conv2d_transpose_2.w_0", "conv2d_transpose_3.w_0"],
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"The sensitivity analysis results of model parameters saved in sen.pickle"
|
||||
)
|
||||
# calculate pruned params's ratio
|
||||
params_sensitive = pruner._get_ratios_by_loss(params_sensitive, loss=0.02)
|
||||
for key in params_sensitive.keys():
|
||||
logger.info("{}, {}".format(key, params_sensitive[key]))
|
||||
|
||||
plan = pruner.prune_vars(params_sensitive, [0])
|
||||
|
||||
flops = paddle.flops(model, input_shape)
|
||||
logger.info("FLOPs after pruning: {}".format(flops))
|
||||
|
||||
# load pretrain model
|
||||
load_model(config, model)
|
||||
metric = program.eval(model, valid_dataloader, post_process_class, eval_class)
|
||||
if config["Architecture"]["model_type"] == "det":
|
||||
main_indicator = "hmean"
|
||||
else:
|
||||
main_indicator = "acc"
|
||||
logger.info("metric['']: {}".format(main_indicator, metric[main_indicator]))
|
||||
|
||||
# start export model
|
||||
from paddle.jit import to_static
|
||||
|
||||
infer_shape = [3, -1, -1]
|
||||
if config["Architecture"]["model_type"] == "rec":
|
||||
infer_shape = [3, 32, -1] # for rec model, H must be 32
|
||||
|
||||
if (
|
||||
"Transform" in config["Architecture"]
|
||||
and config["Architecture"]["Transform"] is not None
|
||||
and config["Architecture"]["Transform"]["name"] == "TPS"
|
||||
):
|
||||
logger.info(
|
||||
"When there is tps in the network, variable length input is not supported, and the input size needs to be the same as during training"
|
||||
)
|
||||
infer_shape[-1] = 100
|
||||
model = to_static(
|
||||
model,
|
||||
input_spec=[
|
||||
paddle.static.InputSpec(shape=[None] + infer_shape, dtype="float32")
|
||||
],
|
||||
)
|
||||
|
||||
save_path = "{}/inference".format(config["Global"]["save_inference_dir"])
|
||||
paddle.jit.save(model, save_path)
|
||||
logger.info("inference model is saved to {}".format(save_path))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
config, device, logger, vdl_writer = program.preprocess(is_train=True)
|
||||
main(config, device, logger, vdl_writer)
|
||||
200
deploy/slim/prune/sensitivity_anal.py
Normal file
200
deploy/slim/prune/sensitivity_anal.py
Normal file
@@ -0,0 +1,200 @@
|
||||
# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from __future__ import absolute_import
|
||||
from __future__ import division
|
||||
from __future__ import print_function
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
__dir__ = os.path.dirname(__file__)
|
||||
sys.path.append(__dir__)
|
||||
sys.path.append(os.path.join(__dir__, "..", "..", ".."))
|
||||
sys.path.append(os.path.join(__dir__, "..", "..", "..", "tools"))
|
||||
|
||||
import paddle
|
||||
import paddle.distributed as dist
|
||||
from ppocr.data import build_dataloader, set_signal_handlers
|
||||
from ppocr.modeling.architectures import build_model
|
||||
from ppocr.losses import build_loss
|
||||
from ppocr.optimizer import build_optimizer
|
||||
from ppocr.postprocess import build_post_process
|
||||
from ppocr.metrics import build_metric
|
||||
from ppocr.utils.save_load import load_model
|
||||
import tools.program as program
|
||||
|
||||
dist.get_world_size()
|
||||
|
||||
|
||||
def get_pruned_params(parameters):
|
||||
params = []
|
||||
|
||||
for param in parameters:
|
||||
if (
|
||||
len(param.shape) == 4
|
||||
and "depthwise" not in param.name
|
||||
and "transpose" not in param.name
|
||||
and "conv2d_57" not in param.name
|
||||
and "conv2d_56" not in param.name
|
||||
):
|
||||
params.append(param.name)
|
||||
return params
|
||||
|
||||
|
||||
def main(config, device, logger, vdl_writer):
|
||||
# init dist environment
|
||||
if config["Global"]["distributed"]:
|
||||
dist.init_parallel_env()
|
||||
|
||||
global_config = config["Global"]
|
||||
|
||||
# build dataloader
|
||||
set_signal_handlers()
|
||||
train_dataloader = build_dataloader(config, "Train", device, logger)
|
||||
if config["Eval"]:
|
||||
valid_dataloader = build_dataloader(config, "Eval", device, logger)
|
||||
else:
|
||||
valid_dataloader = None
|
||||
|
||||
# build post process
|
||||
post_process_class = build_post_process(config["PostProcess"], global_config)
|
||||
|
||||
# build model
|
||||
# for rec algorithm
|
||||
if hasattr(post_process_class, "character"):
|
||||
char_num = len(getattr(post_process_class, "character"))
|
||||
config["Architecture"]["Head"]["out_channels"] = char_num
|
||||
model = build_model(config["Architecture"])
|
||||
if config["Architecture"]["model_type"] == "det":
|
||||
input_shape = [1, 3, 640, 640]
|
||||
elif config["Architecture"]["model_type"] == "rec":
|
||||
input_shape = [1, 3, 32, 320]
|
||||
flops = paddle.flops(model, input_shape)
|
||||
|
||||
logger.info("FLOPs before pruning: {}".format(flops))
|
||||
|
||||
from paddleslim.dygraph import FPGMFilterPruner
|
||||
|
||||
model.train()
|
||||
|
||||
pruner = FPGMFilterPruner(model, input_shape)
|
||||
|
||||
# build loss
|
||||
loss_class = build_loss(config["Loss"])
|
||||
|
||||
# build optim
|
||||
optimizer, lr_scheduler = build_optimizer(
|
||||
config["Optimizer"],
|
||||
epochs=config["Global"]["epoch_num"],
|
||||
step_each_epoch=len(train_dataloader),
|
||||
model=model,
|
||||
)
|
||||
|
||||
# build metric
|
||||
eval_class = build_metric(config["Metric"])
|
||||
# load pretrain model
|
||||
pre_best_model_dict = load_model(config, model, optimizer)
|
||||
|
||||
logger.info(
|
||||
"train dataloader has {} iters, valid dataloader has {} iters".format(
|
||||
len(train_dataloader), len(valid_dataloader)
|
||||
)
|
||||
)
|
||||
# build metric
|
||||
eval_class = build_metric(config["Metric"])
|
||||
|
||||
logger.info(
|
||||
"train dataloader has {} iters, valid dataloader has {} iters".format(
|
||||
len(train_dataloader), len(valid_dataloader)
|
||||
)
|
||||
)
|
||||
|
||||
def eval_fn():
|
||||
metric = program.eval(
|
||||
model, valid_dataloader, post_process_class, eval_class, False
|
||||
)
|
||||
if config["Architecture"]["model_type"] == "det":
|
||||
main_indicator = "hmean"
|
||||
else:
|
||||
main_indicator = "acc"
|
||||
|
||||
logger.info("metric[{}]: {}".format(main_indicator, metric[main_indicator]))
|
||||
return metric[main_indicator]
|
||||
|
||||
run_sensitive_analysis = False
|
||||
"""
|
||||
run_sensitive_analysis=True:
|
||||
Automatically compute the sensitivities of convolutions in a model.
|
||||
The sensitivity of a convolution is the losses of accuracy on test dataset in
|
||||
different pruned ratios. The sensitivities can be used to get a group of best
|
||||
ratios with some condition.
|
||||
|
||||
run_sensitive_analysis=False:
|
||||
Set prune trim ratio to a fixed value, such as 10%. The larger the value,
|
||||
the more convolution weights will be cropped.
|
||||
|
||||
"""
|
||||
|
||||
if run_sensitive_analysis:
|
||||
params_sensitive = pruner.sensitive(
|
||||
eval_func=eval_fn,
|
||||
sen_file="./deploy/slim/prune/sen.pickle",
|
||||
skip_vars=[
|
||||
"conv2d_57.w_0",
|
||||
"conv2d_transpose_2.w_0",
|
||||
"conv2d_transpose_3.w_0",
|
||||
],
|
||||
)
|
||||
logger.info(
|
||||
"The sensitivity analysis results of model parameters saved in sen.pickle"
|
||||
)
|
||||
# calculate pruned params's ratio
|
||||
params_sensitive = pruner._get_ratios_by_loss(params_sensitive, loss=0.02)
|
||||
for key in params_sensitive.keys():
|
||||
logger.info("{}, {}".format(key, params_sensitive[key]))
|
||||
else:
|
||||
params_sensitive = {}
|
||||
for param in model.parameters():
|
||||
if "transpose" not in param.name and "linear" not in param.name:
|
||||
# set prune ratio as 10%. The larger the value, the more convolution weights will be cropped
|
||||
params_sensitive[param.name] = 0.1
|
||||
|
||||
plan = pruner.prune_vars(params_sensitive, [0])
|
||||
|
||||
flops = paddle.flops(model, input_shape)
|
||||
logger.info("FLOPs after pruning: {}".format(flops))
|
||||
|
||||
# start train
|
||||
|
||||
program.train(
|
||||
config,
|
||||
train_dataloader,
|
||||
valid_dataloader,
|
||||
device,
|
||||
model,
|
||||
loss_class,
|
||||
optimizer,
|
||||
lr_scheduler,
|
||||
post_process_class,
|
||||
eval_class,
|
||||
pre_best_model_dict,
|
||||
logger,
|
||||
vdl_writer,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
config, device, logger, vdl_writer = program.preprocess(is_train=True)
|
||||
main(config, device, logger, vdl_writer)
|
||||
60
deploy/slim/quantization/README.md
Normal file
60
deploy/slim/quantization/README.md
Normal file
@@ -0,0 +1,60 @@
|
||||
|
||||
# PP-OCR模型量化
|
||||
复杂的模型有利于提高模型的性能，但也导致模型中存在一定冗余，模型量化将全精度缩减到定点数减少这种冗余，达到减少模型计算复杂度，提高模型推理性能的目的。
|
||||
模型量化可以在基本不损失模型的精度的情况下，将FP32精度的模型参数转换为Int8精度，减小模型参数大小并加速计算，使用量化后的模型在移动端等部署时更具备速度优势。
|
||||
|
||||
本教程将介绍如何使用飞桨模型压缩库PaddleSlim做PaddleOCR模型的压缩。
|
||||
[PaddleSlim](https://github.com/PaddlePaddle/PaddleSlim) 集成了模型剪枝、量化（包括量化训练和离线量化）、蒸馏和神经网络搜索等多种业界常用且领先的模型压缩功能，如果您感兴趣，可以关注并了解。
|
||||
|
||||
在开始本教程之前，建议先了解[PaddleOCR模型的训练方法](../../../doc/doc_ch/training.md)以及[PaddleSlim](https://paddleslim.readthedocs.io/zh_CN/latest/index.html)
|
||||
|
||||
|
||||
## 快速开始
|
||||
量化多适用于轻量模型在移动端的部署，当训练出一个模型后，如果希望进一步的压缩模型大小并加速预测，可使用量化的方法压缩模型。
|
||||
|
||||
模型量化主要包括五个步骤：
|
||||
1. 安装 PaddleSlim
|
||||
2. 准备训练好的模型
|
||||
3. 量化训练
|
||||
4. 导出量化推理模型
|
||||
5. 量化模型预测部署
|
||||
|
||||
### 1. 安装PaddleSlim
|
||||
|
||||
```bash
|
||||
pip3 install paddleslim==2.3.2
|
||||
```
|
||||
|
||||
### 2. 准备训练好的模型
|
||||
|
||||
PaddleOCR提供了一系列训练好的[模型](../../../doc/doc_ch/models_list.md)，如果待量化的模型不在列表中，需要按照[常规训练](../../../doc/doc_ch/quickstart.md)方法得到训练好的模型。
|
||||
|
||||
### 3. 量化训练
|
||||
量化训练包括离线量化训练和在线量化训练，在线量化训练效果更好，需加载预训练模型，在定义好量化策略后即可对模型进行量化。
|
||||
|
||||
|
||||
量化训练的代码位于slim/quantization/quant.py 中，比如训练检测模型，以PPOCRv3检测模型为例，训练指令如下：
|
||||
```
|
||||
# 下载检测预训练模型：
|
||||
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_distill_train.tar
|
||||
tar xf ch_PP-OCRv3_det_distill_train.tar
|
||||
|
||||
python deploy/slim/quantization/quant.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml -o Global.pretrained_model='./ch_PP-OCRv3_det_distill_train/best_accuracy' Global.save_model_dir=./output/quant_model_distill/
|
||||
```
|
||||
如果要训练识别模型的量化，修改配置文件和加载的模型参数即可。
|
||||
|
||||
### 4. 导出模型
|
||||
|
||||
在得到量化训练保存的模型后，我们可以将其导出为inference_model，用于预测部署：
|
||||
|
||||
```bash
|
||||
python deploy/slim/quantization/export_model.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml -o Global.checkpoints=output/quant_model/best_accuracy Global.save_inference_dir=./output/quant_inference_model
|
||||
```
|
||||
|
||||
### 5. 量化模型部署
|
||||
|
||||
上述步骤导出的量化模型，参数精度仍然是FP32，但是参数的数值范围是int8，导出的模型可以通过PaddleLite的opt模型转换工具完成模型转换。
|
||||
|
||||
量化模型移动端部署的可参考 [移动端模型部署](../../lite/readme.md)
|
||||
|
||||
备注：量化训练后的模型参数是float32类型，转inference model预测时相对不量化无加速效果，原因是量化后模型结构之间存在量化和反量化算子，如果要使用量化模型部署，建议使用TensorRT并设置precision为INT8加速量化模型的预测时间。
|
||||
65
deploy/slim/quantization/README_en.md
Normal file
65
deploy/slim/quantization/README_en.md
Normal file
@@ -0,0 +1,65 @@
|
||||
|
||||
# PP-OCR Models Quantization
|
||||
|
||||
Generally, a more complex model would achieve better performance in the task, but it also leads to some redundancy in the model.
|
||||
Quantization is a technique that reduces this redundancy by reducing the full precision data to a fixed number,
|
||||
so as to reduce model calculation complexity and improve model inference performance.
|
||||
|
||||
This example uses PaddleSlim provided [APIs of Quantization](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/zh_cn/api_cn/dygraph/quanter/qat.rst) to compress the OCR model.
|
||||
|
||||
It is recommended that you could understand following pages before reading this example：
|
||||
- [The training strategy of OCR model](../../../doc/doc_en/quickstart_en.md)
|
||||
- [PaddleSlim Document](https://github.com/PaddlePaddle/PaddleSlim/blob/develop/docs/zh_cn/api_cn/dygraph/quanter/qat.rst)
|
||||
|
||||
## Quick Start
|
||||
Quantization is mostly suitable for the deployment of lightweight models on mobile terminals.
|
||||
After training, if you want to further compress the model size and accelerate the prediction, you can use quantization methods to compress the model according to the following steps.
|
||||
|
||||
1. Install PaddleSlim
|
||||
2. Prepare trained model
|
||||
3. Quantization-Aware Training
|
||||
4. Export inference model
|
||||
5. Deploy quantization inference model
|
||||
|
||||
|
||||
### 1. Install PaddleSlim
|
||||
|
||||
```bash
|
||||
pip3 install paddleslim==2.3.2
|
||||
```
|
||||
|
||||
|
||||
### 2. Download Pre-trained Model
|
||||
PaddleOCR provides a series of pre-trained [models](../../../doc/doc_en/models_list_en.md).
|
||||
If the model to be quantified is not in the list, you need to follow the [Regular Training](../../../doc/doc_en/quickstart_en.md) method to get the trained model.
|
||||
|
||||
|
||||
### 3. Quant-Aware Training
|
||||
Quantization training includes offline quantization training and online quantization training.
|
||||
Online quantization training is more effective. It is necessary to load the pre-trained model.
|
||||
After the quantization strategy is defined, the model can be quantified.
|
||||
|
||||
The code for quantization training is located in `slim/quantization/quant.py`. For example, the training instructions of slim PPOCRv3 detection model are as follows:
|
||||
```
|
||||
# download provided model
|
||||
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/chinese/ch_PP-OCRv3_det_distill_train.tar
|
||||
tar xf ch_PP-OCRv3_det_distill_train.tar
|
||||
|
||||
python deploy/slim/quantization/quant.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml -o Global.pretrained_model='./ch_PP-OCRv3_det_distill_train/best_accuracy' Global.save_model_dir=./output/quant_model_distill/
|
||||
```
|
||||
|
||||
If you want to quantify the text recognition model, you can modify the configuration file and loaded model parameters.
|
||||
|
||||
### 4. Export inference model
|
||||
|
||||
Once we got the model after pruning and fine-tuning, we can export it as an inference model for the deployment of predictive tasks:
|
||||
|
||||
```bash
|
||||
python deploy/slim/quantization/export_model.py -c configs/det/ch_PP-OCRv3/ch_PP-OCRv3_det_cml.yml -o Global.checkpoints=output/quant_model/best_accuracy Global.save_inference_dir=./output/quant_inference_model
|
||||
```
|
||||
|
||||
### 5. Deploy
|
||||
The numerical range of the quantized model parameters derived from the above steps is still FP32, but the numerical range of the parameters is int8.
|
||||
The derived model can be converted through the `opt tool` of PaddleLite.
|
||||
|
||||
For quantitative model deployment, please refer to [Mobile terminal model deployment](../../lite/readme.md)
|
||||
189
deploy/slim/quantization/export_model.py
Executable file
189
deploy/slim/quantization/export_model.py
Executable file
@@ -0,0 +1,189 @@
|
||||
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
__dir__ = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.append(__dir__)
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(__dir__, "..", "..", "..")))
|
||||
sys.path.insert(0, os.path.abspath(os.path.join(__dir__, "..", "..", "..", "tools")))
|
||||
|
||||
import argparse
|
||||
|
||||
import paddle
|
||||
from paddle.jit import to_static
|
||||
|
||||
from ppocr.modeling.architectures import build_model
|
||||
from ppocr.postprocess import build_post_process
|
||||
from ppocr.utils.save_load import load_model
|
||||
from ppocr.utils.logging import get_logger
|
||||
from tools.program import load_config, merge_config, ArgsParser
|
||||
from ppocr.metrics import build_metric
|
||||
import tools.program as program
|
||||
from paddleslim.dygraph.quant import QAT
|
||||
from ppocr.data import build_dataloader, set_signal_handlers
|
||||
from ppocr.utils.export_model import export_single_model
|
||||
|
||||
|
||||
def main():
|
||||
############################################################################################################
|
||||
# 1. quantization configs
|
||||
############################################################################################################
|
||||
quant_config = {
|
||||
# weight preprocess type, default is None and no preprocessing is performed.
|
||||
"weight_preprocess_type": None,
|
||||
# activation preprocess type, default is None and no preprocessing is performed.
|
||||
"activation_preprocess_type": None,
|
||||
# weight quantize type, default is 'channel_wise_abs_max'
|
||||
"weight_quantize_type": "channel_wise_abs_max",
|
||||
# activation quantize type, default is 'moving_average_abs_max'
|
||||
"activation_quantize_type": "moving_average_abs_max",
|
||||
# weight quantize bit num, default is 8
|
||||
"weight_bits": 8,
|
||||
# activation quantize bit num, default is 8
|
||||
"activation_bits": 8,
|
||||
# data type after quantization, such as 'uint8', 'int8', etc. default is 'int8'
|
||||
"dtype": "int8",
|
||||
# window size for 'range_abs_max' quantization. default is 10000
|
||||
"window_size": 10000,
|
||||
# The decay coefficient of moving average, default is 0.9
|
||||
"moving_rate": 0.9,
|
||||
# for dygraph quantization, layers of type in quantizable_layer_type will be quantized
|
||||
"quantizable_layer_type": ["Conv2D", "Linear"],
|
||||
}
|
||||
FLAGS = ArgsParser().parse_args()
|
||||
config = load_config(FLAGS.config)
|
||||
config = merge_config(config, FLAGS.opt)
|
||||
logger = get_logger()
|
||||
# build post process
|
||||
|
||||
post_process_class = build_post_process(config["PostProcess"], config["Global"])
|
||||
|
||||
# build model
|
||||
if hasattr(post_process_class, "character"):
|
||||
char_num = len(getattr(post_process_class, "character"))
|
||||
if config["Architecture"]["algorithm"] in [
|
||||
"Distillation",
|
||||
]: # distillation model
|
||||
for key in config["Architecture"]["Models"]:
|
||||
if (
|
||||
config["Architecture"]["Models"][key]["Head"]["name"] == "MultiHead"
|
||||
): # for multi head
|
||||
if config["PostProcess"]["name"] == "DistillationSARLabelDecode":
|
||||
char_num = char_num - 2
|
||||
# update SARLoss params
|
||||
assert (
|
||||
list(config["Loss"]["loss_config_list"][-1].keys())[0]
|
||||
== "DistillationSARLoss"
|
||||
)
|
||||
config["Loss"]["loss_config_list"][-1]["DistillationSARLoss"][
|
||||
"ignore_index"
|
||||
] = (char_num + 1)
|
||||
out_channels_list = {}
|
||||
out_channels_list["CTCLabelDecode"] = char_num
|
||||
out_channels_list["SARLabelDecode"] = char_num + 2
|
||||
config["Architecture"]["Models"][key]["Head"][
|
||||
"out_channels_list"
|
||||
] = out_channels_list
|
||||
else:
|
||||
config["Architecture"]["Models"][key]["Head"][
|
||||
"out_channels"
|
||||
] = char_num
|
||||
elif config["Architecture"]["Head"]["name"] == "MultiHead": # for multi head
|
||||
if config["PostProcess"]["name"] == "SARLabelDecode":
|
||||
char_num = char_num - 2
|
||||
# update SARLoss params
|
||||
assert list(config["Loss"]["loss_config_list"][1].keys())[0] == "SARLoss"
|
||||
if config["Loss"]["loss_config_list"][1]["SARLoss"] is None:
|
||||
config["Loss"]["loss_config_list"][1]["SARLoss"] = {
|
||||
"ignore_index": char_num + 1
|
||||
}
|
||||
else:
|
||||
config["Loss"]["loss_config_list"][1]["SARLoss"]["ignore_index"] = (
|
||||
char_num + 1
|
||||
)
|
||||
out_channels_list = {}
|
||||
out_channels_list["CTCLabelDecode"] = char_num
|
||||
out_channels_list["SARLabelDecode"] = char_num + 2
|
||||
config["Architecture"]["Head"]["out_channels_list"] = out_channels_list
|
||||
else: # base rec model
|
||||
config["Architecture"]["Head"]["out_channels"] = char_num
|
||||
|
||||
if config["PostProcess"]["name"] == "SARLabelDecode": # for SAR model
|
||||
config["Loss"]["ignore_index"] = char_num - 1
|
||||
|
||||
model = build_model(config["Architecture"])
|
||||
|
||||
# get QAT model
|
||||
quanter = QAT(config=quant_config)
|
||||
quanter.quantize(model)
|
||||
|
||||
load_model(config, model)
|
||||
|
||||
# build metric
|
||||
eval_class = build_metric(config["Metric"])
|
||||
|
||||
# build dataloader
|
||||
set_signal_handlers()
|
||||
valid_dataloader = build_dataloader(config, "Eval", device, logger)
|
||||
|
||||
use_srn = config["Architecture"]["algorithm"] == "SRN"
|
||||
model_type = config["Architecture"].get("model_type", None)
|
||||
# start eval
|
||||
metric = program.eval(
|
||||
model, valid_dataloader, post_process_class, eval_class, model_type, use_srn
|
||||
)
|
||||
model.eval()
|
||||
|
||||
logger.info("metric eval ***************")
|
||||
for k, v in metric.items():
|
||||
logger.info("{}:{}".format(k, v))
|
||||
|
||||
save_path = config["Global"]["save_inference_dir"]
|
||||
|
||||
arch_config = config["Architecture"]
|
||||
|
||||
if (
|
||||
arch_config["algorithm"] == "SVTR"
|
||||
and arch_config["Head"]["name"] != "MultiHead"
|
||||
):
|
||||
input_shape = config["Eval"]["dataset"]["transforms"][-2]["SVTRRecResizeImg"][
|
||||
"image_shape"
|
||||
]
|
||||
else:
|
||||
input_shape = None
|
||||
|
||||
if arch_config["algorithm"] in [
|
||||
"Distillation",
|
||||
]: # distillation model
|
||||
archs = list(arch_config["Models"].values())
|
||||
for idx, name in enumerate(model.model_name_list):
|
||||
sub_model_save_path = os.path.join(save_path, name, "inference")
|
||||
export_single_model(
|
||||
model.model_list[idx],
|
||||
archs[idx],
|
||||
sub_model_save_path,
|
||||
logger,
|
||||
input_shape,
|
||||
quanter,
|
||||
)
|
||||
else:
|
||||
save_path = os.path.join(save_path, "inference")
|
||||
export_single_model(model, arch_config, save_path, logger, input_shape, quanter)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
config, device, logger, vdl_writer = program.preprocess()
|
||||
main()
|
||||
226
deploy/slim/quantization/quant.py
Executable file
226
deploy/slim/quantization/quant.py
Executable file
@@ -0,0 +1,226 @@
|
||||
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from __future__ import absolute_import
|
||||
from __future__ import division
|
||||
from __future__ import print_function
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
__dir__ = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.append(__dir__)
|
||||
sys.path.append(os.path.abspath(os.path.join(__dir__, "..", "..", "..")))
|
||||
sys.path.append(os.path.abspath(os.path.join(__dir__, "..", "..", "..", "tools")))
|
||||
|
||||
import yaml
|
||||
import paddle
|
||||
import paddle.distributed as dist
|
||||
|
||||
paddle.seed(2)
|
||||
|
||||
from ppocr.data import build_dataloader, set_signal_handlers
|
||||
from ppocr.modeling.architectures import build_model
|
||||
from ppocr.losses import build_loss
|
||||
from ppocr.optimizer import build_optimizer
|
||||
from ppocr.postprocess import build_post_process
|
||||
from ppocr.metrics import build_metric
|
||||
from ppocr.utils.save_load import load_model
|
||||
import tools.program as program
|
||||
from paddleslim.dygraph.quant import QAT
|
||||
|
||||
dist.get_world_size()
|
||||
|
||||
|
||||
class PACT(paddle.nn.Layer):
|
||||
def __init__(self):
|
||||
super(PACT, self).__init__()
|
||||
alpha_attr = paddle.ParamAttr(
|
||||
name=self.full_name() + ".pact",
|
||||
initializer=paddle.nn.initializer.Constant(value=20),
|
||||
learning_rate=1.0,
|
||||
regularizer=paddle.regularizer.L2Decay(2e-5),
|
||||
)
|
||||
|
||||
self.alpha = self.create_parameter(shape=[1], attr=alpha_attr, dtype="float32")
|
||||
|
||||
def forward(self, x):
|
||||
out_left = paddle.nn.functional.relu(x - self.alpha)
|
||||
out_right = paddle.nn.functional.relu(-self.alpha - x)
|
||||
x = x - out_left + out_right
|
||||
return x
|
||||
|
||||
|
||||
quant_config = {
|
||||
# weight preprocess type, default is None and no preprocessing is performed.
|
||||
"weight_preprocess_type": None,
|
||||
# activation preprocess type, default is None and no preprocessing is performed.
|
||||
"activation_preprocess_type": None,
|
||||
# weight quantize type, default is 'channel_wise_abs_max'
|
||||
"weight_quantize_type": "channel_wise_abs_max",
|
||||
# activation quantize type, default is 'moving_average_abs_max'
|
||||
"activation_quantize_type": "moving_average_abs_max",
|
||||
# weight quantize bit num, default is 8
|
||||
"weight_bits": 8,
|
||||
# activation quantize bit num, default is 8
|
||||
"activation_bits": 8,
|
||||
# data type after quantization, such as 'uint8', 'int8', etc. default is 'int8'
|
||||
"dtype": "int8",
|
||||
# window size for 'range_abs_max' quantization. default is 10000
|
||||
"window_size": 10000,
|
||||
# The decay coefficient of moving average, default is 0.9
|
||||
"moving_rate": 0.9,
|
||||
# for dygraph quantization, layers of type in quantizable_layer_type will be quantized
|
||||
"quantizable_layer_type": ["Conv2D", "Linear"],
|
||||
}
|
||||
|
||||
|
||||
def main(config, device, logger, vdl_writer):
|
||||
# init dist environment
|
||||
if config["Global"]["distributed"]:
|
||||
dist.init_parallel_env()
|
||||
|
||||
global_config = config["Global"]
|
||||
|
||||
# build dataloader
|
||||
set_signal_handlers()
|
||||
train_dataloader = build_dataloader(config, "Train", device, logger)
|
||||
if config["Eval"]:
|
||||
valid_dataloader = build_dataloader(config, "Eval", device, logger)
|
||||
else:
|
||||
valid_dataloader = None
|
||||
|
||||
# build post process
|
||||
post_process_class = build_post_process(config["PostProcess"], global_config)
|
||||
|
||||
# build model
|
||||
# for rec algorithm
|
||||
if hasattr(post_process_class, "character"):
|
||||
char_num = len(getattr(post_process_class, "character"))
|
||||
if config["Architecture"]["algorithm"] in [
|
||||
"Distillation",
|
||||
]: # distillation model
|
||||
for key in config["Architecture"]["Models"]:
|
||||
if (
|
||||
config["Architecture"]["Models"][key]["Head"]["name"] == "MultiHead"
|
||||
): # for multi head
|
||||
if config["PostProcess"]["name"] == "DistillationSARLabelDecode":
|
||||
char_num = char_num - 2
|
||||
# update SARLoss params
|
||||
assert (
|
||||
list(config["Loss"]["loss_config_list"][-1].keys())[0]
|
||||
== "DistillationSARLoss"
|
||||
)
|
||||
config["Loss"]["loss_config_list"][-1]["DistillationSARLoss"][
|
||||
"ignore_index"
|
||||
] = (char_num + 1)
|
||||
out_channels_list = {}
|
||||
out_channels_list["CTCLabelDecode"] = char_num
|
||||
out_channels_list["SARLabelDecode"] = char_num + 2
|
||||
config["Architecture"]["Models"][key]["Head"][
|
||||
"out_channels_list"
|
||||
] = out_channels_list
|
||||
else:
|
||||
config["Architecture"]["Models"][key]["Head"][
|
||||
"out_channels"
|
||||
] = char_num
|
||||
elif config["Architecture"]["Head"]["name"] == "MultiHead": # for multi head
|
||||
if config["PostProcess"]["name"] == "SARLabelDecode":
|
||||
char_num = char_num - 2
|
||||
# update SARLoss params
|
||||
assert list(config["Loss"]["loss_config_list"][1].keys())[0] == "SARLoss"
|
||||
if config["Loss"]["loss_config_list"][1]["SARLoss"] is None:
|
||||
config["Loss"]["loss_config_list"][1]["SARLoss"] = {
|
||||
"ignore_index": char_num + 1
|
||||
}
|
||||
else:
|
||||
config["Loss"]["loss_config_list"][1]["SARLoss"]["ignore_index"] = (
|
||||
char_num + 1
|
||||
)
|
||||
out_channels_list = {}
|
||||
out_channels_list["CTCLabelDecode"] = char_num
|
||||
out_channels_list["SARLabelDecode"] = char_num + 2
|
||||
config["Architecture"]["Head"]["out_channels_list"] = out_channels_list
|
||||
else: # base rec model
|
||||
config["Architecture"]["Head"]["out_channels"] = char_num
|
||||
|
||||
if config["PostProcess"]["name"] == "SARLabelDecode": # for SAR model
|
||||
config["Loss"]["ignore_index"] = char_num - 1
|
||||
model = build_model(config["Architecture"])
|
||||
|
||||
pre_best_model_dict = dict()
|
||||
# load fp32 model to begin quantization
|
||||
pre_best_model_dict = load_model(
|
||||
config, model, None, config["Architecture"]["model_type"]
|
||||
)
|
||||
|
||||
freeze_params = False
|
||||
if config["Architecture"]["algorithm"] in ["Distillation"]:
|
||||
for key in config["Architecture"]["Models"]:
|
||||
freeze_params = freeze_params or config["Architecture"]["Models"][key].get(
|
||||
"freeze_params", False
|
||||
)
|
||||
act = None if freeze_params else PACT
|
||||
quanter = QAT(config=quant_config, act_preprocess=act)
|
||||
quanter.quantize(model)
|
||||
|
||||
if config["Global"]["distributed"]:
|
||||
model = paddle.DataParallel(model)
|
||||
|
||||
# build loss
|
||||
loss_class = build_loss(config["Loss"])
|
||||
|
||||
# build optim
|
||||
optimizer, lr_scheduler = build_optimizer(
|
||||
config["Optimizer"],
|
||||
epochs=config["Global"]["epoch_num"],
|
||||
step_each_epoch=len(train_dataloader),
|
||||
model=model,
|
||||
)
|
||||
|
||||
# resume PACT training process
|
||||
pre_best_model_dict = load_model(
|
||||
config, model, optimizer, config["Architecture"]["model_type"]
|
||||
)
|
||||
|
||||
# build metric
|
||||
eval_class = build_metric(config["Metric"])
|
||||
|
||||
logger.info(
|
||||
"train dataloader has {} iters, valid dataloader has {} iters".format(
|
||||
len(train_dataloader), len(valid_dataloader)
|
||||
)
|
||||
)
|
||||
|
||||
# start train
|
||||
program.train(
|
||||
config,
|
||||
train_dataloader,
|
||||
valid_dataloader,
|
||||
device,
|
||||
model,
|
||||
loss_class,
|
||||
optimizer,
|
||||
lr_scheduler,
|
||||
post_process_class,
|
||||
eval_class,
|
||||
pre_best_model_dict,
|
||||
logger,
|
||||
vdl_writer,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
config, device, logger, vdl_writer = program.preprocess(is_train=True)
|
||||
main(config, device, logger, vdl_writer)
|
||||
172
deploy/slim/quantization/quant_kl.py
Executable file
172
deploy/slim/quantization/quant_kl.py
Executable file
@@ -0,0 +1,172 @@
|
||||
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
from __future__ import absolute_import
|
||||
from __future__ import division
|
||||
from __future__ import print_function
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
__dir__ = os.path.dirname(os.path.abspath(__file__))
|
||||
sys.path.append(__dir__)
|
||||
sys.path.append(os.path.abspath(os.path.join(__dir__, "..", "..", "..")))
|
||||
sys.path.append(os.path.abspath(os.path.join(__dir__, "..", "..", "..", "tools")))
|
||||
|
||||
import yaml
|
||||
import paddle
|
||||
import paddle.distributed as dist
|
||||
|
||||
paddle.seed(2)
|
||||
|
||||
from ppocr.data import build_dataloader, set_signal_handlers
|
||||
from ppocr.modeling.architectures import build_model
|
||||
from ppocr.losses import build_loss
|
||||
from ppocr.optimizer import build_optimizer
|
||||
from ppocr.postprocess import build_post_process
|
||||
from ppocr.metrics import build_metric
|
||||
from ppocr.utils.save_load import load_model
|
||||
import tools.program as program
|
||||
import paddleslim
|
||||
from paddleslim.dygraph.quant import QAT
|
||||
import numpy as np
|
||||
|
||||
dist.get_world_size()
|
||||
|
||||
|
||||
class PACT(paddle.nn.Layer):
|
||||
def __init__(self):
|
||||
super(PACT, self).__init__()
|
||||
alpha_attr = paddle.ParamAttr(
|
||||
name=self.full_name() + ".pact",
|
||||
initializer=paddle.nn.initializer.Constant(value=20),
|
||||
learning_rate=1.0,
|
||||
regularizer=paddle.regularizer.L2Decay(2e-5),
|
||||
)
|
||||
|
||||
self.alpha = self.create_parameter(shape=[1], attr=alpha_attr, dtype="float32")
|
||||
|
||||
def forward(self, x):
|
||||
out_left = paddle.nn.functional.relu(x - self.alpha)
|
||||
out_right = paddle.nn.functional.relu(-self.alpha - x)
|
||||
x = x - out_left + out_right
|
||||
return x
|
||||
|
||||
|
||||
quant_config = {
|
||||
# weight preprocess type, default is None and no preprocessing is performed.
|
||||
"weight_preprocess_type": None,
|
||||
# activation preprocess type, default is None and no preprocessing is performed.
|
||||
"activation_preprocess_type": None,
|
||||
# weight quantize type, default is 'channel_wise_abs_max'
|
||||
"weight_quantize_type": "channel_wise_abs_max",
|
||||
# activation quantize type, default is 'moving_average_abs_max'
|
||||
"activation_quantize_type": "moving_average_abs_max",
|
||||
# weight quantize bit num, default is 8
|
||||
"weight_bits": 8,
|
||||
# activation quantize bit num, default is 8
|
||||
"activation_bits": 8,
|
||||
# data type after quantization, such as 'uint8', 'int8', etc. default is 'int8'
|
||||
"dtype": "int8",
|
||||
# window size for 'range_abs_max' quantization. default is 10000
|
||||
"window_size": 10000,
|
||||
# The decay coefficient of moving average, default is 0.9
|
||||
"moving_rate": 0.9,
|
||||
# for dygraph quantization, layers of type in quantizable_layer_type will be quantized
|
||||
"quantizable_layer_type": ["Conv2D", "Linear"],
|
||||
}
|
||||
|
||||
|
||||
def sample_generator(loader):
|
||||
def __reader__():
|
||||
for indx, data in enumerate(loader):
|
||||
images = np.array(data[0])
|
||||
yield images
|
||||
|
||||
return __reader__
|
||||
|
||||
|
||||
def sample_generator_layoutxlm_ser(loader):
|
||||
def __reader__():
|
||||
for indx, data in enumerate(loader):
|
||||
input_ids = np.array(data[0])
|
||||
bbox = np.array(data[1])
|
||||
attention_mask = np.array(data[2])
|
||||
token_type_ids = np.array(data[3])
|
||||
images = np.array(data[4])
|
||||
yield [input_ids, bbox, attention_mask, token_type_ids, images]
|
||||
|
||||
return __reader__
|
||||
|
||||
|
||||
def main(config, device, logger, vdl_writer):
|
||||
# init dist environment
|
||||
if config["Global"]["distributed"]:
|
||||
dist.init_parallel_env()
|
||||
|
||||
global_config = config["Global"]
|
||||
|
||||
# build dataloader
|
||||
set_signal_handlers()
|
||||
config["Train"]["loader"]["num_workers"] = 0
|
||||
is_layoutxlm_ser = (
|
||||
config["Architecture"]["model_type"] == "kie"
|
||||
and config["Architecture"]["Backbone"]["name"] == "LayoutXLMForSer"
|
||||
)
|
||||
train_dataloader = build_dataloader(config, "Train", device, logger)
|
||||
if config["Eval"]:
|
||||
config["Eval"]["loader"]["num_workers"] = 0
|
||||
valid_dataloader = build_dataloader(config, "Eval", device, logger)
|
||||
if is_layoutxlm_ser:
|
||||
train_dataloader = valid_dataloader
|
||||
else:
|
||||
valid_dataloader = None
|
||||
|
||||
paddle.enable_static()
|
||||
exe = paddle.static.Executor(device)
|
||||
|
||||
if "inference_model" in global_config.keys(): # , 'inference_model'):
|
||||
inference_model_dir = global_config["inference_model"]
|
||||
else:
|
||||
inference_model_dir = os.path.dirname(global_config["pretrained_model"])
|
||||
if not (
|
||||
os.path.exists(os.path.join(inference_model_dir, "inference.pdmodel"))
|
||||
and os.path.exists(os.path.join(inference_model_dir, "inference.pdiparams"))
|
||||
):
|
||||
raise ValueError(
|
||||
"Please set inference model dir in Global.inference_model or Global.pretrained_model for post-quantization"
|
||||
)
|
||||
|
||||
if is_layoutxlm_ser:
|
||||
generator = sample_generator_layoutxlm_ser(train_dataloader)
|
||||
else:
|
||||
generator = sample_generator(train_dataloader)
|
||||
|
||||
paddleslim.quant.quant_post_static(
|
||||
executor=exe,
|
||||
model_dir=inference_model_dir,
|
||||
model_filename="inference.pdmodel",
|
||||
params_filename="inference.pdiparams",
|
||||
quantize_model_path=global_config["save_inference_dir"],
|
||||
sample_generator=generator,
|
||||
save_model_filename="inference.pdmodel",
|
||||
save_params_filename="inference.pdiparams",
|
||||
batch_size=1,
|
||||
batch_nums=None,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
config, device, logger, vdl_writer = program.preprocess(is_train=True)
|
||||
main(config, device, logger, vdl_writer)
|
||||
Reference in New Issue
Block a user