This commit is contained in:
224
docs/version2.x/algorithm/end_to_end/algorithm_e2e_pgnet.en.md
Normal file
224
docs/version2.x/algorithm/end_to_end/algorithm_e2e_pgnet.en.md
Normal file
@@ -0,0 +1,224 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
## 1. Brief Introduction
|
||||
|
||||
OCR algorithms can be divided into two categories: two-stage algorithm and end-to-end algorithm. The two-stage OCR algorithm is generally divided into two parts, text detection and text recognition algorithm. The text detection algorithm locates the box of the text line from the image, and then the recognition algorithm identifies the content of the text box. The end-to-end OCR algorithm combines text detection and recognition in one algorithm. Its basic idea is to design a model with both detection unit and recognition module, share the CNN features of both and train them together. Because one algorithm can complete character recognition, the end-to-end model is smaller and faster.
|
||||
|
||||
### Introduction Of PGNet Algorithm
|
||||
|
||||
During the recent years, the end-to-end OCR algorithm has been well developed, including MaskTextSpotter series, TextSnake, TextDragon, PGNet series and so on. Among these algorithms, PGNet algorithm has some advantages over the other algorithms.
|
||||
|
||||
- PGNet loss is designed to guide training, and no character-level annotations is needed.
|
||||
- NMS and ROI related operations are not needed. It can accelerate the prediction
|
||||
- The reading order prediction module is proposed
|
||||
- A graph based modification module (GRM) is proposed to further improve the performance of model recognition
|
||||
- Higher accuracy and faster prediction speed
|
||||
|
||||
For details of PGNet algorithm, please refer to [paper](https://www.aaai.org/AAAI21Papers/AAAI-2885.WangP.pdf). The schematic diagram of the algorithm is as follows:
|
||||
|
||||

|
||||
|
||||
After feature extraction, the input image is sent to four branches: TBO module for text edge offset prediction, TCL module for text center-line prediction, TDO module for text direction offset prediction, and TCC module for text character classification graph prediction.
|
||||
The output of TBO and TCL can get text detection results after post-processing, and TCL, TDO and TCC are responsible for text recognition.
|
||||
|
||||
The results of detection and recognition are as follows:
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
### Performance
|
||||
|
||||
#### Test set: Total Text
|
||||
|
||||
#### Test environment: NVIDIA Tesla V100-SXM2-16GB
|
||||
|
||||
|PGNetA|det_precision|det_recall|det_f_score|e2e_precision|e2e_recall|e2e_f_score|FPS|download|
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
|Paper|85.30|86.80|86.10|-|-|61.70|38.20 (size=640)|-|
|
||||
|Ours|87.03|82.48|84.69|61.71|58.43|60.03|48.73 (size=768)|[download link](https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/en_server_pgnetA.tar)|
|
||||
|
||||
*note:PGNet in PaddleOCR optimizes the prediction speed, and can significantly improve the end-to-end prediction speed within the acceptable range of accuracy reduction*
|
||||
|
||||
## 2. Environment Configuration
|
||||
|
||||
Please refer to [Operation Environment Preparation](../../ppocr/environment.en.md) to configure PaddleOCR operating environment first, refer to [Project Clone](../../ppocr/blog/clone.en.md) to clone the project
|
||||
|
||||
## 3. Quick Use
|
||||
|
||||
### Inference model download
|
||||
|
||||
This section takes the trained end-to-end model as an example to quickly use the model prediction. First, download the trained end-to-end inference model [download address](https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/e2e_server_pgnetA_infer.tar)
|
||||
|
||||
```bash linenums="1"
|
||||
mkdir inference && cd inference
|
||||
# Download the English end-to-end model and unzip it
|
||||
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/e2e_server_pgnetA_infer.tar && tar xf e2e_server_pgnetA_infer.tar
|
||||
```
|
||||
|
||||
- In Windows environment, if 'wget' is not installed, the link can be copied to the browser when downloading the model, and decompressed and placed in the corresponding directory
|
||||
|
||||
After decompression, there should be the following file structure:
|
||||
|
||||
```text linenums="1"
|
||||
├── e2e_server_pgnetA_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
### Single image or image set prediction
|
||||
|
||||
```bash linenums="1"
|
||||
# Prediction single image specified by image_dir
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/img623.jpg" --e2e_model_dir="./inference/e2e_server_pgnetA_infer/" --e2e_pgnet_valid_set="totaltext"
|
||||
|
||||
# Prediction the collection of images specified by image_dir
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/" --e2e_model_dir="./inference/e2e_server_pgnetA_infer/" --e2e_pgnet_valid_set="totaltext"
|
||||
|
||||
# If you want to use CPU for prediction, you need to set use_gpu parameter is false
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/img623.jpg" --e2e_model_dir="./inference/e2e_server_pgnetA_infer/" --use_gpu=False --e2e_pgnet_valid_set="totaltext"
|
||||
```
|
||||
|
||||
### Visualization results
|
||||
|
||||
The visualized end-to-end results are saved to the `./inference_results` folder by default, and the name of the result file is prefixed with 'e2e_res'. Examples of results are as follows:
|
||||
|
||||

|
||||
|
||||
## 4. Model Training,Evaluation And Inference
|
||||
|
||||
This section takes the totaltext dataset as an example to introduce the training, evaluation and testing of the end-to-end model in PaddleOCR.
|
||||
|
||||
### Data Preparation
|
||||
|
||||
Download and unzip [totaltext](https://paddleocr.bj.bcebos.com/dataset/total_text.tar) dataset to PaddleOCR/train_data/, dataset organization structure is as follow:
|
||||
|
||||
```text linenums="1"
|
||||
/PaddleOCR/train_data/total_text/train/
|
||||
|- rgb/ # total_text training data of dataset
|
||||
|- img11.png
|
||||
| ...
|
||||
|- train.txt # total_text training annotation of dataset
|
||||
```
|
||||
|
||||
total_text.txt: the format of dimension file is as follows,the file name and annotation information are separated by "\t":
|
||||
|
||||
```text linenums="1"
|
||||
" Image file name Image annotation information encoded by json.dumps"
|
||||
rgb/img11.jpg [{"transcription": "ASRAMA", "points": [[214.0, 325.0], [235.0, 308.0], [259.0, 296.0], [286.0, 291.0], [313.0, 295.0], [338.0, 305.0], [362.0, 320.0], [349.0, 347.0], [330.0, 337.0], [310.0, 329.0], [290.0, 324.0], [269.0, 328.0], [249.0, 336.0], [231.0, 346.0]]}, {...}]
|
||||
```
|
||||
|
||||
The image annotation after **json.dumps()** encoding is a list containing multiple dictionaries.
|
||||
|
||||
The `points` in the dictionary represent the coordinates (x, y) of the four points of the text box, arranged clockwise from the point at the upper left corner.
|
||||
|
||||
`transcription` represents the text of the current text box. **When its content is "###" it means that the text box is invalid and will be skipped during training.**
|
||||
|
||||
If you want to train PaddleOCR on other datasets, please build the annotation file according to the above format.
|
||||
|
||||
### Start Training
|
||||
|
||||
PGNet training is divided into two steps: Step 1: training on the synthetic data to get the pretrain_model, and the accuracy of the model is still low; step 2: loading the pretrain_model and training on the totaltext data set; for fast training, we directly provide the pre training model of step 1[download link](https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/train_step1.tar).
|
||||
|
||||
```bash linenums="1"
|
||||
cd PaddleOCR/
|
||||
|
||||
# download step1 pretrain_models
|
||||
wget -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/train_step1.tar
|
||||
|
||||
# You can get the following file format
|
||||
./pretrain_models/train_step1/
|
||||
└─ best_accuracy.pdopt
|
||||
└─ best_accuracy.states
|
||||
└─ best_accuracy.pdparams
|
||||
```
|
||||
|
||||
*If CPU version installed, please set the parameter `use_gpu` to `false` in the configuration.*
|
||||
|
||||
```bash linenums="1"
|
||||
# single GPU training
|
||||
python3 tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.pretrained_model=./pretrain_models/train_step1/best_accuracy Global.load_static_weights=False
|
||||
# multi-GPU training
|
||||
# Set the GPU ID used by the '--gpus' parameter.
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.pretrained_model=./pretrain_models/train_step1/best_accuracy Global.load_static_weights=False
|
||||
```
|
||||
|
||||
In the above instruction, use `-c` to select the training to use the `configs/e2e/e2e_r50_vd_pg.yml` configuration file.
|
||||
For a detailed explanation of the configuration file, please refer to [config](../../ppocr/blog/config.en.md).
|
||||
|
||||
You can also use `-o` to change the training parameters without modifying the yml file. For example, adjust the training learning rate to 0.0001
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Optimizer.base_lr=0.0001
|
||||
```
|
||||
|
||||
#### Load trained model and continue training
|
||||
|
||||
If you would like to load trained model and continue the training again, you can specify the parameter `Global.checkpoints` as the model path to be loaded.
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.checkpoints=./your/trained/model
|
||||
```
|
||||
|
||||
**Note**: The priority of `Global.checkpoints` is higher than that of `Global.pretrain_weights`, that is, when two parameters are specified at the same time, the model specified by `Global.checkpoints` will be loaded first. If the model path specified by `Global.checkpoints` is wrong, the one specified by `Global.pretrain_weights` will be loaded.
|
||||
|
||||
PaddleOCR calculates three indicators for evaluating performance of OCR end-to-end task: Precision, Recall, and Hmean.
|
||||
|
||||
Run the following code to calculate the evaluation indicators. The result will be saved in the test result file specified by `save_res_path` in the configuration file `e2e_r50_vd_pg.yml`
|
||||
When evaluating, set post-processing parameters `max_side_len=768`. If you use different datasets, different models for training.
|
||||
The model parameters during training are saved in the `Global.save_model_dir` directory by default. When evaluating indicators, you need to set `Global.checkpoints` to point to the saved parameter file.
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/eval.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.checkpoints="{path/to/weights}/best_accuracy"
|
||||
```
|
||||
|
||||
### Model Test
|
||||
|
||||
Test the end-to-end result on a single image:
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/infer_e2e.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.infer_img="./doc/imgs_en/img_10.jpg" Global.pretrained_model="./output/e2e_pgnet/best_accuracy" Global.load_static_weights=false
|
||||
```
|
||||
|
||||
Test the end-to-end result on all images in the folder:
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/infer_e2e.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.infer_img="./doc/imgs_en/" Global.pretrained_model="./output/e2e_pgnet/best_accuracy" Global.load_static_weights=false
|
||||
```
|
||||
|
||||
### Model inference
|
||||
|
||||
#### (1).Quadrangle text detection model (ICDAR2015)
|
||||
|
||||
First, convert the model saved in the PGNet end-to-end training process into an inference model. In the first stage of training based on composite dataset, the model of English data set training is taken as an example[model download link](https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/en_server_pgnetA.tar), you can use the following command to convert:
|
||||
|
||||
```bash linenums="1"
|
||||
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/en_server_pgnetA.tar && tar xf en_server_pgnetA.tar
|
||||
python3 tools/export_model.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.pretrained_model=./en_server_pgnetA/best_accuracy Global.load_static_weights=False Global.save_inference_dir=./inference/e2e
|
||||
```
|
||||
|
||||
**For PGNet quadrangle end-to-end model inference, you need to set the parameter `--e2e_algorithm="PGNet"` and `--e2e_pgnet_valid_set="partvgg"`**, run the following command:
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/img_10.jpg" --e2e_model_dir="./inference/e2e/" --e2e_pgnet_valid_set="partvgg"
|
||||
```
|
||||
|
||||
The visualized text detection results are saved to the `./inference_results` folder by default, and the name of the result file is prefixed with 'e2e_res'. Examples of results are as follows:
|
||||
|
||||

|
||||
|
||||
#### (2). Curved text detection model (Total-Text)
|
||||
|
||||
For the curved text example, we use the same model as the quadrilateral
|
||||
**For PGNet end-to-end curved text detection model inference, you need to set the parameter `--e2e_algorithm="PGNet"` and `--e2e_pgnet_valid_set="totaltext"`**, run the following command:
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/img623.jpg" --e2e_model_dir="./inference/e2e/" --e2e_pgnet_valid_set="totaltext"
|
||||
```
|
||||
|
||||
The visualized text detection results are saved to the `./inference_results` folder by default, and the name of the result file is prefixed with 'e2e_res'. Examples of results are as follows:
|
||||
|
||||

|
||||
220
docs/version2.x/algorithm/end_to_end/algorithm_e2e_pgnet.md
Normal file
220
docs/version2.x/algorithm/end_to_end/algorithm_e2e_pgnet.md
Normal file
@@ -0,0 +1,220 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
## 一、简介
|
||||
|
||||
OCR算法可以分为两阶段算法和端对端的算法。二阶段OCR算法一般分为两个部分,文本检测和文本识别算法,文件检测算法从图像中得到文本行的检测框,然后识别算法去识别文本框中的内容。而端对端OCR算法可以在一个算法中完成文字检测和文字识别,其基本思想是设计一个同时具有检测单元和识别模块的模型,共享其中两者的CNN特征,并联合训练。由于一个算法即可完成文字识别,端对端模型更小,速度更快。
|
||||
|
||||
### PGNet算法介绍
|
||||
|
||||
近些年来,端对端OCR算法得到了良好的发展,包括MaskTextSpotter系列、TextSnake、TextDragon、PGNet系列等算法。在这些算法中,PGNet算法具备其他算法不具备的优势,包括:
|
||||
|
||||
- 设计PGNet loss指导训练,不需要字符级别的标注
|
||||
- 不需要NMS和ROI相关操作,加速预测
|
||||
- 提出预测文本行内的阅读顺序模块;
|
||||
- 提出基于图的修正模块(GRM)来进一步提高模型识别性能
|
||||
- 精度更高,预测速度更快
|
||||
|
||||
PGNet算法细节详见[论文](https://www.aaai.org/AAAI21Papers/AAAI-2885.WangP.pdf) ,算法原理图如下所示:
|
||||
|
||||

|
||||
|
||||
输入图像经过特征提取送入四个分支,分别是:文本边缘偏移量预测TBO模块,文本中心线预测TCL模块,文本方向偏移量预测TDO模块,以及文本字符分类图预测TCC模块。
|
||||
其中TBO以及TCL的输出经过后处理后可以得到文本的检测结果,TCL、TDO、TCC负责文本识别。
|
||||
|
||||
其检测识别效果图如下:
|
||||
|
||||

|
||||
|
||||

|
||||
|
||||
### 性能指标
|
||||
|
||||
#### 测试集: Total Text
|
||||
|
||||
#### 测试环境: NVIDIA Tesla V100-SXM2-16GB
|
||||
|
||||
|PGNetA|det_precision|det_recall|det_f_score|e2e_precision|e2e_recall|e2e_f_score|FPS|下载|
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
|Paper|85.30|86.80|86.10|-|-|61.70|38.20 (size=640)|-|
|
||||
|Ours|87.03|82.48|84.69|61.71|58.43|60.03|48.73 (size=768)|[下载链接](https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/en_server_pgnetA.tar)|
|
||||
|
||||
*note:PaddleOCR里的PGNet实现针对预测速度做了优化,在精度下降可接受范围内,可以显著提升端对端预测速度*
|
||||
|
||||
## 二、环境配置
|
||||
|
||||
请先参考[《运行环境准备》](../../ppocr/environment.md)配置PaddleOCR运行环境,参考[《项目克隆》](../../ppocr/blog/clone.md)克隆项目
|
||||
|
||||
## 三、快速使用
|
||||
|
||||
### inference模型下载
|
||||
|
||||
本节以训练好的端到端模型为例,快速使用模型预测,首先下载训练好的端到端inference模型[下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/e2e_server_pgnetA_infer.tar)
|
||||
|
||||
```bash linenums="1"
|
||||
mkdir inference && cd inference
|
||||
# 下载英文端到端模型并解压
|
||||
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/e2e_server_pgnetA_infer.tar && tar xf e2e_server_pgnetA_infer.tar
|
||||
```
|
||||
|
||||
- windows 环境下如果没有安装wget,下载模型时可将链接复制到浏览器中下载,并解压放置在相应目录下
|
||||
|
||||
解压完毕后应有如下文件结构:
|
||||
|
||||
```text linenums="1"
|
||||
├── e2e_server_pgnetA_infer
|
||||
│ ├── inference.pdiparams
|
||||
│ ├── inference.pdiparams.info
|
||||
│ └── inference.pdmodel
|
||||
```
|
||||
|
||||
### 单张图像或者图像集合预测
|
||||
|
||||
```bash linenums="1"
|
||||
# 预测image_dir指定的单张图像
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/img623.jpg" --e2e_model_dir="./inference/e2e_server_pgnetA_infer/" --e2e_pgnet_valid_set="totaltext"
|
||||
|
||||
# 预测image_dir指定的图像集合
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/" --e2e_model_dir="./inference/e2e_server_pgnetA_infer/" --e2e_pgnet_valid_set="totaltext"
|
||||
|
||||
# 如果想使用CPU进行预测,需设置use_gpu参数为False
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/img623.jpg" --e2e_model_dir="./inference/e2e_server_pgnetA_infer/" --e2e_pgnet_valid_set="totaltext" --use_gpu=False
|
||||
```
|
||||
|
||||
### 可视化结果
|
||||
|
||||
可视化文本检测结果默认保存到./inference_results文件夹里面,结果文件的名称前缀为'e2e_res'。结果示例如下:
|
||||
|
||||

|
||||
|
||||
## 四、模型训练、评估、推理
|
||||
|
||||
本节以totaltext数据集为例,介绍PaddleOCR中端到端模型的训练、评估与测试。
|
||||
|
||||
### 准备数据
|
||||
|
||||
下载解压[totaltext](https://paddleocr.bj.bcebos.com/dataset/total_text.tar) 数据集到PaddleOCR/train_data/目录,数据集组织结构:
|
||||
|
||||
```text linenums="1"
|
||||
/PaddleOCR/train_data/total_text/train/
|
||||
|- rgb/ # total_text数据集的训练数据
|
||||
|- img11.jpg
|
||||
| ...
|
||||
|- train.txt # total_text数据集的训练标注
|
||||
```
|
||||
|
||||
train.txt标注文件格式如下,文件名和标注信息中间用"\t"分隔:
|
||||
|
||||
```text linenums="1"
|
||||
" 图像文件名 json.dumps编码的图像标注信息"
|
||||
rgb/img11.jpg [{"transcription": "ASRAMA", "points": [[214.0, 325.0], [235.0, 308.0], [259.0, 296.0], [286.0, 291.0], [313.0, 295.0], [338.0, 305.0], [362.0, 320.0], [349.0, 347.0], [330.0, 337.0], [310.0, 329.0], [290.0, 324.0], [269.0, 328.0], [249.0, 336.0], [231.0, 346.0]]}, {...}]
|
||||
```
|
||||
|
||||
json.dumps编码前的图像标注信息是包含多个字典的list,字典中的 `points` 表示文本框的四个点的坐标(x, y),从左上角的点开始顺时针排列。
|
||||
`transcription` 表示当前文本框的文字,**当其内容为“###”时,表示该文本框无效,在训练时会跳过。**
|
||||
如果您想在其他数据集上训练,可以按照上述形式构建标注文件。
|
||||
|
||||
### 启动训练
|
||||
|
||||
PGNet训练分为两个步骤:step1: 在合成数据上训练,得到预训练模型,此时模型精度依然较低;step2: 加载预训练模型,在totaltext数据集上训练;为快速训练,我们直接提供了step1的预训练模型。
|
||||
|
||||
```bash linenums="1"
|
||||
cd PaddleOCR/
|
||||
# 下载step1 预训练模型
|
||||
wget -P ./pretrain_models/ https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/train_step1.tar
|
||||
|
||||
# 可以得到以下的文件格式
|
||||
./pretrain_models/train_step1/
|
||||
└─ best_accuracy.pdopt
|
||||
└─ best_accuracy.states
|
||||
└─ best_accuracy.pdparams
|
||||
```
|
||||
|
||||
*如果您安装的是cpu版本,请将配置文件中的 `use_gpu` 字段修改为false*
|
||||
|
||||
```bash linenums="1"
|
||||
# 单机单卡训练 e2e 模型
|
||||
python3 tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.pretrained_model=./pretrain_models/train_step1/best_accuracy Global.load_static_weights=False
|
||||
# 单机多卡训练,通过 --gpus 参数设置使用的GPU ID
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.pretrained_model=./pretrain_models/train_step1/best_accuracy Global.load_static_weights=False
|
||||
```
|
||||
|
||||
上述指令中,通过-c 选择训练使用configs/e2e/e2e_r50_vd_pg.yml配置文件。
|
||||
有关配置文件的详细解释,请参考[链接](../../ppocr/blog/config.md)。
|
||||
|
||||
您也可以通过-o参数在不需要修改yml文件的情况下,改变训练的参数,比如,调整训练的学习率为0.0001
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Optimizer.base_lr=0.0001
|
||||
```
|
||||
|
||||
#### 断点训练
|
||||
|
||||
如果训练程序中断,如果希望加载训练中断的模型从而恢复训练,可以通过指定Global.checkpoints指定要加载的模型路径:
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/train.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.checkpoints=./your/trained/model
|
||||
```
|
||||
|
||||
**注意**:`Global.checkpoints`的优先级高于`Global.pretrain_weights`的优先级,即同时指定两个参数时,优先加载`Global.checkpoints`指定的模型,如果`Global.checkpoints`指定的模型路径有误,会加载`Global.pretrain_weights`指定的模型。
|
||||
|
||||
PaddleOCR计算三个OCR端到端相关的指标,分别是:Precision、Recall、Hmean。
|
||||
|
||||
运行如下代码,根据配置文件`e2e_r50_vd_pg.yml`中`save_res_path`指定的测试集检测结果文件,计算评估指标。
|
||||
|
||||
评估时设置后处理参数`max_side_len=768`,使用不同数据集、不同模型训练,可调整参数进行优化
|
||||
训练中模型参数默认保存在`Global.save_model_dir`目录下。在评估指标时,需要设置`Global.checkpoints`指向保存的参数文件。
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/eval.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.checkpoints="{path/to/weights}/best_accuracy"
|
||||
```
|
||||
|
||||
### 模型预测
|
||||
|
||||
测试单张图像的端到端识别效果
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/infer_e2e.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.infer_img="./doc/imgs_en/img_10.jpg" Global.pretrained_model="./output/e2e_pgnet/best_accuracy" Global.load_static_weights=false
|
||||
```
|
||||
|
||||
测试文件夹下所有图像的端到端识别效果
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/infer_e2e.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.infer_img="./doc/imgs_en/" Global.pretrained_model="./output/e2e_pgnet/best_accuracy" Global.load_static_weights=false
|
||||
```
|
||||
|
||||
### 预测推理
|
||||
|
||||
#### (1). 四边形文本检测模型(ICDAR2015)
|
||||
|
||||
首先将PGNet端到端训练过程中保存的模型,转换成inference model。以基于Resnet50_vd骨干网络,以英文数据集训练的模型为例[模型下载地址](https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/en_server_pgnetA.tar) ,可以使用如下命令进行转换:
|
||||
|
||||
```bash linenums="1"
|
||||
wget https://paddleocr.bj.bcebos.com/dygraph_v2.0/pgnet/en_server_pgnetA.tar && tar xf en_server_pgnetA.tar
|
||||
python3 tools/export_model.py -c configs/e2e/e2e_r50_vd_pg.yml -o Global.pretrained_model=./en_server_pgnetA/best_accuracy Global.load_static_weights=False Global.save_inference_dir=./inference/e2e
|
||||
```
|
||||
|
||||
**PGNet端到端模型推理,需要设置参数`--e2e_algorithm="PGNet"` and `--e2e_pgnet_valid_set="partvgg"`**,可以执行如下命令:
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/img_10.jpg" --e2e_model_dir="./inference/e2e/" --e2e_pgnet_valid_set="partvgg" --e2e_pgnet_valid_set="totaltext"
|
||||
```
|
||||
|
||||
可视化文本检测结果默认保存到`./inference_results`文件夹里面,结果文件的名称前缀为'e2e_res'。结果示例如下:
|
||||
|
||||

|
||||
|
||||
#### (2). 弯曲文本检测模型(Total-Text)
|
||||
|
||||
对于弯曲文本样例
|
||||
|
||||
**PGNet端到端模型推理,需要设置参数`--e2e_algorithm="PGNet"`,同时,还需要增加参数`--e2e_pgnet_valid_set="totaltext"`,**可以执行如下命令:
|
||||
|
||||
```bash linenums="1"
|
||||
python3 tools/infer/predict_e2e.py --e2e_algorithm="PGNet" --image_dir="./doc/imgs_en/img623.jpg" --e2e_model_dir="./inference/e2e/" --e2e_pgnet_valid_set="totaltext"
|
||||
```
|
||||
|
||||
可视化文本端到端结果默认保存到`./inference_results`文件夹里面,结果文件的名称前缀为'e2e_res'。结果示例如下:
|
||||
|
||||

|
||||
Binary file not shown.
|
After Width: | Height: | Size: 663 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 467 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 134 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 337 KiB |
BIN
docs/version2.x/algorithm/end_to_end/images/pgnet_framework.png
Normal file
BIN
docs/version2.x/algorithm/end_to_end/images/pgnet_framework.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 242 KiB |
Reference in New Issue
Block a user