This commit is contained in:
64
docs/version3.x/algorithm/PP-ChatOCRv4/PP-ChatOCRv4.en.md
Normal file
64
docs/version3.x/algorithm/PP-ChatOCRv4/PP-ChatOCRv4.en.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# Introduction to PP-ChatOCRV4
|
||||
**PP-ChatOCRv4** is a unique document and image intelligent analysis solution from PaddlePaddle, combining LLM, MLLM, and OCR technologies to address complex document information extraction challenges such as layout analysis, rare characters, multi-page PDFs, tables, and seal recognition. Integrated with ERNIE Bot, it fuses massive data and knowledge, achieving high accuracy and wide applicability. This pipeline also provides flexible service deployment options, supporting deployment on various hardware. Furthermore, it offers custom development capabilities, allowing you to train and fine-tune models on your own datasets, with seamless integration of trained models.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# Key Metrics
|
||||
|
||||
<div align="center">
|
||||
<table>
|
||||
<thead>
|
||||
<tr >
|
||||
<th class>Solution</td>
|
||||
<th class>Avg Recall</td>
|
||||
</tr>
|
||||
<thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>GPT-4o</td>
|
||||
<td>63.47%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-ChatOCRv3</td>
|
||||
<td class>70.08%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>80.26%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-ChatOCRv4</b></td>
|
||||
<td><b>85.55%</b></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
# Demo
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo1.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo2.png" width="350"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo3.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo4.png" width="350"/>
|
||||
</div>
|
||||
|
||||
|
||||
# FAQ
|
||||
|
||||
1. Does support other multimodal models?
|
||||
|
||||
Yes, only set on pipeline configuration.
|
||||
|
||||
2. How to reduce latency and improve throughput?
|
||||
|
||||
Use the High-performance inference plugin, and deploy multi instances.
|
||||
|
||||
3. How to further improve accuracy?
|
||||
|
||||
Firstly, it is necessary to check whether the extracted visual information is correct. If the visual information is incorrect, it is necessary to visualize the visual prediction results to determine which model performs poorly, and then fine-tune train the model with more data. If the visual information is correct but cannot extract the correct information, the prompt needs to be adjusted according to the analysing about the question and answer.
|
||||
65
docs/version3.x/algorithm/PP-ChatOCRv4/PP-ChatOCRv4.md
Normal file
65
docs/version3.x/algorithm/PP-ChatOCRv4/PP-ChatOCRv4.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# 一、PP-ChatOCRV4简介
|
||||
**PP-ChatOCRv4**是飞桨特色的文档图像智能分析解决方案，结合了 LLM、MLLM 和 OCR 等技术，一站式解决版面分析、生僻字识别、多页 PDF 文件批量解析、复杂表格识别、印章识别等常见的复杂文档信息抽取难点问题，结合文心大模型将海量数据和知识相融合，信息抽取准确率高且应用广泛。本产线同时提供了灵活的服务化部署方式，支持在多种硬件上部署。不仅如此，本产线也提供了二次开发的能力，您可以基于本产线在您自己的数据集上训练调优，训练后的模型也可以无缝集成。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# 二、关键指标
|
||||
|
||||
<div align="center">
|
||||
<table>
|
||||
<thead>
|
||||
<tr >
|
||||
<th class>Solution</td>
|
||||
<th class>Avg Recall</td>
|
||||
</tr>
|
||||
<thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>GPT-4o</td>
|
||||
<td>63.47%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-ChatOCRv3</td>
|
||||
<td class>70.08%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>80.26%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-ChatOCRv4</b></td>
|
||||
<td><b>85.55%</b></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
# 三、PP-ChatOCRv4 Demo示例
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo1.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo2.png" width="350"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo3.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo4.png" width="350"/>
|
||||
</div>
|
||||
|
||||
# 四、使用方法和常见问题
|
||||
|
||||
1. 多模态大模型除了DocBee，是否支持其他多模态模型？
|
||||
|
||||
支持，只需在配置文件中进行设置即可。
|
||||
|
||||
2. 如何降低时延、提升吞吐？
|
||||
|
||||
无论使用哪一种服务化部署方案，都可以通过启用高性能推理插件提升模型推理速度，从而降低处理时延。
|
||||
|
||||
此外，对于高稳定性服务化部署方案，通过调整服务配置，设置多个实例，也可以充分利用部署机器的资源，有效提升吞吐。
|
||||
|
||||
3. 如何进一步提升精度？
|
||||
|
||||
首先需要检查提取的视觉信息是否正确，如果视觉信息有误，则需要通过可视化视觉预测结果，判断哪个模型效果较差，从而针对性地训练微调较差的模型；如果视觉信息无误，但无法抽取正确信息，则需要根据问答的具体情况调整Prompt。
|
||||
337
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5.en.md
Normal file
337
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5.en.md
Normal file
@@ -0,0 +1,337 @@
|
||||
# Introduction to PP-OCRv5
|
||||
|
||||
**PP-OCRv5** is the new generation text recognition solution of PP-OCR, focusing on multi-scenario and multi-text type recognition. In terms of text types, PP-OCRv5 supports 5 major mainstream text types: Simplified Chinese, Chinese Pinyin, Traditional Chinese, English, and Japanese. For scenarios, PP-OCRv5 has upgraded recognition capabilities for challenging scenarios such as complex Chinese and English handwriting, vertical text, and uncommon characters. On internal complex evaluation sets across multiple scenarios, PP-OCRv5 achieved a 13 percentage point end-to-end improvement over PP-OCRv4.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/algorithm_ppocrv5.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# Key Metrics
|
||||
|
||||
### 1. Text Detection Metrics
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th>
|
||||
<th>Handwritten Chinese</th>
|
||||
<th>Handwritten English</th>
|
||||
<th>Printed Chinese</th>
|
||||
<th>Printed English</th>
|
||||
<th>Traditional Chinese</th>
|
||||
<th>Ancient Text</th>
|
||||
<th>Japanese</th>
|
||||
<th>General Scenario</th>
|
||||
<th>Pinyin</th>
|
||||
<th>Rotation</th>
|
||||
<th>Distortion</th>
|
||||
<th>Artistic Text</th>
|
||||
<th>Average</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><b>PP-OCRv5_server_det</b></td>
|
||||
<td><b>0.803</b></td>
|
||||
<td><b>0.841</b></td>
|
||||
<td><b>0.945</b></td>
|
||||
<td><b>0.917</b></td>
|
||||
<td><b>0.815</b></td>
|
||||
<td><b>0.676</b></td>
|
||||
<td><b>0.772</b></td>
|
||||
<td><b>0.797</b></td>
|
||||
<td><b>0.671</b></td>
|
||||
<td><b>0.8</b></td>
|
||||
<td><b>0.876</b></td>
|
||||
<td><b>0.673</b></td>
|
||||
<td><b>0.827</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td>0.706</td>
|
||||
<td>0.249</td>
|
||||
<td>0.888</td>
|
||||
<td>0.690</td>
|
||||
<td>0.759</td>
|
||||
<td>0.473</td>
|
||||
<td>0.685</td>
|
||||
<td>0.715</td>
|
||||
<td>0.542</td>
|
||||
<td>0.366</td>
|
||||
<td>0.775</td>
|
||||
<td>0.583</td>
|
||||
<td>0.662</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-OCRv5_mobile_det</b></td>
|
||||
<td><b>0.744</b></td>
|
||||
<td><b>0.777</b></td>
|
||||
<td><b>0.905</b></td>
|
||||
<td><b>0.910</b></td>
|
||||
<td><b>0.823</b></td>
|
||||
<td><b>0.581</b></td>
|
||||
<td><b>0.727</b></td>
|
||||
<td><b>0.721</b></td>
|
||||
<td><b>0.575</b></td>
|
||||
<td><b>0.647</b></td>
|
||||
<td><b>0.827</b></td>
|
||||
<td>0.525</td>
|
||||
<td><b>0.770</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td>0.583</td>
|
||||
<td>0.369</td>
|
||||
<td>0.872</td>
|
||||
<td>0.773</td>
|
||||
<td>0.663</td>
|
||||
<td>0.231</td>
|
||||
<td>0.634</td>
|
||||
<td>0.710</td>
|
||||
<td>0.430</td>
|
||||
<td>0.299</td>
|
||||
<td>0.715</td>
|
||||
<td><b>0.549</b></td>
|
||||
<td>0.624</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
Compared to PP-OCRv4, PP-OCRv5 shows significant improvement in all detection scenarios, especially in handwriting, ancient texts, and Japanese detection capabilities.
|
||||
|
||||
### 2. Text Recognition Metrics
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/ocrv5_rec_acc.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Evaluation Set Category</th>
|
||||
<th>Handwritten Chinese</th>
|
||||
<th>Handwritten English</th>
|
||||
<th>Printed Chinese</th>
|
||||
<th>Printed English</th>
|
||||
<th>Traditional Chinese</th>
|
||||
<th>Ancient Text</th>
|
||||
<th>Japanese</th>
|
||||
<th>Confusable Characters</th>
|
||||
<th>General Scenario</th>
|
||||
<th>Pinyin</th>
|
||||
<th>Vertical Text</th>
|
||||
<th>Artistic Text</th>
|
||||
<th>Weighted Average</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><b>0.5807</b></td>
|
||||
<td><b>0.5806</b></td>
|
||||
<td><b>0.9013</b></td>
|
||||
<td><b>0.8679</b></td>
|
||||
<td><b>0.7472</b></td>
|
||||
<td><b>0.6039</b></td>
|
||||
<td><b>0.7372</b></td>
|
||||
<td><b>0.5946</b></td>
|
||||
<td><b>0.8384</b></td>
|
||||
<td><b>0.7435</b></td>
|
||||
<td><b>0.9314</b></td>
|
||||
<td><b>0.6397</b></td>
|
||||
<td><b>0.8401</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td>0.3626</td>
|
||||
<td>0.2661</td>
|
||||
<td>0.8486</td>
|
||||
<td>0.6677</td>
|
||||
<td>0.4097</td>
|
||||
<td>0.3080</td>
|
||||
<td>0.4623</td>
|
||||
<td>0.5028</td>
|
||||
<td>0.8362</td>
|
||||
<td>0.2694</td>
|
||||
<td>0.5455</td>
|
||||
<td>0.5892</td>
|
||||
<td>0.5735</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><b>0.4166</b></td>
|
||||
<td><b>0.4944</b></td>
|
||||
<td><b>0.8605</b></td>
|
||||
<td><b>0.8753</b></td>
|
||||
<td><b>0.7199</b></td>
|
||||
<td><b>0.5786</b></td>
|
||||
<td><b>0.7577</b></td>
|
||||
<td><b>0.5570</b></td>
|
||||
<td>0.7703</td>
|
||||
<td><b>0.7248</b></td>
|
||||
<td><b>0.8089</b></td>
|
||||
<td>0.5398</td>
|
||||
<td><b>0.8015</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td>0.2980</td>
|
||||
<td>0.2550</td>
|
||||
<td>0.8398</td>
|
||||
<td>0.6598</td>
|
||||
<td>0.3218</td>
|
||||
<td>0.2593</td>
|
||||
<td>0.4724</td>
|
||||
<td>0.4599</td>
|
||||
<td><b>0.8106</b></td>
|
||||
<td>0.2593</td>
|
||||
<td>0.5924</td>
|
||||
<td><b>0.5555</b></td>
|
||||
<td>0.5301</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
A single model can cover multiple languages and text types, with recognition accuracy significantly ahead of previous generation products and mainstream open-source solutions.
|
||||
|
||||
# PP-OCRv5 Demo Examples
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/algorithm_ppocrv5_demo1.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/doc_images/PP-OCRv5/algorithm_ppocrv5_demo.pdf">More Demos</a>
|
||||
|
||||
## Reference Data for Inference Performance
|
||||
|
||||
Test Environment:
|
||||
|
||||
- NVIDIA Tesla V100
|
||||
- Intel Xeon Gold 6271C
|
||||
- PaddlePaddle 3.0.0
|
||||
|
||||
Tested on 200 images (including both general and document images). During testing, images are read from disk, so the image reading time and other associated overhead are also included in the total time consumption. If the images are preloaded into memory, the average time per image can be further reduced by approximately 25 ms.
|
||||
|
||||
Unless otherwise specified:
|
||||
|
||||
- PP-OCRv4_mobile_det and PP-OCRv4_mobile_rec models are used.
|
||||
- Document orientation classification, image correction, and text line orientation classification are not used.
|
||||
- `text_det_limit_type` is set to `"min"` and `text_det_limit_side_len` to `732`.
|
||||
|
||||
### 1. Comparison of Inference Performance Between PP-OCRv5 and PP-OCRv4
|
||||
|
||||
| Config | Description |
|
||||
| --------------- | ------------------------------------------------------------ |
|
||||
| v5_mobile | Uses PP-OCRv5_mobile_det and PP-OCRv5_mobile_rec models. |
|
||||
| v4_mobile | Uses PP-OCRv4_mobile_det and PP-OCRv4_mobile_rec models. |
|
||||
| v5_server | Uses PP-OCRv5_server_det and PP-OCRv5_server_rec models. |
|
||||
| v4_server | Uses PP-OCRv4_server_det and PP-OCRv4_server_rec models. |
|
||||
|
||||
**GPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) | Avg. GPU Utilization (%) | Peak VRAM Usage (MB) | Avg. VRAM Usage (MB) |
|
||||
| ------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- | ------------------------ | -------------------- | -------------------- |
|
||||
| v5_mobile | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| v4_mobile | 0.27 | 2246 | 111.20 | 1392.22 | 1318.76 | 28.90 | 1304.00 | 1166.46 |
|
||||
| v5_server | 0.70 | 929 | 105.31 | 1634.85 | 1428.55 | 36.21 | 5402.00 | 4685.13 |
|
||||
| v4_server | 0.44 | 1418 | 106.96 | 1455.34 | 1346.95 | 58.82 | 6760.00 | 5817.46 |
|
||||
|
||||
**GPU, with high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) | Avg. GPU Utilization (%) | Peak VRAM Usage (MB) | Avg. VRAM Usage (MB) |
|
||||
| ------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- | ------------------------ | -------------------- | -------------------- |
|
||||
| v5_mobile | 0.50 | 1301 | 106.50 | 1338.12 | 1155.86 | 11.97 | 4112.00 | 3536.36 |
|
||||
| v4_mobile | 0.21 | 2887 | 114.09 | 1113.27 | 1054.46 | 15.22 | 2072.00 | 1840.59 |
|
||||
| v5_server | 0.60 | 1084 | 105.73 | 1980.73 | 1776.20 | 22.10 | 12150.00 | 11849.40 |
|
||||
| v4_server | 0.36 | 1687 | 104.15 | 1186.42 | 1065.67 | 38.12 | 13058.00 | 12679.00 |
|
||||
|
||||
**CPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) |
|
||||
| ------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- |
|
||||
| v5_mobile | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| v4_mobile | 1.09 | 556 | 813.16 | 11996.30 | 6834.25 |
|
||||
| v5_server | 3.79 | 172 | 799.24 | 50216.00 | 27902.40 |
|
||||
| v4_server | 4.22 | 148 | 803.74 | 51428.70 | 28593.60 |
|
||||
|
||||
**CPU, with high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) |
|
||||
| ------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- |
|
||||
| v5_mobile | 1.14 | 571 | 339.68 | 3245.17 | 2560.55 |
|
||||
| v4_mobile | 0.68 | 892 | 443.00 | 3057.38 | 2329.44 |
|
||||
| v5_server | 3.56 | 183 | 797.03 | 45664.70 | 26905.90 |
|
||||
| v4_server | 4.22 | 148 | 803.74 | 51428.70 | 28593.60 |
|
||||
|
||||
> Note: PP-OCRv5 uses a larger dictionary in the recognition model, which increases inference time and causes slower performance compared to PP-OCRv4.
|
||||
|
||||
### 2. Impact of Auxiliary Features on PP-OCRv5 Inference Performance
|
||||
|
||||
| Config | Description |
|
||||
| --------------- | --------------------------------------------------------------------------------------------------------- |
|
||||
| base | No document orientation classification, no image correction, no text line orientation classification. |
|
||||
| with_textline | Includes text line orientation classification only. |
|
||||
| with_all | Includes document orientation classification, image correction, and text line orientation classification. |
|
||||
|
||||
**GPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) | Avg. GPU Utilization (%) | Peak VRAM Usage (MB) | Avg. VRAM Usage (MB) |
|
||||
| -------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- | ------------------------ | -------------------- | -------------------- |
|
||||
| base | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| with_textline | 0.59 | 1104 | 105.58 | 1765.64 | 1478.53 | 19.48 | 4350.00 | 3267.77 |
|
||||
| with_all | 1.02 | 600 | 104.92 | 1924.23 | 1628.50 | 10.96 | 2632.00 | 2217.01 |
|
||||
|
||||
**CPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) |
|
||||
| -------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- |
|
||||
| base | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| with_textline | 1.50 | 434 | 799.47 | 12007.20 | 6882.22 |
|
||||
| with_all | 1.93 | 316 | 646.49 | 11759.60 | 6940.54 |
|
||||
|
||||
> Note: Auxiliary features such as image unwarping can impact inference accuracy. More features do not necessarily yield better results and may increase resource usage.
|
||||
|
||||
### 3. Impact of Input Scaling Strategy in Text Detection Module on PP-OCRv5 Inference Performance
|
||||
|
||||
| Config | Description |
|
||||
| ----------------- | -------------------------------------------------------------------------------------- |
|
||||
| mobile_min_1280 | Uses `min` limit type and `text_det_limit_side_len=1280` with PP-OCRv5_mobile models. |
|
||||
| mobile_min_736 | Same as default, `min`, `side_len=736`. |
|
||||
| mobile_max_960 | Uses `max` limit type and `side_len=960`. |
|
||||
| mobile_max_640 | Uses `max` limit type and `side_len=640`. |
|
||||
| server_min_1280 | Uses `min`, `side_len=1280` with PP-OCRv5_server models. |
|
||||
| server_min_736 | Same as default, `min`, `side_len=736`. |
|
||||
| server_max_960 | Uses `max`, `side_len=960`. |
|
||||
| server_max_640 | Uses `max`, `side_len=640`. |
|
||||
|
||||
**GPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) | Avg. GPU Utilization (%) | Peak VRAM Usage (MB) | Avg. VRAM Usage (MB) |
|
||||
| ----------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- | ------------------------ | -------------------- | -------------------- |
|
||||
| mobile_min_1280 | 0.61 | 1071 | 109.12 | 1663.71 | 1439.72 | 19.27 | 4202.00 | 3550.32 |
|
||||
| mobile_min_736 | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| mobile_max_960 | 0.48 | 1313 | 103.49 | 1587.25 | 1395.48 | 19.37 | 2642.00 | 2319.03 |
|
||||
| mobile_max_640 | 0.42 | 1436 | 103.07 | 1651.14 | 1422.62 | 18.95 | 2530.00 | 2149.11 |
|
||||
| server_min_1280 | 0.82 | 795 | 107.17 | 1678.16 | 1428.94 | 40.43 | 10368.00 | 8320.43 |
|
||||
| server_min_736 | 0.70 | 929 | 105.31 | 1634.85 | 1428.55 | 36.21 | 5402.00 | 4685.13 |
|
||||
| server_max_960 | 0.59 | 1073 | 103.03 | 1590.19 | 1383.62 | 33.42 | 2928.00 | 2079.47 |
|
||||
| server_max_640 | 0.54 | 1099 | 102.63 | 1602.09 | 1416.49 | 30.77 | 3152.00 | 2737.81 |
|
||||
|
||||
**CPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) |
|
||||
| ----------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- |
|
||||
| mobile_min_1280 | 1.64 | 398 | 799.45 | 12344.10 | 7100.60 |
|
||||
| mobile_min_736 | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| mobile_max_960 | 1.21 | 521 | 800.13 | 11099.10 | 6369.49 |
|
||||
| mobile_max_640 | 1.01 | 597 | 802.52 | 9585.48 | 5573.52 |
|
||||
| server_min_1280 | 4.48 | 145 | 800.49 | 50683.10 | 28273.30 |
|
||||
| server_min_736 | 3.79 | 172 | 799.24 | 50216.00 | 27902.40 |
|
||||
| server_max_960 | 2.67 | 237 | 797.63 | 49362.50 | 26075.60 |
|
||||
| server_max_640 | 2.36 | 251 | 795.18 | 45656.10 | 24900.80 |
|
||||
|
||||
# Deployment and Secondary Development
|
||||
* **Multiple System Support**: Compatible with mainstream operating systems including Windows, Linux, and Mac.
|
||||
* **Multiple Hardware Support**: Besides NVIDIA GPUs, it also supports inference and deployment on Intel CPU, Kunlun chips, Ascend, and other new hardware.
|
||||
* **High-Performance Inference Plugin**: Recommended to combine with high-performance inference plugins to further improve inference speed. See [High-Performance Inference Guide](../../deployment/high_performance_inference.md) for details.
|
||||
* **Service Deployment**: Supports highly stable service deployment solutions. See [Service Deployment Guide](../../deployment/serving.md) for details.
|
||||
* **Secondary Development Capability**: Supports custom dataset training, dictionary extension, and model fine-tuning. Example: To add Korean recognition, you can extend the dictionary and fine-tune the model, seamlessly integrating into existing pipelines. See [Text Detection Module Usage Tutorial](../../module_usage/text_detection.en.md) and [Text Recognition Module Usage Tutorial](../../module_usage/text_recognition.en.md) for details.
|
||||
340
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5.md
Normal file
340
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5.md
Normal file
@@ -0,0 +1,340 @@
|
||||
# 一、PP-OCRv5简介
|
||||
**PP-OCRv5** 是PP-OCR新一代文字识别解决方案，该方案聚焦于多场景、多文字类型的文字识别。在文字类型方面，PP-OCRv5支持简体中文、中文拼音、繁体中文、英文、日文5大主流文字类型，在场景方面，PP-OCRv5升级了中英复杂手写体、竖排文本、生僻字等多种挑战性场景的识别能力。在内部多场景复杂评估集上，PP-OCRv5较PP-OCRv4端到端提升13个百分点。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/algorithm_ppocrv5.png" width="600"/>
|
||||
</div>
|
||||
|
||||
|
||||
# 二、关键指标
|
||||
### 1. 文本检测指标
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>手写中文</th>
|
||||
<th>手写英文</th>
|
||||
<th>印刷中文</th>
|
||||
<th>印刷英文</th>
|
||||
<th>繁体中文</th>
|
||||
<th>古籍文本</th>
|
||||
<th>日文</th>
|
||||
<th>通用场景</th>
|
||||
<th>拼音</th>
|
||||
<th>旋转</th>
|
||||
<th>扭曲</th>
|
||||
<th>艺术字</th>
|
||||
<th>平均</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><b>PP-OCRv5_server_det</b></td>
|
||||
<td><b>0.803</b></td>
|
||||
<td><b>0.841</b></td>
|
||||
<td><b>0.945</b></td>
|
||||
<td><b>0.917</b></td>
|
||||
<td><b>0.815</b></td>
|
||||
<td><b>0.676</b></td>
|
||||
<td><b>0.772</b></td>
|
||||
<td><b>0.797</b></td>
|
||||
<td><b>0.671</b></td>
|
||||
<td><b>0.8</b></td>
|
||||
<td><b>0.876</b></td>
|
||||
<td><b>0.673</b></td>
|
||||
<td><b>0.827</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td>0.706</td>
|
||||
<td>0.249</td>
|
||||
<td>0.888</td>
|
||||
<td>0.690</td>
|
||||
<td>0.759</td>
|
||||
<td>0.473</td>
|
||||
<td>0.685</td>
|
||||
<td>0.715</td>
|
||||
<td>0.542</td>
|
||||
<td>0.366</td>
|
||||
<td>0.775</td>
|
||||
<td>0.583</td>
|
||||
<td>0.662</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-OCRv5_mobile_det</b></td>
|
||||
<td><b>0.744</b></td>
|
||||
<td><b>0.777</b></td>
|
||||
<td><b>0.905</b></td>
|
||||
<td><b>0.910</b></td>
|
||||
<td><b>0.823</b></td>
|
||||
<td><b>0.581</b></td>
|
||||
<td><b>0.727</b></td>
|
||||
<td><b>0.721</b></td>
|
||||
<td><b>0.575</b></td>
|
||||
<td><b>0.647</b></td>
|
||||
<td><b>0.827</b></td>
|
||||
<td>0.525</td>
|
||||
<td><b>0.770</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td>0.583</td>
|
||||
<td>0.369</td>
|
||||
<td>0.872</td>
|
||||
<td>0.773</td>
|
||||
<td>0.663</td>
|
||||
<td>0.231</td>
|
||||
<td>0.634</td>
|
||||
<td>0.710</td>
|
||||
<td>0.430</td>
|
||||
<td>0.299</td>
|
||||
<td>0.715</td>
|
||||
<td><b>0.549</b></td>
|
||||
<td>0.624</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
对比PP-OCRv4，PP-OCRv5在所有检测场景下均有明显提升，尤其在手写、古籍、日文检测能力上表现更优。
|
||||
|
||||
### 2. 文本识别指标
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/ocrv5_rec_acc.png" width="600"/>
|
||||
</div>
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>评估集类别</th>
|
||||
<th>手写中文</th>
|
||||
<th>手写英文</th>
|
||||
<th>印刷中文</th>
|
||||
<th>印刷英文</th>
|
||||
<th>繁体中文</th>
|
||||
<th>古籍文本</th>
|
||||
<th>日文</th>
|
||||
<th>易混淆字符</th>
|
||||
<th>通用场景</th>
|
||||
<th>拼音</th>
|
||||
<th>竖直文本</th>
|
||||
<th>艺术字</th>
|
||||
<th>加权平均</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><b>0.5807</b></td>
|
||||
<td><b>0.5806</b></td>
|
||||
<td><b>0.9013</b></td>
|
||||
<td><b>0.8679</b></td>
|
||||
<td><b>0.7472</b></td>
|
||||
<td><b>0.6039</b></td>
|
||||
<td><b>0.7372</b></td>
|
||||
<td><b>0.5946</b></td>
|
||||
<td><b>0.8384</b></td>
|
||||
<td><b>0.7435</b></td>
|
||||
<td><b>0.9314</b></td>
|
||||
<td><b>0.6397</b></td>
|
||||
<td><b>0.8401</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td>0.3626</td>
|
||||
<td>0.2661</td>
|
||||
<td>0.8486</td>
|
||||
<td>0.6677</td>
|
||||
<td>0.4097</td>
|
||||
<td>0.3080</td>
|
||||
<td>0.4623</td>
|
||||
<td>0.5028</td>
|
||||
<td>0.8362</td>
|
||||
<td>0.2694</td>
|
||||
<td>0.5455</td>
|
||||
<td>0.5892</td>
|
||||
<td>0.5735</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><b>0.4166</b></td>
|
||||
<td><b>0.4944</b></td>
|
||||
<td><b>0.8605</b></td>
|
||||
<td><b>0.8753</b></td>
|
||||
<td><b>0.7199</b></td>
|
||||
<td><b>0.5786</b></td>
|
||||
<td><b>0.7577</b></td>
|
||||
<td><b>0.5570</b></td>
|
||||
<td>0.7703</td>
|
||||
<td><b>0.7248</b></td>
|
||||
<td><b>0.8089</b></td>
|
||||
<td>0.5398</td>
|
||||
<td><b>0.8015</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td>0.2980</td>
|
||||
<td>0.2550</td>
|
||||
<td>0.8398</td>
|
||||
<td>0.6598</td>
|
||||
<td>0.3218</td>
|
||||
<td>0.2593</td>
|
||||
<td>0.4724</td>
|
||||
<td>0.4599</td>
|
||||
<td><b>0.8106</b></td>
|
||||
<td>0.2593</td>
|
||||
<td>0.5924</td>
|
||||
<td><b>0.5555</b></td>
|
||||
<td>0.5301</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
单模型即可覆盖多语言和多类型文本，识别精度大幅领先前代产品和主流开源方案。
|
||||
|
||||
|
||||
# 三、PP-OCRv5 Demo示例
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/algorithm_ppocrv5_demo1.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/doc_images/PP-OCRv5/algorithm_ppocrv5_demo.pdf">更多示例</a>
|
||||
|
||||
## 四、推理性能参考数据
|
||||
|
||||
测试环境：
|
||||
|
||||
- NVIDIA Tesla V100
|
||||
- Intel Xeon Gold 6271C
|
||||
- PaddlePaddle 3.0.0
|
||||
|
||||
在 200 张图像（包括通用图像与文档图像）上测试。测试时从磁盘读取图像，因此读图时间及其他额外开销也被包含在总耗时内。如果将图像提前载入到内存，可进一步减少平均每图约 25 ms 的时间开销。
|
||||
|
||||
如果不特别说明，则：
|
||||
|
||||
- 使用 PP-OCRv4_mobile_det 和 PP-OCRv4_mobile_rec 模型。
|
||||
- 不使用文档图像方向分类、文本图像矫正、文本行方向分类。
|
||||
- 将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `732`。
|
||||
|
||||
### 1. PP-OCRv5 与 PP-OCRv4 推理性能对比
|
||||
|
||||
| 配置 | 说明 |
|
||||
| --- | --- |
|
||||
| v5_mobile | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型。 |
|
||||
| v4_mobile | 使用 PP-OCRv4_mobile_det 和 PP-OCRv4_mobile_rec 模型。 |
|
||||
| v5_server | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型。 |
|
||||
| v4_server | 使用 PP-OCRv4_server_det 和 PP-OCRv4_server_rec 模型。 |
|
||||
|
||||
**GPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） | 平均GPU利用率（%） | 峰值 VRAM 用量（MB） | 平均 VRAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| v5_mobile | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| v4_mobile | 0.27 | 2246 | 111.20 | 1392.22 | 1318.76 | 28.90 | 1304.00 | 1166.46 |
|
||||
| v5_server | 0.70 | 929 | 105.31 | 1634.85 | 1428.55 | 36.21 | 5402.00 | 4685.13 |
|
||||
| v4_server | 0.44 | 1418 | 106.96 | 1455.34 | 1346.95 | 58.82 | 6760.00 | 5817.46 |
|
||||
|
||||
**GPU，使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） | 平均GPU利用率（%） | 峰值 VRAM 用量（MB） | 平均 VRAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| v5_mobile | 0.50 | 1301 | 106.50 | 1338.12 | 1155.86 | 11.97 | 4112.00 | 3536.36 |
|
||||
| v4_mobile | 0.21 | 2887 | 114.09 | 1113.27 | 1054.46 | 15.22 | 2072.00 | 1840.59 |
|
||||
| v5_server | 0.60 | 1084 | 105.73 | 1980.73 | 1776.20 | 22.10 | 12150.00 | 11849.40 |
|
||||
| v4_server | 0.36 | 1687 | 104.15 | 1186.42 | 1065.67 | 38.12 | 13058.00 | 12679.00 |
|
||||
|
||||
**CPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| v5_mobile | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| v4_mobile | 1.09 | 556 | 813.16 | 11996.30 | 6834.25 |
|
||||
| v5_server | 3.79 | 172 | 799.24 | 50216.00 | 27902.40 |
|
||||
| v4_server | 4.22 | 148 | 803.74 | 51428.70 | 28593.60 |
|
||||
|
||||
**CPU，使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| v5_mobile | 1.14 | 571 | 339.68 | 3245.17 | 2560.55 |
|
||||
| v4_mobile | 0.68 | 892 | 443.00 | 3057.38 | 2329.44 |
|
||||
| v5_server | 3.56 | 183 | 797.03 | 45664.70 | 26905.90 |
|
||||
| v4_server | 4.22 | 148 | 803.74 | 51428.70 | 28593.60 |
|
||||
|
||||
> 说明：PP-OCRv5 的识别模型使用了更大的字典，需要更长的推理时间，导致 PP-OCRv5 的推理速度慢于 PP-OCRv4。
|
||||
|
||||
### 2. 使用辅助功能对 PP-OCRv5 推理性能的影响
|
||||
|
||||
| 配置 | 说明 |
|
||||
| --- | --- |
|
||||
| base | 不使用文档图像方向分类、文本图像矫正、文本行方向分类。 |
|
||||
| with_textline | 使用文本行方向分类，不使用文档图像方向分类、文本图像矫正。 |
|
||||
| with_all | 使用文档图像方向分类、文本图像矫正、文本行方向分类。 |
|
||||
|
||||
**GPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） | 平均GPU利用率（%） | 峰值 VRAM 用量（MB） | 平均 VRAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| base | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| with_textline | 0.59 | 1104 | 105.58 | 1765.64 | 1478.53 | 19.48 | 4350.00 | 3267.77 |
|
||||
| with_all | 1.02 | 600 | 104.92 | 1924.23 | 1628.50 | 10.96 | 2632.00 | 2217.01 |
|
||||
|
||||
**CPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| base | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| with_textline | 1.50 | 434 | 799.47 | 12007.20 | 6882.22 |
|
||||
| with_all | 1.93 | 316 | 646.49 | 11759.60 | 6940.54 |
|
||||
|
||||
> 说明：文本图像矫正等辅助功能会对端到端推理精度造成影响，因此并不一定使用的辅助功能越多、资源用量越大。
|
||||
|
||||
### 3. 文本检测模块输入缩放尺寸策略对 PP-OCRv5 推理性能的影响
|
||||
|
||||
| 配置 | 说明 |
|
||||
| --- | --- |
|
||||
| mobile_min_1280 | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型，将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `1280`。 |
|
||||
| mobile_min_736 | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型，将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `1280`。 |
|
||||
| mobile_max_960 | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型，将 `text_det_limit_type` 设置为 `"max"`、`text_det_limit_side_len` 设置为 `960`。 |
|
||||
| mobile_max_640 | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型，将 `text_det_limit_type` 设置为 `"max"`、`text_det_limit_side_len` 设置为 `640`。 |
|
||||
| server_min_1280 | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型，将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `1280`。 |
|
||||
| server_min_736 | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型，将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `1280`。 |
|
||||
| server_max_960 | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型，将 `text_det_limit_type` 设置为 `"max"`、`text_det_limit_side_len` 设置为 `960`。 |
|
||||
| server_max_640 | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型，将 `text_det_limit_type` 设置为 `"max"`、`text_det_limit_side_len` 设置为 `640`。 |
|
||||
|
||||
**GPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） | 平均GPU利用率（%） | 峰值 VRAM 用量（MB） | 平均 VRAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| mobile_min_1280 | 0.61 | 1071 | 109.12 | 1663.71 | 1439.72 | 19.27 | 4202.00 | 3550.32 |
|
||||
| mobile_min_736 | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| mobile_max_960 | 0.48 | 1313 | 103.49 | 1587.25 | 1395.48 | 19.37 | 2642.00 | 2319.03 |
|
||||
| mobile_max_640 | 0.42 | 1436 | 103.07 | 1651.14 | 1422.62 | 18.95 | 2530.00 | 2149.11 |
|
||||
| server_min_1280 | 0.82 | 795 | 107.17 | 1678.16 | 1428.94 | 40.43 | 10368.00 | 8320.43 |
|
||||
| server_min_736 | 0.70 | 929 | 105.31 | 1634.85 | 1428.55 | 36.21 | 5402.00 | 4685.13 |
|
||||
| server_max_960 | 0.59 | 1073 | 103.03 | 1590.19 | 1383.62 | 33.42 | 2928.00 | 2079.47 |
|
||||
| server_max_640 | 0.54 | 1099 | 102.63 | 1602.09 | 1416.49 | 30.77 | 3152.00 | 2737.81 |
|
||||
|
||||
**CPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| mobile_min_1280 | 1.64 | 398 | 799.45 | 12344.10 | 7100.60 |
|
||||
| mobile_min_736 | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| mobile_max_960 | 1.21 | 521 | 800.13 | 11099.10 | 6369.49 |
|
||||
| mobile_max_640 | 1.01 | 597 | 802.52 | 9585.48 | 5573.52 |
|
||||
| server_min_1280 | 4.48 | 145 | 800.49 | 50683.10 | 28273.30 |
|
||||
| server_min_736 | 3.79 | 172 | 799.24 | 50216.00 | 27902.40 |
|
||||
| server_max_960 | 2.67 | 237 | 797.63 | 49362.50 | 26075.60 |
|
||||
| server_max_640 | 2.36 | 251 | 795.18 | 45656.10 | 24900.80 |
|
||||
|
||||
|
||||
# 五、部署与二次开发
|
||||
* **多系统支持**：兼容Windows、Linux、Mac等主流操作系统。
|
||||
* **多硬件支持**：除了英伟达GPU外，还支持Intel CPU、昆仑芯、昇腾等新硬件推理和部署。
|
||||
* **高性能推理插件**：推荐结合高性能推理插件进一步提升推理速度，详见[高性能推理指南](../../deployment/high_performance_inference.md)。
|
||||
* **服务化部署**：支持高稳定性服务化部署方案，详见[服务化部署指南](../../deployment/serving.md)。
|
||||
* **二次开发能力**：支持自定义数据集训练、字典扩展、模型微调。举例：如需增加韩文识别，可扩展字典并微调模型，无缝集成到现有产线，详见[文本检测模块使用教程](../../module_usage/text_detection.md)及[文本识别模块使用教程](../../module_usage/text_recognition.md)
|
||||
@@ -0,0 +1,116 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 1. Introduction to PP-OCRv5 Multilingual Text Recognition
|
||||
|
||||
PP-OCRv5 is the latest generation of the PP-OCR series text recognition solutions, focusing on text recognition tasks across multiple scenarios and languages. By default, the recognition model supports accurate recognition of five mainstream text types: Simplified Chinese, Chinese Pinyin, Traditional Chinese, English, and Japanese. In addition, PP-OCRv5 provides multilingual recognition capabilities covering 37 languages, including Korean, Spanish, French, Portuguese, German, Italian, Russian, and more (see [Section 4](#4-supported-languages-and-abbreviations) for the full list of supported languages and abbreviations). Compared to the previous PP-OCRv3 version, PP-OCRv5 achieves more than a 30% improvement in recognition accuracy for multilingual tasks.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## 2. Quick Start
|
||||
|
||||
You can use the `--lang` parameter in the command line to specify the text recognition model for your target language when running the general OCR pipeline:
|
||||
|
||||
```bash
|
||||
# Use the `--lang` parameter to specify the French recognition model
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png \
|
||||
--lang fr \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False \
|
||||
--save_path ./output \
|
||||
--device gpu:0
|
||||
```
|
||||
For explanations of other command line parameters, please refer to the [command line usage](../../pipeline_usage/OCR.en.md#21-command-line) of the general OCR pipeline. After execution, results will be printed to the terminal:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_ocr_french01.png', 'page_index': None, 'model_settings': {'use_doc_preprocessor': True, 'use_textline_orientation': False}, 'doc_preprocessor_res': {'input_path': None, 'page_index': None, 'model_settings': {'use_doc_orientation_classify': False, 'use_doc_unwarping': False}, 'angle': -1}, 'dt_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'text_det_params': {'limit_side_len': 64, 'limit_type': 'min', 'thresh': 0.3, 'max_side_limit': 4000, 'box_thresh': 0.6, 'unclip_ratio': 1.5}, 'text_type': 'general', 'textline_orientation_angles': array([-1, ..., -1]), 'text_rec_score_thresh': 0.0, 'rec_texts': ['mifere; la profpérité & les fuccès ac-', 'compagnent l’homme induftrieux.', 'Quel eft celui qui a acquis des ri-', 'cheffes, qui eft devenu puiffant, qui', 's’eft couvert de gloire, dont l’éloge', 'retentit par-tout, qui fiege au confeil', "du Roi? C'eft celui qui bannit la pa-", "reffe de fa maifon, & qui a dit à l'oifi-", 'veté : tu es mon ennemie.'], 'rec_scores': array([0.98409832, ..., 0.98091048]), 'rec_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'rec_boxes': array([[118, ..., 81],
|
||||
...,
|
||||
[108, ..., 562]], dtype=int16)}}
|
||||
```
|
||||
|
||||
If you specify `save_path`, the visualization results will be saved in the `save_path` directory. An example visualization is shown below:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/general_ocr_french01_res.png"/>
|
||||
|
||||
You can also use Python code to specify the recognition model for your target language using the `lang` parameter when initializing the general OCR pipeline:
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(
|
||||
lang="fr", # Specify the French recognition model via the lang parameter
|
||||
use_doc_orientation_classify=False, # Disable document orientation classification
|
||||
use_doc_unwarping=False, # Disable text image unwarping
|
||||
use_textline_orientation=False, # Disable textline orientation classification
|
||||
)
|
||||
result = ocr.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png")
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
```
|
||||
For more details on the `PaddleOCR` class parameters, refer to the [Python script integration](../../pipeline_usage/OCR.en.md#22-python-script-integration) of the general OCR pipeline.
|
||||
|
||||
## 3. Benchmark Comparison
|
||||
|
||||
| Model | Korean Dataset Accuracy (%) | | Model | Latin Script Languages Dataset Accuracy (%) | | Model | East Slavic Languages Dataset Accuracy (%) |
|
||||
|--|--|--|--|--|--|--|--|
|
||||
| korean_PP-OCRv5_mobile_rec | 88.0 | | latin_PP-OCRv5_mobile_rec | 84.7 | | eslav_PP-OCRv5_mobile_rec | 85.8 |
|
||||
| korean_PP-OCRv3_mobile_rec | 23.0 | | latin_PP-OCRv3_mobile_rec | 37.9 | | cyrillic_PP-OCRv3_mobile_rec| 50.2 |
|
||||
|
||||
**Notes:**
|
||||
- Korean Dataset: PP-OCRv5's latest dataset containing 5,007 Korean text images.
|
||||
- Latin Script Languages Dataset: The latest PP-OCRv5 recognition dataset, containing 3,111 text images in Latin script languages.
|
||||
- East Slavic Languages Dataset: PP-OCRv5's latest dataset containing a total of 7,031 Russian, Belarusian, and Ukrainian text images.
|
||||
|
||||
## 4. Supported Languages and Abbreviations
|
||||
|
||||
| Language | Description | Abbreviation | | Language | Description | Abbreviation |
|
||||
| --- | --- | --- | ---|--- | --- | --- |
|
||||
| Chinese | Chinese & English | ch | | Hungarian | Hungarian | hu |
|
||||
| English | English | en | | Serbian (Latin) | Serbian(latin) | rslatin |
|
||||
| French | French | fr | | Indonesian | Indonesian | id |
|
||||
| German | German | de | | Occitan | Occitan | oc |
|
||||
| Japanese | Japanese | japan | | Icelandic | Icelandic | is |
|
||||
| Korean | Korean | korean | | Lithuanian | Lithuanian | lt |
|
||||
| Chinese Traditional | Chinese Traditional | chinese_cht | | Maori | Maori | mi |
|
||||
| Afrikaans | Afrikaans | af | | Malay | Malay | ms |
|
||||
| Italian | Italian | it | | Dutch | Dutch | nl |
|
||||
| Spanish | Spanish | es | | Norwegian | Norwegian | no |
|
||||
| Bosnian | Bosnian | bs | | Polish | Polish | pl |
|
||||
| Portuguese | Portuguese | pt | | Slovak | Slovak | sk |
|
||||
| Czech | Czech | cs | | Slovenian | Slovenian | sl |
|
||||
| Welsh | Welsh | cy | | Albanian | Albanian | sq |
|
||||
| Danish | Danish | da | | Swedish | Swedish | sv |
|
||||
| Estonian | Estonian | et | | Swahili | Swahili | sw |
|
||||
| Irish | Irish | ga | | Tagalog | Tagalog | tl |
|
||||
| Croatian | Croatian | hr | | Turkish | Turkish | tr |
|
||||
| Uzbek | Uzbek | uz | | Latin | Latin | la |
|
||||
| Russian | Russian | ru | | Belarusian | Belarusian | be |
|
||||
| Ukrainian | Ukranian | uk | | | | |
|
||||
120
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5_multi_languages.md
Normal file
120
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5_multi_languages.md
Normal file
@@ -0,0 +1,120 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 一、PP-OCRv5多语种文本识别介绍
|
||||
|
||||
|
||||
PP-OCRv5 是 PP-OCR 系列的最新一代文字识别解决方案，专注于多场景、多语种的文字识别任务。在文字类型支持方面，默认配置的识别模型可准确识别简体中文、中文拼音、繁体中文、英文和日文这五大主流文字类型。同时，PP-OCRv5还提供了覆盖37种语言的多语种识别能力，包括韩文、西班牙文、法文、葡萄牙文、德文、意大利文、俄罗斯文等（具体支持语种及缩写详见[第四节](#四-支持语种及缩写)）。相较于前代 PP-OCRv3 版本，PP-OCRv5 在多语言识别准确率上实现了超过30%的提升。
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## 二、快速使用
|
||||
|
||||
您可以通过在命令行中使用 `--lang` 参数，来使用指定语种的文本识别模型进行通用 OCR 产线的推理：
|
||||
|
||||
```bash
|
||||
# 通过 `--lang` 参数指定使用法语的识别模型
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png \
|
||||
--lang fr \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False \
|
||||
--save_path ./output \
|
||||
--device gpu:0
|
||||
```
|
||||
上述命令行的其他参数说明请参考通用 OCR 产线的[命令行使用方式](../../pipeline_usage/OCR.md#21-命令行方式), 运行后结果会被打印到终端上：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_ocr_french01.png', 'page_index': None, 'model_settings': {'use_doc_preprocessor': True, 'use_textline_orientation': False}, 'doc_preprocessor_res': {'input_path': None, 'page_index': None, 'model_settings': {'use_doc_orientation_classify': False, 'use_doc_unwarping': False}, 'angle': -1}, 'dt_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'text_det_params': {'limit_side_len': 64, 'limit_type': 'min', 'thresh': 0.3, 'max_side_limit': 4000, 'box_thresh': 0.6, 'unclip_ratio': 1.5}, 'text_type': 'general', 'textline_orientation_angles': array([-1, ..., -1]), 'text_rec_score_thresh': 0.0, 'rec_texts': ['mifere; la profpérité & les fuccès ac-', 'compagnent l’homme induftrieux.', 'Quel eft celui qui a acquis des ri-', 'cheffes, qui eft devenu puiffant, qui', 's’eft couvert de gloire, dont l’éloge', 'retentit par-tout, qui fiege au confeil', "du Roi? C'eft celui qui bannit la pa-", "reffe de fa maifon, & qui a dit à l'oifi-", 'veté : tu es mon ennemie.'], 'rec_scores': array([0.98409832, ..., 0.98091048]), 'rec_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'rec_boxes': array([[118, ..., 81],
|
||||
...,
|
||||
[108, ..., 562]], dtype=int16)}}
|
||||
```
|
||||
|
||||
若指定了`save_path`，则会保存可视化结果在`save_path`下。可视化结果如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/general_ocr_french01_res.png"/>
|
||||
|
||||
|
||||
您也可以使用 Python 代码，在通用 OCR 产线初始化时，通过 `lang` 参数来使用指定语种的识别模型：
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(
|
||||
lang="fr" # 通过 lang 参数指定使用法语的识别模型
|
||||
use_doc_orientation_classify=False, # 通过 use_doc_orientation_classify 参数指定不使用文档方向分类模型
|
||||
use_doc_unwarping=False, # 通过 use_doc_unwarping 参数指定不使用文本图像矫正模型
|
||||
use_textline_orientation=False, # 通过 use_textline_orientation 参数指定不使用文本行方向分类模型
|
||||
)
|
||||
result = ocr.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png")
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
```
|
||||
更过关于 `PaddleOCR` 类参数的说明参考通用 OCR 产线的[脚本方式集成](../../pipeline_usage/OCR.md#22-python脚本方式集成)。
|
||||
|
||||
|
||||
## 三、指标对比
|
||||
|
||||
| 模型 |韩语数据集 精度 (%)| | 模型 | 拉丁字母语言数据集 精度 (%)| | 模型| 东斯拉夫语言数据集 精度 (%) |
|
||||
|--|--|--|--|--|--|--|--|
|
||||
| korean_PP-OCRv5_mobile_rec | 88.0 | | latin_PP-OCRv5_mobile_rec | 84.7 | | eslav_PP-OCRv5_mobile_rec | 85.8 |
|
||||
| korean_PP-OCRv3_mobile_rec | 23.0 | | latin_PP-OCRv3_mobile_rec | 37.9 | | cyrillic_PP-OCRv3_mobile_rec| 50.2 |
|
||||
|
||||
**注：**
|
||||
- 韩语数据集：PP-OCRv5 最新构建的包含了 5007 张韩语文本图片的识别数据集。
|
||||
- 拉丁字母语言数据集：PP-OCRv5 最新构建的包含了 3111 张拉丁字母语言的文本图片识别数据集。
|
||||
- 东斯拉夫语言数据集：PP-OCRv5 最新构建的包含了俄语、 白俄罗斯语和乌克兰语共计 7031 张文本图片的识别数据集。
|
||||
|
||||
## 四、 支持语种及缩写
|
||||
|
||||
| 语种 | 描述 | 缩写 | | 语种 | 描述 | 缩写 |
|
||||
| --- | --- | --- | ---|--- | --- | --- |
|
||||
| 中文 | Chinese & English | ch | | 匈牙利文 | Hungarian | hu |
|
||||
| 英文 | English | en | | 塞尔维亚文（latin） | Serbian(latin) | rslatin |
|
||||
| 法文 | French | fr | | 印度尼西亚文 | Indonesian | id |
|
||||
| 德文 | German | de | | 欧西坦文 | Occitan | oc |
|
||||
| 日文 | Japanese | japan | | 冰岛文 | Icelandic | is |
|
||||
| 韩文 | Korean | korean | | 立陶宛文 | Lithuanian | lt |
|
||||
| 中文繁体 | Chinese Traditional | chinese_cht | | 毛利文 | Maori | mi |
|
||||
| 南非荷兰文 | Afrikaans | af | | 马来文 | Malay | ms |
|
||||
| 意大利文 | Italian | it | | 荷兰文 | Dutch | nl |
|
||||
| 西班牙文 | Spanish | es | | 挪威文 | Norwegian | no |
|
||||
| 波斯尼亚文 | Bosnian | bs | | 波兰文 | Polish | pl |
|
||||
| 葡萄牙文 | Portuguese | pt | | 斯洛伐克文 | Slovak | sk |
|
||||
| 捷克文 | Czech | cs | | 斯洛文尼亚文 | Slovenian | sl |
|
||||
| 威尔士文 | Welsh | cy | | 阿尔巴尼亚文 | Albanian | sq |
|
||||
| 丹麦文 | Danish | da | | 瑞典文 | Swedish | sv |
|
||||
| 爱沙尼亚文 | Estonian | et | | 西瓦希里文 | Swahili | sw |
|
||||
| 爱尔兰文 | Irish | ga | | 塔加洛文 | Tagalog | tl |
|
||||
| 克罗地亚文 | Croatian | hr | | 土耳其文 | Turkish | tr |
|
||||
| 乌兹别克文 | Uzbek | uz | | 拉丁文 | Latin | la |
|
||||
| 俄罗斯文 | Russian | ru | | 白俄罗斯文 | Belarusian | be |
|
||||
| 乌克兰文 | Ukranian | uk | | | | |
|
||||
981
docs/version3.x/algorithm/PP-StructureV3/PP-StructureV3.en.md
Normal file
981
docs/version3.x/algorithm/PP-StructureV3/PP-StructureV3.en.md
Normal file
@@ -0,0 +1,981 @@
|
||||
# Introduction to PP-StructureV3
|
||||
|
||||
**PP-StructureV3** pipeline, based on the Layout Parsing v1 pipeline, has strengthened the ability of layout detection, table recognition, and formula recognition. It has also added the ability to understand charts and restore reading order, as well as the ability to convert results into Markdown files. In various document data, it performs excellently and can handle more complex document data. This pipeline also provides flexible service-oriented deployment methods, supporting the use of multiple programming languages on various hardware. Moreover, it also provides the ability for secondary development. You can train and optimize on your own dataset based on this pipeline, and the trained model can be seamlessly integrated.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-StructureV3/algorithm_ppstructurev3.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# Key Metrics
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th rowspan="2">Method Type</th>
|
||||
<th rowspan="2">Methods</th>
|
||||
<th colspan="2">Overall<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Text<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Formula<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Table<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Read Order<sup>Edit</sup>↓</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="9">Pipeline Tools</td>
|
||||
<td><b>PP-structureV3</b></td>
|
||||
<td><b>0.147</b></td>
|
||||
<td><b>0.212</b></td>
|
||||
<td>0.059</td>
|
||||
<td><b>0.09</b></td>
|
||||
<td>0.295</td>
|
||||
<td>0.535</td>
|
||||
<td>0.159</td>
|
||||
<td><b>0.109</b></td>
|
||||
<td>0.075</td>
|
||||
<td><b>0.114</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MinerU-0.9.3</td>
|
||||
<td>0.15</td>
|
||||
<td>0.357</td>
|
||||
<td>0.061</td>
|
||||
<td>0.215</td>
|
||||
<td>0.278</td>
|
||||
<td>0.577</td>
|
||||
<td>0.18</td>
|
||||
<td>0.344</td>
|
||||
<td>0.079</td>
|
||||
<td>0.292</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MinerU-1.3.11</td>
|
||||
<td>0.166</td>
|
||||
<td>0.310</td>
|
||||
<td>0.0826</td>
|
||||
<td>0.2000</td>
|
||||
<td>0.3368</td>
|
||||
<td>0.6236</td>
|
||||
<td>0.1613</td>
|
||||
<td>0.1833</td>
|
||||
<td>0.0834</td>
|
||||
<td>0.2316</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Marker-1.2.3</td>
|
||||
<td>0.336</td>
|
||||
<td>0.556</td>
|
||||
<td>0.08</td>
|
||||
<td>0.315</td>
|
||||
<td>0.53</td>
|
||||
<td>0.883</td>
|
||||
<td>0.619</td>
|
||||
<td>0.685</td>
|
||||
<td>0.114</td>
|
||||
<td>0.34</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mathpix</td>
|
||||
<td>0.191</td>
|
||||
<td>0.365</td>
|
||||
<td>0.105</td>
|
||||
<td>0.384</td>
|
||||
<td>0.306</td>
|
||||
<td>0.454</td>
|
||||
<td>0.243</td>
|
||||
<td>0.32</td>
|
||||
<td>0.108</td>
|
||||
<td>0.304</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Docling-2.14.0</td>
|
||||
<td>0.589</td>
|
||||
<td>0.909</td>
|
||||
<td>0.416</td>
|
||||
<td>0.987</td>
|
||||
<td>0.999</td>
|
||||
<td>1</td>
|
||||
<td>0.627</td>
|
||||
<td>0.81</td>
|
||||
<td>0.313</td>
|
||||
<td>0.837</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Pix2Text-1.1.2.3</td>
|
||||
<td>0.32</td>
|
||||
<td>0.528</td>
|
||||
<td>0.138</td>
|
||||
<td>0.356</td>
|
||||
<td><b>0.276</b></td>
|
||||
<td>0.611</td>
|
||||
<td>0.584</td>
|
||||
<td>0.645</td>
|
||||
<td>0.281</td>
|
||||
<td>0.499</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Unstructured-0.17.2</td>
|
||||
<td>0.586</td>
|
||||
<td>0.716</td>
|
||||
<td>0.198</td>
|
||||
<td>0.481</td>
|
||||
<td>0.999</td>
|
||||
<td>1</td>
|
||||
<td>1</td>
|
||||
<td>0.998</td>
|
||||
<td>0.145</td>
|
||||
<td>0.387</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenParse-0.7.0</td>
|
||||
<td>0.646</td>
|
||||
<td>0.814</td>
|
||||
<td>0.681</td>
|
||||
<td>0.974</td>
|
||||
<td>0.996</td>
|
||||
<td>1</td>
|
||||
<td>0.284</td>
|
||||
<td>0.639</td>
|
||||
<td>0.595</td>
|
||||
<td>0.641</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="5">Expert VLMs</td>
|
||||
<td>GOT-OCR</td>
|
||||
<td>0.287</td>
|
||||
<td>0.411</td>
|
||||
<td>0.189</td>
|
||||
<td>0.315</td>
|
||||
<td>0.36</td>
|
||||
<td>0.528</td>
|
||||
<td>0.459</td>
|
||||
<td>0.52</td>
|
||||
<td>0.141</td>
|
||||
<td>0.28</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Nougat</td>
|
||||
<td>0.452</td>
|
||||
<td>0.973</td>
|
||||
<td>0.365</td>
|
||||
<td>0.998</td>
|
||||
<td>0.488</td>
|
||||
<td>0.941</td>
|
||||
<td>0.572</td>
|
||||
<td>1</td>
|
||||
<td>0.382</td>
|
||||
<td>0.954</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mistral OCR</td>
|
||||
<td>0.268</td>
|
||||
<td>0.439</td>
|
||||
<td>0.072</td>
|
||||
<td>0.325</td>
|
||||
<td>0.318</td>
|
||||
<td>0.495</td>
|
||||
<td>0.6</td>
|
||||
<td>0.65</td>
|
||||
<td>0.083</td>
|
||||
<td>0.284</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OLMOCR-sglang</td>
|
||||
<td>0.326</td>
|
||||
<td>0.469</td>
|
||||
<td>0.097</td>
|
||||
<td>0.293</td>
|
||||
<td>0.455</td>
|
||||
<td>0.655</td>
|
||||
<td>0.608</td>
|
||||
<td>0.652</td>
|
||||
<td>0.145</td>
|
||||
<td>0.277</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SmolDocling-256M_transformer</td>
|
||||
<td>0.493</td>
|
||||
<td>0.816</td>
|
||||
<td>0.262</td>
|
||||
<td>0.838</td>
|
||||
<td>0.753</td>
|
||||
<td>0.997</td>
|
||||
<td>0.729</td>
|
||||
<td>0.907</td>
|
||||
<td>0.227</td>
|
||||
<td>0.522</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="6">General VLMs</td>
|
||||
<td>Gemini2.0-flash</td>
|
||||
<td>0.191</td>
|
||||
<td>0.264</td>
|
||||
<td>0.091</td>
|
||||
<td>0.139</td>
|
||||
<td>0.389</td>
|
||||
<td>0.584</td>
|
||||
<td>0.193</td>
|
||||
<td>0.206</td>
|
||||
<td>0.092</td>
|
||||
<td>0.128</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini2.5-Pro</td>
|
||||
<td>0.148</td>
|
||||
<td><b>0.212</b></td>
|
||||
<td><b>0.055</b></td>
|
||||
<td>0.168</td>
|
||||
<td>0.356</td>
|
||||
<td>0.439</td>
|
||||
<td><b>0.13</b></td>
|
||||
<td>0.119</td>
|
||||
<td><b>0.049</b></td>
|
||||
<td>0.121</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GPT4o</td>
|
||||
<td>0.233</td>
|
||||
<td>0.399</td>
|
||||
<td>0.144</td>
|
||||
<td>0.409</td>
|
||||
<td>0.425</td>
|
||||
<td>0.606</td>
|
||||
<td>0.234</td>
|
||||
<td>0.329</td>
|
||||
<td>0.128</td>
|
||||
<td>0.251</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2-VL-72B</td>
|
||||
<td>0.252</td>
|
||||
<td>0.327</td>
|
||||
<td>0.096</td>
|
||||
<td>0.218</td>
|
||||
<td>0.404</td>
|
||||
<td>0.487</td>
|
||||
<td>0.387</td>
|
||||
<td>0.408</td>
|
||||
<td>0.119</td>
|
||||
<td>0.193</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>0.214</td>
|
||||
<td>0.261</td>
|
||||
<td>0.092</td>
|
||||
<td>0.18</td>
|
||||
<td>0.315</td>
|
||||
<td><b>0.434</b></td>
|
||||
<td>0.341</td>
|
||||
<td>0.262</td>
|
||||
<td>0.106</td>
|
||||
<td>0.168</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>InternVL2-76B</td>
|
||||
<td>0.44</td>
|
||||
<td>0.443</td>
|
||||
<td>0.353</td>
|
||||
<td>0.29</td>
|
||||
<td>0.543</td>
|
||||
<td>0.701</td>
|
||||
<td>0.547</td>
|
||||
<td>0.555</td>
|
||||
<td>0.317</td>
|
||||
<td>0.228</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
The above data is from:
|
||||
* <a href="https://github.com/opendatalab/OmniDocBench">OmniDocBench</a>
|
||||
* <a href="https://arxiv.org/abs/2412.07626">OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations</a>
|
||||
|
||||
# End to End Benchmark
|
||||
|
||||
The performance of PP-StructureV3 and MinerU with different configurations under different GPU environments are as follows.
|
||||
|
||||
Requirements:
|
||||
* Paddle 3.0
|
||||
* PaddleOCR 3.0.0
|
||||
* MinerU 1.3.10
|
||||
* CUDA 11.8
|
||||
* cuDNN 8.9
|
||||
|
||||
## Local inference
|
||||
|
||||
Local inference was tested with both V100 and A100 GPU, evaluating the performance of PP-StructureV3 under 6 different configurations. The test data consists of 15 PDF files, totaling 925 pages, including elements such as tables, formulas, seals, and charts.
|
||||
|
||||
In the following PP-StructureV3 configuration, please refer to [PP-OCRv5](../PP-OCRv5/PP-OCRv5.en.md) for OCR model details, see [Formula Recognition](../../module_usage/formula_recognition.en.md) for formula recognition model details, and refer to [Text Detection](../../module_usage/text_detection.en.md) for the max_side_limit setting of the text detection module.
|
||||
|
||||
### Env: NVIDIA Tesla V100 + Intel Xeon Gold 6271C
|
||||
|
||||
<table border="1">
|
||||
<tr>
|
||||
<td>
|
||||
Methods
|
||||
</td>
|
||||
<td colspan="4">
|
||||
Configurations
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average time per
|
||||
page
|
||||
(s)
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average CPU
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Peak RAM Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average RAM
|
||||
Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average GPU
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Peak VRAM Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average VRAM
|
||||
Usage
|
||||
（GB）
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="7">
|
||||
PP-StructureV3
|
||||
</td>
|
||||
<td>
|
||||
OCR Models
|
||||
</td>
|
||||
<td>
|
||||
Formula Recognition Model
|
||||
</td>
|
||||
<td>
|
||||
Chart Recognition Model
|
||||
</td>
|
||||
<td>
|
||||
text detection module max_side_limit
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.77
|
||||
</td>
|
||||
<td>
|
||||
111.4
|
||||
</td>
|
||||
<td>
|
||||
6.7
|
||||
</td>
|
||||
<td>
|
||||
5.2
|
||||
</td>
|
||||
<td>
|
||||
38.9
|
||||
</td>
|
||||
<td>
|
||||
17.0
|
||||
</td>
|
||||
<td>
|
||||
16.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✔
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
4.09
|
||||
</td>
|
||||
<td>
|
||||
105.3
|
||||
</td>
|
||||
<td>
|
||||
5.5
|
||||
</td>
|
||||
<td>
|
||||
4.0
|
||||
</td>
|
||||
<td>
|
||||
24.7
|
||||
</td>
|
||||
<td>
|
||||
17.0
|
||||
</td>
|
||||
<td>
|
||||
16.6
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.56
|
||||
</td>
|
||||
<td>
|
||||
113.7
|
||||
</td>
|
||||
<td>
|
||||
6.6
|
||||
</td>
|
||||
<td>
|
||||
4.9
|
||||
</td>
|
||||
<td>
|
||||
29.1
|
||||
</td>
|
||||
<td>
|
||||
10.7
|
||||
</td>
|
||||
<td>
|
||||
10.6
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.42
|
||||
</td>
|
||||
<td>
|
||||
112.9
|
||||
</td>
|
||||
<td>
|
||||
6.8
|
||||
</td>
|
||||
<td>
|
||||
5.1
|
||||
</td>
|
||||
<td>
|
||||
38
|
||||
</td>
|
||||
<td>
|
||||
16.0
|
||||
</td>
|
||||
<td>
|
||||
15.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.15
|
||||
</td>
|
||||
<td>
|
||||
114.8
|
||||
</td>
|
||||
<td>
|
||||
6.5
|
||||
</td>
|
||||
<td>
|
||||
5.0
|
||||
</td>
|
||||
<td>
|
||||
26.1
|
||||
</td>
|
||||
<td>
|
||||
8.4
|
||||
</td>
|
||||
<td>
|
||||
8.3
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
1200
|
||||
</td>
|
||||
<td>
|
||||
0.99
|
||||
</td>
|
||||
<td>
|
||||
113
|
||||
</td>
|
||||
<td>
|
||||
7.0
|
||||
</td>
|
||||
<td>
|
||||
5.6
|
||||
</td>
|
||||
<td>
|
||||
29.2
|
||||
</td>
|
||||
<td>
|
||||
8.6
|
||||
</td>
|
||||
<td>
|
||||
8.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
MinerU
|
||||
</td>
|
||||
<td colspan="4">
|
||||
-
|
||||
</td>
|
||||
<td>
|
||||
1.57
|
||||
</td>
|
||||
<td>
|
||||
142.9
|
||||
</td>
|
||||
<td>
|
||||
13.3
|
||||
</td>
|
||||
<td>
|
||||
11.8
|
||||
</td>
|
||||
<td>
|
||||
43.3
|
||||
</td>
|
||||
<td>
|
||||
31.6
|
||||
</td>
|
||||
<td>
|
||||
9.7
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
### NVIDIA A100 + Intel Xeon Platinum 8350C
|
||||
|
||||
<table border="1">
|
||||
<tr>
|
||||
<td>
|
||||
Methods
|
||||
</td>
|
||||
<td colspan="4">
|
||||
Configurations
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average time per
|
||||
page
|
||||
(s)
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average CPU
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Peak RAM Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average RAM
|
||||
Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average GPU
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Peak VRAM Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average VRAM
|
||||
Usage
|
||||
（GB）
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="7">
|
||||
PP-StructureV3
|
||||
</td>
|
||||
<td>
|
||||
OCR Models
|
||||
</td>
|
||||
<td>
|
||||
Formula Recognition Model
|
||||
</td>
|
||||
<td>
|
||||
Chart Recognition Model
|
||||
</td>
|
||||
<td>
|
||||
text detection module max_side_limit
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.12
|
||||
</td>
|
||||
<td>
|
||||
109.8
|
||||
</td>
|
||||
<td>
|
||||
9.2
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
29.8
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✔
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
2.76
|
||||
</td>
|
||||
<td>
|
||||
103.7
|
||||
</td>
|
||||
<td>
|
||||
9.0
|
||||
</td>
|
||||
<td>
|
||||
7.7
|
||||
</td>
|
||||
<td>
|
||||
24
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.04
|
||||
</td>
|
||||
<td>
|
||||
110.7
|
||||
</td>
|
||||
<td>
|
||||
9.3
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
22
|
||||
</td>
|
||||
<td>
|
||||
12.2
|
||||
</td>
|
||||
<td>
|
||||
12.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
0.95
|
||||
</td>
|
||||
<td>
|
||||
111.4
|
||||
</td>
|
||||
<td>
|
||||
9.1
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
28.1
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.0
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
0.89
|
||||
</td>
|
||||
<td>
|
||||
112.1
|
||||
</td>
|
||||
<td>
|
||||
9.2
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
18.5
|
||||
</td>
|
||||
<td>
|
||||
11.4
|
||||
</td>
|
||||
<td>
|
||||
11.2
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
1200
|
||||
</td>
|
||||
<td>
|
||||
0.64
|
||||
</td>
|
||||
<td>
|
||||
113.5
|
||||
</td>
|
||||
<td>
|
||||
10.2
|
||||
</td>
|
||||
<td>
|
||||
8.5
|
||||
</td>
|
||||
<td>
|
||||
23.7
|
||||
</td>
|
||||
<td>
|
||||
11.4
|
||||
</td>
|
||||
<td>
|
||||
11.2
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
MinerU
|
||||
</td>
|
||||
<td colspan="4">
|
||||
-
|
||||
</td>
|
||||
<td>
|
||||
1.06
|
||||
</td>
|
||||
<td>
|
||||
168.3
|
||||
</td>
|
||||
<td>
|
||||
18.3
|
||||
</td>
|
||||
<td>
|
||||
16.8
|
||||
</td>
|
||||
<td>
|
||||
27.5
|
||||
</td>
|
||||
<td>
|
||||
76.9
|
||||
</td>
|
||||
<td>
|
||||
14.8
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## Serving Inference
|
||||
|
||||
The serving inference test is based on the NVIDIA A100 + Intel Xeon Platinum 8350C environment, with test data consisting of 1500 images, including tables, formulas, seals, charts, and other elements.
|
||||
|
||||
<table>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Instances Number</td>
|
||||
<td>Concurrent Requests Number</td>
|
||||
<td>Throughput</td>
|
||||
<td>Average Latency (s)</td>
|
||||
<td>Success Number/Total Number</td>
|
||||
</tr>
|
||||
<tr">
|
||||
<td>4 GPUs ✖️ 1 instance/gpu</td>
|
||||
<td>4</td>
|
||||
<td>1.69</td>
|
||||
<td>2.36</td>
|
||||
<td>100%</td>
|
||||
</tr>
|
||||
<tr">
|
||||
<td>4 GPUs ✖️ 4 instances/gpu</td>
|
||||
<td>16</td>
|
||||
<td>4.05</td>
|
||||
<td>3.87</td>
|
||||
<td>100%</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
# PP-StructureV3 Demo
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-StructureV3/algorithm_ppstructurev3_demo.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex%2FPaddleX3.0%2Fdoc_images%2FPP-StructureV3%2Falgorithm_ppstructurev3_demo.pdf">More Demos</a>
|
||||
|
||||
|
||||
# FAQ
|
||||
|
||||
1. What is the default configuration? How to get higher accuracy, faster speed, or smaller GPU memory?
|
||||
|
||||
When using mobile OCR models + PP-FormulaNet_plus-M, and max length of text detection set to 1200, if set use_chart_recognition to False and dont not load the chart recognition model, the GPU memory would be reduced.
|
||||
|
||||
On the V100, the peak and average GPU memory would be reduced from 8776.0 MB and 8680.8 MB to 6118.0 MB and 6016.7 MB, respectively; On the A100, the peak and average GPU memory would be reduced from 11716.0 MB and 11453.9 MB to 9850.0 MB and 9593.5 MB, respectively.
|
||||
|
||||
You can using multi-gpus by setting `device` to `gpu:<no.>,<no.>`, such as `gpu:0,1,2,3`. And about multi-process parallel inference, you can refer: [Multi-Process Parallel Inference](https://github.com/PaddlePaddle/PaddleX/blob/develop/docs/pipeline_usage/instructions/parallel_inference.en.md#example-of-multi-process-parallel-inference).
|
||||
|
||||
2. About serving deployment
|
||||
|
||||
(1) Can the service handle requests concurrently?
|
||||
|
||||
For the basic serving deployment solution, the service processes only one request at a time. This plan is mainly used for rapid verification, to establish the development chain, or for scenarios where concurrent requests are not required.
|
||||
|
||||
For high-stability serving deployment solution, the service process only one request at a time by default, but you can refer to the related docs to adjust achieve scaling.
|
||||
|
||||
（2）How to reduce latency and improve throughput?
|
||||
|
||||
Use the High-performance inference plugin, and deploy multi instances.
|
||||
971
docs/version3.x/algorithm/PP-StructureV3/PP-StructureV3.md
Normal file
971
docs/version3.x/algorithm/PP-StructureV3/PP-StructureV3.md
Normal file
@@ -0,0 +1,971 @@
|
||||
# 一、PP-StructureV3 简介
|
||||
**PP-StructureV3** 产线在通用版面解析v1产线的基础上，强化了版面区域检测、表格识别、公式识别的能力，增加了图表理解和多栏阅读顺序的恢复能力、结果转换 Markdown 文件的能力，在多种文档数据中，表现优异，可以处理较复杂的文档数据。本产线同时提供了灵活的服务化部署方式，支持在多种硬件上使用多种编程语言调用。不仅如此，本产线也提供了二次开发的能力，您可以基于本产线在您自己的数据集上训练调优，训练后的模型也可以无缝集成。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-StructureV3/algorithm_ppstructurev3.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# 二、关键指标
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th rowspan="2">Method Type</th>
|
||||
<th rowspan="2">Methods</th>
|
||||
<th colspan="2">Overall<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Text<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Formula<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Table<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Read Order<sup>Edit</sup>↓</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="9">Pipeline Tools</td>
|
||||
<td><b>PP-structureV3</b></td>
|
||||
<td><b>0.147</b></td>
|
||||
<td><b>0.212</b></td>
|
||||
<td>0.059</td>
|
||||
<td><b>0.09</b></td>
|
||||
<td>0.295</td>
|
||||
<td>0.535</td>
|
||||
<td>0.159</td>
|
||||
<td><b>0.109</b></td>
|
||||
<td>0.075</td>
|
||||
<td><b>0.114</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MinerU-0.9.3</td>
|
||||
<td>0.15</td>
|
||||
<td>0.357</td>
|
||||
<td>0.061</td>
|
||||
<td>0.215</td>
|
||||
<td>0.278</td>
|
||||
<td>0.577</td>
|
||||
<td>0.18</td>
|
||||
<td>0.344</td>
|
||||
<td>0.079</td>
|
||||
<td>0.292</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MinerU-1.3.11</td>
|
||||
<td>0.166</td>
|
||||
<td>0.310</td>
|
||||
<td>0.0826</td>
|
||||
<td>0.2000</td>
|
||||
<td>0.3368</td>
|
||||
<td>0.6236</td>
|
||||
<td>0.1613</td>
|
||||
<td>0.1833</td>
|
||||
<td>0.0834</td>
|
||||
<td>0.2316</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Marker-1.2.3</td>
|
||||
<td>0.336</td>
|
||||
<td>0.556</td>
|
||||
<td>0.08</td>
|
||||
<td>0.315</td>
|
||||
<td>0.53</td>
|
||||
<td>0.883</td>
|
||||
<td>0.619</td>
|
||||
<td>0.685</td>
|
||||
<td>0.114</td>
|
||||
<td>0.34</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mathpix</td>
|
||||
<td>0.191</td>
|
||||
<td>0.365</td>
|
||||
<td>0.105</td>
|
||||
<td>0.384</td>
|
||||
<td>0.306</td>
|
||||
<td>0.454</td>
|
||||
<td>0.243</td>
|
||||
<td>0.32</td>
|
||||
<td>0.108</td>
|
||||
<td>0.304</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Docling-2.14.0</td>
|
||||
<td>0.589</td>
|
||||
<td>0.909</td>
|
||||
<td>0.416</td>
|
||||
<td>0.987</td>
|
||||
<td>0.999</td>
|
||||
<td>1</td>
|
||||
<td>0.627</td>
|
||||
<td>0.81</td>
|
||||
<td>0.313</td>
|
||||
<td>0.837</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Pix2Text-1.1.2.3</td>
|
||||
<td>0.32</td>
|
||||
<td>0.528</td>
|
||||
<td>0.138</td>
|
||||
<td>0.356</td>
|
||||
<td><b>0.276</b></td>
|
||||
<td>0.611</td>
|
||||
<td>0.584</td>
|
||||
<td>0.645</td>
|
||||
<td>0.281</td>
|
||||
<td>0.499</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Unstructured-0.17.2</td>
|
||||
<td>0.586</td>
|
||||
<td>0.716</td>
|
||||
<td>0.198</td>
|
||||
<td>0.481</td>
|
||||
<td>0.999</td>
|
||||
<td>1</td>
|
||||
<td>1</td>
|
||||
<td>0.998</td>
|
||||
<td>0.145</td>
|
||||
<td>0.387</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenParse-0.7.0</td>
|
||||
<td>0.646</td>
|
||||
<td>0.814</td>
|
||||
<td>0.681</td>
|
||||
<td>0.974</td>
|
||||
<td>0.996</td>
|
||||
<td>1</td>
|
||||
<td>0.284</td>
|
||||
<td>0.639</td>
|
||||
<td>0.595</td>
|
||||
<td>0.641</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="5">Expert VLMs</td>
|
||||
<td>GOT-OCR</td>
|
||||
<td>0.287</td>
|
||||
<td>0.411</td>
|
||||
<td>0.189</td>
|
||||
<td>0.315</td>
|
||||
<td>0.36</td>
|
||||
<td>0.528</td>
|
||||
<td>0.459</td>
|
||||
<td>0.52</td>
|
||||
<td>0.141</td>
|
||||
<td>0.28</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Nougat</td>
|
||||
<td>0.452</td>
|
||||
<td>0.973</td>
|
||||
<td>0.365</td>
|
||||
<td>0.998</td>
|
||||
<td>0.488</td>
|
||||
<td>0.941</td>
|
||||
<td>0.572</td>
|
||||
<td>1</td>
|
||||
<td>0.382</td>
|
||||
<td>0.954</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mistral OCR</td>
|
||||
<td>0.268</td>
|
||||
<td>0.439</td>
|
||||
<td>0.072</td>
|
||||
<td>0.325</td>
|
||||
<td>0.318</td>
|
||||
<td>0.495</td>
|
||||
<td>0.6</td>
|
||||
<td>0.65</td>
|
||||
<td>0.083</td>
|
||||
<td>0.284</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OLMOCR-sglang</td>
|
||||
<td>0.326</td>
|
||||
<td>0.469</td>
|
||||
<td>0.097</td>
|
||||
<td>0.293</td>
|
||||
<td>0.455</td>
|
||||
<td>0.655</td>
|
||||
<td>0.608</td>
|
||||
<td>0.652</td>
|
||||
<td>0.145</td>
|
||||
<td>0.277</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SmolDocling-256M_transformer</td>
|
||||
<td>0.493</td>
|
||||
<td>0.816</td>
|
||||
<td>0.262</td>
|
||||
<td>0.838</td>
|
||||
<td>0.753</td>
|
||||
<td>0.997</td>
|
||||
<td>0.729</td>
|
||||
<td>0.907</td>
|
||||
<td>0.227</td>
|
||||
<td>0.522</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="6">General VLMs</td>
|
||||
<td>Gemini2.0-flash</td>
|
||||
<td>0.191</td>
|
||||
<td>0.264</td>
|
||||
<td>0.091</td>
|
||||
<td>0.139</td>
|
||||
<td>0.389</td>
|
||||
<td>0.584</td>
|
||||
<td>0.193</td>
|
||||
<td>0.206</td>
|
||||
<td>0.092</td>
|
||||
<td>0.128</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini2.5-Pro</td>
|
||||
<td>0.148</td>
|
||||
<td><b>0.212</b></td>
|
||||
<td><b>0.055</b></td>
|
||||
<td>0.168</td>
|
||||
<td>0.356</td>
|
||||
<td>0.439</td>
|
||||
<td><b>0.13</b></td>
|
||||
<td>0.119</td>
|
||||
<td><b>0.049</b></td>
|
||||
<td>0.121</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GPT4o</td>
|
||||
<td>0.233</td>
|
||||
<td>0.399</td>
|
||||
<td>0.144</td>
|
||||
<td>0.409</td>
|
||||
<td>0.425</td>
|
||||
<td>0.606</td>
|
||||
<td>0.234</td>
|
||||
<td>0.329</td>
|
||||
<td>0.128</td>
|
||||
<td>0.251</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2-VL-72B</td>
|
||||
<td>0.252</td>
|
||||
<td>0.327</td>
|
||||
<td>0.096</td>
|
||||
<td>0.218</td>
|
||||
<td>0.404</td>
|
||||
<td>0.487</td>
|
||||
<td>0.387</td>
|
||||
<td>0.408</td>
|
||||
<td>0.119</td>
|
||||
<td>0.193</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>0.214</td>
|
||||
<td>0.261</td>
|
||||
<td>0.092</td>
|
||||
<td>0.18</td>
|
||||
<td>0.315</td>
|
||||
<td><b>0.434</b></td>
|
||||
<td>0.341</td>
|
||||
<td>0.262</td>
|
||||
<td>0.106</td>
|
||||
<td>0.168</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>InternVL2-76B</td>
|
||||
<td>0.44</td>
|
||||
<td>0.443</td>
|
||||
<td>0.353</td>
|
||||
<td>0.29</td>
|
||||
<td>0.543</td>
|
||||
<td>0.701</td>
|
||||
<td>0.547</td>
|
||||
<td>0.555</td>
|
||||
<td>0.317</td>
|
||||
<td>0.228</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
以上部分数据出自：
|
||||
* <a href="https://github.com/opendatalab/OmniDocBench">OmniDocBench</a>
|
||||
* <a href="https://arxiv.org/abs/2412.07626">OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations</a>
|
||||
|
||||
|
||||
# 三、推理 Benchmark
|
||||
|
||||
在不同GPU环境下，不同配置的 PP-StructureV3 和 MinerU 对比的性能指标如下。
|
||||
|
||||
基本测试环境：
|
||||
* Paddle 3.0正式版
|
||||
* PaddleOCR 3.0.0正式版
|
||||
* MinerU 1.3.10
|
||||
* CUDA 11.8
|
||||
* cuDNN 8.9
|
||||
|
||||
## 3.1 本地推理
|
||||
|
||||
本地推理分别在 V100 和 A100 两种 GPU机器上，测试了 6 种不同配置下 PP-StructureV3 的性能，测试数据为15个PDF文件，共925页，包含表格、公式、印章、图表等元素。
|
||||
|
||||
下述 PP-StructureV3 配置中，OCR 模型详情请见[PP-OCRv5](../PP-OCRv5/PP-OCRv5.md)，公式识别模型详情请见[公式识别](../../module_usage/formula_recognition.md)，文本检测模块 max_side_limit 设置请见[文本检测](../../module_usage/text_detection.md)。
|
||||
|
||||
### NVIDIA Tesla V100 + Intel Xeon Gold 6271C
|
||||
<table border="1">
|
||||
<tr>
|
||||
<td>
|
||||
方案
|
||||
</td>
|
||||
<td colspan="4">
|
||||
配置
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均每页耗时
|
||||
（s）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均CPU利用率
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
峰值RAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均RAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均GPU利用率
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
峰值VRAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均VRAM用量
|
||||
（GB）
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="7">
|
||||
PP-StructureV3
|
||||
</td>
|
||||
<td>
|
||||
OCR模型
|
||||
</td>
|
||||
<td>
|
||||
公式识别模型
|
||||
</td>
|
||||
<td>
|
||||
是否启用图表识别模块
|
||||
</td>
|
||||
<td>
|
||||
文本检测max_side_limit
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.77
|
||||
</td>
|
||||
<td>
|
||||
111.4
|
||||
</td>
|
||||
<td>
|
||||
6.7
|
||||
</td>
|
||||
<td>
|
||||
5.2
|
||||
</td>
|
||||
<td>
|
||||
38.9
|
||||
</td>
|
||||
<td>
|
||||
17.0
|
||||
</td>
|
||||
<td>
|
||||
16.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✔
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
4.09
|
||||
</td>
|
||||
<td>
|
||||
105.3
|
||||
</td>
|
||||
<td>
|
||||
5.5
|
||||
</td>
|
||||
<td>
|
||||
4.0
|
||||
</td>
|
||||
<td>
|
||||
24.7
|
||||
</td>
|
||||
<td>
|
||||
17.0
|
||||
</td>
|
||||
<td>
|
||||
16.6
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.56
|
||||
</td>
|
||||
<td>
|
||||
113.7
|
||||
</td>
|
||||
<td>
|
||||
6.6
|
||||
</td>
|
||||
<td>
|
||||
4.9
|
||||
</td>
|
||||
<td>
|
||||
29.1
|
||||
</td>
|
||||
<td>
|
||||
10.7
|
||||
</td>
|
||||
<td>
|
||||
10.6
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.42
|
||||
</td>
|
||||
<td>
|
||||
112.9
|
||||
</td>
|
||||
<td>
|
||||
6.8
|
||||
</td>
|
||||
<td>
|
||||
5.1
|
||||
</td>
|
||||
<td>
|
||||
38
|
||||
</td>
|
||||
<td>
|
||||
16.0
|
||||
</td>
|
||||
<td>
|
||||
15.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.15
|
||||
</td>
|
||||
<td>
|
||||
114.8
|
||||
</td>
|
||||
<td>
|
||||
6.5
|
||||
</td>
|
||||
<td>
|
||||
5.0
|
||||
</td>
|
||||
<td>
|
||||
26.1
|
||||
</td>
|
||||
<td>
|
||||
8.4
|
||||
</td>
|
||||
<td>
|
||||
8.3
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
1200
|
||||
</td>
|
||||
<td>
|
||||
0.99
|
||||
</td>
|
||||
<td>
|
||||
113
|
||||
</td>
|
||||
<td>
|
||||
7.0
|
||||
</td>
|
||||
<td>
|
||||
5.6
|
||||
</td>
|
||||
<td>
|
||||
29.2
|
||||
</td>
|
||||
<td>
|
||||
8.6
|
||||
</td>
|
||||
<td>
|
||||
8.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
MinerU
|
||||
</td>
|
||||
<td colspan="4">
|
||||
-
|
||||
</td>
|
||||
<td>
|
||||
1.57
|
||||
</td>
|
||||
<td>
|
||||
142.9
|
||||
</td>
|
||||
<td>
|
||||
13.3
|
||||
</td>
|
||||
<td>
|
||||
11.8
|
||||
</td>
|
||||
<td>
|
||||
43.3
|
||||
</td>
|
||||
<td>
|
||||
31.6
|
||||
</td>
|
||||
<td>
|
||||
9.7
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
### NVIDIA A100 + Intel Xeon Platinum 8350C
|
||||
<table border="1">
|
||||
<tr>
|
||||
<td>
|
||||
方案
|
||||
</td>
|
||||
<td colspan="4">
|
||||
配置
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均每页耗时
|
||||
（s）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均CPU利用率
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
峰值RAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均RAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均GPU利用率
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
峰值VRAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均VRAM用量
|
||||
（GB）
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="7">
|
||||
PP-StructureV3
|
||||
</td>
|
||||
<td>
|
||||
OCR模型
|
||||
</td>
|
||||
<td>
|
||||
公式识别模型
|
||||
</td>
|
||||
<td>
|
||||
是否启用图表识别模块
|
||||
</td>
|
||||
<td>
|
||||
文本检测max_side_limit
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.12
|
||||
</td>
|
||||
<td>
|
||||
109.8
|
||||
</td>
|
||||
<td>
|
||||
9.2
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
29.8
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✔
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
2.76
|
||||
</td>
|
||||
<td>
|
||||
103.7
|
||||
</td>
|
||||
<td>
|
||||
9.0
|
||||
</td>
|
||||
<td>
|
||||
7.7
|
||||
</td>
|
||||
<td>
|
||||
24
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.04
|
||||
</td>
|
||||
<td>
|
||||
110.7
|
||||
</td>
|
||||
<td>
|
||||
9.3
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
22
|
||||
</td>
|
||||
<td>
|
||||
12.2
|
||||
</td>
|
||||
<td>
|
||||
12.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
0.95
|
||||
</td>
|
||||
<td>
|
||||
111.4
|
||||
</td>
|
||||
<td>
|
||||
9.1
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
28.1
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.0
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
0.89
|
||||
</td>
|
||||
<td>
|
||||
112.1
|
||||
</td>
|
||||
<td>
|
||||
9.2
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
18.5
|
||||
</td>
|
||||
<td>
|
||||
11.4
|
||||
</td>
|
||||
<td>
|
||||
11.2
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
1200
|
||||
</td>
|
||||
<td>
|
||||
0.64
|
||||
</td>
|
||||
<td>
|
||||
113.5
|
||||
</td>
|
||||
<td>
|
||||
10.2
|
||||
</td>
|
||||
<td>
|
||||
8.5
|
||||
</td>
|
||||
<td>
|
||||
23.7
|
||||
</td>
|
||||
<td>
|
||||
11.4
|
||||
</td>
|
||||
<td>
|
||||
11.2
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
MinerU
|
||||
</td>
|
||||
<td colspan="4">
|
||||
-
|
||||
</td>
|
||||
<td>
|
||||
1.06
|
||||
</td>
|
||||
<td>
|
||||
168.3
|
||||
</td>
|
||||
<td>
|
||||
18.3
|
||||
</td>
|
||||
<td>
|
||||
16.8
|
||||
</td>
|
||||
<td>
|
||||
27.5
|
||||
</td>
|
||||
<td>
|
||||
76.9
|
||||
</td>
|
||||
<td>
|
||||
14.8
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 3.2 服务化部署
|
||||
|
||||
服务化部署测试基于 NVIDIA A100 + Intel Xeon Platinum 8350C 环境，测试数据为 1500 张图像，包含表格、公式、印章、图表等元素。
|
||||
|
||||
<table>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>实例数</td>
|
||||
<td>并发请求数</td>
|
||||
<td>吞吐</td>
|
||||
<td>平均时延（s）</td>
|
||||
<td>成功请求数/总请求数</td>
|
||||
</tr>
|
||||
<tr">
|
||||
<td>4卡 ✖️ 1实例/卡</td>
|
||||
<td>4</td>
|
||||
<td>1.69</td>
|
||||
<td>2.36</td>
|
||||
<td>100%</td>
|
||||
</tr>
|
||||
<tr">
|
||||
<td>4卡 ✖️ 4实例/卡</td>
|
||||
<td>16</td>
|
||||
<td>4.05</td>
|
||||
<td>3.87</td>
|
||||
<td>100%</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
# 四、PP-StructureV3 Demo示例
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-StructureV3/algorithm_ppstructurev3_demo.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex%2FPaddleX3.0%2Fdoc_images%2FPP-StructureV3%2Falgorithm_ppstructurev3_demo.pdf">更多示例</a>
|
||||
|
||||
# 五、使用方法和常见问题
|
||||
|
||||
1. 默认模型是什么配置，如果需要更高精度、更快速度、或者更小显存，应该调哪些参数或者更换哪些模型，对结果影响大概有多大？
|
||||
|
||||
在“使用轻量OCR模型+轻量公式模型，文本检测max 1200”的基础上，将产线配置文件中的use_chart_recognition设置为False，不加载图表识别模型，可以进一步减少显存用量。在V100测试环境中，峰值和平均显存用量分别从8776.0 MB和8680.8 MB降低到6118.0 MB和6016.7 MB；在A100测试环境中，峰值和平均显存用量分别从11716.0 MB和11453.9 MB降低到9850.0 MB和9593.5 MB。
|
||||
在Python API或CLI设置device为<设备类型>:<设备编号1>,<设备编号2>...（例如gpu:0,1,2,3）可实现多卡并行推理。如果内置的多卡并行推理功能提速效果仍不满足预期，可参考多进程并行推理示例代码，结合具体场景进行进一步优化：[多进程并行推理](https://github.com/PaddlePaddle/PaddleX/blob/develop/docs/pipeline_usage/instructions/parallel_inference.md#%E5%A4%9A%E8%BF%9B%E7%A8%8B%E5%B9%B6%E8%A1%8C%E6%8E%A8%E7%90%86%E7%A4%BA%E4%BE%8B)。
|
||||
|
||||
2. 服务化部署的常见问题
|
||||
|
||||
（1）服务可以并发处理请求吗？
|
||||
|
||||
对于基础服务化部署方案，服务同一时间只处理一个请求，该方案主要用于快速验证、打通开发链路，或者用在不需要并发请求的场景；
|
||||
|
||||
对于高稳定性服务化部署方案，服务默认在同一时间只处理一个请求，但用户可以参考服务化部署指南，通过调整配置实现水平扩展，以使服务同时处理多个请求。
|
||||
|
||||
（2）如何降低时延、提升吞吐？
|
||||
|
||||
无论使用哪一种服务化部署方案，都可以通过启用高性能推理插件提升模型推理速度，从而降低处理时延。
|
||||
|
||||
此外，对于高稳定性服务化部署方案，通过调整服务配置，设置多个实例，也可以充分利用部署机器的资源，有效提升吞吐。
|
||||
94
docs/version3.x/deployment/high_performance_inference.en.md
Normal file
94
docs/version3.x/deployment/high_performance_inference.en.md
Normal file
@@ -0,0 +1,94 @@
|
||||
# High-Performance Inference
|
||||
|
||||
In real-world production environments, many applications have stringent performance requirements for deployment strategies, particularly regarding response speed, to ensure efficient system operation and a smooth user experience. PaddleOCR provides high-performance inference capabilities, allowing users to enhance model inference speed with a single click without worrying about complex configurations or underlying details. Specifically, PaddleOCR's high-performance inference functionality can:
|
||||
|
||||
- Automatically select an appropriate inference backend (e.g., Paddle Inference, OpenVINO, ONNX Runtime, TensorRT) based on prior knowledge and configure acceleration strategies (e.g., increasing the number of inference threads, setting FP16 precision inference);
|
||||
- Automatically convert PaddlePaddle static graph models to ONNX format as needed to leverage better inference backends for acceleration.
|
||||
|
||||
This document primarily introduces the installation and usage methods for high-performance inference.
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
### 1.1 Install High-Performance Inference Dependencies
|
||||
|
||||
Install the dependencies required for high-performance inference using the PaddleOCR CLI:
|
||||
|
||||
```bash
|
||||
paddleocr install_hpi_deps {device_type}
|
||||
```
|
||||
|
||||
The supported device types are:
|
||||
|
||||
- `cpu`: For CPU-only inference. Currently supports Linux systems, x86-64 architecture processors, and Python 3.8-3.12.
|
||||
- `gpu`: For inference using either CPU or NVIDIA GPU. Currently supports Linux systems, x86-64 architecture processors, and Python 3.8-3.12. If you want to use the full high-performance inference capabilities, you also need to ensure that a compatible version of TensorRT is installed in your environment. Refer to the next subsection for detailed instructions.
|
||||
|
||||
Only one type of device dependency should exist in the same environment. For Windows systems, it is currently recommended to install within a Docker container or [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) environment.
|
||||
|
||||
**It is recommended to use the official PaddlePaddle Docker image to install high-performance inference dependencies.** The corresponding images for each device type are as follows:
|
||||
|
||||
- `cpu`: `ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddle:3.0.0`
|
||||
- `gpu`:
|
||||
- CUDA 11.8: `ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddle:3.0.0-gpu-cuda11.8-cudnn8.9-trt8.6`
|
||||
|
||||
### 1.2 Detailed GPU Environment Instructions
|
||||
|
||||
First, ensure that the environment has the required CUDA and cuDNN versions installed. Currently, PaddleOCR only supports CUDA and cuDNN versions compatible with CUDA 11.8 + cuDNN 8.9. Below are the installation instructions for CUDA 11.8 and cuDNN 8.9:
|
||||
|
||||
- [Install CUDA 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive)
|
||||
- [Install cuDNN 8.9](https://docs.nvidia.com/deeplearning/cudnn/archives/cudnn-890/install-guide/index.html)
|
||||
|
||||
If using the official PaddlePaddle image, the CUDA and cuDNN versions in the image already meet the requirements, and no additional installation is needed.
|
||||
|
||||
If installing PaddlePaddle via pip, the relevant Python packages for CUDA and cuDNN will typically be installed automatically. In this case, **you still need to install the non-Python-specific CUDA and cuDNN versions.** It is also recommended to install CUDA and cuDNN versions that match the Python package versions in your environment to avoid potential issues caused by coexisting library versions. You can check the versions of the CUDA and cuDNN-related Python packages with the following commands:
|
||||
|
||||
```bash
|
||||
# CUDA-related Python package versions
|
||||
pip list | grep nvidia-cuda
|
||||
# cuDNN-related Python package versions
|
||||
pip list | grep nvidia-cudnn
|
||||
```
|
||||
|
||||
Secondly, it is recommended to ensure that a compatible version of TensorRT is installed in the environment; otherwise, the Paddle Inference TensorRT subgraph engine will be unavailable, and the program may not achieve optimal inference performance. Currently, PaddleOCR only supports TensorRT 8.6.1.6. If using the official PaddlePaddle image, you can install the TensorRT wheel package with the following command:
|
||||
|
||||
```bash
|
||||
python -m pip install /usr/local/TensorRT-*/python/tensorrt-*-cp310-none-linux_x86_64.whl
|
||||
```
|
||||
|
||||
For other environments, refer to the [TensorRT documentation](https://docs.nvidia.com/deeplearning/tensorrt/archives/index.html) to install TensorRT. Here is an example:
|
||||
|
||||
```bash
|
||||
# Download the TensorRT tar file
|
||||
wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/secure/8.6.1/tars/TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-11.8.tar.gz
|
||||
# Extract the TensorRT tar file
|
||||
tar xvf TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-11.8.tar.gz
|
||||
# Install the TensorRT wheel package
|
||||
python -m pip install TensorRT-8.6.1.6/python/tensorrt-8.6.1-cp310-none-linux_x86_64.whl
|
||||
# Add the absolute path of the TensorRT `lib` directory to LD_LIBRARY_PATH
|
||||
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:TensorRT-8.6.1.6/lib"
|
||||
```
|
||||
|
||||
## 2. Executing High-Performance Inference
|
||||
|
||||
For the PaddleOCR CLI, specify `--enable_hpi` as `True` to execute high-performance inference. For example:
|
||||
|
||||
```bash
|
||||
paddleocr ocr --enable_hpi True ...
|
||||
```
|
||||
|
||||
For the PaddleOCR Python API, set `enable_hpi` to `True` when initializing the pipeline or module object to enable high-performance inference when calling the inference method. For example:
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
pipeline = PaddleOCR(enable_hpi=True)
|
||||
result = pipeline.predict(...)
|
||||
```
|
||||
|
||||
## 3. Notes
|
||||
|
||||
1. For some models, the first execution of high-performance inference may take longer to complete the construction of the inference engine. Relevant information about the inference engine will be cached in the model directory after the first construction, and subsequent initializations can reuse the cached content to improve speed.
|
||||
|
||||
2. Currently, due to reasons such as not using static graph format models or the presence of unsupported operators, some models may not achieve inference acceleration.
|
||||
|
||||
3. During high-performance inference, PaddleOCR automatically handles the conversion of model formats and selects the optimal inference backend whenever possible. Additionally, PaddleOCR supports users specifying ONNX models. For information on converting PaddlePaddle static graph models to ONNX format, refer to [Obtaining ONNX Models](./obtaining_onnx_models.en.md).
|
||||
|
||||
4. The high-performance inference capabilities of PaddleOCR rely on PaddleX and its high-performance inference plugins. By passing in a custom PaddleX production line configuration file, you can configure the inference backend and other related settings. Please refer to [Using PaddleX Production Line Configuration Files](../paddleocr_and_paddlex.en.md#3-Using-PaddleX-Pipeline-Configuration-Files) and the [PaddleX High-Performance Inference Guide](https://paddlepaddle.github.io/PaddleX/3.0/en/pipeline_deploy/high_performance_inference.html#22) to learn how to adjust the high-performance inference configurations.
|
||||
91
docs/version3.x/deployment/high_performance_inference.md
Normal file
91
docs/version3.x/deployment/high_performance_inference.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# 高性能推理
|
||||
|
||||
在实际生产环境中，许多应用对部署策略的性能指标（尤其是响应速度）有着较严苛的标准，以确保系统的高效运行与用户体验的流畅性。PaddleOCR 提供高性能推理能力，让用户无需关注复杂的配置和底层细节，一键提升模型的推理速度。具体而言，PaddleOCR 的高性能推理功能能够：
|
||||
|
||||
- 结合先验知识自动选择合适的推理后端（Paddle Inference、OpenVINO、ONNX Runtime、TensorRT等），并配置加速策略（如增大推理线程数、设置 FP16 精度推理）；
|
||||
- 根据需要自动将飞桨静态图模型转换为 ONNX 格式，以使用更优的推理后端实现加速。
|
||||
|
||||
本文档主要介绍高性能推理功能的安装与使用方法。
|
||||
|
||||
## 1. 前置条件
|
||||
|
||||
## 1.1 安装高性能推理依赖
|
||||
|
||||
通过 PaddleOCR CLI 安装高性能推理所需依赖：
|
||||
|
||||
```bash
|
||||
paddleocr install_hpi_deps {设备类型}
|
||||
```
|
||||
|
||||
支持的设备类型包括：
|
||||
|
||||
- `cpu`：仅使用 CPU 推理。目前支持 Linux 系统、x86-64 架构处理器、Python 3.8-3.12。
|
||||
- `gpu`：使用 CPU 或 NVIDIA GPU 推理。目前支持 Linux 系统、x86-64 架构处理器、Python 3.8-3.12。如果希望使用完整的高性能推理功能，还需要确保环境中安装有符合要求的 TensorRT。请查看下一小节的详细说明。
|
||||
|
||||
同一环境中只应该存在一种设备类型的依赖。对于 Windows 系统，目前建议在 Docker 容器或者 [WSL](https://learn.microsoft.com/zh-cn/windows/wsl/install) 环境中安装。
|
||||
|
||||
**推荐使用飞桨官方 Docker 镜像安装高性能推理依赖。** 各设备类型对应的镜像如下：
|
||||
|
||||
- `cpu`：`ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddle:3.0.0`
|
||||
- `gpu`：
|
||||
- CUDA 11.8：`ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddle:3.0.0-gpu-cuda11.8-cudnn8.9-trt8.6`
|
||||
|
||||
## 1.2 GPU 环境详细说明
|
||||
|
||||
首先，需要确保环境中安装有符合要求的 CUDA 与 cuDNN。目前 PaddleOCR 仅支持与 CUDA 11.8 + cuDNN 8.9 兼容的 CUDA 和 cuDNN版本。以下分别是 CUDA 11.8 和 cuDNN 8.9 的安装说明文档：
|
||||
|
||||
- [安装 CUDA 11.8](https://developer.nvidia.com/cuda-11-8-0-download-archive)
|
||||
- [安装 cuDNN 8.9](https://docs.nvidia.com/deeplearning/cudnn/archives/cudnn-890/install-guide/index.html)
|
||||
|
||||
如果使用飞桨官方镜像，则镜像中的 CUDA 和 cuDNN 版本已经是满足要求的，无需额外安装。
|
||||
|
||||
如果通过 pip 安装飞桨，通常 CUDA、cuDNN 的相关 Python 包将被自动安装。在这种情况下，**仍需要通过安装非 Python 专用的 CUDA 与 cuDNN**。同时，建议安装的 CUDA 和 cuDNN 版本与环境中存在的 Python 包版本保持一致，以避免不同版本的库共存导致的潜在问题。可以通过如下方式可以查看 CUDA 和 cuDNN 相关 Python 包的版本：
|
||||
|
||||
```bash
|
||||
# CUDA 相关 Python 包版本
|
||||
pip list | grep nvidia-cuda
|
||||
# cuDNN 相关 Python 包版本
|
||||
pip list | grep nvidia-cudnn
|
||||
```
|
||||
|
||||
其次，建议确保环境中安装有符合要求的 TensorRT，否则 Paddle Inference TensorRT 子图引擎将不可用，程序可能无法取得最佳推理性能。目前 PaddleOCR 仅支持 TensorRT 8.6.1.6。如果使用飞桨官方镜像，可执行如下命令安装 TensorRT wheel 包：
|
||||
|
||||
```bash
|
||||
python -m pip install /usr/local/TensorRT-*/python/tensorrt-*-cp310-none-linux_x86_64.whl
|
||||
```
|
||||
|
||||
对于其他环境，请参考 [TensorRT 文档](https://docs.nvidia.com/deeplearning/tensorrt/archives/index.html) 安装 TensorRT。示例如下：
|
||||
|
||||
```bash
|
||||
# 下载 TensorRT tar 文件
|
||||
wget https://developer.nvidia.com/downloads/compute/machine-learning/tensorrt/secure/8.6.1/tars/TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-11.8.tar.gz
|
||||
# 解压 TensorRT tar 文件
|
||||
tar xvf TensorRT-8.6.1.6.Linux.x86_64-gnu.cuda-11.8.tar.gz
|
||||
# 安装 TensorRT wheel 包
|
||||
python -m pip install TensorRT-8.6.1.6/python/tensorrt-8.6.1-cp310-none-linux_x86_64.whl
|
||||
# 添加 TensorRT 的 `lib` 目录的绝对路径到 LD_LIBRARY_PATH 中
|
||||
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:TensorRT-8.6.1.6/lib"
|
||||
```
|
||||
|
||||
## 2. 执行高性能推理
|
||||
|
||||
对于 PaddleOCR CLI，指定 `--enable_hpi` 为 `True` 即可执行高性能推理。例如：
|
||||
|
||||
```bash
|
||||
paddleocr ocr --enable_hpi True ...
|
||||
```
|
||||
|
||||
对于 PaddleOCR Python API，在初始化产线对象或者模块对象时，设置 `enable_hpi` 为 `True` 即可在调用推理方法时执行高性能推理。例如：
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
pipeline = PaddleOCR(enable_hpi=True)
|
||||
result = pipeline.predict(...)
|
||||
```
|
||||
|
||||
## 3. 说明
|
||||
|
||||
1. 对于部分模型，在首次执行高性能推理时，可能需要花费较长时间完成推理引擎的构建。推理引擎相关信息将在第一次构建完成后被缓存在模型目录，后续可复用缓存中的内容以提升初始化速度。
|
||||
2. 目前，由于使用的不是静态图格式模型、存在不支持算子等原因，部分模型可能无法获得推理加速。
|
||||
3. 在进行高性能推理时，PaddleOCR 会自动处理模型格式的转换，并尽可能选择最优的推理后端。同时，PaddleOCR 也支持用户指定 ONNX 模型。有关如何飞桨静态图模型转换为 ONNX 格式，可参考 [获取 ONNX 模型](./obtaining_onnx_models.md)。
|
||||
4. PaddleOCR 的高性能推理能力依托于 PaddleX 及其高性能推理插件。通过传入自定义 PaddleX 产线配置文件，可以对推理后端等进行配置。请参考 [使用 PaddleX 产线配置文件](../paddleocr_and_paddlex.md#3-使用-paddlex-产线配置文件) 和 [PaddleX 高性能推理指南](https://paddlepaddle.github.io/PaddleX/3.0/pipeline_deploy/high_performance_inference.html#22) 了解如何调整高性能推理配置。
|
||||
312
docs/version3.x/deployment/mcp_server.en.md
Normal file
312
docs/version3.x/deployment/mcp_server.en.md
Normal file
@@ -0,0 +1,312 @@
|
||||
# PaddleOCR MCP Server
|
||||
|
||||
[](https://github.com/PaddlePaddle/PaddleOCR)
|
||||
[](https://gofastmcp.com)
|
||||
|
||||
This project provides a lightweight [Model Context Protocol (MCP)](https://modelcontextprotocol.io/introduction) server designed to integrate PaddleOCR capabilities into various LLM applications.
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Currently Supported Tools**
|
||||
- **OCR**: Performs text detection and recognition on images and PDF files.
|
||||
- **PP-StructureV3**: Identifies and extracts text blocks, titles, paragraphs, images, tables, and other layout elements from images or PDF files, converting the input into Markdown documents.
|
||||
- **Supported Working Modes**
|
||||
- **Local Python Library**: Runs PaddleOCR pipelines directly on the local machine. This mode requires a suitable local environment and hardware, and is ideal for offline use or privacy-sensitive scenarios.
|
||||
- **AI Studio Community Service**: Invokes services hosted on the [PaddlePaddle AI Studio Community](https://aistudio.baidu.com/pipeline/mine). This is suitable for quick testing, prototyping, or no-code scenarios.
|
||||
- **Self-hosted Service**: Invokes the user's self-hosted PaddleOCR services. This mode offers the advantages of serving and high flexibility. It is suitable for scenarios requiring customized service configurations, as well as those with strict data privacy requirements. **Currently, only the basic serving solution is supported.**
|
||||
|
||||
### Table of Contents
|
||||
|
||||
- [Table of Contents](#table-of-contents)
|
||||
- [1. Installation](#1-installation)
|
||||
- [2. Using with Claude for Desktop](#2-using-with-claude-for-desktop)
|
||||
- [2.1 Quick Start](#21-quick-start)
|
||||
- [2.2 MCP Host Configuration Details](#22-mcp-host-configuration-details)
|
||||
- [2.3 Working Modes Explained](#23-working-modes-explained)
|
||||
- [2.4 Using `uvx`](#24-using-uvx)
|
||||
- [3. Running the Server](#3-running-the-server)
|
||||
- [4. Parameter Reference](#4-parameter-reference)
|
||||
- [5. Known Limitations](#5-known-limitations)
|
||||
|
||||
## 1. Installation
|
||||
|
||||
This section explains how to install the `paddleocr-mcp` library via pip.
|
||||
|
||||
- For the local Python library mode, you need to install both `paddleocr-mcp` and the PaddlePaddle framework along with PaddleOCR, as per the [PaddleOCR installation documentation](../installation.en.md).
|
||||
- For the AI Studio community service or the self-hosted service modes, if used within MCP hosts like Claude for Desktop, the server can also be run without installation via tools like `uvx`. See [2. Using with Claude for Desktop](#2-using-with-claude-for-desktop) for details.
|
||||
|
||||
To install `paddleocr-mcp` using pip:
|
||||
|
||||
```bash
|
||||
# Install the wheel
|
||||
pip install https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/mcp/paddleocr_mcp/releases/v0.1.0/paddleocr_mcp-0.1.0-py3-none-any.whl
|
||||
|
||||
# Or install from source
|
||||
# git clone https://github.com/PaddlePaddle/PaddleOCR.git
|
||||
# pip install -e mcp_server
|
||||
```
|
||||
|
||||
To verify successful installation:
|
||||
|
||||
```bash
|
||||
paddleocr_mcp --help
|
||||
```
|
||||
|
||||
If the help message is printed, the installation succeeded. This project depends on the `python-magic` library. If you see the following error:
|
||||
|
||||
```
|
||||
...
|
||||
ImportError: failed to find libmagic. Check your installation
|
||||
```
|
||||
|
||||
You are likely missing a required native library for python-magic. Please refer to the [official python-magic documentation](https://github.com/ahupp/python-magic?tab=readme-ov-file#installation) for installation instructions.
|
||||
|
||||
## 2. Using with Claude for Desktop
|
||||
|
||||
This section explains how to use the PaddleOCR MCP server within Claude for Desktop. The steps are also applicable to other MCP hosts with minor adjustments.
|
||||
|
||||
### 2.1 Quick Start
|
||||
|
||||
1. **Install `paddleocr-mcp`**
|
||||
|
||||
Refer to [1. Installation](#1-installation). To avoid dependency conflicts, **it is strongly recommended to install in an isolated virtual environment**.
|
||||
|
||||
2. **Install PaddleOCR**
|
||||
|
||||
Install the PaddlePaddle framework and PaddleOCR, as per the [PaddleOCR installation documentation](../installation.en.md).
|
||||
|
||||
3. **Add MCP Server Configuration**
|
||||
|
||||
Locate the `claude_desktop_config.json` configuration file:
|
||||
|
||||
- **macOS**: `~/Library/Application Support/Claude/claude_desktop_config.json`
|
||||
- **Windows**: `%APPDATA%\Claude\claude_desktop_config.json`
|
||||
- **Linux**: `~/.config/Claude/claude_desktop_config.json`
|
||||
|
||||
Edit the file as follows:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"paddleocr-ocr": {
|
||||
"command": "paddleocr_mcp",
|
||||
"args": [],
|
||||
"env": {
|
||||
"PADDLEOCR_MCP_PIPELINE": "OCR",
|
||||
"PADDLEOCR_MCP_PPOCR_SOURCE": "local"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**:
|
||||
|
||||
- `PADDLEOCR_MCP_PIPELINE_CONFIG` is optional; if not set, the default pipeline configuration will be used. If you need to adjust the configuration, such as changing the model, please refer to the [PaddleOCR documentation](../paddleocr_and_paddlex.md) to export the pipeline configuration file, and set `PADDLEOCR_MCP_PIPELINE_CONFIG` to the absolute path of this configuration file.
|
||||
|
||||
- **Inference Performance Tips**:
|
||||
|
||||
If you encounter issues such as long inference time or insufficient memory during use, you may consider adjusting the pipeline configuration according to the following recommendations.
|
||||
|
||||
- **OCR Pipeline**: It is recommended to switch to the `mobile` series models. For example, you can modify the pipeline configuration file to use `PP-OCRv5_mobile_det` for detection and `PP-OCRv5_mobile_rec` for recognition.
|
||||
|
||||
- **PP-StructureV3 Pipeline**:
|
||||
|
||||
- Disable unused features, e.g., set `use_formula_recognition` to `False` to disable formula recognition.
|
||||
- Use lightweight models, such as replacing the OCR model with the `mobile` version or switching to a lightweight formula recognition model like PP-FormulaNet-S.
|
||||
|
||||
The following sample code can be used to obtain the pipeline configuration file, in which most optional features of the PP-StructureV3 pipeline are disabled, and some key models are replaced with lightweight versions.
|
||||
|
||||
```python
|
||||
from paddleocr import PPStructureV3
|
||||
|
||||
pipeline = PPStructureV3(
|
||||
use_doc_orientation_classify=False, # Disable document image orientation classification
|
||||
use_doc_unwarping=False, # Disable text image unwarping
|
||||
use_textline_orientation=False, # Disable text line orientation classification
|
||||
use_formula_recognition=False, # Disable formula recognition
|
||||
use_seal_recognition=False, # Disable seal text recognition
|
||||
use_table_recognition=False, # Disable table recognition
|
||||
use_chart_recognition=False, # Disable chart parsing
|
||||
# Use lightweight models
|
||||
text_detection_model_name="PP-OCRv5_mobile_det",
|
||||
text_recognition_model_name="PP-OCRv5_mobile_rec",
|
||||
layout_detection_model_name="PP-DocLayout-S",
|
||||
)
|
||||
|
||||
# The configuration file is saved to `PP-StructureV3.yaml`
|
||||
pipeline.export_paddlex_config_to_yaml("PP-StructureV3.yaml")
|
||||
```
|
||||
|
||||
**Important**:
|
||||
|
||||
- If `paddleocr_mcp` is not in your system's `PATH`, set `command` to the absolute path of the executable.
|
||||
|
||||
4. **Restart the MCP Host**
|
||||
|
||||
Restart Claude for Desktop. The `paddleocr-ocr` tool should now be available in the application.
|
||||
|
||||
### 2.2 MCP Host Configuration Details
|
||||
|
||||
In the configuration file for Claude for Desktop, you need to define how the MCP server is started. The key fields are as follows:
|
||||
|
||||
- `command`: `paddleocr_mcp` (if the executable can be found in the `PATH`) or the absolute path.
|
||||
- `args`: Configurable command-line arguments, such as `["--verbose"]`. See [4. Parameter Reference](#4-parameter-reference) for details.
|
||||
- `env`: Configurable environment variables. See [4. Parameter Reference](#4-parameter-reference) for details.
|
||||
|
||||
### 2.3 Working Modes Explained
|
||||
|
||||
You can configure the MCP server according to your requirements to run in different working modes. The operational procedures vary for different modes, which will be explained in detail below.
|
||||
|
||||
#### Mode 1: Local Python Library
|
||||
|
||||
See [2.1 Quick Start](#21-quick-start).
|
||||
|
||||
#### Mode 2: AI Studio Community Service
|
||||
|
||||
1. Install `paddleocr-mcp`.
|
||||
2. Set up AI Studio community service.
|
||||
- Visit [PaddlePaddle AI Studio Community](https://aistudio.baidu.com/pipeline/mine) and log in. **Note: A China mainland phone number is required to use this service.** If unavailable, consider other modes.
|
||||
- Under "PaddleX Pipeline" in the "More" section on the left, click in sequence: [Create Pipeline] - [OCR] - [General OCR] - [Deploy Directly] - [Start Deployment].
|
||||
- After deployment, obtain your **service base URL** (e.g., `https://xxxxxx.aistudio-hub.baidu.com`).
|
||||
- Get your **access token** from [this page](https://aistudio.baidu.com/index/accessToken).
|
||||
3. Refer to the configuration example below to modify the contents of the `claude_desktop_config.json` file.
|
||||
3. Restart the MCP host.
|
||||
|
||||
Configuration example:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"paddleocr-ocr": {
|
||||
"command": "paddleocr_mcp",
|
||||
"args": [],
|
||||
"env": {
|
||||
"PADDLEOCR_MCP_PIPELINE": "OCR",
|
||||
"PADDLEOCR_MCP_PPOCR_SOURCE": "aistudio",
|
||||
"PADDLEOCR_MCP_SERVER_URL": "<your-server-url>",
|
||||
"PADDLEOCR_MCP_AISTUDIO_ACCESS_TOKEN": "<your-access-token>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Notes**:
|
||||
|
||||
- Replace `<your-server-url>` with your AI Studio service base URL, e.g., `https://xxxxx.aistudio-hub.baidu.com`. Make sure not to include the endpoint path (such as `/ocr`).
|
||||
- Replace `<your-access-token>` with your access token.
|
||||
|
||||
**Important**:
|
||||
|
||||
- Do not expose your access token.
|
||||
|
||||
You may also train and deploy custom models on the platform.
|
||||
|
||||
#### Mode 3: Self-hosted Service
|
||||
|
||||
1. In the environment where you need to run the PaddleOCR inference server, run the inference server as per the [PaddleOCR serving documentation](./serving.en.md).
|
||||
2. Install `paddleocr-mcp` where the MCP server will run.
|
||||
3. Refer to the configuration example below to modify the contents of the `claude_desktop_config.json` file.
|
||||
4. Set `PADDLEOCR_MCP_SERVER_URL` (e.g., `"http://127.0.0.1:8000"`).
|
||||
5. Restart the MCP host.
|
||||
|
||||
Configuration example:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"paddleocr-ocr": {
|
||||
"command": "paddleocr_mcp",
|
||||
"args": [],
|
||||
"env": {
|
||||
"PADDLEOCR_MCP_PIPELINE": "OCR",
|
||||
"PADDLEOCR_MCP_PPOCR_SOURCE": "self_hosted",
|
||||
"PADDLEOCR_MCP_SERVER_URL": "<your-server-url>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Note**:
|
||||
|
||||
- Replace `<your-server-url>` with your service’s base URL (e.g., `http://127.0.0.1:8000`).
|
||||
|
||||
### 2.4 Using `uvx`
|
||||
|
||||
Currently, for both the AI Studio and self-hosted modes, starting the MCP server via `uvx` is also supported. With this approach, manual installation of `paddleocr-mcp` is not required. The main steps are as follows:
|
||||
|
||||
1. Install [uv](https://docs.astral.sh/uv/#installation).
|
||||
2. Modify `claude_desktop_config.json`. Example for self-hosted mode:
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"paddleocr-ocr": {
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"--from",
|
||||
"paddleocr-mcp@https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/mcp/paddleocr_mcp/releases/v0.1.0/paddleocr_mcp-0.1.0-py3-none-any.whl",
|
||||
"paddleocr_mcp"
|
||||
],
|
||||
"env": {
|
||||
"PADDLEOCR_MCP_PIPELINE": "OCR",
|
||||
"PADDLEOCR_MCP_PPOCR_SOURCE": "self_hosted",
|
||||
"PADDLEOCR_MCP_SERVER_URL": "<your-server-url>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Due to the different startup methods used, the settings for `command` and `args` in the configuration file differ significantly from those described in [2.1 Quick Start](#21-quick-start). However, the command-line arguments and environment variables (such as `PADDLEOCR_MCP_SERVER_URL`) supported by the MCP service itself can still be set in the same way.
|
||||
|
||||
## 3. Running the Server
|
||||
|
||||
In addition to MCP hosts like Claude for Desktop, you can also run the PaddleOCR MCP server via the CLI.
|
||||
|
||||
To view help:
|
||||
|
||||
```bash
|
||||
paddleocr_mcp --help
|
||||
```
|
||||
|
||||
Example commands:
|
||||
|
||||
```bash
|
||||
# OCR + AI Studio community service + stdio
|
||||
PADDLEOCR_MCP_AISTUDIO_ACCESS_TOKEN=xxxxxx paddleocr_mcp --pipeline OCR --ppocr_source aistudio --server_url https://xxxxxx.aistudio-hub.baidu.com
|
||||
|
||||
# PP-StructureV3 + local Python library + stdio
|
||||
paddleocr_mcp --pipeline PP-StructureV3 --ppocr_source local
|
||||
|
||||
# OCR + self-hosted service + Streamable HTTP
|
||||
paddleocr_mcp --pipeline OCR --ppocr_source self_hosted --server_url http://127.0.0.1:8080 --http
|
||||
```
|
||||
|
||||
You can find all the supported parameters of the PaddleOCR MCP server in [4. Parameter Reference](#4-parameter-reference).
|
||||
|
||||
## 4. Parameter Reference
|
||||
|
||||
You can control the MCP server via environment variables or CLI arguments.
|
||||
|
||||
| Environment Variable | CLI Argument | Type | Description | Options | Default |
|
||||
| ------------------------------------- | ------------------------- | ------ | --------------------------------------------------------------------- | ---------------------------------------- | ------------- |
|
||||
| `PADDLEOCR_MCP_PIPELINE` | `--pipeline` | `str` | Pipeline to run. | `"OCR"`, `"PP-StructureV3"` | `"OCR"` |
|
||||
| `PADDLEOCR_MCP_PPOCR_SOURCE` | `--ppocr_source` | `str` | Source of PaddleOCR capabilities. | `"local"` (local Python library), `"aistudio"` (AI Studio community service), `"self_hosted"` (self-hosted service) | `"local"` |
|
||||
| `PADDLEOCR_MCP_SERVER_URL` | `--server_url` | `str` | Base URL for the underlying service (`aistudio` or `self_hosted` mode only). | - | `None` |
|
||||
| `PADDLEOCR_MCP_AISTUDIO_ACCESS_TOKEN` | `--aistudio_access_token` | `str` | AI Studio access token (`aistudio` mode only). | - | `None` |
|
||||
| `PADDLEOCR_MCP_TIMEOUT` | `--timeout` | `int` | Read timeout for the underlying requests (seconds). | - | `60` |
|
||||
| `PADDLEOCR_MCP_DEVICE` | `--device` | `str` | Device for inference (`local` mode only). | - | `None` |
|
||||
| `PADDLEOCR_MCP_PIPELINE_CONFIG` | `--pipeline_config` | `str` | Path to pipeline config file (`local` mode only). | - | `None` |
|
||||
| - | `--http` | `bool` | Use Streamable HTTP instead of stdio (for remote/multi-client use). | - | `False` |
|
||||
| - | `--host` | `str` | Host for the Stremable HTTP mode. | - | `"127.0.0.1"` |
|
||||
| - | `--port` | `int` | Port for the Streamable HTTP mode. | - | `8000` |
|
||||
| - | `--verbose` | `bool` | Enable verbose logging for debugging. | - | `False` |
|
||||
|
||||
## 5. Known Limitations
|
||||
|
||||
- In the local Python library mode, the current tools cannot process PDF document inputs that are Base64 encoded.
|
||||
- In the local Python library mode, the current tools do not infer the file type based on the model's `file_type` prompt, and may fail to process some complex URLs.
|
||||
- For the PP-StructureV3 pipeline, if the input file contains images, the returned results may significantly increase token usage. If image content is not needed, you can explicitly exclude it through prompts to reduce resource consumption.
|
||||
308
docs/version3.x/deployment/mcp_server.md
Normal file
308
docs/version3.x/deployment/mcp_server.md
Normal file
@@ -0,0 +1,308 @@
|
||||
# PaddleOCR MCP 服务器
|
||||
|
||||
[](https://github.com/PaddlePaddle/PaddleOCR)
|
||||
[](https://gofastmcp.com)
|
||||
|
||||
本项目提供轻量级的 [Model Context Protocol（MCP）](https://modelcontextprotocol.io/introduction) 服务器，旨在将 PaddleOCR 的能力集成到各种大模型应用中。
|
||||
|
||||
主要功能如下：
|
||||
|
||||
- **当前支持的工具**
|
||||
- **OCR**：对图像和 PDF 文件进行文本检测与识别。
|
||||
- **PP-StructureV3**：从图像或 PDF 文件中识别和提取文本块、标题、段落、图片、表格以及其他版面元素，将输入转换为 Markdown 文档。
|
||||
- **支持运行在如下工作模式**
|
||||
- **本地 Python 库**：在本机直接运行 PaddleOCR 产线。此模式对本地环境与计算机性能有一定要求，适用于需要离线使用、对数据隐私有严格要求的场景。
|
||||
- **星河社区服务**：调用托管在 [飞桨星河社区](https://aistudio.baidu.com/pipeline/mine) 的服务。此模式适合快速体验功能、快速验证方案等，也适用于零代码开发场景。
|
||||
- **自托管服务**：调用用户自托管的 PaddleOCR 服务。此模式具备服务化部署优势及高度灵活性，适用于需要自定义服务配置的场景，同时也适用于对数据隐私有严格要求的场景。**目前暂时只支持基础服务化部署方案。**
|
||||
|
||||
### 目录
|
||||
|
||||
- [目录](#目录)
|
||||
- [1. 安装](#1-安装)
|
||||
- [2. 在 Claude for Desktop 中使用](#2-在-claude-for-desktop-中使用)
|
||||
- [2.1 快速开始](#21-快速开始)
|
||||
- [2.2 MCP 主机配置说明](#22-mcp-主机配置说明)
|
||||
- [2.3 工作模式说明](#23-工作模式说明)
|
||||
- [2.4 使用 `uvx`](#24-使用-uvx)
|
||||
- [3. 运行服务器](#3-运行服务器)
|
||||
- [4. 参数说明](#4-参数说明)
|
||||
- [5. 已知局限性](#5-已知局限性)
|
||||
|
||||
## 1. 安装
|
||||
|
||||
本节将介绍如何通过 pip 安装 `paddleocr-mcp` 库。
|
||||
|
||||
- 对于本地 Python 库模式，除了安装 `paddleocr-mcp` 外，还需要参考 [PaddleOCR 安装文档](../installation.md) 安装飞桨框架和 PaddleOCR。
|
||||
- 对于星河社区服务和自托管服务模式，如果希望在 Claude for Desktop 等 MCP 主机中使用，也支持通过 `uvx` 等方式免安装运行服务器。详情请参考 [2. 在 Claude for Desktop 中使用](#2-在-claude-for-desktop-中使用) 中的说明。
|
||||
|
||||
使用 pip 安装 `paddleocr-mcp` 库的命令如下：
|
||||
|
||||
```bash
|
||||
# 安装 wheel 包
|
||||
pip install https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/mcp/paddleocr_mcp/releases/v0.1.0/paddleocr_mcp-0.1.0-py3-none-any.whl
|
||||
|
||||
# 或者，从项目源码安装
|
||||
# git clone https://github.com/PaddlePaddle/PaddleOCR.git
|
||||
# pip install -e mcp_server
|
||||
```
|
||||
|
||||
可通过以下命令检查是否安装成功：
|
||||
|
||||
```bash
|
||||
paddleocr_mcp --help
|
||||
```
|
||||
|
||||
如果执行上述命令后打印出了帮助信息，则说明安装成功。本项目依赖 python-magic 库。如果在执行上述命令时出现如下错误提示：
|
||||
|
||||
```
|
||||
...
|
||||
ImportError: failed to find libmagic. Check your installation
|
||||
```
|
||||
|
||||
很可能是因为缺少 python-magic 库所需的底层库。请参考 [python-magic 官方文档](https://github.com/ahupp/python-magic?tab=readme-ov-file#installation) 完成相应依赖库的安装。
|
||||
|
||||
## 2. 在 Claude for Desktop 中使用
|
||||
|
||||
本节将介绍如何在 Claude for Desktop 中使用 PaddleOCR MCP 服务器。对于其他 MCP 主机，也可参照本节的步骤，并根据实际情况进行相应调整。
|
||||
|
||||
### 2.1 快速开始
|
||||
|
||||
接下来以 **星河社区服务** 工作模式为例，引导您快速上手。此模式无需在本地安装复杂的依赖，因此比较适合用于快速体验。
|
||||
|
||||
1. **安装 `paddleocr-mcp`**
|
||||
|
||||
请参考 [1. 安装](#1-安装)。
|
||||
|
||||
2. **准备星河社区服务**
|
||||
|
||||
- 访问 [飞桨星河社区](https://aistudio.baidu.com/pipeline/mine) 并登录。**请注意，目前星河社区要求用户绑定中国大陆手机号。** 如果您不具备此条件，请考虑使用其他工作模式。
|
||||
- 在左侧"更多内容"下的 "PaddleX 产线" 部分，依次点击：【创建产线】 - 【OCR】- 【通用OCR】-【直接部署】-【开始部署】。
|
||||
- 部署成功后，获取您的 **服务基础 URL**（示例：`https://xxxxxx.aistudio-hub.baidu.com`）。
|
||||
- 在 [此页面](https://aistudio.baidu.com/index/accessToken) 获取您的 **访问令牌**。
|
||||
|
||||
3. **添加 MCP 服务器配置**
|
||||
|
||||
在以下位置之一找到 Claude for Desktop 配置文件：
|
||||
|
||||
- **macOS**：`~/Library/Application Support/Claude/claude_desktop_config.json`
|
||||
- **Windows**：`%APPDATA%\Claude\claude_desktop_config.json`
|
||||
- **Linux**：`~/.config/Claude/claude_desktop_config.json`
|
||||
|
||||
打开 `claude_desktop_config.json` 文件，参考如下示例调整配置，填充到 `claude_desktop_config.json` 中。
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"paddleocr-ocr": {
|
||||
"command": "paddleocr_mcp",
|
||||
"args": [],
|
||||
"env": {
|
||||
"PADDLEOCR_MCP_PIPELINE": "OCR",
|
||||
"PADDLEOCR_MCP_PPOCR_SOURCE": "aistudio",
|
||||
"PADDLEOCR_MCP_SERVER_URL": "<your-server-url>",
|
||||
"PADDLEOCR_MCP_AISTUDIO_ACCESS_TOKEN": "<your-access-token>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**说明**：
|
||||
|
||||
- 将 `<your-server-url>` 替换为您的星河社区服务的基础 URL，例如 `https://xxxxx.aistudio-hub.baidu.com`，注意不要带有端点路径（如 `/ocr`）。
|
||||
- 将 `<your-access-token>` 替换为您的访问令牌。
|
||||
|
||||
**注意**：
|
||||
|
||||
- 请勿泄漏您的 **访问令牌**。
|
||||
- 如果 `paddleocr_mcp` 无法在系统 `PATH` 中找到，请将 `command` 设置为可执行文件的绝对路径。
|
||||
|
||||
5. **重启 MCP 主机**
|
||||
|
||||
重启 Claude for Desktop。新的 `paddleocr-ocr` 工具现在应该可以在应用中使用了。
|
||||
|
||||
### 2.2 MCP 主机配置说明
|
||||
|
||||
在 Claude for Desktop 的配置文件中，您需要定义 MCP 服务器的启动方式。关键字段如下：
|
||||
|
||||
- `command`：`paddleocr_mcp`（如果可执行文件可在 `PATH` 中找到）或绝对路径。
|
||||
- `args`：可配置命令行参数，如 `["--verbose"]`。详见 [4. 参数说明](#4-参数说明)。
|
||||
- `env`：可配置环境变量。详见 [4. 参数说明](#4-参数说明)。
|
||||
|
||||
### 2.3 工作模式说明
|
||||
|
||||
您可以根据需求配置 MCP 服务器，使其运行在不同的工作模式。不同工作模式需要的操作流程有所不同，下面将详细介绍。
|
||||
|
||||
#### 模式一：本地 Python 库
|
||||
|
||||
1. 安装 `paddleocr-mcp`。
|
||||
2. 安装飞桨框架和 PaddleOCR。为避免依赖冲突，**强烈建议在独立的虚拟环境中安装**。
|
||||
3. 参考下方的配置示例更改 `claude_desktop_config.json` 文件内容。
|
||||
4. 重启 MCP 主机。
|
||||
|
||||
配置示例：
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"paddleocr-ocr": {
|
||||
"command": "paddleocr_mcp",
|
||||
"args": [],
|
||||
"env": {
|
||||
"PADDLEOCR_MCP_PIPELINE": "OCR",
|
||||
"PADDLEOCR_MCP_PPOCR_SOURCE": "local"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**说明**：
|
||||
|
||||
- `PADDLEOCR_MCP_PIPELINE_CONFIG` 为可选项，不设置时使用产线默认配置。如需调整配置，例如更换模型，请参考 [PaddleOCR 文档](../paddleocr_and_paddlex.md) 导出产线配置文件，并将 `PADDLEOCR_MCP_PIPELINE_CONFIG` 设置为配置文件的绝对路径。
|
||||
- **推理性能提示**：
|
||||
|
||||
如果使用过程中出现推理耗时过长、内存不足等问题，可考虑参考如下建议调整产线配置：
|
||||
|
||||
- **OCR 产线**：建议更换 `mobile` 系列模型。例如，您可以在产线配置文件中将检测和识别模型分别修改为 `PP-OCRv5_mobile_det` 和 `PP-OCRv5_mobile_rec`。
|
||||
- **PP-StructureV3 产线**：
|
||||
|
||||
- 关闭不需要用到的功能，例如设置 `use_formula_recognition` 为 `False` 以禁用公式识别。
|
||||
- 使用轻量级的模型，例如将 OCR 模型替换为 `mobile` 版本、换用轻量的公式识别模型 PP-FormulaNet-S 等。
|
||||
|
||||
以下示例代码可用于获取产线配置文件，其中关闭了 PP-StructureV3 产线的大部分可选功能，同时将部分关键模型更换为轻量级版本。
|
||||
|
||||
```python
|
||||
from paddleocr import PPStructureV3
|
||||
|
||||
pipeline = PPStructureV3(
|
||||
use_doc_orientation_classify=False, # 禁用文档图像方向分类
|
||||
use_doc_unwarping=False, # 禁用文本图像矫正
|
||||
use_textline_orientation=False, # 禁用文本行方向分类
|
||||
use_formula_recognition=False, # 禁用公式识别
|
||||
use_seal_recognition=False, # 禁用印章文本识别
|
||||
use_table_recognition=False, # 禁用表格识别
|
||||
use_chart_recognition=False, # 禁用图表解析
|
||||
# 使用轻量级模型
|
||||
text_detection_model_name="PP-OCRv5_mobile_det",
|
||||
text_recognition_model_name="PP-OCRv5_mobile_rec",
|
||||
layout_detection_model_name="PP-DocLayout-S",
|
||||
)
|
||||
|
||||
# 配置文件保存到 `PP-StructureV3.yaml` 中
|
||||
pipeline.export_paddlex_config_to_yaml("PP-StructureV3.yaml")
|
||||
```
|
||||
|
||||
#### 模式二：星河社区服务
|
||||
|
||||
请参考 [2.1 快速开始](#21-快速开始)。
|
||||
|
||||
除了使用平台预设的模型方案，您也可以在平台上自行训练并部署自定义模型。
|
||||
|
||||
#### 模式三：自托管服务
|
||||
|
||||
1. 在需要运行 PaddleOCR 推理服务器的环境中，参考 [PaddleOCR 服务化部署文档](./serving.md) 运行推理服务器。
|
||||
3. 在需要运行 MCP 服务器的环境中安装 `paddleocr-mcp`。
|
||||
4. 参考下方的配置示例更改 `claude_desktop_config.json` 文件内容。
|
||||
5. 将您的服务地址填入 `PADDLEOCR_MCP_SERVER_URL` (例如：`"http://127.0.0.1:8000"`)。
|
||||
6. 重启 MCP 主机。
|
||||
|
||||
配置示例：
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"paddleocr-ocr": {
|
||||
"command": "paddleocr_mcp",
|
||||
"args": [],
|
||||
"env": {
|
||||
"PADDLEOCR_MCP_PIPELINE": "OCR",
|
||||
"PADDLEOCR_MCP_PPOCR_SOURCE": "self_hosted",
|
||||
"PADDLEOCR_MCP_SERVER_URL": "<your-server-url>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**说明**：
|
||||
|
||||
- 将 `<your-server-url>` 替换为底层服务的基础 URL（如：`http://127.0.0.1:8000`）。
|
||||
|
||||
### 2.4 使用 `uvx`
|
||||
|
||||
对于星河社区服务和自托管服务模式，目前也支持通过 `uvx` 启动 MCP 服务器。这种方式不需要手动安装 `paddleocr-mcp`。主要步骤如下：
|
||||
|
||||
1. 安装 [uv](https://docs.astral.sh/uv/#installation)。
|
||||
2. 修改 `claude_desktop_config.json` 文件的内容。以自托管服务模式为例：
|
||||
|
||||
```json
|
||||
{
|
||||
"mcpServers": {
|
||||
"paddleocr-ocr": {
|
||||
"command": "uvx",
|
||||
"args": [
|
||||
"--from",
|
||||
"paddleocr-mcp@https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/mcp/paddleocr_mcp/releases/v0.1.0/paddleocr_mcp-0.1.0-py3-none-any.whl",
|
||||
"paddleocr_mcp"
|
||||
],
|
||||
"env": {
|
||||
"PADDLEOCR_MCP_PIPELINE": "OCR",
|
||||
"PADDLEOCR_MCP_PPOCR_SOURCE": "self_hosted",
|
||||
"PADDLEOCR_MCP_SERVER_URL": "<your-server-url>"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
由于使用了不一样的启动方式，配置文件中 `command` 和 `args` 的设置都与 [2.1 快速开始](#21-快速开始) 介绍的方式存在显著不同，但 MCP 服务本身支持的命令行参数与环境变量（如 `PADDLEOCR_MCP_SERVER_URL`）仍然可以以相同的方式设置。
|
||||
|
||||
## 3. 运行服务器
|
||||
|
||||
除了在 Claude for Desktop 等 MCP 主机中使用外，您也可以通过 CLI 运行 PaddleOCR MCP 服务器。
|
||||
|
||||
执行以下命令可以打印帮助信息：
|
||||
|
||||
```bash
|
||||
paddleocr_mcp --help
|
||||
```
|
||||
|
||||
示例命令如下：
|
||||
|
||||
```bash
|
||||
# OCR + 星河社区服务 + stdio
|
||||
PADDLEOCR_MCP_AISTUDIO_ACCESS_TOKEN=xxxxxx paddleocr_mcp --pipeline OCR --ppocr_source aistudio --server_url https://xxxxxx.aistudio-hub.baidu.com
|
||||
|
||||
# PP-StructureV3 + 本地 Python 库 + stdio
|
||||
paddleocr_mcp --pipeline PP-StructureV3 --ppocr_source local
|
||||
|
||||
# OCR + 本地服务 + Streamable HTTP
|
||||
paddleocr_mcp --pipeline OCR --ppocr_source self_hosted --server_url http://127.0.0.1:8080 --http
|
||||
```
|
||||
|
||||
在 [4. 参数说明](#4-参数说明) 中可以了解 PaddleOCR MCP 服务器支持的全部参数。
|
||||
|
||||
## 4. 参数说明
|
||||
|
||||
您可以通过环境变量或命令行参数来控制 MCP 服务器的行为。
|
||||
|
||||
| 环境变量 | 命令行参数 | 类型 | 描述 | 可选值 | 默认值 |
|
||||
|:---------|:-----------|:-----|:-----|:-------|:-------|
|
||||
| `PADDLEOCR_MCP_PIPELINE` | `--pipeline` | `str` | 要运行的产线。 | `"OCR"`，`"PP-StructureV3"` | `"OCR"` |
|
||||
| `PADDLEOCR_MCP_PPOCR_SOURCE` | `--ppocr_source` | `str` | PaddleOCR 能力来源。 | `"local"`（本地 Python 库），`"aistudio"`（星河社区服务），`"self_hosted"`（自托管服务） | `"local"` |
|
||||
| `PADDLEOCR_MCP_SERVER_URL` | `--server_url` | `str` | 底层服务基础 URL（`aistudio` 或 `self_hosted` 模式下必需）。 | - | `None` |
|
||||
| `PADDLEOCR_MCP_AISTUDIO_ACCESS_TOKEN` | `--aistudio_access_token` | `str` | AI Studio 访问令牌（`aistudio` 模式下必需）。 | - | `None` |
|
||||
| `PADDLEOCR_MCP_TIMEOUT` | `--timeout` | `int` | 底层服务请求的读取超时时间（秒）。 | - | `60` |
|
||||
| `PADDLEOCR_MCP_DEVICE` | `--device` | `str` | 指定运行推理的设备（仅在 `local` 模式下生效）。 | - | `None` |
|
||||
| `PADDLEOCR_MCP_PIPELINE_CONFIG` | `--pipeline_config` | `str` | PaddleOCR 产线配置文件路径（仅在 `local` 模式下生效）。 | - | `None` |
|
||||
| - | `--http` | `bool` | 使用 Streamable HTTP 传输而非 stdio（适用于远程部署和多客户端）。 | - | `False` |
|
||||
| - | `--host` | `str` | Streamable HTTP 模式的主机地址。 | - | `"127.0.0.1"` |
|
||||
| - | `--port` | `int` | Streamable HTTP 模式的端口。 | - | `8000` |
|
||||
| - | `--verbose` | `bool` | 启用详细日志记录，便于调试。 | - | `False` |
|
||||
|
||||
## 5. 已知局限性
|
||||
|
||||
- 在本地 Python 库模式下，当前提供的工具无法处理 Base64 编码的 PDF 文档输入。
|
||||
- 在本地 Python 库模式下，当前提供的工具不会根据模型提示的 `file_type` 推断文件类型，对于一些复杂 URL 可能处理失败。
|
||||
- 对于 PP-StructureV3 产线，若输入文件中包含图像，返回结果可能会显著增加 token 使用量。若无需图像内容，可通过提示词明确排除，以降低资源消耗。
|
||||
51
docs/version3.x/deployment/obtaining_onnx_models.en.md
Normal file
51
docs/version3.x/deployment/obtaining_onnx_models.en.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# Obtaining ONNX Models
|
||||
|
||||
PaddleOCR provides a rich collection of pre-trained models, all stored in PaddlePaddle's static graph format. To use these models in ONNX format during deployment, you can convert them using the Paddle2ONNX plugin provided by PaddleX. For more information about PaddleX and its relationship with PaddleOCR, refer to [Differences and Connections Between PaddleOCR and PaddleX](../paddleocr_and_paddlex.en.md#1-Differences-and-Connections-Between-PaddleOCR-and-PaddleX).
|
||||
|
||||
First, install the Paddle2ONNX plugin for PaddleX using the following command via the PaddleX CLI:
|
||||
|
||||
```bash
|
||||
# Windows users need to use the following command to install the dev version of paddlepaddle
|
||||
# python -m pip install --pre paddlepaddle -i https://www.paddlepaddle.org.cn/packages/nightly/cpu/
|
||||
|
||||
paddlex --install paddle2onnx
|
||||
```
|
||||
|
||||
Then, execute the following command to complete the model conversion:
|
||||
|
||||
```bash
|
||||
paddlex \
|
||||
--paddle2onnx \ # Use the paddle2onnx feature
|
||||
--paddle_model_dir /your/paddle_model/dir \ # Specify the directory containing the Paddle model
|
||||
--onnx_model_dir /your/onnx_model/output/dir \ # Specify the output directory for the converted ONNX model
|
||||
--opset_version 7 # Specify the ONNX opset version to use
|
||||
```
|
||||
|
||||
The parameters are described as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>paddle_model_dir</td>
|
||||
<td>str</td>
|
||||
<td>The directory containing the Paddle model.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnx_model_dir</td>
|
||||
<td>str</td>
|
||||
<td>The output directory for the ONNX model. It can be the same as the Paddle model directory. Defaults to <code>onnx</code>.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>opset_version</td>
|
||||
<td>int</td>
|
||||
<td>The ONNX opset version to use. If conversion fails with a lower opset version, a higher version will be automatically selected for conversion. Defaults to <code>7</code>.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
51
docs/version3.x/deployment/obtaining_onnx_models.md
Normal file
51
docs/version3.x/deployment/obtaining_onnx_models.md
Normal file
@@ -0,0 +1,51 @@
|
||||
# 获取 ONNX 模型
|
||||
|
||||
PaddleOCR 提供了丰富的预训练模型，这些模型均采用飞桨的静态图格式进行存储。若需在部署阶段使用 ONNX 格式的模型，可借助 PaddleX 提供的 Paddle2ONNX 插件进行转换。关于 PaddleX 及其与 PaddleOCR 之间的关系，请参考 [PaddleOCR 与 PaddleX 的区别与联系](../paddleocr_and_paddlex.md#1-paddleocr-与-paddlex-的区别与联系)。
|
||||
|
||||
首先，执行如下命令，通过 PaddleX CLI 安装 PaddleX 的 Paddle2ONNX 插件：
|
||||
|
||||
```bash
|
||||
# Windows 用户需使用以下命令安装 paddlepaddle dev版本
|
||||
# python -m pip install --pre paddlepaddle -i https://www.paddlepaddle.org.cn/packages/nightly/cpu/
|
||||
|
||||
paddlex --install paddle2onnx
|
||||
```
|
||||
|
||||
然后，执行如下命令完成模型转换：
|
||||
|
||||
```bash
|
||||
paddlex \
|
||||
--paddle2onnx \ # 使用paddle2onnx功能
|
||||
--paddle_model_dir /your/paddle_model/dir \ # 指定 Paddle 模型所在的目录
|
||||
--onnx_model_dir /your/onnx_model/output/dir \ # 指定转换后 ONNX 模型的输出目录
|
||||
--opset_version 7 # 指定要使用的 ONNX opset 版本
|
||||
```
|
||||
|
||||
参数说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>类型</th>
|
||||
<th>描述</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>paddle_model_dir</td>
|
||||
<td>str</td>
|
||||
<td>包含 Paddle 模型的目录。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>onnx_model_dir</td>
|
||||
<td>str</td>
|
||||
<td>ONNX 模型的输出目录，可以与 Paddle 模型目录相同。默认为 <code>onnx</code>。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>opset_version</td>
|
||||
<td>int</td>
|
||||
<td>使用的 ONNX opset 版本。当使用低版本 opset 无法完成转换时，将自动选择更高版本的 opset 进行转换。默认为 <code>7</code>。</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
475
docs/version3.x/deployment/on_device_deployment.en.md
Normal file
475
docs/version3.x/deployment/on_device_deployment.en.md
Normal file
@@ -0,0 +1,475 @@
|
||||
# OCR On-Device Deployment Demo Usage Guide
|
||||
|
||||
- [Quick Start](#quick-start)
|
||||
- [Environment Preparation](#environment-preparation)
|
||||
- [Deployment Steps](#deployment-steps)
|
||||
- [Code Introduction](#code-introduction)
|
||||
- [Project Explanation](#project-explanation)
|
||||
- [Advanced Usage](#advanced-usage)
|
||||
- [Update Prediction Library](#update-prediction-library)
|
||||
- [Convert NB Model](#convert-nb-model)
|
||||
- [Update Model, Label File, and Prediction Image](#update-model-label-file-and-prediction-image)
|
||||
- [Update Model](#update-model)
|
||||
- [Update Label File](#update-label-file)
|
||||
- [Update Prediction Image](#update-prediction-image)
|
||||
- [Update Input/Output Preprocessing](#update-inputoutput-preprocessing)
|
||||
|
||||
This guide mainly introduces how to run the PaddleX on-device deployment demo for OCR text recognition on an Android shell.
|
||||
|
||||
The following OCR models are supported in this guide:
|
||||
|
||||
- PP-OCRv3_mobile (cpu)
|
||||
- PP-OCRv4_mobile (cpu)
|
||||
- PP-OCRv5_mobile (cpu)
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Environment Preparation
|
||||
|
||||
1. Install the CMAKE compilation tool in your local environment and download an NDK package for your current system from the [Android NDK official website](https://developer.android.google.cn/ndk/downloads). For example, if developing on a Mac, download the NDK package for the Mac platform from the Android NDK official website.
|
||||
|
||||
**Environment Requirements**
|
||||
|
||||
- `CMake >= 3.10` (the minimum version has not been verified; 3.20 or above is recommended)
|
||||
- `Android NDK >= r17c` (the minimum version has not been verified; r20b or above is recommended)
|
||||
|
||||
**Test Environment Used in This Guide**:
|
||||
|
||||
- `cmake == 3.20.0`
|
||||
- `android-ndk == r20b`
|
||||
|
||||
2. Prepare an Android phone and enable USB debugging mode. Method: `Phone Settings -> Find Developer Options -> Enable Developer Options and USB Debugging Mode`
|
||||
|
||||
3. Install the ADB tool on your computer for debugging. The installation methods for ADB are as follows:
|
||||
|
||||
3.1. Install ADB on a Mac:
|
||||
|
||||
```shell
|
||||
brew cask install android-platform-tools
|
||||
```
|
||||
|
||||
3.2. Install ADB on Linux:
|
||||
|
||||
```shell
|
||||
sudo apt update
|
||||
sudo apt install -y wget adb
|
||||
```
|
||||
|
||||
3.3. Install ADB on Windows:
|
||||
|
||||
For installation on Windows, download and install the ADB software package from Google's Android platform: [Link](https://developer.android.com/studio)
|
||||
|
||||
Open a terminal, connect your phone to the computer, and enter the following command in the terminal:
|
||||
|
||||
```shell
|
||||
adb devices
|
||||
```
|
||||
|
||||
If there is a `device` output, the installation is successful.
|
||||
|
||||
```shell
|
||||
List of devices attached
|
||||
744be294 device
|
||||
```
|
||||
|
||||
### Material Preparation
|
||||
|
||||
1. Clone the `feature/paddle-x` branch of the `Paddle-Lite-Demo` repository to the `PaddleX-Lite-Deploy` directory.
|
||||
|
||||
```shell
|
||||
git clone -b feature/paddle-x https://github.com/PaddlePaddle/Paddle-Lite-Demo.git PaddleX-Lite-Deploy
|
||||
```
|
||||
|
||||
2. Fill out the [questionnaire](https://paddle.wjx.cn/vm/eaaBo0H.aspx#) to download the compressed package. Place the compressed package in the specified extraction directory, switch to the specified extraction directory, and execute the extraction command.
|
||||
|
||||
```shell
|
||||
# 1. Switch to the specified extraction directory
|
||||
cd PaddleX-Lite-Deploy/ocr/android/shell/ppocr_demo
|
||||
|
||||
# 2. Execute the extraction command
|
||||
unzip ocr.zip
|
||||
```
|
||||
|
||||
### Deployment Steps
|
||||
|
||||
1. Switch the working directory to `PaddleX-Lite-Deploy/libs` and run the `download.sh` script to download the required Paddle Lite prediction library. This step only needs to be executed once to support each demo.
|
||||
|
||||
2. Switch the working directory to `PaddleX-Lite-Deploy/ocr/assets` and run the `download.sh` script to download the [paddle_lite_opt tool](https://www.paddlepaddle.org.cn/lite/v2.10/user_guides/model_optimize_tool.html)-optimized NB model file, prediction images, dictionary files, and other materials.
|
||||
|
||||
3. Switch the working directory to `PaddleX-Lite-Deploy/ocr/android/shell/cxx/ppocr_demo` and run the `build.sh` script to complete the compilation of the executable file.
|
||||
|
||||
4. Switch the working directory to `PaddleX-Lite-Deploy/ocr/android/shell/cxx/ppocr_demo` and run the `run.sh` script to complete the on-device prediction.
|
||||
|
||||
**Notes**:
|
||||
|
||||
- Before running the `build.sh` script, change the path specified by `NDK_ROOT` to the actual installation path of NDK.
|
||||
- On Windows systems, you can use Git Bash to execute the deployment steps.
|
||||
- If compiling on a Windows system, set `CMAKE_SYSTEM_NAME` to `windows` in `CMakeLists.txt`.
|
||||
- If compiling on a Mac system, set `CMAKE_SYSTEM_NAME` to `darwin` in `CMakeLists.txt`.
|
||||
- Keep the ADB connection active when running the `run.sh` script.
|
||||
- The `download.sh` and `run.sh` scripts support passing parameters to specify the model. If no model is specified, the `PP-OCRv5_mobile` model is used by default. The following models are currently supported:
|
||||
- `PP-OCRv3_mobile`
|
||||
- `PP-OCRv4_mobile`
|
||||
- `PP-OCRv5_mobile`
|
||||
|
||||
Here is an example of the actual operation:
|
||||
|
||||
```shell
|
||||
# 1. Download the required Paddle Lite prediction library
|
||||
cd PaddleX-Lite-Deploy/libs
|
||||
sh download.sh
|
||||
|
||||
# 2. Download the paddle_lite_opt tool-optimized NB model file, prediction images, dictionary files, and other materials
|
||||
cd ../ocr/assets
|
||||
sh download.sh PP-OCRv5_mobile
|
||||
|
||||
# 3. Complete the compilation of the executable file
|
||||
cd ../android/shell/ppocr_demo
|
||||
sh build.sh
|
||||
|
||||
# 4. Prediction
|
||||
sh run.sh PP-OCRv5_mobile
|
||||
```
|
||||
|
||||
The output is as follows:
|
||||
|
||||
```text
|
||||
The detection visualized image saved in ./test_img_result.jpg
|
||||
0 纯臻营养护发素 0.998541
|
||||
1 产品信息/参数 0.999094
|
||||
2 (45元/每公斤，100公斤起订） 0.948841
|
||||
3 每瓶22元，1000瓶起订) 0.961245
|
||||
4 【品牌】：代加工方式/OEMODM 0.970401
|
||||
5 【品名】：纯臻营养护发素 0.977496
|
||||
6 ODMOEM 0.955396
|
||||
7 【产品编号】：YM-X-3011 0.977864
|
||||
8 【净含量】：220ml 0.970538
|
||||
9 【适用人群】：适合所有肤质 0.995907
|
||||
10 【主要成分】：鲸蜡硬脂醇、燕麦β-葡聚 0.975813
|
||||
11 糖、椰油酰胺丙基甜菜碱、泛醌 0.964397
|
||||
12 （成品包材） 0.97298
|
||||
13 【主要功能】：可紧致头发磷层，从而达到 0.989097
|
||||
14 即时持久改善头发光泽的效果，给干燥的头 0.990088
|
||||
15 发足够的滋养 0.998037
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Code Introduction
|
||||
|
||||
```
|
||||
.
|
||||
├── ...
|
||||
├── ocr
|
||||
│ ├── ...
|
||||
│ ├── android
|
||||
│ │ ├── ...
|
||||
│ │ └── shell
|
||||
│ │ └── ppocr_demo
|
||||
│ │ ├── src # Contains prediction code
|
||||
│ │ | ├── cls_process.cc # Full inference process for orientation classifier, including preprocessing, prediction, and postprocessing
|
||||
│ │ | ├── rec_process.cc # Full inference process for recognition model CRNN, including preprocessing, prediction, and postprocessing
|
||||
│ │ | ├── det_process.cc # Full inference process for detection model CRNN, including preprocessing, prediction, and postprocessing
|
||||
│ │ | ├── det_post_process.cc # Postprocessing file for detection model DB
|
||||
│ │ | ├── pipeline.cc # Full inference process code for OCR text recognition demo
|
||||
│ │ | └── MakeFile # MakeFile file for prediction code
|
||||
│ │ |
|
||||
│ │ ├── CMakeLists.txt # CMake file that defines the compilation method for the executable
|
||||
│ │ ├── README.md
|
||||
│ │ ├── build.sh # Used for compiling the executable
|
||||
│ │ └── run.sh # Used for prediction
|
||||
│ └── assets # Stores models, test images, label files, and config files
|
||||
│ ├── images # Stores test images
|
||||
│ ├── labels # Stores dictionary files (see remarks below for details)
|
||||
│ ├── models # Stores nb models
|
||||
│ ├── config.txt
|
||||
│ └── download.sh # Download script for paddle_lite_opt tool-optimized models
|
||||
└── libs # Stores prediction libraries and OpenCV libraries for different platforms.
|
||||
├── ...
|
||||
└── download.sh # Download script for Paddle Lite prediction libraries and OpenCV libraries
|
||||
```
|
||||
|
||||
**Remarks**:
|
||||
|
||||
- The `PaddleX-Lite-Deploy/ocr/assets/labels/` directory contains the dictionary files `ppocr_keys_v1.txt` for PP-OCRv3 and PP-OCRv4 models, and `ppocr_keys_ocrv5.txt` for the PP-OCRv5 model. The appropriate dictionary file is automatically selected during inference based on the model name, so no manual intervention is required.
|
||||
- If you are using an English/numeric or other language model, you need to replace it with the corresponding language dictionary. The PaddleOCR repository provides [some dictionary files](https://github.com/PaddlePaddle/PaddleOCR/tree/release/2.3/ppocr/utils).
|
||||
|
||||
```shell
|
||||
# Parameters of the executable in run.sh script:
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"./models/${MODEL_NAME}_rec.nb\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/test.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/${LABEL_FILE} \
|
||||
./config.txt"
|
||||
|
||||
First parameter: ppocr_demo executable
|
||||
Second parameter: ./models/${MODEL_NAME}_det.nb Detection model .nb file
|
||||
Third parameter: ./models/${MODEL_NAME}_rec.nb Recognition model .nb file
|
||||
Fourth parameter: ./models/${CLS_MODEL_FILE} Text line orientation classification model .nb file (automatically selected based on model name by default)
|
||||
Fifth parameter: ./images/test.jpg Test image
|
||||
Sixth parameter: ./test_img_result.jpg Result save file
|
||||
Seventh parameter: ./labels/${LABEL_FILE} Label file (automatically selected based on model name by default)
|
||||
Eighth parameter: ./config.txt Configuration file containing hyperparameters for the detection and classification models
|
||||
```
|
||||
|
||||
```shell
|
||||
# List of Specific Parameters in config.txt:
|
||||
max_side_len 960 # When the width or height of the input image is greater than 960, the image is scaled proportionally so that the longest side of the image is 960.
|
||||
det_db_thresh 0.3 # Used to filter the binarized images predicted by DB; setting it to 0.3 has no significant impact on the results.
|
||||
det_db_box_thresh 0.5 # Threshold for filtering boxes in the DB post-processing; if there are missing boxes in detection, you may reduce this value.
|
||||
det_db_unclip_ratio 1.6 # Represents the compactness of the text box; the smaller the value, the closer the text box is to the text.
|
||||
use_direction_classify 0 # Whether to use a direction classifier: 0 means not using it, 1 means using it.
|
||||
```
|
||||
|
||||
## Engineering Details
|
||||
|
||||
The OCR text recognition demo accomplishes the OCR text recognition function using three models collaboratively. First, the input image undergoes detection processing via the `${MODEL_NAME}_det.nb` model, followed by text direction classification using the `ch_ppocr_mobile_v2.0_cls_slim_opt.nb` model, and finally, text recognition with the `${MODEL_NAME}_rec.nb` model.
|
||||
|
||||
1. `pipeline.cc`: Full-process prediction code for the OCR text recognition demo
|
||||
This file handles the entire process control for serial inference of the three models, including scheduling for the entire processing flow.
|
||||
|
||||
- The `Pipeline::Pipeline(...)` method initializes the three model class constructors, accomplishes model loading, thread count, core binding, and predictor creation.
|
||||
- The `Pipeline::Process(...)` method manages the entire process control for serial inference of the three models.
|
||||
|
||||
2. `cls_process.cc`: Prediction file for the direction classifier
|
||||
This file handles the preprocessing, prediction, and postprocessing for the direction classifier.
|
||||
|
||||
- The `ClsPredictor::ClsPredictor()` method initializes model loading, thread count, core binding, and predictor creation.
|
||||
- The `ClsPredictor::Preprocess()` method handles model preprocessing.
|
||||
- The `ClsPredictor::Postprocess()` method handles model postprocessing.
|
||||
|
||||
3. `rec_process.cc`: Prediction file for the CRNN recognition model
|
||||
This file handles the preprocessing, prediction, and postprocessing for the CRNN recognition model.
|
||||
|
||||
- The `RecPredictor::RecPredictor()` method initializes model loading, thread count, core binding, and predictor creation.
|
||||
- The `RecPredictor::Preprocess()` method handles model preprocessing.
|
||||
- The `RecPredictor::Postprocess()` method handles model postprocessing.
|
||||
|
||||
4. `det_process.cc`: Prediction file for the DB detection model
|
||||
This file handles the preprocessing, prediction, and postprocessing for the DB detection model.
|
||||
|
||||
- The `DetPredictor::DetPredictor()` method initializes model loading, thread count, core binding, and predictor creation.
|
||||
- The `DetPredictor::Preprocess()` method handles model preprocessing.
|
||||
- The `DetPredictor::Postprocess()` method handles model postprocessing.
|
||||
|
||||
5. `db_post_process`: Postprocessing functions for the DB detection model, including calls to the clipper library
|
||||
This file implements third-party library calls and other postprocessing methods for the DB detection model.
|
||||
|
||||
- The `std::vector<std::vector<std::vector<int>>> BoxesFromBitmap(...)` method retrieves detection boxes from a Bitmap.
|
||||
- The `std::vector<std::vector<std::vector<int>>> FilterTagDetRes(...)` method retrieves target box positions based on recognition results.
|
||||
|
||||
## Advanced Usage
|
||||
|
||||
If the quick start section does not meet your needs, refer to this section for custom modifications to the demo.
|
||||
|
||||
This section mainly includes four parts:
|
||||
|
||||
- Updating the prediction library;
|
||||
- Converting `.nb` models;
|
||||
- Updating models, label files, and prediction images;
|
||||
- Updating input/output preprocessing.
|
||||
|
||||
### Updating the Prediction Library
|
||||
|
||||
The prediction library used in this guide is the latest version (214rc), and manual updates are not recommended.
|
||||
|
||||
If you need to use a different version, follow these steps to update the prediction library:
|
||||
|
||||
* Paddle Lite project: https://github.com/PaddlePaddle/Paddle-Lite
|
||||
* Refer to the [Paddle Lite Source Code Compilation Documentation](https://www.paddlepaddle.org.cn/lite/develop/source_compile/compile_env.html) to compile the Android prediction library.
|
||||
* The final compilation output is located in `build.lite.xxx.xxx.xxx` under `inference_lite_lib.xxx.xxx`.
|
||||
* Replace the C++ library:
|
||||
* Header files:
|
||||
Replace the `PaddleX-Lite-Deploy/libs/android/cxx/include` folder in the demo with the generated `build.lite.android.xxx.gcc/inference_lite_lib.android.xxx/cxx/include` folder.
|
||||
* armeabi-v7a:
|
||||
Replace the `PaddleX-Lite-Deploy/libs/android/cxx/libs/armeabi-v7a/libpaddle_lite_api_shared.so` library in the demo with the generated `build.lite.android.armv7.gcc/inference_lite_lib.android.armv7/cxx/libs/libpaddle_lite_api_shared.so` library.
|
||||
* arm64-v8a:
|
||||
Replace the `PaddleX-Lite-Deploy/libs/android/cxx/libs/arm64-v8a/libpaddle_lite_api_shared.so` library in the demo with the generated `build.lite.android.armv8.gcc/inference_lite_lib.android.armv8/cxx/libs/libpaddle_lite_api_shared.so` library.
|
||||
|
||||
### Converting .nb Models
|
||||
|
||||
If you want to use your own trained models, follow the process below to obtain `.nb` models.
|
||||
|
||||
#### Terminal Command Method (Supports Mac/Ubuntu)
|
||||
|
||||
1. Navigate to the [release interface](https://github.com/PaddlePaddle/Paddle-Lite/releases) of the Paddle-Lite GitHub repository and download the corresponding conversion tool, opt, for the desired version (the latest version is recommended).
|
||||
|
||||
2. After downloading the opt tool, execute the following command (using the 2.14rc version of the linux_x86 opt tool to convert the PP-OCRv5_mobile_det model as an example):
|
||||
|
||||
```bash
|
||||
./opt_linux_x86 \
|
||||
--model_file=PP-OCRv5_mobile_det/inference.pdmodel \
|
||||
--param_file=PP-OCRv5_mobile_det/inference.pdiparams \
|
||||
--optimize_out=PP-OCRv5_mobile_det \
|
||||
--valid_targets=arm
|
||||
```
|
||||
|
||||
For detailed instructions on converting `.nb` models using the terminal command method, refer to the [Using the Executable opt](https://www.paddlepaddle.org.cn/lite/v2.12/user_guides/opt/opt_bin.html) section in the Paddle-Lite repository.
|
||||
|
||||
#### Python Script Method (Supports Windows/Mac/Ubuntu)
|
||||
|
||||
1. Install the latest version of the paddlelite wheel package.
|
||||
|
||||
```bash
|
||||
pip install --pre paddlelite
|
||||
```
|
||||
|
||||
2. Use the Python script to convert the model. Below is an example code snippet for converting the PP-OCRv5_mobile_det model:
|
||||
|
||||
```python
|
||||
from paddlelite.lite import Opt
|
||||
|
||||
# 1. Create an Opt instance
|
||||
opt = Opt()
|
||||
# 2. Specify the input model paths
|
||||
opt.set_model_file("./PP-OCRv5_mobile_det/inference.pdmodel")
|
||||
opt.set_param_file("./PP-OCRv5_mobile_det/inference.pdiparams")
|
||||
# 3. Specify the target platform for optimization
|
||||
opt.set_valid_places("arm")
|
||||
# 4. Specify the output path for the optimized model
|
||||
opt.set_optimize_out("./PP-OCRv5_mobile_det")
|
||||
# 5. Execute model optimization
|
||||
opt.run()
|
||||
```
|
||||
|
||||
For detailed instructions on converting `.nb` models using the Python script method, refer to the [Python Script opt Usage](https://www.paddlepaddle.org.cn/lite/v2.12/api_reference/python_api/opt.html) section in the Paddle-Lite repository.
|
||||
|
||||
**Notes**
|
||||
|
||||
- For detailed information about the model optimization tool `opt`, refer to Paddle-Lite's [Model Optimization Tool opt](https://www.paddlepaddle.org.cn/lite/v2.12/user_guides/model_optimize_tool.html).
|
||||
- Currently, only static graph models in `.pdmodel` format can be converted to `.nb` format.
|
||||
|
||||
### Updating Models, Label Files, and Prediction Images
|
||||
|
||||
#### Updating Models
|
||||
|
||||
This guide has only validated the `PP-OCRv3_mobile`, `PP-OCRv4_mobile`, and `PP-OCRv5_mobile` models. Other models may not be compatible.
|
||||
|
||||
If you fine-tune the `PP-OCRv5_mobile` model and generate a new model named `PP-OCRv5_mobile_ft`, follow these steps to replace the original model with your fine-tuned model:
|
||||
|
||||
1. Place the `.nb` models of `PP-OCRv5_mobile_ft` into the directory `PaddleX-Lite-Deploy/ocr/assets/models/`. The resulting file structure should be:
|
||||
|
||||
```text
|
||||
.
|
||||
├── ocr
|
||||
│ ├── ...
|
||||
│ └── assets
|
||||
│ ├── models
|
||||
│ │ ├── ...
|
||||
│ │ ├── PP-OCRv5_mobile_ft_det.nb
|
||||
│ │ └── PP-OCRv5_mobile_ft_rec.nb
|
||||
│ └── ...
|
||||
└── ...
|
||||
```
|
||||
|
||||
2. Add the model name to the `MODEL_LIST` in the `run.sh` script.
|
||||
|
||||
```shell
|
||||
MODEL_LIST="PP-OCRv3_mobile PP-OCRv4_mobile PP-OCRv5_mobile PP-OCRv5_mobile_ft" # Models are separated by spaces
|
||||
```
|
||||
|
||||
3. Specify the model directory name when running the `run.sh` script.
|
||||
|
||||
```shell
|
||||
sh run.sh PP-OCRv5_mobile_ft
|
||||
```
|
||||
|
||||
**Notes**:
|
||||
|
||||
- If the input Tensor, Shape, or Dtype of the model is updated:
|
||||
|
||||
- For the text direction classifier model, update the `ClsPredictor::Preprocess` function in `ppocr_demo/src/cls_process.cc`.
|
||||
- For the detection model, update the `DetPredictor::Preprocess` function in `ppocr_demo/src/det_process.cc`.
|
||||
- For the recognition model, update the `RecPredictor::Preprocess` function in `ppocr_demo/src/rec_process.cc`.
|
||||
|
||||
- If the output Tensor or Dtype of the model is updated:
|
||||
|
||||
- For the text direction classifier model, update the `ClsPredictor::Postprocess` function in `ppocr_demo/src/cls_process.cc`.
|
||||
- For the detection model, update the `DetPredictor::Postprocess` function in `ppocr_demo/src/det_process.cc`.
|
||||
- For the recognition model, update the `RecPredictor::Postprocess` function in `ppocr_demo/src/rec_process.cc`.
|
||||
|
||||
#### Updating Label Files
|
||||
|
||||
To update the label file, place the new label file in the directory `PaddleX-Lite-Deploy/ocr/assets/labels/` and update the execution command in `PaddleX-Lite-Deploy/ocr/android/shell/ppocr_demo/run.sh` following the model update method.
|
||||
|
||||
For example, to update to `new_labels.txt`:
|
||||
|
||||
```shell
|
||||
# File: `PaddleX-Lite-Deploy/ocr/android/shell/ppocr_demo/run.sh`
|
||||
# Original command
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"./models/${MODEL_NAME}_rec.nb\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/test.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/${LABEL_FILE} \
|
||||
./config.txt"
|
||||
# Updated command
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"./models/${MODEL_NAME}_rec.nb\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/test.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/new_labels.txt \
|
||||
./config.txt"
|
||||
```
|
||||
|
||||
#### Updating Prediction Images
|
||||
|
||||
If you need to update the prediction images, place the updated images in the `PaddleX-Lite-Deploy/ocr/assets/images/` directory and update the execution command in the `PaddleX-Lite-Deploy/ocr/android/shell/ppocr_demo/run.sh` file.
|
||||
|
||||
Here is an example of updating to `new_pics.jpg`:
|
||||
|
||||
```shell
|
||||
# File: `PaddleX-Lite-Deploy/ocr/android/shell/ppocr_demo/run.sh`
|
||||
## Original command
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"./models/${MODEL_NAME}_rec.nb\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/test.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/${LABEL_FILE} \
|
||||
./config.txt"
|
||||
# Updated command
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"./models/${MODEL_NAME}_rec.nb\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/new_pics.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/${LABEL_FILE} \
|
||||
./config.txt"
|
||||
```
|
||||
|
||||
### Updating Input/Output Preprocessing
|
||||
|
||||
- Updating Input Preprocessing
|
||||
- For the text direction classifier model, update the `ClsPredictor::Preprocess` function in `ppocr_demo/src/cls_process.cc`.
|
||||
- For the detection model, update the `DetPredictor::Preprocess` function in `ppocr_demo/src/det_process.cc`.
|
||||
- For the recognition model, update the `RecPredictor::Preprocess` function in `ppocr_demo/src/rec_process.cc`.
|
||||
|
||||
- Updating Output Preprocessing
|
||||
- For the text direction classifier model, update the `ClsPredictor::Postprocess` function in `ppocr_demo/src/cls_process.cc`.
|
||||
- For the detection model, update the `DetPredictor::Postprocess` function in `ppocr_demo/src/det_process.cc`.
|
||||
- For the recognition model, update the `RecPredictor::Postprocess` function in `ppocr_demo/src/rec_process.cc`.
|
||||
475
docs/version3.x/deployment/on_device_deployment.md
Normal file
475
docs/version3.x/deployment/on_device_deployment.md
Normal file
@@ -0,0 +1,475 @@
|
||||
# OCR 端侧部署 demo 使用指南
|
||||
|
||||
- [快速开始](#快速开始)
|
||||
- [环境准备](#环境准备)
|
||||
- [部署步骤](#部署步骤)
|
||||
- [代码介绍](#代码介绍)
|
||||
- [工程详解](#工程详解)
|
||||
- [进阶使用](#进阶使用)
|
||||
- [更新预测库](#更新预测库)
|
||||
- [转换 nb 模型](#转换-nb-模型)
|
||||
- [更新模型、标签文件和预测图片](#更新模型标签文件和预测图片)
|
||||
- [更新模型](#更新模型)
|
||||
- [更新标签文件](#更新标签文件)
|
||||
- [更新预测图片](#更新预测图片)
|
||||
- [更新输入/输出预处理](#更新输入输出预处理)
|
||||
|
||||
本指南主要介绍 PaddleX 端侧部署——OCR文字识别 demo 在 Android shell 上的运行方法。
|
||||
|
||||
本指南适配了以下 OCR 模型：
|
||||
|
||||
- PP-OCRv3_mobile（cpu）
|
||||
- PP-OCRv4_mobile（cpu）
|
||||
- PP-OCRv5_mobile（cpu）
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 环境准备
|
||||
|
||||
1. 在本地环境安装好 CMAKE 编译工具，并在 [Android NDK 官网](https://developer.android.google.cn/ndk/downloads)下载当前系统的某个版本的 NDK 软件包。例如，在 Mac 上开发，需要在 Android NDK 官网下载 Mac 平台的 NDK 软件包
|
||||
|
||||
**环境要求**
|
||||
|
||||
- `CMake >= 3.10`（最低版本未经验证，推荐 3.20 及以上）
|
||||
- `Android NDK >= r17c`（最低版本未经验证，推荐 r20b 及以上）
|
||||
|
||||
**本指南所使用的测试环境：**
|
||||
|
||||
- `cmake == 3.20.0`
|
||||
- `android-ndk == r20b`
|
||||
|
||||
2. 准备一部 Android 手机，并开启 USB 调试模式。开启方法: `手机设置 -> 查找开发者选项 -> 打开开发者选项和 USB 调试模式`
|
||||
|
||||
3. 电脑上安装 ADB 工具，用于调试。ADB 安装方式如下：
|
||||
|
||||
3.1. Mac 电脑安装 ADB:
|
||||
|
||||
```shell
|
||||
brew cask install android-platform-tools
|
||||
```
|
||||
|
||||
3.2. Linux 安装 ADB
|
||||
|
||||
```shell
|
||||
sudo apt update
|
||||
sudo apt install -y wget adb
|
||||
```
|
||||
|
||||
3.3. Windows 安装 ADB
|
||||
|
||||
win 上安装需要去谷歌的安卓平台下载 ADB 软件包进行安装：[链接](https://developer.android.com/studio)
|
||||
|
||||
打开终端，手机连接电脑，在终端中输入
|
||||
|
||||
```shell
|
||||
adb devices
|
||||
```
|
||||
|
||||
如果有 device 输出，则表示安装成功。
|
||||
|
||||
```shell
|
||||
List of devices attached
|
||||
744be294 device
|
||||
```
|
||||
|
||||
### 物料准备
|
||||
|
||||
1. 克隆 `Paddle-Lite-Demo` 仓库的 `feature/paddle-x` 分支到 `PaddleX-Lite-Deploy` 目录。
|
||||
|
||||
```shell
|
||||
git clone -b feature/paddle-x https://github.com/PaddlePaddle/Paddle-Lite-Demo.git PaddleX-Lite-Deploy
|
||||
```
|
||||
|
||||
2. 填写 [问卷](https://paddle.wjx.cn/vm/eaaBo0H.aspx#) 下载压缩包，将压缩包放到指定解压目录，切换到指定解压目录后执行解压命令。
|
||||
|
||||
```shell
|
||||
# 1. 切换到指定解压目录
|
||||
cd PaddleX-Lite-Deploy/ocr/android/shell/ppocr_demo
|
||||
|
||||
# 2. 执行解压命令
|
||||
unzip ocr.zip
|
||||
```
|
||||
|
||||
### 部署步骤
|
||||
|
||||
1. 将工作目录切换到 `PaddleX-Lite-Deploy/libs` 目录，运行 `download.sh` 脚本，下载需要的 Paddle Lite 预测库。此步骤只需执行一次，即可支持每个 demo 使用。
|
||||
|
||||
2. 将工作目录切换到 `PaddleX-Lite-Deploy/ocr/assets` 目录，运行 `download.sh` 脚本，下载 [paddle_lite_opt 工具](https://www.paddlepaddle.org.cn/lite/v2.10/user_guides/model_optimize_tool.html) 优化后的 nb 模型文件及预测图片、字典文件等物料。
|
||||
|
||||
3. 将工作目录切换到 `PaddleX-Lite-Deploy/ocr/android/shell/cxx/ppocr_demo` 目录，运行 `build.sh` 脚本，完成可执行文件的编译。
|
||||
|
||||
4. 将工作目录切换到 `PaddleX-Lite-Deploy/ocr/android/shell/cxx/ppocr_demo`，运行 `run.sh` 脚本，完成在端侧的预测。
|
||||
|
||||
**注意事项：**
|
||||
|
||||
- 在运行 `build.sh` 脚本前，需要更改 `NDK_ROOT` 指定的路径为实际安装的 NDK 路径。
|
||||
- 在 Windows 系统上可以使用 Git Bash 执行部署步骤。
|
||||
- 若在 Windows 系统上编译，需要将 `CMakeLists.txt` 中的 `CMAKE_SYSTEM_NAME` 设置为 `windows`。
|
||||
- 若在 Mac 系统上编译，需要将 `CMakeLists.txt` 中的 `CMAKE_SYSTEM_NAME` 设置为 `darwin`。
|
||||
- 在运行 `run.sh` 脚本时需保持 ADB 连接。
|
||||
- `download.sh` 和 `run.sh` 支持传入参数来指定模型，若不指定则默认使用 `PP-OCRv5_mobile` 模型。目前适配了以下模型：
|
||||
- `PP-OCRv3_mobile`
|
||||
- `PP-OCRv4_mobile`
|
||||
- `PP-OCRv5_mobile`
|
||||
|
||||
以下为实际操作时的示例：
|
||||
|
||||
```shell
|
||||
# 1. 下载需要的 Paddle Lite 预测库
|
||||
cd PaddleX-Lite-Deploy/libs
|
||||
sh download.sh
|
||||
|
||||
# 2. 下载 paddle_lite_opt 工具优化后的 nb 模型文件及预测图片、字典文件等物料
|
||||
cd ../ocr/assets
|
||||
sh download.sh PP-OCRv5_mobile
|
||||
|
||||
# 3. 完成可执行文件的编译
|
||||
cd ../android/shell/ppocr_demo
|
||||
sh build.sh
|
||||
|
||||
# 4. 预测
|
||||
sh run.sh PP-OCRv5_mobile
|
||||
```
|
||||
|
||||
运行结果如下所示：
|
||||
|
||||
```text
|
||||
The detection visualized image saved in ./test_img_result.jpg
|
||||
0 纯臻营养护发素 0.998541
|
||||
1 产品信息/参数 0.999094
|
||||
2 (45元/每公斤，100公斤起订） 0.948841
|
||||
3 每瓶22元，1000瓶起订) 0.961245
|
||||
4 【品牌】：代加工方式/OEMODM 0.970401
|
||||
5 【品名】：纯臻营养护发素 0.977496
|
||||
6 ODMOEM 0.955396
|
||||
7 【产品编号】：YM-X-3011 0.977864
|
||||
8 【净含量】：220ml 0.970538
|
||||
9 【适用人群】：适合所有肤质 0.995907
|
||||
10 【主要成分】：鲸蜡硬脂醇、燕麦β-葡聚 0.975813
|
||||
11 糖、椰油酰胺丙基甜菜碱、泛醌 0.964397
|
||||
12 （成品包材） 0.97298
|
||||
13 【主要功能】：可紧致头发磷层，从而达到 0.989097
|
||||
14 即时持久改善头发光泽的效果，给干燥的头 0.990088
|
||||
15 发足够的滋养 0.998037
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 代码介绍
|
||||
|
||||
```
|
||||
.
|
||||
├── ...
|
||||
├── ocr
|
||||
│ ├── ...
|
||||
│ ├── android
|
||||
│ │ ├── ...
|
||||
│ │ └── shell
|
||||
│ │ └── ppocr_demo
|
||||
│ │ ├── src # 存放预测代码
|
||||
│ │ | ├── cls_process.cc # 方向分类器的推理全流程，包含预处理、预测和后处理三部分
|
||||
│ │ | ├── rec_process.cc # 识别模型 CRNN 的推理全流程，包含预处理、预测和后处理三部分
|
||||
│ │ | ├── det_process.cc # 检测模型 CRNN 的推理全流程，包含预处理、预测和后处理三部分
|
||||
│ │ | ├── det_post_process.cc # 检测模型 DB 的后处理文件
|
||||
│ │ | ├── pipeline.cc # OCR 文字识别 demo 推理全流程代码
|
||||
│ │ | └── MakeFile # 预测代码的 MakeFile 文件
|
||||
│ │ |
|
||||
│ │ ├── CMakeLists.txt # CMake 文件，约束可执行文件的编译方法
|
||||
│ │ ├── README.md
|
||||
│ │ ├── build.sh # 用于可执行文件的编译
|
||||
│ │ └── run.sh # 用于预测
|
||||
│ └── assets # 存放模型、测试图片、标签文件、config 文件
|
||||
│ ├── images # 存放测试图片
|
||||
│ ├── labels # 存放字典文件，更多详情可参考下文备注
|
||||
│ ├── models # 存放 nb 模型
|
||||
│ ├── config.txt
|
||||
│ └── download.sh # 下载脚本，用于下载 paddle_lite_opt 工具优化后的模型
|
||||
└── libs # 存放不同端的预测库和 OpenCV 库。
|
||||
├── ...
|
||||
└── download.sh # 下载脚本，用于下载 Paddle Lite 预测库和 OpenCV 库
|
||||
```
|
||||
|
||||
**备注：**
|
||||
|
||||
- `PaddleX-Lite-Deploy/ocr/assets/labels/` 目录下存放了 PP-OCRv3、PP-OCRv4 模型的字典文件 `ppocr_keys_v1.txt` 以及 PP-OCRv5 模型的字典文件 `ppocr_keys_ocrv5.txt`。在实际推理过程中，会根据模型名称自动选择相应的字典文件，因此无需手动干预。
|
||||
- 如果使用的 nb 模型是英文数字或其他语言的模型，需要更换为对应语言的字典。PaddleOCR 仓库提供了[部分字典文件](https://github.com/PaddlePaddle/PaddleOCR/tree/release/2.3/ppocr/utils)。
|
||||
|
||||
```shell
|
||||
# run.sh 脚本中可执行文件的参数含义：
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"./models/${MODEL_NAME}_rec.nb\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/test.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/${LABEL_FILE} \
|
||||
./config.txt"
|
||||
|
||||
第一个参数：ppocr_demo 可执行文件
|
||||
第二个参数：./models/${MODEL_NAME}_det.nb 检测模型的.nb文件
|
||||
第三个参数：./models/${MODEL_NAME}_rec.nb 识别模型的.nb文件
|
||||
第四个参数：./models/${CLS_MODEL_FILE} 文本行方向分类模型的.nb文件，默认根据模型名自动选择
|
||||
第五个参数：./images/test.jpg 测试图片
|
||||
第六个参数：./test_img_result.jpg 结果保存文件
|
||||
第七个参数：./labels/${LABEL_FILE} label 文件，默认根据模型名自动选择
|
||||
第八个参数：./config.txt 配置文件，模型的超参数配置文件，包含了检测器、分类器的超参数
|
||||
```
|
||||
|
||||
```shell
|
||||
# config.txt 具体参数 List：
|
||||
max_side_len 960 # 输入图像长宽大于 960 时，等比例缩放图像，使得图像最长边为 960
|
||||
det_db_thresh 0.3 # 用于过滤 DB 预测的二值化图像，设置为 0.3 对结果影响不明显
|
||||
det_db_box_thresh 0.5 # DB 后处理过滤 box 的阈值，如果检测存在漏框情况，可酌情减小
|
||||
det_db_unclip_ratio 1.6 # 表示文本框的紧致程度，越小则文本框更靠近文本
|
||||
use_direction_classify 0 # 是否使用方向分类器，0 表示不使用，1 表示使用
|
||||
```
|
||||
|
||||
## 工程详解
|
||||
|
||||
OCR 文字识别 demo 由三个模型一起完成 OCR 文字识别功能，对输入图片先通过 `${MODEL_NAME}_det.nb` 模型做检测处理，然后通过 `ch_ppocr_mobile_v2.0_cls_slim_opt.nb` 模型做文字方向分类处理，最后通过 `${MODEL_NAME}_rec.nb` 模型完成文字识别处理。
|
||||
|
||||
1. `pipeline.cc` : OCR 文字识别 demo 预测全流程代码
|
||||
该文件完成了三个模型串行推理的全流程控制处理，包含整个处理过程的调度处理。
|
||||
|
||||
- `Pipeline::Pipeline(...)` 方法完成调用三个模型类构造函数，完成模型加载和线程数、绑核处理及 predictor 创建处理
|
||||
- `Pipeline::Process(...)` 方法用于完成这三个模型串行推理的全流程控制处理
|
||||
|
||||
2. `cls_process.cc` 方向分类器的预测文件
|
||||
该文件完成了方向分类器的预处理、预测和后处理过程
|
||||
|
||||
- `ClsPredictor::ClsPredictor()` 方法用于完成模型加载和线程数、绑核处理及 predictor 创建处理
|
||||
- `ClsPredictor::Preprocess()` 方法用于模型的预处理
|
||||
- `ClsPredictor::Postprocess()` 方法用于模型的后处理
|
||||
|
||||
3. `rec_process.cc` 识别模型 CRNN 的预测文件
|
||||
该文件完成了识别模型 CRNN 的预处理、预测和后处理过程
|
||||
|
||||
- `RecPredictor::RecPredictor()` 方法用于完成模型加载和线程数、绑核处理及 predictor 创建处理
|
||||
- `RecPredictor::Preprocess()` 方法用于模型的预处理
|
||||
- `RecPredictor::Postprocess()` 方法用于模型的后处理
|
||||
|
||||
4. `det_process.cc` 检测模型 DB 的预测文件
|
||||
该文件完成了检测模型 DB 的预处理、预测和后处理过程
|
||||
|
||||
- `DetPredictor::DetPredictor()` 方法用于完成模型加载和线程数、绑核处理及 predictor 创建处理
|
||||
- `DetPredictor::Preprocess()` 方法用于模型的预处理
|
||||
- `DetPredictor::Postprocess()` 方法用于模型的后处理
|
||||
|
||||
5. `db_post_process` 检测模型 DB 的后处理函数，包含 clipper 库的调用
|
||||
该文件完成了检测模型 DB 的第三方库调用和其他后处理方法实现
|
||||
|
||||
- `std::vector<std::vector<std::vector<int>>> BoxesFromBitmap(...)` 方法从 Bitmap 图中获取检测框
|
||||
- `std::vector<std::vector<std::vector<int>>> FilterTagDetRes(...)` 方法根据识别结果获取目标框位置
|
||||
|
||||
## 进阶使用
|
||||
|
||||
如果快速开始部分无法满足你的需求，可以参考本节对 demo 进行自定义修改。
|
||||
|
||||
本节主要包含四部分：
|
||||
|
||||
- 更新预测库；
|
||||
- 转换 `.nb` 模型；
|
||||
- 更新模型、标签文件和预测图片；
|
||||
- 更新输入/输出预处理。
|
||||
|
||||
### 更新预测库
|
||||
|
||||
本指南所使用的预测库为最新版本（214rc），不推荐自行更新预测库。
|
||||
|
||||
若有使用其他版本的需求，可参考如下步骤更新预测库：
|
||||
|
||||
* Paddle Lite 项目：https://github.com/PaddlePaddle/Paddle-Lite
|
||||
* 参考 [Paddle Lite 源码编译文档](https://www.paddlepaddle.org.cn/lite/develop/source_compile/compile_env.html)，编译 Android 预测库
|
||||
* 编译最终产物位于 `build.lite.xxx.xxx.xxx` 下的 `inference_lite_lib.xxx.xxx`
|
||||
* 替换 c++ 库
|
||||
* 头文件
|
||||
将生成的 `build.lite.android.xxx.gcc/inference_lite_lib.android.xxx/cxx/include` 文件夹替换 demo 中的 `PaddleX-Lite-Deploy/libs/android/cxx/include`
|
||||
* armeabi-v7a
|
||||
将生成的 `build.lite.android.armv7.gcc/inference_lite_lib.android.armv7/cxx/libs/libpaddle_lite_api_shared.so` 库替换 demo 中的 `PaddleX-Lite-Deploy/libs/android/cxx/libs/armeabi-v7a/libpaddle_lite_api_shared.so`
|
||||
* arm64-v8a
|
||||
将生成的 `build.lite.android.armv8.gcc/inference_lite_lib.android.armv8/cxx/libs/libpaddle_lite_api_shared.so` 库替换 demo 中的 `PaddleX-Lite-Deploy/libs/android/cxx/libs/arm64-v8a/libpaddle_lite_api_shared.so`
|
||||
|
||||
### 转换 .nb 模型
|
||||
|
||||
若想使用自己训练的模型，可先参考以下流程得到 `.nb` 模型。
|
||||
|
||||
#### 终端命令方法（支持Mac/Ubuntu）
|
||||
|
||||
1. 进入Paddle-Lite Github仓库的[release界面](https://github.com/PaddlePaddle/Paddle-Lite/releases)，选择所需版本下载对应的转化工具opt（推荐使用最新版本）。
|
||||
|
||||
2. 下载 opt 工具后，执行以下命令（此处以 2.14rc 版本的 linux_x86 opt 工具转换 PP-OCRv5_mobile_det 模型为例）：
|
||||
|
||||
```bash
|
||||
./opt_linux_x86 \
|
||||
--model_file=PP-OCRv5_mobile_det/inference.pdmodel \
|
||||
--param_file=PP-OCRv5_mobile_det/inference.pdiparams \
|
||||
--optimize_out=PP-OCRv5_mobile_det \
|
||||
--valid_targets=arm
|
||||
```
|
||||
|
||||
有关使用终端命令方法转换 `.nb` 模型的详细介绍，可参考 Paddle-Lite 仓库的[使用可执行文件 opt](https://www.paddlepaddle.org.cn/lite/v2.12/user_guides/opt/opt_bin.html)。
|
||||
|
||||
#### python 脚本方法（支持Windows/Mac/Ubuntu）
|
||||
|
||||
1. 安装最新版本的 paddlelite wheel 包。
|
||||
|
||||
```bash
|
||||
pip install --pre paddlelite
|
||||
```
|
||||
|
||||
2. 使用 python 脚本进行模型转换。以下为转换 PP-OCRv5_mobile_det 模型的示例代码：
|
||||
|
||||
```python
|
||||
from paddlelite.lite import Opt
|
||||
|
||||
# 1. 创建opt实例
|
||||
opt = Opt()
|
||||
# 2. 指定输入模型地址
|
||||
opt.set_model_file("./PP-OCRv5_mobile_det/inference.pdmodel")
|
||||
opt.set_param_file("./PP-OCRv5_mobile_det/inference.pdiparams")
|
||||
# 3. 指定转化类型
|
||||
opt.set_valid_places("arm")
|
||||
# 4. 指定输出模型地址
|
||||
opt.set_optimize_out("./PP-OCRv5_mobile_det")
|
||||
# 5. 执行模型优化
|
||||
opt.run()
|
||||
```
|
||||
|
||||
有关使用 python 脚本方法转换 `.nb` 模型的详细介绍，可参考 Paddle-Lite 仓库的[使用 Python 脚本 opt](https://www.paddlepaddle.org.cn/lite/v2.12/api_reference/python_api/opt.html)。
|
||||
|
||||
**注意**
|
||||
|
||||
- 有关模型优化工具 opt 的详细介绍，可参考 Paddle-Lite 仓库的[模型优化工具 opt](https://www.paddlepaddle.org.cn/lite/v2.12/user_guides/model_optimize_tool.html)
|
||||
- 目前仅支持将 `.pdmodel` 格式的静态图模型转换为 `.nb` 格式。
|
||||
|
||||
### 更新模型、标签文件和预测图片
|
||||
|
||||
#### 更新模型
|
||||
|
||||
本指南只对 `PP-OCRv3_mobile`、`PP-OCRv4_mobile`、`PP-OCRv5_mobile` 模型进行了验证，其他模型不保证适用性。
|
||||
|
||||
如果你对 `PP-OCRv5_mobile` 模型进行了微调，并生成了一个名为 `PP-OCRv5_mobile_ft` 的新模型，可以按照以下步骤将原有模型替换为你的微调模型：
|
||||
|
||||
1. 将 `PP-OCRv5_mobile_ft` 的 nb 模型存放到目录 `PaddleX-Lite-Deploy/ocr/assets/models/` 下，最终得到的文件结构如下：
|
||||
|
||||
```text
|
||||
.
|
||||
├── ocr
|
||||
│ ├── ...
|
||||
│ └── assets
|
||||
│ ├── models
|
||||
│ │ ├── ...
|
||||
│ │ ├── PP-OCRv5_mobile_ft_det.nb
|
||||
│ │ └── PP-OCRv5_mobile_ft_rec.nb
|
||||
│ └── ...
|
||||
└── ...
|
||||
```
|
||||
|
||||
2. 将模型名加入到 `run.sh` 脚本中的 `MODEL_LIST`。
|
||||
|
||||
```shell
|
||||
MODEL_LIST="PP-OCRv3_mobile PP-OCRv4_mobile PP-OCRv5_mobile PP-OCRv5_mobile_ft" # 模型之间以单个空格为间隔
|
||||
```
|
||||
|
||||
3. 运行 `run.sh` 脚本时使用模型目录名。
|
||||
|
||||
```shell
|
||||
sh run.sh PP-OCRv5_mobile_ft
|
||||
```
|
||||
|
||||
**注意：**
|
||||
|
||||
- 如果更新模型中的输入 Tensor、Shape、和 Dtype 发生更新:
|
||||
|
||||
- 更新文字方向分类器模型，则需要更新 `ppocr_demo/src/cls_process.cc` 中 `ClsPredictor::Preprocss` 函数
|
||||
- 更新检测模型，则需要更新 `ppocr_demo/src/det_process.cc` 中 `DetPredictor::Preprocss` 函数
|
||||
- 更新识别器模型，则需要更新 `ppocr_demo/src/rec_process.cc` 中 `RecPredictor::Preprocss` 函数
|
||||
|
||||
- 如果更新模型中的输出 Tensor 和 Dtype 发生更新:
|
||||
|
||||
- 更新文字方向分类器模型，则需要更新 `ppocr_demo/src/cls_process.cc` 中 `ClsPredictor::Postprocss` 函数
|
||||
- 更新检测模型，则需要更新 `ppocr_demo/src/det_process.cc` 中 `DetPredictor::Postprocss` 函数
|
||||
- 更新识别器模型，则需要更新 `ppocr_demo/src/rec_process.cc` 中 `RecPredictor::Postprocss` 函数
|
||||
|
||||
#### 更新标签文件
|
||||
|
||||
如果需要更新标签文件，则需要将新的标签文件存放在目录 `PaddleX-Lite-Deploy/ocr/assets/labels/` 下，并参考模型更新方法更新 `PaddleX-Lite-Deploy/ocr/android/shell/ppocr_demo/run.sh` 中执行命令；
|
||||
|
||||
以更新 `new_labels.txt` 为例：
|
||||
|
||||
```shell
|
||||
# 代码文件 `PaddleX-Lite-Deploy/ocr/android/shell/ppocr_demo/run.sh`
|
||||
# old
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"./models/${MODEL_NAME}_rec.nb\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/test.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/${LABEL_FILE} \
|
||||
./config.txt"
|
||||
# update
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"./models/${MODEL_NAME}_rec.nb\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/test.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/new_labels.txt \
|
||||
./config.txt"
|
||||
```
|
||||
|
||||
#### 更新预测图片
|
||||
|
||||
如果需要更新预测图片，将更新的图片存放在 `PaddleX-Lite-Deploy/ocr/assets/images/` 下，更新文件 `PaddleX-Lite-Deploy/ocr/android/shell/ppocr_demo/rush.sh` 中执行命令；
|
||||
|
||||
以更新 `new_pics.jpg` 为例：
|
||||
|
||||
```shell
|
||||
# 代码文件 `PaddleX-Lite-Deploy/ocr/assets/images/run.sh`
|
||||
## old
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"./models/${MODEL_NAME}_rec.nb\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/test.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/${LABEL_FILE} \
|
||||
./config.txt"
|
||||
# update
|
||||
adb shell "cd ${ppocr_demo_path} \
|
||||
&& chmod +x ./ppocr_demo \
|
||||
&& export LD_LIBRARY_PATH=${ppocr_demo_path}:${LD_LIBRARY_PATH} \
|
||||
&& ./ppocr_demo \
|
||||
\"./models/${MODEL_NAME}_det.nb\" \
|
||||
\"\"./models/${MODEL_NAME}_rec.nb\"\" \
|
||||
./models/${CLS_MODEL_FILE} \
|
||||
./images/new_pics.jpg \
|
||||
./test_img_result.jpg \
|
||||
./labels/${LABEL_FILE} \
|
||||
./config.txt"
|
||||
```
|
||||
|
||||
### 更新输入/输出预处理
|
||||
|
||||
- 更新输入预处理
|
||||
- 更新文字方向分类器模型，则需要更新 `ppocr_demo/src/cls_process.cc` 中 `ClsPredictor::Preprocss` 函数
|
||||
- 更新检测模型，则需要更新 `ppocr_demo/src/det_process.cc` 中 `DetPredictor::Preprocss` 函数
|
||||
- 更新识别器模型，则需要更新 `ppocr_demo/src/rec_process.cc` 中 `RecPredictor::Preprocss` 函数
|
||||
|
||||
- 更新输出预处理
|
||||
- 更新文字方向分类器模型，则需要更新 `ppocr_demo/src/cls_process.cc` 中 `ClsPredictor::Postprocss` 函数
|
||||
- 更新检测模型，则需要更新 `ppocr_demo/src/det_process.cc` 中 `DetPredictor::Postprocss` 函数
|
||||
- 更新识别器模型，则需要更新 `ppocr_demo/src/rec_process.cc` 中 `RecPredictor::Postprocss` 函数
|
||||
11
docs/version3.x/deployment/python_and_cpp_infer.en.md
Normal file
11
docs/version3.x/deployment/python_and_cpp_infer.en.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# Inference with Python or C++ Prediction Engine
|
||||
|
||||
Since the 2.x branch, inference with Python or C++ prediction engines has been a significant feature. This functionality allows users to load OCR-related models and perform inference without installing the wheel package.
|
||||
|
||||
Due to differences in pre-processing, post-processing, and concatenation details compared to the wheel package, the inference results may slightly vary, and the two cannot be directly interchanged.
|
||||
|
||||
For specific usage instructions, please refer to the following documents:
|
||||
|
||||
* [Inference with Python Prediction Engine](../../version2.x/legacy/python_infer.md)
|
||||
* [Inference with C++ Prediction Engine](../../version2.x/legacy/cpp_infer.md)
|
||||
* [List of Supported Models](../../version2.x/legacy/model_list_2.x.md)
|
||||
11
docs/version3.x/deployment/python_and_cpp_infer.md
Normal file
11
docs/version3.x/deployment/python_and_cpp_infer.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# 基于Python或C++预测引擎推理
|
||||
|
||||
自 2.x 分支以来，基于 Python 或 C++ 的预测引擎推理一直是一项重要功能。该功能允许用户在无需安装 wheel 包的情况下加载OCR相关模型并进行推理。
|
||||
|
||||
由于其在前后处理和串联细节上与 wheel 包存在差异，导致推理效果略有不同，两者无法直接互换使用。
|
||||
|
||||
有关具体使用方法，请参考以下文档：
|
||||
|
||||
* [基于Python预测引擎推理](../../version2.x/legacy/python_infer.md)
|
||||
* [基于C++预测引擎推理](../../version2.x/legacy/cpp_infer.md)
|
||||
* [支持的模型列表](../../version2.x/legacy/model_list_2.x.md)
|
||||
92
docs/version3.x/deployment/serving.en.md
Normal file
92
docs/version3.x/deployment/serving.en.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# Serving
|
||||
|
||||
Serving is a common deployment method in real-world production environments. By encapsulating inference capabilities as services, clients can access these services via network requests to obtain inference results. PaddleOCR recommends using [PaddleX](https://github.com/PaddlePaddle/PaddleX) for serving. Please refer to [Differences and Connections between PaddleOCR and PaddleX](../paddleocr_and_paddlex.en.md#1-Differences-and-Connections-Between-PaddleOCR-and-PaddleX) to understand the relationship between PaddleOCR and PaddleX.
|
||||
|
||||
PaddleX provides the following serving solutions:
|
||||
|
||||
- **Basic Serving**: An easy-to-use serving solution with low development costs.
|
||||
- **High-Stability Serving**: Built based on [NVIDIA Triton Inference Server](https://developer.nvidia.com/triton-inference-server). Compared to the basic serving, this solution offers higher stability and allows users to adjust configurations to optimize performance.
|
||||
|
||||
**It is recommended to first use the basic serving solution for quick validation**, and then evaluate whether to try more complex solutions based on actual needs.
|
||||
|
||||
## 1. Basic Serving
|
||||
|
||||
### 1.1 Install Dependencies
|
||||
|
||||
Run the following command to install the PaddleX serving plugin via PaddleX CLI:
|
||||
|
||||
```bash
|
||||
paddlex --install serving
|
||||
```
|
||||
|
||||
### 1.2 Run the Server
|
||||
|
||||
Run the server via PaddleX CLI:
|
||||
|
||||
```bash
|
||||
paddlex --serve --pipeline {PaddleX pipeline registration name or pipeline configuration file path} [{other command-line options}]
|
||||
```
|
||||
|
||||
Take the general OCR pipeline as an example:
|
||||
|
||||
```bash
|
||||
paddlex --serve --pipeline OCR
|
||||
```
|
||||
|
||||
You should see information similar to the following:
|
||||
|
||||
```text
|
||||
INFO: Started server process [63108]
|
||||
INFO: Waiting for application startup.
|
||||
INFO: Application startup complete.
|
||||
INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
To adjust configurations (such as model path, batch size, deployment device, etc.), specify `--pipeline` as a custom configuration file. Refer to [PaddleOCR and PaddleX](../paddleocr_and_paddlex.en.md) for the mapping between PaddleOCR pipelines and PaddleX pipeline registration names, as well as how to obtain and modify PaddleX pipeline configuration files.
|
||||
|
||||
The command-line options related to serving are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>--pipeline</code></td>
|
||||
<td>PaddleX pipeline registration name or pipeline configuration file path.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--device</code></td>
|
||||
<td>Deployment device for the pipeline. By default, a GPU will be used if available; otherwise, a CPU will be used."</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--host</code></td>
|
||||
<td>Hostname or IP address to which the server is bound. Defaults to <code>0.0.0.0</code>.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--port</code></td>
|
||||
<td>Port number on which the server listens. Defaults to <code>8080</code>.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--use_hpip</code></td>
|
||||
<td>If specified, uses high-performance inference. Refer to the High-Performance Inference documentation for more information.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--hpi_config</code></td>
|
||||
<td>High-performance inference configuration. Refer to the High-Performance Inference documentation for more information.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
### 1.3 Invoke the Service
|
||||
|
||||
The <b>"Development Integration/Deployment"</b> section in the PaddleOCR pipeline tutorial provides API references and multi-language invocation examples for the service.
|
||||
|
||||
## 2. High-Stability Serving
|
||||
|
||||
Please refer to the [PaddleX Serving Guide](https://paddlepaddle.github.io/PaddleX/3.0/en/pipeline_deploy/serving.html#2). More information about PaddleX pipeline configuration files can be found in [Using PaddleX Pipeline Configuration Files](../paddleocr_and_paddlex.en.md#3-using-paddlex-pipeline-configuration-files).
|
||||
|
||||
It should be noted that, due to the lack of fine-grained optimization and other reasons, the current high-stability serving deployment solution provided by PaddleOCR may not match the performance of the 2.x version based on PaddleServing. However, this new solution fully supports the PaddlePaddle 3.0 framework. We will continue to optimize it and consider introducing more performant deployment solutions in the future.
|
||||
93
docs/version3.x/deployment/serving.md
Normal file
93
docs/version3.x/deployment/serving.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# 服务化部署
|
||||
|
||||
服务化部署是实际生产环境中常见的一种部署形式。通过将推理功能封装为服务，客户端可以通过网络请求来访问这些服务，以获取推理结果。PaddleOCR 推荐用户使用 [PaddleX](https://github.com/PaddlePaddle/PaddleX) 进行服务化部署。请阅读 [PaddleOCR 与 PaddleX 的区别与联系](../paddleocr_and_paddlex.md#1-paddleocr-与-paddlex-的区别与联系) 了解 PaddleOCR 与 PaddleX 的关系。
|
||||
|
||||
PaddleX 提供以下服务化部署方案：
|
||||
|
||||
- **基础服务化部署**：简单易用的服务化部署方案，开发成本低。
|
||||
- **高稳定性服务化部署**：基于 [NVIDIA Triton Inference Server](https://developer.nvidia.com/triton-inference-server) 打造。与基础服务化部署相比，该方案提供更高的稳定性，并允许用户调整配置以优化性能。
|
||||
|
||||
**建议首先使用基础服务化部署方案进行快速验证**，然后根据实际需要，评估是否尝试更复杂的方案。
|
||||
|
||||
## 1. 基础服务化部署
|
||||
|
||||
### 1.1 安装依赖
|
||||
|
||||
执行如下命令，通过 PaddleX CLI 安装 PaddleX 服务化部署插件：
|
||||
|
||||
```bash
|
||||
paddlex --install serving
|
||||
```
|
||||
|
||||
### 1.2 运行服务器
|
||||
|
||||
通过 PaddleX CLI 运行服务器：
|
||||
|
||||
```bash
|
||||
paddlex --serve --pipeline {PaddleX 产线注册名或产线配置文件路径} [{其他命令行选项}]
|
||||
```
|
||||
|
||||
以通用 OCR 产线为例：
|
||||
|
||||
```bash
|
||||
paddlex --serve --pipeline OCR
|
||||
```
|
||||
|
||||
可以看到类似以下展示的信息：
|
||||
|
||||
```text
|
||||
INFO: Started server process [63108]
|
||||
INFO: Waiting for application startup.
|
||||
INFO: Application startup complete.
|
||||
INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
|
||||
```
|
||||
|
||||
如需调整配置（如模型路径、batch size、部署设备等），可指定 `--pipeline` 为自定义配置文件。请参考 [PaddleOCR 与 PaddleX](../paddleocr_and_paddlex.md) 了解 PaddleOCR 产线与 PaddleX 产线注册名的对应关系，以及 PaddleX 产线配置文件的获取与修改方式。
|
||||
|
||||
与服务化部署相关的命令行选项如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>--pipeline</code></td>
|
||||
<td>PaddleX 产线注册名或产线配置文件路径。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--device</code></td>
|
||||
<td>产线部署设备。默认情况下，当 GPU 可用时，将使用 GPU；否则使用 CPU。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--host</code></td>
|
||||
<td>服务器绑定的主机名或 IP 地址。默认为 <code>0.0.0.0</code>。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--port</code></td>
|
||||
<td>服务器监听的端口号。默认为 <code>8080</code>。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--use_hpip</code></td>
|
||||
<td>如果指定，则使用高性能推理。请参考高性能推理文档了解更多信息。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>--hpi_config</code></td>
|
||||
<td>高性能推理配置。请参考高性能推理文档了解更多信息。</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</table>
|
||||
|
||||
### 1.3 调用服务
|
||||
|
||||
PaddleOCR 产线使用教程中的 <b>“开发集成/部署”</b> 部分提供了服务的 API 参考与多语言调用示例。
|
||||
|
||||
## 2. 高稳定性服务化部署
|
||||
|
||||
请参考 [PaddleX 服务化部署指南](https://paddlepaddle.github.io/PaddleX/3.0/pipeline_deploy/serving.html#2)。在 [使用 PaddleX 产线配置文件](../paddleocr_and_paddlex.md#3-使用-paddlex-产线配置文件) 中，可以了解关于 PaddleX 产线配置文件的更多信息。
|
||||
|
||||
需要说明的是，由于缺乏细粒度优化等原因，当前 PaddleOCR 提供的高稳定性服务化部署方案在性能上可能不及 2.x 版本基于 PaddleServing 的方案；但该新方案已对飞桨 3.0 框架提供了全面支持，我们也将持续优化，后续考虑推出性能更优的部署方案。
|
||||
50
docs/version3.x/installation.en.md
Normal file
50
docs/version3.x/installation.en.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Installation
|
||||
|
||||
# 1. Install PaddlePaddle Framework
|
||||
|
||||
Please refer to the [PaddlePaddle Official Website](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/en/develop/install/pip/linux-pip.html) to install PaddlePaddle framework version `3.0` or above. **Using the official PaddlePaddle Docker image is recommended.**
|
||||
|
||||
# 2. Install PaddleOCR
|
||||
|
||||
If you only want to use the inference capabilities of PaddleOCR, please refer to [Install Inference Package](#21-install-inference-package); if you want to perform model training, exporting, etc., please refer to [Install Training Dependencies](#22-install-training-dependencies). It is allowed to install both the inference package and training dependencies in the same environment without the need for environment isolation.
|
||||
|
||||
## 2.1 Install Inference Package
|
||||
|
||||
Install the latest version of the PaddleOCR inference package from PyPI:
|
||||
|
||||
```bash
|
||||
python -m pip install paddleocr
|
||||
```
|
||||
|
||||
Or install from source (default is the development branch):
|
||||
|
||||
```bash
|
||||
python -m pip install "git+https://github.com/PaddlePaddle/PaddleOCR.git"
|
||||
```
|
||||
|
||||
## 2.2 Install Training Dependencies
|
||||
|
||||
To perform model training, exporting, etc., first clone the repository to your local machine:
|
||||
|
||||
```bash
|
||||
# Recommended method
|
||||
git clone https://github.com/PaddlePaddle/PaddleOCR
|
||||
|
||||
# (Optional) Switch to a specific branch
|
||||
git checkout release/3.0
|
||||
|
||||
# If you encounter network issues preventing successful cloning, you can also use the repository on Gitee:
|
||||
git clone https://gitee.com/paddlepaddle/PaddleOCR
|
||||
|
||||
# Note: The code hosted on Gitee may not be synchronized in real-time with updates from this GitHub project, with a delay of 3~5 days. Please prioritize using the recommended method.
|
||||
```
|
||||
|
||||
Run the following command to install the dependencies:
|
||||
|
||||
```bash
|
||||
python -m pip install -r requirements.txt
|
||||
```
|
||||
50
docs/version3.x/installation.md
Normal file
50
docs/version3.x/installation.md
Normal file
@@ -0,0 +1,50 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 安装
|
||||
|
||||
# 1. 安装飞桨框架
|
||||
|
||||
请参考 [飞桨官网](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation/docs/zh/develop/install/pip/linux-pip.html) 安装 `3.0` 及以上版本的飞桨框架。**推荐使用飞桨官方 Docker 镜像。**
|
||||
|
||||
# 2. 安装 PaddleOCR
|
||||
|
||||
如果只希望使用 PaddleOCR 的推理功能，请参考 [安装推理包](#21-安装推理包)；如果希望进行模型训练、导出等，请参考 [安装训练依赖](#22-安装训练依赖)。在同一环境中安装推理包和训练依赖是允许的，无需进行环境隔离。
|
||||
|
||||
## 2.1 安装推理包
|
||||
|
||||
从 PyPI 安装最新版本 PaddleOCR 推理包：
|
||||
|
||||
```bash
|
||||
python -m pip install paddleocr
|
||||
```
|
||||
|
||||
或者从源码安装（默认为开发分支）：
|
||||
|
||||
```bash
|
||||
python -m pip install "git+https://github.com/PaddlePaddle/PaddleOCR.git"
|
||||
```
|
||||
|
||||
## 2.2 安装训练依赖
|
||||
|
||||
要进行模型训练、导出等，需要首先将仓库克隆到本地：
|
||||
|
||||
```bash
|
||||
# 推荐方式
|
||||
git clone https://github.com/PaddlePaddle/PaddleOCR
|
||||
|
||||
# （可选）切换到指定分支
|
||||
git checkout release/3.0
|
||||
|
||||
# 如果因为网络问题无法克隆成功，也可选择使用码云上的仓库：
|
||||
git clone https://gitee.com/paddlepaddle/PaddleOCR
|
||||
|
||||
# 注：码云托管代码可能无法实时同步本 GitHub 项目更新，存在3~5天延时，请优先使用推荐方式。
|
||||
```
|
||||
|
||||
执行如下命令安装依赖：
|
||||
|
||||
```bash
|
||||
python -m pip install -r requirements.txt
|
||||
```
|
||||
22
docs/version3.x/logging.en.md
Normal file
22
docs/version3.x/logging.en.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Logging
|
||||
|
||||
This document mainly introduces how to configure the logging system for the PaddleOCR inference package. It's important to note that PaddleOCR's inference package uses a different logging system than the training scripts, and this document does not cover the configuration of the logging system used in the training scripts.
|
||||
|
||||
PaddleOCR has built a centralized logging system based on Python's [`logging` standard library](https://docs.python.org/3/library/logging.html#module-logging). In other words, PaddleOCR uses a single logger, which can be accessed and configured via `paddleocr.logger`.
|
||||
|
||||
By default, the logging level in PaddleOCR is set to `ERROR`, meaning that log messages will only be output if their level is `ERROR` or higher (e.g., `CRITICAL`). PaddleOCR also configures a `StreamHandler` for this logger, which outputs logs to the standard error stream, and sets the logger's `propagate` attribute to `False` to prevent log messages from being passed to its parent logger.
|
||||
|
||||
If you wish to disable PaddleOCR's automatic logging configuration behavior, you can set the environment variable `DISABLE_AUTO_LOGGING_CONFIG` to `1`. In this case, PaddleOCR will not perform any additional configuration of the logger.
|
||||
|
||||
For more flexible customization of logging behavior, refer to the relevant documentation of the `logging` standard library. Below is an example of writing logs to a file:
|
||||
|
||||
```python
|
||||
import logging
|
||||
from paddleocr import logger
|
||||
|
||||
# Write logs to the file `paddleocr.log`
|
||||
fh = logging.FileHandler("paddleocr.log")
|
||||
logger.addHandler(fh)
|
||||
```
|
||||
|
||||
Please note that other libraries that PaddleOCR depends on (such as [PaddleX](./paddleocr_and_paddlex.en.md)) have their own independent logging systems, and the above configuration will not affect the log output of these libraries.
|
||||
22
docs/version3.x/logging.md
Normal file
22
docs/version3.x/logging.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# 日志
|
||||
|
||||
本文档主要介绍如何配置 PaddleOCR 推理包的日志系统。需要注意的是，PaddleOCR 推理包与训练脚本使用的是不同的日志系统，本文档不涉及训练脚本所使用的日志系统的配置方法。
|
||||
|
||||
PaddleOCR 构建了一个基于 Python [`logging` 标准库](https://docs.python.org/zh-cn/3/library/logging.html#module-logging) 的集中式日志系统。换言之，PaddleOCR 使用唯一的日志记录器（logger），可通过 `paddleocr.logger` 访问和配置。
|
||||
|
||||
默认情况下，PaddleOCR 的日志级别设为 `ERROR`，这意味着仅当日志级别为 `ERROR` 或更高（如 `CRITICAL`）时，日志信息才会输出。PaddleOCR 同时为该日志记录器配置了一个 `StreamHandler`，将日志输出到标准错误流，并将记录器的 `propagate` 属性设为 `False`，以避免日志信息传递到其父记录器。
|
||||
|
||||
若希望禁止 PaddleOCR 对日志系统的自动配置行为，可将环境变量 `DISABLE_AUTO_LOGGING_CONFIG` 设为 `1`。此时，PaddleOCR 将不会对日志记录器进行任何额外配置。
|
||||
|
||||
如需更灵活地定制日志行为，可参考 `logging` 标准库的相关文档。以下是一个将日志写入文件的示例：
|
||||
|
||||
```python
|
||||
import logging
|
||||
from paddleocr import logger
|
||||
|
||||
# 将日志写入文件 `paddleocr.log`
|
||||
fh = logging.FileHandler("paddleocr.log")
|
||||
logger.addHandler(fh)
|
||||
```
|
||||
|
||||
请注意，PaddleOCR 依赖的其他库（如 [PaddleX](./paddleocr_and_paddlex.md)）拥有各自独立的日志系统，以上配置不会影响这些库的日志输出。
|
||||
922
docs/version3.x/model_list.md
Normal file
922
docs/version3.x/model_list.md
Normal file
@@ -0,0 +1,922 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# PaddleOCR模型列表（CPU/GPU）
|
||||
|
||||
PaddleOCR 内置了多条产线，每条产线都包含了若干模块，每个模块包含若干模型，具体使用哪些模型，您可以根据下边的 benchmark 数据来选择。如您更考虑模型精度，请选择精度较高的模型，如您更考虑模型推理速度，请选择推理速度较快的模型，如您更考虑模型存储大小，请选择存储大小较小的模型。
|
||||
|
||||
## [文本检测模块](./module_usage/text_detection.md)
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>检测Hmean（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_det</td>
|
||||
<td>83.8</td>
|
||||
<td>89.55 / 70.19</td>
|
||||
<td>383.15 / 383.15</td>
|
||||
<td>101</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_detection/PP-OCRv5_server_det.yaml">PP-OCRv5_server_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_server_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_det</td>
|
||||
<td>79.0</td>
|
||||
<td>10.67 / 6.36</td>
|
||||
<td>57.77 / 28.15</td>
|
||||
<td>4.7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_detection/PP-OCRv5_mobile_det.yaml">PP-OCRv5_mobile_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_mobile_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td>82.56</td>
|
||||
<td>127.82 / 98.87</td>
|
||||
<td>585.95 / 489.77</td>
|
||||
<td>109</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_detection/PP-OCRv4_server_det.yaml">PP-OCRv4_server_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td>63.8</td>
|
||||
<td>9.87 / 4.17</td>
|
||||
<td>56.60 / 20.79</td>
|
||||
<td>4.7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_detection/PP-OCRv4_mobile_det.yaml">PP-OCRv4_mobile_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv3_mobile_det</td>
|
||||
<td>78.68</td>
|
||||
<td>9.90 / 3.60</td>
|
||||
<td>41.93 / 20.76</td>
|
||||
<td>2.1</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_detection/PP-OCRv3_mobile_det.yaml">PP-OCRv3_mobile_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv3_mobile_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv3_mobile_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv3_server_det</td>
|
||||
<td>80.11</td>
|
||||
<td>119.50 / 75.00</td>
|
||||
<td>379.35 / 318.35</td>
|
||||
<td>102.1</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_detection/PP-OCRv3_server_det.yaml">PP-OCRv3_server_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv3_server_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv3_server_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<b>注：以上精度指标的评估集是 PaddleOCR 自建的中英文数据集，覆盖街景、网图、文档、手写多个场景，其中文本识别包含 593 张图片。</b>
|
||||
|
||||
## [印章文本检测模块](./module_usage/seal_text_detection.md)
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型名称</th>
|
||||
<th>检测Hmean（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_seal_det</td>
|
||||
<td>96.36</td>
|
||||
<td>9.70 / 3.56</td>
|
||||
<td>50.38 / 19.64</td>
|
||||
<td>4.7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/seal_text_detection/PP-OCRv4_mobile_seal_det.yaml">PP-OCRv4_mobile_seal_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_seal_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_seal_det_pretrained.pdparams">训练模型</a></td></tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_seal_det</td>
|
||||
<td>98.40</td>
|
||||
<td>124.64 / 91.57</td>
|
||||
<td>545.68 / 439.86</td>
|
||||
<td>109</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/seal_text_detection/PP-OCRv4_server_seal_det.yaml">PP-OCRv4_server_seal_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_seal_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams">训练模型</a></td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<b>注：以上精度指标的评估集是 PaddleOCR 自建的印章数据集，包含500印章图像。</b>
|
||||
|
||||
## [文本识别模块](./module_usage/text_recognition.md)
|
||||
|
||||
* <b>中文识别模型</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td>86.38</td>
|
||||
<td>8.46 / 2.36</td>
|
||||
<td>31.21 / 31.21</td>
|
||||
<td>81</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/PP-OCRv5_server_rec.yaml">PP-OCRv5_server_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td>81.29</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>16</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/PP-OCRv5_mobile_rec.yaml">PP-OCRv5_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec_doc</td>
|
||||
<td>86.58</td>
|
||||
<td>8.69 / 2.78</td>
|
||||
<td>37.93 / 37.93</td>
|
||||
<td>182</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/PP-OCRv4_server_rec_doc.yaml">PP-OCRv4_server_rec_doc.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv4_server_rec_doc_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td>78.74</td>
|
||||
<td>5.26 / 1.12</td>
|
||||
<td>17.48 / 3.61</td>
|
||||
<td>10.5</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/PP-OCRv4_mobile_rec.yaml">PP-OCRv4_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td>85.19</td>
|
||||
<td>8.75 / 2.49</td>
|
||||
<td>36.93 / 36.93</td>
|
||||
<td>173</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/PP-OCRv4_server_rec.yaml">PP-OCRv4_server_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv3_mobile_rec</td>
|
||||
<td>72.96</td>
|
||||
<td>3.89 / 1.16</td>
|
||||
<td>8.72 / 3.56</td>
|
||||
<td>10.3</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/PP-OCRv3_mobile_rec.yaml">PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
<p><b>注：以上精度指标的评估集是 PaddleOCR 自建的中文数据集，覆盖街景、网图、文档、手写多个场景，其中文本识别包含 8367 张图片。</b></p>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_SVTRv2_rec</td>
|
||||
<td>68.81</td>
|
||||
<td>10.38 / 8.31</td>
|
||||
<td>66.52 / 30.83</td>
|
||||
<td>80.5</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/ch_SVTRv2_rec.yaml">ch_SVTRv2_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_SVTRv2_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_SVTRv2_rec_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
<p><b>注：以上精度指标的评估集是 <a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务</a>A榜。 </b></p>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_RepSVTR_rec</td>
|
||||
<td>65.07</td>
|
||||
<td>6.29 / 1.57</td>
|
||||
<td>20.64 / 5.40</td>
|
||||
<td>48.8</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/ch_RepSVTR_rec.yaml">ch_RepSVTR_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_RepSVTR_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_RepSVTR_rec_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
<p><b>注：以上精度指标的评估集是 <a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务</a>B榜。 </b></p>
|
||||
|
||||
* <b>英文识别模型</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv4_mobile_rec</td>
|
||||
<td> 70.39</td>
|
||||
<td>4.81 / 1.23</td>
|
||||
<td>17.20 / 4.18</td>
|
||||
<td>7.5</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/en_PP-OCRv4_mobile_rec.yaml">en_PP-OCRv4_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv3_mobile_rec</td>
|
||||
<td>70.69</td>
|
||||
<td>3.56 / 0.78</td>
|
||||
<td>8.44 / 5.78</td>
|
||||
<td>17.3</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/en_PP-OCRv3_mobile_rec.yaml">en_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<p><b>注：以上精度指标的评估集是 PaddleOCR 自建的英文数据集。 </b></p>
|
||||
|
||||
* <b>多语言识别模型</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>korean_PP-OCRv3_mobile_rec</td>
|
||||
<td>60.21</td>
|
||||
<td>3.73 / 0.98</td>
|
||||
<td>8.76 / 2.91</td>
|
||||
<td>9.6</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/korean_PP-OCRv3_mobile_rec.yaml">korean_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
korean_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>japan_PP-OCRv3_mobile_rec</td>
|
||||
<td>45.69</td>
|
||||
<td>3.86 / 1.01</td>
|
||||
<td>8.62 / 2.92</td>
|
||||
<td>9.8</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/japan_PP-OCRv3_mobile_rec.yaml">japan_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
japan_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>chinese_cht_PP-OCRv3_mobile_rec</td>
|
||||
<td>82.06</td>
|
||||
<td>3.90 / 1.16</td>
|
||||
<td>9.24 / 3.18</td>
|
||||
<td>10.8</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/chinese_cht_PP-OCRv3_mobile_rec.yaml">chinese_cht_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
chinese_cht_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>te_PP-OCRv3_mobile_rec</td>
|
||||
<td>95.88</td>
|
||||
<td>3.59 / 0.81</td>
|
||||
<td>8.28 / 6.21</td>
|
||||
<td>8.7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/te_PP-OCRv3_mobile_rec.yaml">te_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
te_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ka_PP-OCRv3_mobile_rec</td>
|
||||
<td>96.96</td>
|
||||
<td>3.49 / 0.89</td>
|
||||
<td>8.63 / 2.77</td>
|
||||
<td>17.4</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/ka_PP-OCRv3_mobile_rec.yaml">ka_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ka_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ta_PP-OCRv3_mobile_rec</td>
|
||||
<td>76.83</td>
|
||||
<td>3.49 / 0.86</td>
|
||||
<td>8.35 / 3.41</td>
|
||||
<td>8.7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/ta_PP-OCRv3_mobile_rec.yaml">ta_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ta_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>latin_PP-OCRv3_mobile_rec</td>
|
||||
<td>76.93</td>
|
||||
<td>3.53 / 0.78</td>
|
||||
<td>8.50 / 6.83</td>
|
||||
<td>8.7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/latin_PP-OCRv3_mobile_rec.yaml">latin_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
latin_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>arabic_PP-OCRv3_mobile_rec</td>
|
||||
<td>73.55</td>
|
||||
<td>3.60 / 0.83</td>
|
||||
<td>8.44 / 4.69</td>
|
||||
<td>17.3</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/arabic_PP-OCRv3_mobile_rec.yaml">arabic_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
arabic_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>cyrillic_PP-OCRv3_mobile_rec</td>
|
||||
<td>94.28</td>
|
||||
<td>3.56 / 0.79</td>
|
||||
<td>8.22 / 2.76</td>
|
||||
<td>8.7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/cyrillic_PP-OCRv3_mobile_rec.yaml">cyrillic_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
cyrillic_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>devanagari_PP-OCRv3_mobile_rec</td>
|
||||
<td>96.44</td>
|
||||
<td>3.60 / 0.78</td>
|
||||
<td>6.95 / 2.87</td>
|
||||
<td>8.7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/devanagari_PP-OCRv3_mobile_rec.yaml">devanagari_PP-OCRv3_mobile_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
devanagari_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
<p><b>注：以上精度指标的评估集是 PaddleOCR 自建的多语种数据集。</b></p>
|
||||
|
||||
## [公式识别模块](./module_usage/formula_recognition.md)
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>En-BLEU(%)</th>
|
||||
<th>Zh-BLEU(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UniMERNet</td>
|
||||
<td>85.91</td>
|
||||
<td>43.50</td>
|
||||
<td>1311.84 / 1311.84</td>
|
||||
<td>- / 8288.07</td>
|
||||
<td>1530</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/formula_recognition/UniMERNet.yaml">UniMERNet.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UniMERNet_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UniMERNet_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-S</td>
|
||||
<td>87.00</td>
|
||||
<td>45.71</td>
|
||||
<td>182.25 / 182.25</td>
|
||||
<td>- / 254.39</td>
|
||||
<td>224</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/formula_recognition/PP-FormulaNet-S.yaml">PP-FormulaNet-S.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-S_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-L</td>
|
||||
<td>90.36</td>
|
||||
<td>45.78</td>
|
||||
<td>1482.03 / 1482.03</td>
|
||||
<td>- / 3131.54</td>
|
||||
<td>695</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/formula_recognition/PP-FormulaNet-L.yaml">PP-FormulaNet-L.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-L_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet_plus-S</td>
|
||||
<td>88.71</td>
|
||||
<td>53.32</td>
|
||||
<td>179.20 / 179.20</td>
|
||||
<td>- / 260.99</td>
|
||||
<td>248</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/formula_recognition/PP-FormulaNet_plus-S.yaml">PP-FormulaNet_plus-S.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-S_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-M</td>
|
||||
<td>91.45</td>
|
||||
<td>89.76</td>
|
||||
<td>1040.27 / 1040.27</td>
|
||||
<td>- / 1615.80</td>
|
||||
<td>592</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/formula_recognition/PP-FormulaNet_plus-M.yaml">PP-FormulaNet_plus-M.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-M_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-M_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-L</td>
|
||||
<td>92.22</td>
|
||||
<td>90.64</td>
|
||||
<td>1476.07 / 1476.07</td>
|
||||
<td>- / 3125.58</td>
|
||||
<td>698</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/formula_recognition/PP-FormulaNet_plus-L.yaml">PP-FormulaNet_plus-L.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LaTeX_OCR_rec</td>
|
||||
<td>74.55</td>
|
||||
<td>39.96</td>
|
||||
<td>1088.89 / 1088.89</td>
|
||||
<td>- / -</td>
|
||||
<td>99</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/formula_recognition/LaTeX_OCR_rec.yaml">LaTeX_OCR_rec.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/LaTeX_OCR_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/LaTeX_OCR_rec_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
<b>注：以上精度指标测量自 PaddleX 内部自建公式识别测试集。LaTeX_OCR_rec在LaTeX-OCR公式识别测试集的BLEU score为 0.8821。</b>
|
||||
|
||||
## [表格结构识别模块](./module_usage/table_structure_recognition.md)
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>精度（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet</td>
|
||||
<td>59.52</td>
|
||||
<td>23.96 / 21.75</td>
|
||||
<td>- / 43.12</td>
|
||||
<td>6.9</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/table_structure_recognition/SLANet.yaml">SLANet.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet_plus</td>
|
||||
<td>63.69</td>
|
||||
<td>23.43 / 22.16</td>
|
||||
<td>- / 41.80</td>
|
||||
<td>6.9</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/table_structure_recognition/SLANet_plus.yaml">SLANet_plus.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_plus_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wired</td>
|
||||
<td rowspan="2">69.65</td>
|
||||
<td rowspan="2">85.92 / 85.92</td>
|
||||
<td rowspan="2">- / 501.66</td>
|
||||
<td rowspan="2">351</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/table_structure_recognition/SLANeXt_wired.yaml">SLANeXt_wired.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wired_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wired_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wireless</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/table_structure_recognition/SLANeXt_wireless.yaml">SLANeXt_wireless.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wireless_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wireless_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
<b>注：以上精度指标测量自 PaddleX 内部自建高难度中文表格识别数据集。</b>
|
||||
|
||||
|
||||
## [表格单元格检测模块](./module_usage/table_cells_detection.md)
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>mAP(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wired_table_cell_det</td>
|
||||
<td rowspan="2">82.7</td>
|
||||
<td rowspan="2">33.47 / 27.02</td>
|
||||
<td rowspan="2">402.55 / 256.56</td>
|
||||
<td rowspan="2">124</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/table_cells_detection/RT-DETR-L_wired_table_cell_det.yaml">RT-DETR-L_wired_table_cell_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wired_table_cell_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wired_table_cell_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wireless_table_cell_det</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/table_cells_detection/RT-DETR-L_wireless_table_cell_det.yaml">RT-DETR-L_wireless_table_cell_det.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wireless_table_cell_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wireless_table_cell_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
<p><b>注：以上精度指标测量自 PaddleX 内部自建表格单元格检测数据集。</b></p>
|
||||
|
||||
## [表格分类模块](./module_usage/table_classification.md)
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>Top1 Acc(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_table_cls</td>
|
||||
<td>94.2</td>
|
||||
<td>2.62 / 0.60</td>
|
||||
<td>3.17 / 1.14</td>
|
||||
<td>6.6</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/table_classification/PP-LCNet_x1_0_table_cls.yaml">PP-LCNet_x1_0_table_cls.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_table_cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_table_cls_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
<p><b>注：以上精度指标测量自 PaddleX 内部自建表格分类数据集。</b></p>
|
||||
|
||||
## [文本图像矫正模块](./module_usage/text_image_unwarping.md)
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型名称</th>
|
||||
<th>CER</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml 文件</th>
|
||||
<th>模型下载链接</th></tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>UVDoc</td>
|
||||
<td>0.179</td>
|
||||
<td>19.05 / 19.05</td>
|
||||
<td>- / 869.82</td>
|
||||
<td>30.3</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/image_unwarping/UVDoc.yaml">UVDoc.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UVDoc_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UVDoc_pretrained.pdparams">训练模型</a></td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<b>注：以上精度指标测量自 </b><b>PaddleX自建的图像矫正数据集</b><b>。</b>
|
||||
|
||||
## [版面区域检测模块](./module_usage/layout_detection.md)
|
||||
|
||||
* <b>版面检测模型，包含20个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、表格、图和表标题（图标题、表格标题和图表标题）、印章、图表、侧栏文本和参考文献内容</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td>
|
||||
<td>83.2</td>
|
||||
<td>53.03 / 17.23</td>
|
||||
<td>634.62 / 378.32</td>
|
||||
<td>126.01 </td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PP-DocLayout_plus-L.yaml">PP-DocLayout_plus-L.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。</b>
|
||||
|
||||
* <b>文档图像版面子模块检测，包含1个 版面区域 类别，能检测多栏的报纸、杂志的每个子文章的文本区域:</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocBlockLayout</td>
|
||||
<td>95.9</td>
|
||||
<td>34.60 / 28.54</td>
|
||||
<td>506.43 / 256.83</td>
|
||||
<td>123.92</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PP-DocBlockLayout.yaml">PP-DocBlockLayout.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBlockLayout_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocBlockLayout_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。</b>
|
||||
|
||||
|
||||
* <b>版面检测模型，包含23个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、图表标题、表格、表格标题、印章、图表标题、图表、页眉图像、页脚图像、侧栏文本</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout-L</td>
|
||||
<td>90.4</td>
|
||||
<td>33.59 / 33.59</td>
|
||||
<td>503.01 / 251.08</td>
|
||||
<td>123.76 </td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PP-DocLayout-L.yaml">PP-DocLayout-L.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-L_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-M</td>
|
||||
<td>75.2</td>
|
||||
<td>13.03 / 4.72</td>
|
||||
<td>43.39 / 24.44</td>
|
||||
<td>22.578</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PP-DocLayout-M.yaml">PP-DocLayout-M.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-M_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-M_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-S</td>
|
||||
<td>70.9</td>
|
||||
<td>11.54 / 3.86</td>
|
||||
<td>18.53 / 6.29</td>
|
||||
<td>4.834</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PP-DocLayout-S.yaml">PP-DocLayout-S.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-S_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 500 张文档类型图片。</b>
|
||||
|
||||
* <b>表格版面检测模型</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x_table</td>
|
||||
<td>97.5</td>
|
||||
<td>9.57 / 6.63</td>
|
||||
<td>27.66 / 16.75</td>
|
||||
<td>7.4</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PicoDet_layout_1x_table.yaml">PicoDet_layout_1x_table.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_table_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_table_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
<b>注：以上精度指标的评估集是 PaddleOCR 自建的版面表格区域检测数据集，包含中英文 7835 张带有表格的论文文档类型图片。</b>
|
||||
|
||||
* <b>3类版面检测模型，包含表格、图像、印章</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_3cls</td>
|
||||
<td>88.2</td>
|
||||
<td>8.43 / 3.44</td>
|
||||
<td>17.60 / 6.51</td>
|
||||
<td>4.8</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PicoDet-S_layout_3cls.yaml">PicoDet-S_layout_3cls.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_3cls</td>
|
||||
<td>89.0</td>
|
||||
<td>12.80 / 9.57</td>
|
||||
<td>45.04 / 23.86</td>
|
||||
<td>22.6</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PicoDet-L_layout_3cls.yaml">PicoDet-L_layout_3cls.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_3cls</td>
|
||||
<td>95.8</td>
|
||||
<td>114.80 / 25.65</td>
|
||||
<td>924.38 / 924.38</td>
|
||||
<td>470.1</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/RT-DETR-H_layout_3cls.yaml">RT-DETR-H_layout_3cls.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
<b>注：以上精度指标的评估集是 PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 1154 张文档类型图片。</b>
|
||||
|
||||
* <b>5类英文文档区域检测模型，包含文字、标题、表格、图片以及列表</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x</td>
|
||||
<td>97.8</td>
|
||||
<td>9.62 / 6.75</td>
|
||||
<td>26.96 / 12.77</td>
|
||||
<td>7.4</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PicoDet_layout_1x.yaml">PicoDet_layout_1x.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
<b>注：以上精度指标的评估集是 [PubLayNet](https://developer.ibm.com/exchanges/data/all/publaynet/) 的评估数据集，包含英文文档的 11245 张图片。</b>
|
||||
|
||||
* <b>17类区域检测模型，包含17个版面常见类别，分别是：段落标题、图片、文本、数字、摘要、内容、图表标题、公式、表格、表格标题、参考文献、文档标题、脚注、页眉、算法、页脚、印章</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_17cls</td>
|
||||
<td>87.4</td>
|
||||
<td>8.80 / 3.62</td>
|
||||
<td>17.51 / 6.35</td>
|
||||
<td>4.8</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PicoDet-S_layout_17cls.yaml">PicoDet-S_layout_17cls.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_17cls</td>
|
||||
<td>89.0</td>
|
||||
<td>12.60 / 10.27</td>
|
||||
<td>43.70 / 24.42</td>
|
||||
<td>22.6</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/PicoDet-L_layout_17cls.yaml">PicoDet-L_layout_17cls.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_17cls</td>
|
||||
<td>98.3</td>
|
||||
<td>115.29 / 101.18</td>
|
||||
<td>964.75 / 964.75</td>
|
||||
<td>470.2</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/layout_detection/RT-DETR-H_layout_17cls.yaml">RT-DETR-H_layout_17cls.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<b>注：以上精度指标的评估集是 PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 892 张文档类型图片。</b>
|
||||
|
||||
## [文档图像方向分类模块](./module_usage/doc_img_orientation_classification.md)
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>Top-1 Acc（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_doc_ori</td>
|
||||
<td>99.06</td>
|
||||
<td>2.62 / 0.59</td>
|
||||
<td>3.24 / 1.19</td>
|
||||
<td>7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/doc_text_orientation/PP-LCNet_x1_0_doc_ori.yaml">PP-LCNet_x1_0_doc_ori.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_doc_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_doc_ori_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<b>注：以上精度指标的评估集是自建的数据集，覆盖证件和文档等多个场景，包含 1000 张图片。</b>
|
||||
|
||||
|
||||
## [文本行方向分类模块](./module_usage/doc_img_orientation_classification.md)
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>Top-1 Acc（%）</th>
|
||||
<th>GPU推理耗时（ms）<br>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_doc_ori</td>
|
||||
<td>99.06</td>
|
||||
<td>2.62 / 0.59</td>
|
||||
<td>3.24 / 1.19</td>
|
||||
<td>7</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/textline_orientation/PP-LCNet_x0_25_textline_ori.yaml">PP-LCNet_x0_25_textline_ori.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_doc_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_doc_ori_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的数据集，覆盖证件和文档等多个场景，包含 1000 张图片。</b>
|
||||
|
||||
## [文档类视觉语言模型模块](./module_usage/doc_vlm.md)
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>模型参数尺寸（B）</th>
|
||||
<th>模型存储大小（GB）</th>
|
||||
<th>yaml文件</th>
|
||||
<th>模型下载链接</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-2B</td>
|
||||
<td>2</td>
|
||||
<td>4.2</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/doc_vlm/PP-DocBee-2B.yaml">PP-DocBee-2B.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-2B_infer.tar">推理模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-7B</td>
|
||||
<td>7</td>
|
||||
<td>15.8</td>
|
||||
<td><a href="https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/doc_vlm/PP-DocBee-7B.yaml">PP-DocBee-7B.yaml</a></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-7B_infer.tar">推理模型</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee2-3B</td>
|
||||
<td>3</td>
|
||||
<td>7.6</td>
|
||||
<td></td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee2-3B_infer.tar">推理模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
@@ -0,0 +1,327 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Document Image Orientation Classification Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The Document Image Orientation Classification Module is primarily designed to distinguish the orientation of document images and correct them through post-processing. During processes such as document scanning or ID photo capturing, the device might be rotated to achieve clearer images, resulting in images with various orientations. Standard OCR pipelines may not handle these images effectively. By leveraging image classification techniques, the orientation of documents or IDs containing text regions can be pre-determined and adjusted, thereby improving the accuracy of OCR processing.
|
||||
|
||||
## 2. Supported Models List
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Top-1 Acc (%)</th>
|
||||
<th>GPU Inference Time (ms)<br>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_doc_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_doc_ori_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_doc_ori_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>99.06</td>
|
||||
<td>2.62 / 0.59</td>
|
||||
<td>3.24 / 1.19</td>
|
||||
<td>7</td>
|
||||
<td>A document image classification model based on PP-LCNet_x1_0, with four categories: 0°, 90°, 180°, and 270°.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> Self-built multi-scenario dataset (1000 images, including ID/document scenarios)</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Optimal backend selected (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting, please install the PaddleOCR wheel package. For details, refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference of the Document Image Orientation Classification Module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg) to your local machine.
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(model_name="PP-LCNet_x1_0_doc_ori")
|
||||
output = model.predict("img_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result will be:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'img_rot180_demo.jpg', 'page_index': None, 'class_ids': array([2], dtype=int32), 'scores': array([0.88164], dtype=float32), 'label_names': ['180']}}
|
||||
```
|
||||
|
||||
The meaning of the output parameters is as follows:
|
||||
- `input_path`: Represents the path of the input image.
|
||||
- `class_ids`: Represents the predicted class ID, with four categories: 0°, 90°, 180°, and 270°.
|
||||
```- `scores`: Represents the confidence level of the prediction result.
|
||||
- `label_names`: Represents the category names of the prediction results.
|
||||
|
||||
Here is the visualization of the image:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/doc_img_ori_classification/img_rot180_demo_res.jpg">
|
||||
|
||||
The explanations of relevant methods and parameters are as follows:
|
||||
|
||||
* Instantiate the document image orientation classification model with `DocImgOrientationClassification` (taking `PP-LCNet_x1_0_doc_ori` as an example here). The specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-LCNet_x1_0_doc_ori</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b><code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the document image orientation classification model for inference prediction. This method will return a list of results. In addition, this module also provides the `predict_iter()` method. The two methods are completely consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for scenarios where large datasets need to be processed or memory needs to be saved. You can choose either of these two methods according to your actual needs. The parameters of the `predict()` method are `input` and `batch_size`, and the specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is the corresponding Result object, and it supports operations such as printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data and make it more readable. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; when set to <code>False</code>, the original characters will be retained. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a file in <code>json</code> format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data and make it more readable. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; when set to <code>False</code>, the original characters will be retained. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as a file in image format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining the visualization image with results and the prediction results through attributes. The specifics are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualization image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## IV. Secondary Development
|
||||
|
||||
Since PaddleOCR does not directly provide training functionality for document image orientation classification, if you need to train a document image orientation classification model, you can refer to the [PaddleX Secondary Development for Document Image Orientation Classification](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html#iv-custom-development) section for training guidance. The trained model can be seamlessly integrated into PaddleOCR's API for inference purposes.
|
||||
|
||||
## V. FAQ
|
||||
@@ -0,0 +1,326 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文档图像方向分类模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文档图像方向分类模块主要是将文档图像的方向区分出来，并使用后处理将其矫正。在诸如文档扫描、证照拍摄等过程中，有时为了拍摄更清晰，会将拍摄设备进行旋转，导致得到的图片也是不同方向的。此时，标准的OCR流程无法很好地应对这些数据。利用图像分类技术，可以预先判断含文字区域的文档或证件的方向，并将其进行方向调整，从而提高OCR处理的准确性。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>Top-1 Acc（%）</th>
|
||||
<th>GPU推理耗时（ms）<br>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_doc_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_doc_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_doc_ori_pretrained.pdparams">训练模型</a></td>
|
||||
<td>99.06</td>
|
||||
<td>2.62 / 0.59</td>
|
||||
<td>3.24 / 1.19</td>
|
||||
<td>7</td>
|
||||
<td>基于PP-LCNet_x1_0的文档图像分类模型，含有四个类别，即0度，90度，180度，270度</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建多场景数据集（1000张图片，含证件/文档等场景）</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg
|
||||
```
|
||||
|
||||
您也可以将文档图像方向分类模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(model_name="PP-LCNet_x1_0_doc_ori")
|
||||
output = model.predict("img_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'img_rot180_demo.jpg', 'page_index': None, 'class_ids': array([2], dtype=int32), 'scores': array([0.88164], dtype=float32), 'label_names': ['180']}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入图片的路径。
|
||||
- `class_ids`：表示预测结果的类别id，含有四个类别，即0度，90度，180度和270度。
|
||||
- `scores`：表示预测结果的置信度。
|
||||
- `label_names`：表示预测结果的类别名。
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/doc_img_ori_classification/img_rot180_demo_res.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `DocImgOrientationClassification`实例化文档图像方向分类模型（此处以`PP-LCNet_x1_0_doc_ori`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>说明</th>
|
||||
<th>类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-LCNet_x1_0_doc_ori</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文档图像方向分类模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供文档图像方向分类的训练，因此，如果需要训练文档图像方向分类模型，可以参考 [PaddleX 文档图像方向分类二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 五、FAQ
|
||||
238
docs/version3.x/module_usage/doc_vlm.en.md
Normal file
238
docs/version3.x/module_usage/doc_vlm.en.md
Normal file
@@ -0,0 +1,238 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Document Visual Language Model Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
|
||||
Document visual language models are a cutting-edge multimodal processing technology aimed at addressing the limitations of traditional document processing methods. Traditional methods are often limited to processing document information in specific formats or predefined categories, whereas document visual language models can integrate visual and linguistic information to understand and handle diverse document content. By combining computer vision and natural language processing technologies, these models can recognize images, text, and their relationships within documents, and even understand semantic information within complex layout structures. This makes document processing more intelligent and flexible, with stronger generalization capabilities, showing broad application prospects in automated office work, information extraction, and other fields.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Model Storage Size (GB)</th>
|
||||
<th>Total Score</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-2B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-2B_infer.tar">Inference Model</a></td>
|
||||
<td>4.2</td>
|
||||
<td>765</td>
|
||||
<td rowspan="2">PP-DocBee is a self-developed multimodal large model by the PaddlePaddle team, focusing on document understanding, and it performs excellently in Chinese document understanding tasks. The model is fine-tuned and optimized using nearly 5 million multimodal datasets for document understanding, including general VQA, OCR, charts, text-rich documents, mathematics and complex reasoning, synthetic data, and pure text data, with different training data ratios set. On several authoritative English document understanding evaluation lists in academia, PP-DocBee has basically achieved SOTA for models of the same parameter scale. In terms of internal business Chinese scenario indicators, PP-DocBee also outperforms the current popular open-source and closed-source models.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-7B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-7B_infer.tar">Inference Model</a></td>
|
||||
<td>15.8</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee2-3B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee2-3B_infer.tar">Inference Model</a></td>
|
||||
<td>7.6</td>
|
||||
<td>852</td>
|
||||
<td>PP-DocBee2 is a self-developed multimodal large model by the PaddlePaddle team, further optimizing the base model on the foundation of PP-DocBee and introducing a new data optimization scheme to improve data quality. Using a small amount of 470,000 data generated by a self-developed data synthesis strategy, PP-DocBee2 performs better in Chinese document understanding tasks. In terms of internal business Chinese scenario indicators, PP-DocBee2 improves by about 11.4% compared to PP-DocBee, and also outperforms the current popular open-source and closed-source models of the same scale.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<b>Note: The total scores of the above models are test results from an internal evaluation set, where all images have a resolution (height, width) of (1680, 1204), with a total of 1196 data entries, covering scenarios such as financial reports, laws and regulations, scientific and technical papers, manuals, humanities papers, contracts, research reports, etc. There are no plans for public release at the moment.</b>
|
||||
|
||||
## III. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please install the PaddleOCR wheel package. For details, please refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one line of command:
|
||||
|
||||
```bash
|
||||
paddleocr doc_vlm -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出'}"
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the open document visual language model module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import DocVLM
|
||||
model = DocVLM(model_name="PP-DocBee2-3B")
|
||||
results = model.predict(
|
||||
input={"image": "medal_table.png", "query": "识别这份表格的内容, 以markdown格式输出"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出', 'result': '| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n'}}
|
||||
```
|
||||
|
||||
The meaning of the result parameters is as follows:
|
||||
- `image`: Indicates the path of the input image to be predicted
|
||||
- `query`: Represents the input text information to be predicted
|
||||
- `result`: Information of the model's prediction result
|
||||
|
||||
The visualization of the prediction result is as follows:
|
||||
|
||||
```bash
|
||||
| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |
|
||||
| 2 | 美国（USA） | 36 | 39 | 37 | 112 |
|
||||
| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |
|
||||
| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |
|
||||
| 5 | 德国（GER） | 16 | 11 | 14 | 41 |
|
||||
| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |
|
||||
| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |
|
||||
| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |
|
||||
| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |
|
||||
| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |
|
||||
| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |
|
||||
| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |
|
||||
| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |
|
||||
| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |
|
||||
| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |
|
||||
```
|
||||
|
||||
Explanations of related methods, parameters, etc., are as follows:
|
||||
|
||||
* `DocVLM` instantiates the document visual language model (taking `PP-DocBee-2B` as an example), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-DocBee-2B</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the document visual language model for inference prediction. This method will return a result list. Additionally, this module also provides the `predict_iter()` method. Both are completely consistent in terms of parameter acceptance and result return, the difference being that `predict_iter()` returns a `generator`, capable of gradually processing and obtaining prediction results, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods based on actual needs. The `predict()` method parameters include `input`, `batch_size`, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data. Required. Since multimodal models have different input requirements, please refer to the specific model for the correct format.<br/>
|
||||
For example, for the PP-DocBee series models, the input format should be: <code>{'image': image_path, 'query': query_text}</code>
|
||||
</td>
|
||||
<td><code>dict</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is the corresponding Result object, and it supports operations such as printing and saving as `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print results to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path of the file to be saved. When it is a directory, the naming of the saved file is consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, it also supports obtaining prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## IV. Secondary Development
|
||||
|
||||
The current module does not support fine-tuning training temporarily, only inference integration is supported. The fine-tuning training of this module is planned to be supported in the future.
|
||||
|
||||
## V. FAQ
|
||||
241
docs/version3.x/module_usage/doc_vlm.md
Normal file
241
docs/version3.x/module_usage/doc_vlm.md
Normal file
@@ -0,0 +1,241 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文档类视觉语言模型模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文档类视觉语言模型是当前一种前沿的多模态处理技术，旨在解决传统文档处理方法的局限性。传统方法往往局限于处理特定格式或预定义类别的文档信息，而文档类视觉语言模型能够融合视觉与语言信息，理解并处理多样化的文档内容。通过结合计算机视觉与自然语言处理技术，模型可以识别文档中的图像、文本及其相互关系，甚至能理解复杂版面结构中的语义信息。这使得文档处理更加智能化、灵活化，具备更强的泛化能力，在自动化办公、信息提取等领域展现出广阔的应用前景。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>模型存储大小（GB）</th>
|
||||
<th>模型总分</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-2B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-2B_infer.tar">推理模型</a></td>
|
||||
<td>4.2</td>
|
||||
<td>765</td>
|
||||
<td rowspan="2">PP-DocBee 是飞桨团队自研的一款专注于文档理解的多模态大模型，在中文文档理解任务上具有卓越表现。该模型通过近 500 万条文档理解类多模态数据集进行微调优化，各种数据集包括了通用VQA类、OCR类、图表类、text-rich文档类、数学和复杂推理类、合成数据类、纯文本数据等，并设置了不同训练数据配比。在学术界权威的几个英文文档理解评测榜单上，PP-DocBee基本都达到了同参数量级别模型的SOTA。在内部业务中文场景类的指标上，PP-DocBee也高于目前的热门开源和闭源模型。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-7B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-7B_infer.tar">推理模型</a></td>
|
||||
<td>15.8</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee2-3B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee2-3B_infer.tar">推理模型</a></td>
|
||||
<td>7.6</td>
|
||||
<td>852</td>
|
||||
<td>PP-DocBee2 是飞桨团队自研的一款专注于文档理解的多模态大模型，在PP-DocBee的基础上进一步优化了基础模型，并引入了新的数据优化方案，提高了数据质量，使用自研数据合成策略生成的少量的47万数据便使得PP-DocBee2在中文文档理解任务上表现更佳。在内部业务中文场景类的指标上，PP-DocBee2相较于PP-DocBee提升了约11.4%，同时也高于目前的同规模热门开源和闭源模型。</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<b>注：以上模型总分为内部评估集模型测试结果，内部评估集所有图像分辨率 (height, width) 为 (1680,1204)，共1196条数据，包括了财报、法律法规、理工科论文、说明书、文科论文、合同、研报等场景，暂时未有计划公开。</b>
|
||||
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr doc_vlm -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出'}"
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将开放文档类视觉语言模型模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import DocVLM
|
||||
model = DocVLM(model_name="PP-DocBee2-3B")
|
||||
results = model.predict(
|
||||
input={"image": "medal_table.png", "query": "识别这份表格的内容, 以markdown格式输出"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出', 'result': '| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n'}}
|
||||
```
|
||||
运行结果参数含义如下：
|
||||
- `image`: 表示输入待预测图像的路径
|
||||
- `query`: 表述输入待预测的文本信息
|
||||
- `result`: 模型预测的结果信息
|
||||
|
||||
预测结果打印可视化如下：
|
||||
|
||||
```bash
|
||||
| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |
|
||||
| 2 | 美国（USA） | 36 | 39 | 37 | 112 |
|
||||
| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |
|
||||
| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |
|
||||
| 5 | 德国（GER） | 16 | 11 | 14 | 41 |
|
||||
| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |
|
||||
| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |
|
||||
| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |
|
||||
| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |
|
||||
| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |
|
||||
| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |
|
||||
| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |
|
||||
| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |
|
||||
| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |
|
||||
| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |
|
||||
```
|
||||
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `DocVLM`实例化文档类视觉语言模型（此处以`PP-DocBee-2B`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>>模型名称。如果设置为<code>None</code>，则使用<code>PP-DocBee-2B</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code></code>。
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文档类视觉语言模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 、 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，必填。由于多模态模型对输入要求不同，请根据具体模型设定输入格式。<br/>
|
||||
例如：对于 PP-DocBee 系列模型，输入形式应为：<code>{'image': image_path, 'query': query_text}</code>
|
||||
</td>
|
||||
<td><code>dict</code></td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
当前模块暂时不支持微调训练，仅支持推理集成。关于该模块的微调训练，计划在未来支持。
|
||||
|
||||
## 五、FAQ
|
||||
473
docs/version3.x/module_usage/formula_recognition.en.md
Normal file
473
docs/version3.x/module_usage/formula_recognition.en.md
Normal file
@@ -0,0 +1,473 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Formula Recognition Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
|
||||
The formula recognition module is a key component of an OCR (Optical Character Recognition) system, responsible for converting mathematical formulas in images into editable text or computer-readable formats. The performance of this module directly affects the accuracy and efficiency of the entire OCR system. The formula recognition module typically outputs LaTeX or MathML code of the mathematical formulas, which will be passed as input to the text understanding module for further processing.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>En-BLEU(%)</th>
|
||||
<th>Zh-BLEU(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UniMERNet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UniMERNet_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UniMERNet_pretrained.pdparams">Training Model</a></td>
|
||||
<td>85.91</td>
|
||||
<td>43.50</td>
|
||||
<td>1311.84 / 1311.84</td>
|
||||
<td>- / 8288.07</td>
|
||||
<td>1530</td>
|
||||
<td>UniMERNet is a formula recognition model developed by Shanghai AI Lab. It uses Donut Swin as the encoder and MBartDecoder as the decoder. The model is trained on a dataset of one million samples, including simple formulas, complex formulas, scanned formulas, and handwritten formulas, significantly improving the recognition accuracy of real-world formulas.</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>87.00</td>
|
||||
<td>45.71</td>
|
||||
<td>182.25 / 182.25</td>
|
||||
<td>- / 254.39</td>
|
||||
<td>224</td>
|
||||
<td rowspan="2">PP-FormulaNet is an advanced formula recognition model developed by the Baidu PaddlePaddle Vision Team. The PP-FormulaNet-S version uses PP-HGNetV2-B4 as its backbone network. Through parallel masking and model distillation techniques, it significantly improves inference speed while maintaining high recognition accuracy, making it suitable for applications requiring fast inference. The PP-FormulaNet-L version, on the other hand, uses Vary_VIT_B as its backbone network and is trained on a large-scale formula dataset, showing significant improvements in recognizing complex formulas compared to PP-FormulaNet-S.</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>90.36</td>
|
||||
<td>45.78</td>
|
||||
<td>1482.03 / 1482.03</td>
|
||||
<td>- / 3131.54</td>
|
||||
<td>695</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet_plus-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>88.71</td>
|
||||
<td>53.32</td>
|
||||
<td>179.20 / 179.20</td>
|
||||
<td>- / 260.99</td>
|
||||
<td>248</td>
|
||||
<td rowspan="3">PP-FormulaNet_plus is an enhanced version of the formula recognition model developed by the Baidu PaddlePaddle Vision Team, building upon the original PP-FormulaNet. Compared to the original version, PP-FormulaNet_plus utilizes a more diverse formula dataset during training, including sources such as Chinese dissertations, professional books, textbooks, exam papers, and mathematics journals. This expansion significantly improves the model’s recognition capabilities. Among the models, PP-FormulaNet_plus-M and PP-FormulaNet_plus-L have added support for Chinese formulas and increased the maximum number of predicted tokens for formulas from 1,024 to 2,560, greatly enhancing the recognition performance for complex formulas. Meanwhile, the PP-FormulaNet_plus-S model focuses on improving the recognition of English formulas. With these improvements, the PP-FormulaNet_plus series models perform exceptionally well in handling complex and diverse formula recognition tasks. </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-M_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-M_pretrained.pdparams">Training Model</a></td>
|
||||
<td>91.45</td>
|
||||
<td>89.76</td>
|
||||
<td>1040.27 / 1040.27</td>
|
||||
<td>- / 1615.80</td>
|
||||
<td>592</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>92.22</td>
|
||||
<td>90.64</td>
|
||||
<td>1476.07 / 1476.07</td>
|
||||
<td>- / 3125.58</td>
|
||||
<td>698</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LaTeX_OCR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/LaTeX_OCR_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/LaTeX_OCR_rec_pretrained.pdparams">Training Model</a></td>
|
||||
<td>74.55</td>
|
||||
<td>39.96</td>
|
||||
<td>1088.89 / 1088.89</td>
|
||||
<td>- / -</td>
|
||||
<td>99</td>
|
||||
<td>LaTeX-OCR is a formula recognition algorithm based on an autoregressive large model. It uses Hybrid ViT as the backbone network and a transformer as the decoder, significantly improving the accuracy of formula recognition.</td>
|
||||
</tr>
|
||||
</table>
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> PaddleOCR internal custom formula recognition test set</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technique Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Standard Mode</td>
|
||||
<td>FP32 precision / No TRT acceleration</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of predefined precision type and acceleration strategy</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>Optimal predefined backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## III. Quick Start
|
||||
|
||||
> ❗ Before getting started, please install the PaddleOCR wheel package. For details, refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly try it out with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr formula_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the formula recognition module into your own project.Before running the code below, please download the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import FormulaRecognition
|
||||
model = FormulaRecognition(model_name="PP-FormulaNet_plus-M")
|
||||
output = model.predict(input="general_formula_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
After running, the output is:
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_formula_rec_001.png', 'page_index': None, 'rec_formula': '\\zeta_{0}(\\nu)=-\\frac{\\nu\\varrho^{-2\\nu}}{\\pi}\\int_{\\mu}^{\\infty}d\\omega\\int_{C_{+}}d z\\frac{2z^{2}}{(z^{2}+\\omega^{2})^{\\nu+1}}\\breve{\\Psi}(\\omega;z)e^{i\\epsilon z}\\quad,'}}
|
||||
```
|
||||
Explanation of the result parameters:
|
||||
|
||||
- `input_path`： Indicates the path to the input formula image to be predicted
|
||||
- `page_index`： If the input is a PDF file, this represents the page number; otherwise, it is None
|
||||
- `rec_formula`：Indicates the predicted LaTeX source code of the formula image
|
||||
The visualization image is as follows. The left side is the input formula image, and the right side is the rendered formula from the prediction:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/formula_recog/general_formula_rec_001_res_paddleocr3.png">
|
||||
|
||||
<b>Note: If you need to visualize the formula recognition module, you must install the LaTeX rendering environment by running the following command. Currently, visualization is only supported on Ubuntu. Other environments are not supported for now. For complex formulas, the LaTeX result may contain advanced representations that may not render successfully in Markdown or similar environments:</b>
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install texlive texlive-latex-base texlive-xetex latex-cjk-all texlive-latex-extra -y
|
||||
```
|
||||
|
||||
Related methods and parameter descriptions are as follows:
|
||||
|
||||
* `FormulaRecognition` instantiates the formula recognition model (here using `PP-FormulaNet_plus-M` as an example), with detailed description as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-FormulaNet_plus-M</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b><code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the formula recognition model to perform inference, which returns a result list.
|
||||
Additionally, this module provides the `predict_iter()` method. Both accept the same parameters and return the same result format.
|
||||
The difference is that `predict_iter()` returns a `generator`, which can process and retrieve results step-by-step, suitable for large datasets or memory-efficient scenarios.
|
||||
|
||||
You can choose either method based on your actual needs. The `predict()` method takes parameters `input` and `batch_size`, described as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* The prediction results can be processed. Each result corresponds to a `Result` object, which supports printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Details</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the <code>JSON</code> output; only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. If set to <code>True</code>, all non-ASCII characters are escaped; if <code>False</code>, original characters are kept. Only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json-formatted file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path to save the file. If it is a directory, the saved file name will match the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the <code>JSON</code> output; only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. If set to <code>True</code>, all non-ASCII characters are escaped; if <code>False</code>, original characters are kept. Only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as an image file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path to save the file. If it is a directory, the saved file name will match the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, you can also access the visualized image and prediction result via attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
If the models above do not perform well in your scenario, you can try the following steps for custom development.
|
||||
Here we take training `PP-FormulaNet_plus-M` as an example. For other models, just replace the corresponding config file. First, you need to prepare a formula recognition dataset. You can follow the format of the [formula recognition demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar). Once the data is ready, follow the steps below to train and export the model. After export, the model can be quickly integrated into the API described above. This example uses the demo dataset. Before training the model, please ensure you have installed all PaddleOCR dependencies as described in the [installation documentation](../installation.en.md).
|
||||
|
||||
### 4.1 Environment Setup
|
||||
|
||||
To train the formula recognition model, you need to install additional Python and Linux dependencies. Run the following commands:
|
||||
|
||||
```shell
|
||||
sudo apt-get update
|
||||
sudo apt-get install libmagickwand-dev
|
||||
pip install tokenizers==0.19.1 imagesize ftfy Wand
|
||||
```
|
||||
|
||||
### 4.2 Dataset and Pretrained Model Preparation
|
||||
|
||||
#### 4.2.1 Prepare the Dataset
|
||||
```shell
|
||||
# Download the demo dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar
|
||||
tar -xf ocr_rec_latexocr_dataset_example.tar
|
||||
```
|
||||
|
||||
#### 4.2.2 Download the Pretrained Model
|
||||
```shell
|
||||
# Download the PP-FormulaNet_plus-M pre-trained model
|
||||
wget https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_plus_m_train.tar
|
||||
tar -xf rec_ppformulanet_plus_m_train.tar
|
||||
```
|
||||
|
||||
### 4.3 Model Training
|
||||
PaddleOCR is modularized. To train the `PP-FormulaNet_plus-M` model, you need to use its [config file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml).
|
||||
|
||||
Training commands are as follows:
|
||||
```bash
|
||||
# Single GPU training (default)
|
||||
python3 tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
|
||||
# Multi-GPU training, specify GPU IDs with --gpus
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
|
||||
```
|
||||
**Note:**
|
||||
|
||||
- By default, evaluation is performed every 1 epoch.If you change the batch size or dataset, modify the following accordingly:
|
||||
```bash
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.eval_batch_step=[0,{length_of_dataset//batch_size//4}] \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.4 Model Evaluation
|
||||
You can evaluate trained weights, e.g., output/xxx/xxx.pdparams, or use the downloaded [model](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_plus_m_train.tar ) with the following command:
|
||||
|
||||
```bash
|
||||
# Make sure pretrained_model is set to the local path.
|
||||
# For custom-trained models, modify the path and file name as {path/to/weights}/{model_name}
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.5 Model Export
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-FormulaNet_plus-M_infer/"
|
||||
```
|
||||
|
||||
After exporting, the static graph model will be saved in `./PP-FormulaNet_plus-M_infer/`, and you will see the following files:
|
||||
```
|
||||
./PP-FormulaNet_plus-M_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
At this point, the secondary development is complete. This static graph model can be directly integrated into the PaddleOCR API.
|
||||
|
||||
## V. FAQ
|
||||
|
||||
**Q1: Which formula recognition model does PaddleOCR recommend?**
|
||||
|
||||
A1: It is recommended to use the PP-FormulaNet series.
|
||||
If your scenario is mainly in English and inference speed is not a concern, use PP-FormulaNet-L or PP-FormulaNet_plus-L.
|
||||
For mainly Chinese use cases, use PP-FormulaNet_plus-L or PP-FormulaNet_plus-M.
|
||||
If your device has limited computing power and you are working with English formulas, use PP-FormulaNet-S.
|
||||
|
||||
**Q2: Why does the inference report an error?**
|
||||
A2: The formula recognition model depends heavily on Paddle 3.0 official release.
|
||||
Please ensure the correct version is installed.
|
||||
|
||||
**Q3: Why is there no visualization image after prediction?**
|
||||
A3: This may be due to LaTeX not being installed.You need to refer to Section III and install the LaTeX rendering tools.
|
||||
476
docs/version3.x/module_usage/formula_recognition.md
Normal file
476
docs/version3.x/module_usage/formula_recognition.md
Normal file
@@ -0,0 +1,476 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 公式识别模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
公式识别模块是OCR（光学字符识别）系统中的关键组成部分，负责将图像中的数学公式转换为可编辑的文本或计算机可识别的格式。该模块的性能直接影响到整个OCR系统的准确性和效率。公式识别模块通常会输出数学公式的 LaTeX 或 MathML 代码，这些代码将作为输入传递给文本理解模块进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>En-BLEU(%)</th>
|
||||
<th>Zh-BLEU(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UniMERNet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UniMERNet_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UniMERNet_pretrained.pdparams">训练模型</a></td>
|
||||
<td>85.91</td>
|
||||
<td>43.50</td>
|
||||
<td>1311.84 / 1311.84</td>
|
||||
<td>- / 8288.07</td>
|
||||
<td>1530</td>
|
||||
<td>UniMERNet是由上海AI Lab研发的一款公式识别模型。该模型采用Donut Swin作为编码器，MBartDecoder作为解码器，并通过在包含简单公式、复杂公式、扫描捕捉公式和手写公式在内的一百万数据集上进行训练，大幅提升了模型对真实场景公式的识别准确率</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-S_pretrained.pdparams">训练模型</a></td>
|
||||
<td>87.00</td>
|
||||
<td>45.71</td>
|
||||
<td>182.25 / 182.25</td>
|
||||
<td>- / 254.39</td>
|
||||
<td>224</td>
|
||||
<td rowspan="2">PP-FormulaNet 是由百度飞桨视觉团队开发的一款先进的公式识别模型，支持5万个常见LateX源码词汇的识别。PP-FormulaNet-S 版本采用了 PP-HGNetV2-B4 作为其骨干网络，通过并行掩码和模型蒸馏等技术，大幅提升了模型的推理速度，同时保持了较高的识别精度，适用于简单印刷公式、跨行简单印刷公式等场景。而 PP-FormulaNet-L 版本则基于 Vary_VIT_B 作为骨干网络，并在大规模公式数据集上进行了深入训练，在复杂公式的识别方面，相较于PP-FormulaNet-S表现出显著的提升，适用于简单印刷公式、复杂印刷公式、手写公式等场景。 </td>
|
||||
|
||||
</tr>
|
||||
<td>PP-FormulaNet-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>90.36</td>
|
||||
<td>45.78</td>
|
||||
<td>1482.03 / 1482.03</td>
|
||||
<td>- / 3131.54</td>
|
||||
<td>695</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet_plus-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-S_pretrained.pdparams">训练模型</a></td>
|
||||
<td>88.71</td>
|
||||
<td>53.32</td>
|
||||
<td>179.20 / 179.20</td>
|
||||
<td>- / 260.99</td>
|
||||
<td>248</td>
|
||||
<td rowspan="3">PP-FormulaNet_plus 是百度飞桨视觉团队在 PP-FormulaNet 的基础上开发的增强版公式识别模型。与原版相比，PP-FormulaNet_plus 在训练中使用了更为丰富的公式数据集，包括中文学位论文、专业书籍、教材试卷以及数学期刊等多种来源。这一扩展显著提升了模型的识别能力。
|
||||
|
||||
其中，PP-FormulaNet_plus-M 和 PP-FormulaNet_plus-L 模型新增了对中文公式的支持，并将公式的最大预测 token 数从 1024 扩大至 2560，大幅提升了对复杂公式的识别性能。同时，PP-FormulaNet_plus-S 模型则专注于增强英文公式的识别能力。通过这些改进，PP-FormulaNet_plus 系列模型在处理复杂多样的公式识别任务时表现更加出色。 </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-M_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-M_pretrained.pdparams">训练模型</a></td>
|
||||
<td>91.45</td>
|
||||
<td>89.76</td>
|
||||
<td>1040.27 / 1040.27</td>
|
||||
<td>- / 1615.80</td>
|
||||
<td>592</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>92.22</td>
|
||||
<td>90.64</td>
|
||||
<td>1476.07 / 1476.07</td>
|
||||
<td>- / 3125.58</td>
|
||||
<td>698</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LaTeX_OCR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/LaTeX_OCR_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/LaTeX_OCR_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>74.55</td>
|
||||
<td>39.96</td>
|
||||
<td>1088.89 / 1088.89</td>
|
||||
<td>- / -</td>
|
||||
<td>99</td>
|
||||
<td>LaTeX-OCR是一种基于自回归大模型的公式识别算法，通过采用 Hybrid ViT 作为骨干网络，transformer作为解码器，显著提升了公式识别的准确性。</td>
|
||||
</tr>
|
||||
</table>
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>PaddleOCR 内部自建公式识别测试集</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr formula_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将公式识别的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import FormulaRecognition
|
||||
model = FormulaRecognition(model_name="PP-FormulaNet_plus-M")
|
||||
output = model.predict(input="general_formula_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_formula_rec_001.png', 'page_index': None, 'rec_formula': '\\zeta_{0}(\\nu)=-\\frac{\\nu\\varrho^{-2\\nu}}{\\pi}\\int_{\\mu}^{\\infty}d\\omega\\int_{C_{+}}d z\\frac{2z^{2}}{(z^{2}+\\omega^{2})^{\\nu+1}}\\breve{\\Psi}(\\omega;z)e^{i\\epsilon z}\\quad,'}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待预测公式图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `rec_formula`：表示公式图像的预测LaTeX源码
|
||||
|
||||
|
||||
可视化图片如下，左侧是待预测的公式图像，右边是预测的结果渲染后的公式图像：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/formula_recog/general_formula_rec_001_res_paddleocr3.png">
|
||||
|
||||
<b> 注：如果您需要对公式识别模块进行可视化，需要运行如下命令来对LaTeX渲染环境进行安装。目前公式识别模块可视化只支持Ubuntu环境，其他环境暂不支持。对于复杂公式，LaTeX 结果可能包含部分高级的表示，Markdown等环境中未必可以成功显示：</b>
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install texlive texlive-latex-base texlive-xetex latex-cjk-all texlive-latex-extra -y
|
||||
```
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `FormulaRecognition`实例化公式识别模型（此处以`PP-FormulaNet_plus-M`为例），具体说明如下：
|
||||
<table><thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr></thead><tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-FormulaNet_plus-M</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用公式识别模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
如果以上模型在您的场景下效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-FormulaNet-S` 举例，其他模型替换对应配置文件即可。首先，您需要准备公式识别的数据集，可以参考[公式识别 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述API中。此处以公式识别 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
### 4.1 环境配置
|
||||
|
||||
训练公式识别模型需要安装额外的Python依赖和linux依赖，执行如下命令安装：
|
||||
```shell
|
||||
sudo apt-get update
|
||||
sudo apt-get install libmagickwand-dev
|
||||
pip install tokenizers==0.19.1 imagesize ftfy Wand
|
||||
```
|
||||
|
||||
### 4.2 数据集、预训练模型准备
|
||||
|
||||
#### 4.2.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar
|
||||
tar -xf ocr_rec_latexocr_dataset_example.tar
|
||||
```
|
||||
|
||||
#### 4.2.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-FormulaNet_plus-M 预训练模型
|
||||
wget https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_plus_m_train.tar
|
||||
tar -xf rec_ppformulanet_plus_m_train.tar
|
||||
```
|
||||
|
||||
### 4.3 模型训练
|
||||
|
||||
PaddleOCR对代码进行了模块化，训练 `PP-FormulaNet_plus-M` 识别模型时需要使用 `PP-FormulaNet_plus-M` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
**注意：**
|
||||
|
||||
- 默认每训练 1个 epoch 进行 1 次评估，若您更改训练的 batch_size，或更换数据集，请在训练时作出如下修改
|
||||
```bash
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.eval_batch_step=[0,{length_of_dataset//batch_size//4}] \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.4 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，也可以使用已经下载的[模型文件](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_s_train.tar)，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
|
||||
#注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
#demo 测试集评估
|
||||
python3 tools/eval.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.5 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-FormulaNet_plus-M_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-FormulaNet_plus-M_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-FormulaNet_plus-M_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
|
||||
**Q1: PaddleOCR 更推荐哪个公式识别模型？**
|
||||
|
||||
A1: 更推荐使用 PP-FormulaNet 系列模型，如果是英文场景居多且不考虑推理耗时，则可以使用 PP-FormulaNet-L 或者 PP-FormulaNet_plus-L 模型，如果中文场景居多，则可以使用 PP-FormulaNet_plus-L 或者 PP-FormulaNet_plus-M，如果推理设备算力有限且是英文场景，则可以使用 PP-FormulaNet-S。
|
||||
|
||||
**Q2: 为什么推理报错？**
|
||||
|
||||
A2: 公式识别模型的推理强依赖于 Paddle 框架 3.0 正式版，请确保版本一致。
|
||||
|
||||
|
||||
**Q3: 为什么预测后没有可视化图像？**
|
||||
|
||||
A3: 可能是因为没有安装LaTeX导致，您需要参考第三节安装LaTeX渲染工具。
|
||||
638
docs/version3.x/module_usage/layout_detection.en.md
Normal file
638
docs/version3.x/module_usage/layout_detection.en.md
Normal file
@@ -0,0 +1,638 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Layout Detection Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
The core task of structure analysis is to parse and segment the content of input document images. By identifying different elements in the image (such as text, charts, images, etc.), they are classified into predefined categories (e.g., pure text area, title area, table area, image area, list area, etc.), and the position and size of these regions in the document are determined.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
* <b>The layout detection model includes 20 common categories: document title, paragraph title, text, page number, abstract, table, references, footnotes, header, footer, algorithm, formula, formula number, image, table, seal, figure_table title, chart, and sidebar text and lists of references</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>83.2</td>
|
||||
<td>53.03 / 17.23</td>
|
||||
<td>634.62 / 378.32</td>
|
||||
<td>126.01</td>
|
||||
<td>A higher-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>The layout detection model includes 1 category: Block:</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocBlockLayout</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBlockLayout_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocBlockLayout_pretrained.pdparams">Training Model</a></td>
|
||||
<td>95.9</td>
|
||||
<td>34.60 / 28.54</td>
|
||||
<td>506.43 / 256.83</td>
|
||||
<td>123.92</td>
|
||||
<td>A layout block localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>The layout detection model includes 23 common categories: document title, paragraph title, text, page number, abstract, table of contents, references, footnotes, header, footer, algorithm, formula, formula number, image, figure caption, table, table caption, seal, figure title, figure, header image, footer image, and sidebar text</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>90.4</td>
|
||||
<td>33.59 / 33.59</td>
|
||||
<td>503.01 / 251.08</td>
|
||||
<td>123.76</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using RT-DETR-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-M_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-M_pretrained.pdparams">Training Model</a></td>
|
||||
<td>75.2</td>
|
||||
<td>13.03 / 4.72</td>
|
||||
<td>43.39 / 24.44</td>
|
||||
<td>22.578</td>
|
||||
<td>A layout area localization model with balanced precision and efficiency, trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>70.9</td>
|
||||
<td>11.54 / 3.86</td>
|
||||
<td>18.53 / 6.29</td>
|
||||
<td>4.834</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ The above list includes the <b>4 core models</b> that are key supported by the text recognition module. The module actually supports a total of <b>12 full models</b>, including several predefined models with different categories. The complete model list is as follows:
|
||||
|
||||
<details><summary> 👉 Details of Model List</summary>
|
||||
|
||||
* <b>Table Layout Detection Model</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x_table</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_table_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_table_pretrained.pdparams">Training Model</a></td>
|
||||
<td>97.5</td>
|
||||
<td>9.57 / 6.63</td>
|
||||
<td>27.66 / 16.75</td>
|
||||
<td>7.4</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset using PicoDet-1x, capable of detecting table regions.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>3-Class Layout Detection Model, including Table, Image, and Stamp</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>88.2</td>
|
||||
<td>8.43 / 3.44</td>
|
||||
<td>17.60 / 6.51</td>
|
||||
<td>4.8</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.80 / 9.57</td>
|
||||
<td>45.04 / 23.86</td>
|
||||
<td>22.6</td>
|
||||
<td>A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>95.8</td>
|
||||
<td>114.80 / 25.65</td>
|
||||
<td>924.38 / 924.38</td>
|
||||
<td>470.1</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>5-Class English Document Area Detection Model, including Text, Title, Table, Image, and List</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_pretrained.pdparams">Training Model</a></td>
|
||||
<td>97.8</td>
|
||||
<td>9.62 / 6.75</td>
|
||||
<td>26.96 / 12.77</td>
|
||||
<td>7.4</td>
|
||||
<td>A high-efficiency English document layout area localization model trained on the PubLayNet dataset using PicoDet-1x.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>17-Class Area Detection Model, including 17 common layout categories: Paragraph Title, Image, Text, Number, Abstract, Content, Figure Caption, Formula, Table, Table Caption, References, Document Title, Footnote, Header, Algorithm, Footer, and Stamp</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>87.4</td>
|
||||
<td>8.80 / 3.62</td>
|
||||
<td>17.51 / 6.35</td>
|
||||
<td>4.8</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.60 / 10.27</td>
|
||||
<td>43.70 / 24.42</td>
|
||||
<td>22.6</td>
|
||||
<td>A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.3</td>
|
||||
<td>115.29 / 101.18</td>
|
||||
<td>964.75 / 964.75</td>
|
||||
<td>470.2</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong>
|
||||
<ul>
|
||||
<li>20 types of layout detection models: PaddleOCR's self built layout area detection dataset, including Chinese and English papers, magazines, newspapers, research papers PPT、 1300 images of document types such as test papers and textbooks. </li>
|
||||
<li>Type 1 version face region detection model: PaddleOCR's self built version face region detection dataset, including Chinese and English papers, magazines, newspapers, research reports PPT、 1000 document type images such as test papers and textbooks. </li>
|
||||
<li>23 categories Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, containing 500 common document type images such as Chinese and English papers, magazines, contracts, books, exam papers, and research reports.</li>
|
||||
<li>Table Layout Detection Model: A self-built table area detection dataset by PaddleOCR, including 7,835 Chinese and English paper document type images with tables.</li>
|
||||
<li> 3-Class Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, comprising 1,154 common document type images such as Chinese and English papers, magazines, and research reports.</li>
|
||||
<li>5-Class English Document Area Detection Model: The evaluation dataset of <a href="https://developer.ibm.com/exchanges/data/all/publaynet">PubLayNet</a>, containing 11,245 images of English documents.</li>
|
||||
<li>17-Class Area Detection Model: A self-built layout area detection dataset by PaddleOCR, including 892 common document type images such as Chinese and English papers, magazines, and research reports.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
## III. Quick Integration <a id="quick"> </a>
|
||||
|
||||
> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../installation.en.md)。
|
||||
|
||||
Quickly experience with just one command:
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg) Go to the local area.
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayout_plus-L")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}}
|
||||
```
|
||||
|
||||
The meanings of the parameters are as follows:
|
||||
- `input_path`: The path to the input image for prediction.
|
||||
- `page_index`: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is `None`.
|
||||
- `boxes`: Information about the predicted bounding boxes, a list of dictionaries. Each dictionary represents a detected object and contains the following information:
|
||||
- `cls_id`: Class ID, an integer.
|
||||
- `label`: Class label, a string.
|
||||
- `score`: Confidence score of the bounding box, a float.
|
||||
- `coordinate`: Coordinates of the bounding box, a list of floats in the format <code>[xmin, ymin, xmax, ymax]</code>.
|
||||
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/layout_det/layout_res_plus.jpg"/>
|
||||
|
||||
Relevant methods, parameters, and explanations are as follows:
|
||||
|
||||
* `LayoutDetection` instantiates a target detection model (here, `PP-DocLayout_plus-L` is used as an example). The detailed explanation is as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-DocLayout-L</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td>Input image size.<ul>
|
||||
<li><b>int</b>: e.g. <code>640</code>, resizes input image to 640x640.</li>
|
||||
<li><b>list</b>: e.g. <code>[640, 512]</code>, resizes input image to width 640 and height 512.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Threshold for filtering low-confidence predictions.<ul>
|
||||
<li><b>float</b>: e.g. <code>0.2</code>, filters out all boxes with confidence below 0.2.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>float</code> (threshold). For example, <code>{0: 0.45, 2: 0.48, 7: 0.4}</code> means class 0 uses threshold 0.45, class 2 uses 0.48, class 7 uses 0.4.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>Whether to use NMS post-processing to filter overlapping boxes.
|
||||
<ul>
|
||||
<li><b>bool</b>: whether to use NMS for post-processing to filter overlapping boxes.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>Scaling factor for the side length of the detection box.<ul>
|
||||
<li><b>float</b>: A float greater than 0, e.g. <code>1.1</code>, expands width and height by 1.1 times.</li>
|
||||
<li><b>list</b>: e.g. <code>[1.2, 1.5]</code>, expands width by 1.2x and height by 1.5x.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>tuple</code> of two floats (width ratio, height ratio). For example, <code>{0: (1.1, 2.0)}</code> means for class 0, width is expanded by 1.1x and height by 2.0x.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>Merge mode for model output bounding boxes.<ul>
|
||||
<li><b>"large"</b>: Only keep the largest outer box among overlapping boxes, remove inner boxes.</li>
|
||||
<li><b>"small"</b>: Only keep the smallest inner box among overlapping boxes, remove outer boxes.</li>
|
||||
<li><b>"union"</b>: Keep all boxes, no filtering.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>str</code> (mode). For example, <code>{0: "large", 2: "small"}</code> means class 0 uses "large" mode, class 2 uses "small" mode.</li>
|
||||
<li><b>None</b>: Use the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The `predict()` method of the target detection model is called for inference prediction. The parameters of the `predict()` method are `input`, `batch_size`, and `threshold`, which are explained as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Process the prediction results, with each sample's prediction result being the corresponding Result object, and supporting operations such as printing, saving as an image, and saving as a 'json' file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameters</th>
|
||||
<th>Parameter type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Do you want to use <code>JSON</code> indentation formatting for the output content</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non ASCII </code>characters will be escaped; <code>False</code> preserves the original characters and is only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a JSON format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non <code>ASCII</code> characters will be escaped; <code>False</code> preserves the original characters and is only valid when<code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the results as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
* Additionally, it also supports obtaining the visualized image with results and the prediction results via attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the layout detection module, if you need to train the layout area detection model, you can refer to [PaddleX Layout Detection Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/layout_detection.html#iv-custom-development)Partially conduct training. The trained model can be seamlessly integrated into PaddleOCR's API for inference.
|
||||
|
||||
## V. FAQ
|
||||
641
docs/version3.x/module_usage/layout_detection.md
Normal file
641
docs/version3.x/module_usage/layout_detection.md
Normal file
@@ -0,0 +1,641 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 版面区域检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
版面区域检测任务的核心是对输入的文档图像进行内容解析和区域划分。通过识别图像中的不同元素（如文字、图表、图像、公式、段落、摘要、参考文献等），将其归类为预定义的类别，并确定这些区域在文档中的位置。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
* <b>版面检测模型，包含20个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、表格、图和表标题（图标题、表格标题和图表标题）、印章、图表、侧栏文本和参考文献内容</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>83.2</td>
|
||||
<td>53.03 / 17.23</td>
|
||||
<td>634.62 / 378.32</td>
|
||||
<td>126.01</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的更高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。</b>
|
||||
|
||||
* <b>文档图像版面子模块检测，包含1个 版面区域 类别，能检测多栏的报纸、杂志的每个子文章的文本区域：</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocBlockLayout</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBlockLayout_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocBlockLayout_pretrained.pdparams">训练模型</a></td>
|
||||
<td>95.9</td>
|
||||
<td>34.60 / 28.54</td>
|
||||
<td>506.43 / 256.83</td>
|
||||
<td>123.92</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的文档图像版面子模块检测模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。</b>
|
||||
|
||||
* <b>版面检测模型，包含23个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、图表标题、表格、表格标题、印章、图表标题、图表、页眉图像、页脚图像、侧栏文本</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>90.4</td>
|
||||
<td>33.59 / 33.59</td>
|
||||
<td>503.01 / 251.08</td>
|
||||
<td>123.76</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、杂志、合同、书本、试卷和研报等场景的自建数据集训练的高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-M_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-M_pretrained.pdparams">训练模型</a></td>
|
||||
<td>75.2</td>
|
||||
<td>13.03 / 4.72</td>
|
||||
<td>43.39 / 24.44</td>
|
||||
<td>22.578</td>
|
||||
<td>基于PicoDet-L在包含中英文论文、杂志、合同、书本、试卷和研报等场景的自建数据集训练的精度效率平衡的版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-S_pretrained.pdparams">训练模型</a></td>
|
||||
<td>70.9</td>
|
||||
<td>11.54 / 3.86</td>
|
||||
<td>18.53 / 6.29</td>
|
||||
<td>4.834</td>
|
||||
<td>基于PicoDet-S在中英文论文、杂志、合同、书本、试卷和研报等场景上自建数据集训练的高效率版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、报纸、研报和试卷等 500 张文档类型图片。</b>
|
||||
|
||||
> ❗ 以上列出的是版面检测模块重点支持的<b>5个核心模型</b>，该模块总共支持<b>13个全量模型</b>，包含多个预定义了不同类别的模型，完整的模型列表如下：
|
||||
|
||||
<details><summary> 👉模型列表详情</summary>
|
||||
|
||||
* <b>表格版面检测模型</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x_table</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_table_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_table_pretrained.pdparams">训练模型</a></td>
|
||||
<td>97.5</td>
|
||||
<td>9.57 / 6.63</td>
|
||||
<td>27.66 / 16.75</td>
|
||||
<td>7.4</td>
|
||||
<td>基于PicoDet-1x在自建数据集训练的高效率版面区域定位模型，可定位表格这1类区域</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>3类版面检测模型，包含表格、图像、印章</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>88.2</td>
|
||||
<td>8.43 / 3.44</td>
|
||||
<td>17.60 / 6.51</td>
|
||||
<td>4.8</td>
|
||||
<td>基于PicoDet-S轻量模型在中英文论文、杂志和研报等场景上自建数据集训练的高效率版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.80 / 9.57</td>
|
||||
<td>45.04 / 23.86</td>
|
||||
<td>22.6</td>
|
||||
<td>基于PicoDet-L在中英文论文、杂志和研报等场景上自建数据集训练的效率精度均衡版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>95.8</td>
|
||||
<td>114.80 / 25.65</td>
|
||||
<td>924.38 / 924.38</td>
|
||||
<td>470.1</td>
|
||||
<td>基于RT-DETR-H在中英文论文、杂志和研报等场景上自建数据集训练的高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>5类英文文档区域检测模型，包含文字、标题、表格、图片以及列表</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_pretrained.pdparams">训练模型</a></td>
|
||||
<td>97.8</td>
|
||||
<td>9.62 / 6.75</td>
|
||||
<td>26.96 / 12.77</td>
|
||||
<td>7.4</td>
|
||||
<td>基于PicoDet-1x在PubLayNet数据集训练的高效率英文文档版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>17类区域检测模型，包含17个版面常见类别，分别是：段落标题、图片、文本、数字、摘要、内容、图表标题、公式、表格、表格标题、参考文献、文档标题、脚注、页眉、算法、页脚、印章</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>87.4</td>
|
||||
<td>8.80 / 3.62</td>
|
||||
<td>17.51 / 6.35</td>
|
||||
<td>4.8</td>
|
||||
<td>基于PicoDet-S轻量模型在中英文论文、杂志和研报等场景上自建数据集训练的高效率版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.60 / 10.27</td>
|
||||
<td>43.70 / 24.42</td>
|
||||
<td>22.6</td>
|
||||
<td>基于PicoDet-L在中英文论文、杂志和研报等场景上自建数据集训练的效率精度均衡版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>98.3</td>
|
||||
<td>115.29 / 101.18</td>
|
||||
<td>964.75 / 964.75</td>
|
||||
<td>470.2</td>
|
||||
<td>基于RT-DETR-H在中英文论文、杂志和研报等场景上自建数据集训练的高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>
|
||||
<ul>
|
||||
<li>20类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。</li>
|
||||
<li>1类版面子区域检测模型： PaddleOCR 自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。</li>
|
||||
<li>23类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、合同、书本、试卷和研报等常见的 500 张文档类型图片。</li>
|
||||
<li>3类版面检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 1154 张文档类型图片。</li>
|
||||
<li>5类英文文档区域检测模型： <a href="https://developer.ibm.com/exchanges/data/all/publaynet" target="_blank">PubLayNet</a> 的评估数据集，包含英文文档的 11245 张图片。</li>
|
||||
<li>17类区域检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 892 张文档类型图片。</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</details>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将版面区域检测模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayout_plus-L")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
- `input_path`：输入的待预测图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `boxes`：预测的目标框信息，一个字典列表。每个字典代表一个检出的目标，包含以下信息：
|
||||
- `cls_id`：类别ID，一个整数
|
||||
- `label`：类别标签，一个字符串
|
||||
- `score`：目标框置信度，一个浮点数
|
||||
- `coordinate`：目标框坐标，一个浮点数列表，格式为<code>[xmin, ymin, xmax, ymax]</code>
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/layout_det/layout_res_plus.jpg"/>
|
||||
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `LayoutDetection`实例化目标检测模型（此处以`PP-DocLayout_plus-L`为例），具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-DocLayout-L</code></td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td>输入图像大小。
|
||||
<ul>
|
||||
<li><b>int</b>：如<code>640</code>，表示将输入图像resize到640x640大小。</li>
|
||||
<li><b>list</b>：如<code>[640, 512]</code>，表示将输入图像resize到宽为640、高为512。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>用于过滤掉低置信度预测结果的阈值。
|
||||
<ul>
|
||||
<li><b>float</b>：如<code>0.2</code>，表示过滤掉所有阈值小于0.2的目标框。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>float</code>类型阈值。如<code>{0: 0.45, 2: 0.48, 7: 0.4}</code>，表示对ID为0的类别应用阈值0.45、ID为1的类别应用阈值0.48、ID为7的类别应用阈值0.4。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>是否使用NMS后处理，过滤重叠框。
|
||||
<ul>
|
||||
<li><b>bool</b>表示使用/不使用NMS进行检测框的后处理过滤重叠框。</li>
|
||||
<li><b>None</b>使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>检测框的边长缩放倍数。</b>
|
||||
<ul>
|
||||
<li><b>float</b>：大于0的浮点数，如<code>1.1</code>，表示将模型输出的检测框中心不变，宽和高都扩张1.1倍。</li>
|
||||
<li><b>list</b>：如<code>[1.2, 1.5]</code>，表示将模型输出的检测框中心不变，宽度扩张1.2倍，高度扩张1.5倍。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>tuple</code>类型，如<code>{0: (1.1, 2.0)}</code>，表示将模型输出的第0类别检测框中心不变，宽度扩张1.1倍，高度扩张2.0倍。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>模型输出的检测框的合并处理模式。
|
||||
<ul>
|
||||
<li><b>"large"</b>：设置为<code>"large"</code>，表示在模型输出的检测框中，对于互相重叠包含的检测框，只保留外部最大的框，删除重叠的内部框。</li>
|
||||
<li><b>"small"</b>：设置为<code>"small"</code>，表示在模型输出的检测框中，对于互相重叠包含的检测框，只保留内部被包含的小框，删除重叠的外部框。</li>
|
||||
<li><b>"union"</b>：不进行框的过滤处理，内外框都保留。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>str</code>类型, 如<code>{0: "large", 2: "small"}</code>, 表示对第0类别检测框使用<code>large</code>模式，对第2类别检测框使用<code>small</code>。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* 调用目标检测模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input`、`batch_size`和`threshold`，具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</b>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供版面区域检测模块的训练，因此，如果需要训练版面区域测模型，可以参考 [PaddleX 版面区域检测模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/layout_detection.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 五、FAQ
|
||||
3
docs/version3.x/module_usage/module_overview.en.md
Normal file
3
docs/version3.x/module_usage/module_overview.en.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# Module Overview
|
||||
|
||||
A module is the smallest unit that implements basic functionality. Modules typically use a single model to accomplish specific tasks, such as text detection, image classification, and other basic functions. As fundamental building blocks, modules provide the necessary functional support for more complex application scenarios. This design approach allows users to flexibly select and combine different modules according to their needs, thereby simplifying the development process and enhancing development flexibility and efficiency.
|
||||
3
docs/version3.x/module_usage/module_overview.md
Normal file
3
docs/version3.x/module_usage/module_overview.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# 模块概述
|
||||
|
||||
模块是实现基本功能的最小单位。模块通常使用单个模型去完成特定的任务，比如文本检测、图像分类等基本功能。模块作为基础构建单元，为更复杂的应用场景提供了必要的功能支持。这种设计方式使得用户可以根据需要灵活选择和组合不同的模块，从而简化了开发流程，并提高了开发的灵活性和效率。
|
||||
469
docs/version3.x/module_usage/seal_text_detection.en.md
Normal file
469
docs/version3.x/module_usage/seal_text_detection.en.md
Normal file
@@ -0,0 +1,469 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Seal Text Detection Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
The seal text detection module typically outputs multi-point bounding boxes around text regions, which are then passed as inputs to the distortion correction and text recognition modules for subsequent processing to identify the textual content of the seal. Recognizing seal text is an integral part of document processing and finds applications in various scenarios such as contract comparison, inventory access auditing, and invoice reimbursement verification. The seal text detection module serves as a subtask within OCR (Optical Character Recognition), responsible for locating and marking the regions containing seal text within an image. The performance of this module directly impacts the accuracy and efficiency of the entire seal text OCR system.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model Name</th><th>Model Download Link</th>
|
||||
<th>Hmean（%）</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_seal_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.40</td>
|
||||
<td>124.64 / 91.57</td>
|
||||
<td>545.68 / 439.86</td>
|
||||
<td>109</td>
|
||||
<td>The server-side seal text detection model of PP-OCRv4 boasts higher accuracy and is suitable for deployment on better-equipped servers.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_seal_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_seal_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>96.36</td>
|
||||
<td>9.70 / 3.56</td>
|
||||
<td>50.38 / 19.64</td>
|
||||
<td>4.6</td>
|
||||
<td>The mobile-side seal text detection model of PP-OCRv4, on the other hand, offers greater efficiency and is suitable for deployment on end devices.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong> A Self-built Internal Dataset, Containing 500 Images of Circular Stamps.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## III. Quick Integration <a id="quick"> </a>
|
||||
|
||||
> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../installation.en.md)。
|
||||
|
||||
Quickly experience with just one command:
|
||||
|
||||
```bash
|
||||
paddleocr seal_text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png) Go to the local area.
|
||||
|
||||
```python
|
||||
from paddleocr import SealTextDetection
|
||||
model = SealTextDetection(model_name="PP-OCRv4_server_seal_det")
|
||||
output = model.predict("seal_text_det.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'seal_text_det.png', 'page_index': None, 'dt_polys': [array([[463, 477],
|
||||
...,
|
||||
[428, 505]]), array([[297, 444],
|
||||
...,
|
||||
[230, 443]]), array([[457, 346],
|
||||
...,
|
||||
[267, 345]]), array([[325, 38],
|
||||
...,
|
||||
[322, 37]])], 'dt_scores': [0.9912680344777314, 0.9906849624837963, 0.9847219455533163, 0.9914791724153904]}}
|
||||
```
|
||||
|
||||
The meanings of the parameters are as follows:
|
||||
- `input_path`: represents the path of the input image to be predicted
|
||||
- `dt_polys`: represents the predicted text detection boxes, where each text detection box contains multiple vertices of a polygon. Each vertex is a list of two elements, representing the x and y coordinates of the vertex respectively
|
||||
- `dt_scores`: represents the confidence scores of the predicted text detection boxes
|
||||
|
||||
The visualization image is as follows:
|
||||
|
||||
<img alt="Visualization Image" src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/seal_text_det/seal_text_det_res.png"/>
|
||||
|
||||
The explanations of related methods and parameters are as follows:
|
||||
|
||||
* `SealTextDetection` instantiates a text detection model (here we take `PP-OCRv4_server_seal_det` as an example), and the specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-OCRv4_mobile_seal_det</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Limit on the side length of the input image for detection. <code>int</code> specifies the value. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Type of image side length limitation. <code>"min"</code> ensures the shortest side of the image is no less than <code>det_limit_side_len</code>; <code>"max"</code> ensures the longest side is no greater than <code>limit_side_len</code>. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>Pixel score threshold. Pixels in the output probability map with scores greater than this threshold are considered text pixels. Accepts any float value greater than 0. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>If the average score of all pixels inside the bounding box is greater than this threshold, the result is considered a text region. Accepts any float value greater than 0. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Expansion ratio for the Vatti clipping algorithm, used to expand the text region. Accepts any float value greater than 0. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>Input image size for the model in the format <code>(C, H, W)</code>. If set to <code>None</code>, the model's default size will be used.</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The `predict()` method of the seal text detection model is called for inference prediction. The parameters of the `predict()` method include `input`, `batch_size`, `limit_side_len`, `limit_type`, `thresh`, `box_thresh`, `max_candidates`, `unclip_ratio`. The specific descriptions are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, can be set to any positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Process the prediction results. Each sample's prediction result is a corresponding Result object, and it supports operations such as printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a file in JSON format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path for saving. When it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as a file in image format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path for saving. When it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining visual images with results and prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Attribute Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visual image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
If the above model is still not performing well in your scenario, you can try the following steps for secondary development. Here, we'll use training `PP-OCRv4_server_seal_det` as an example; you can replace it with the corresponding configuration files for other models. First, you need to prepare a text detection dataset. You can refer to the format of the [seal text detection demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar) for preparation. Once prepared, you can follow the steps below for model training and export. After export, you can quickly integrate the model into the above API. This example uses a seal text detection demo dataset. Before training the model, please ensure that you have installed the dependencies required by PaddleOCR as per the [installation documentation](../installation.en.md).
|
||||
|
||||
### 4.1 Dataset and Pre-trained Model Preparation
|
||||
|
||||
#### 4.1.1 Preparing the Dataset
|
||||
|
||||
```shell
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset
|
||||
tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/
|
||||
```
|
||||
|
||||
#### 4.1.1 Preparing the pre-trained model
|
||||
|
||||
|
||||
```shell
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR has modularized the code, and when training the `PP-OCRv4_server_seal_det` model, you need to use the [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml) for `PP-OCRv4_server_seal_det`.
|
||||
|
||||
The training commands are as follows:
|
||||
|
||||
```bash
|
||||
# Single GPU training (default training method)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
|
||||
# Multi-GPU training, specify GPU ids using the --gpus parameter
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command:
|
||||
|
||||
```bash
|
||||
# Make sure to set the pretrained_model path to the local path. If using a model that was trained and saved by yourself, be sure to modify the path and filename to {path/to/weights}/{model_name}.
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
save_inference_dir="./PP-OCRv4_server_seal_det_infer/"
|
||||
```
|
||||
|
||||
After exporting the model, the static graph model will be stored in the `./PP-OCRv4_server_seal_det_infer/` directory. In this directory, you will see the following files:
|
||||
```
|
||||
./PP-OCRv4_server_seal_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
With this, the secondary development is complete, and the static graph model can be directly integrated into PaddleOCR's API.
|
||||
|
||||
## 5. FAQ
|
||||
468
docs/version3.x/module_usage/seal_text_detection.md
Normal file
468
docs/version3.x/module_usage/seal_text_detection.md
Normal file
@@ -0,0 +1,468 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 印章文本检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
印章文本检测模块通常会输出文本区域的多点边界框（Bounding Boxes），这些边界框将作为输入传递给弯曲矫正和文本检测模块进行后续处理，识别出印章的文字内容。印章文本的识别是文档处理的一部分，在很多场景都有用途，例如合同比对，出入库审核以及发票报销审核等场景。印章文本检测模块是OCR（光学字符识别）中的子任务，负责在图像中定位和标记出包含印章文本的区域。该模块的性能直接影响到整个印章文本OCR系统的准确性和效率。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>检测Hmean（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_seal_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>98.40</td>
|
||||
<td>124.64 / 91.57</td>
|
||||
<td>545.68 / 439.86</td>
|
||||
<td>109</td>
|
||||
<td>PP-OCRv4的服务端印章文本检测模型，精度更高，适合在较好的服务器上部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_seal_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_seal_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>96.36</td>
|
||||
<td>9.70 / 3.56</td>
|
||||
<td>50.38 / 19.64</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv4的移动端印章文本检测模型，效率更高，适合在端侧部署</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建的内部数据集，包含500张圆形印章图像。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr seal_text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将印章文本检测的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import SealTextDetection
|
||||
model = SealTextDetection(model_name="PP-OCRv4_server_seal_det")
|
||||
output = model.predict("seal_text_det.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'seal_text_det.png', 'page_index': None, 'dt_polys': [array([[463, 477],
|
||||
...,
|
||||
[428, 505]]), array([[297, 444],
|
||||
...,
|
||||
[230, 443]]), array([[457, 346],
|
||||
...,
|
||||
[267, 345]]), array([[325, 38],
|
||||
...,
|
||||
[322, 37]])], 'dt_scores': [0.9912680344777314, 0.9906849624837963, 0.9847219455533163, 0.9914791724153904]}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待预测图像的路径
|
||||
- `dt_polys`：表示预测的文本检测框，其中每个文本检测框包含一个多边形的多个顶点。其中每个顶点都是一个列表，分别表示该顶点的x坐标和y坐标
|
||||
- `dt_scores`：表示预测的文本检测框的置信度
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/seal_text_det/seal_text_det_res.png"/>
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `SealTextDetection`实例化文本检测模型（此处以`PP-OCRv4_server_seal_det`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。模型名称。如果设置为<code>None</code>，则使用<code>PP-OCRv4_mobile_seal_det</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>检测的图像边长限制：<code>int</code> 表示边长限制数值，如果设置为<code>None</code>, 将使用模型默认配置。</td>
|
||||
<td><code>int|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>检测的图像边长限制，检测的边长限制类型，<code>"min"</code> 表示保证图像最短边不小于 det_limit_side_len，<code>"max"</code> 表示保证图像最长边不大于 <code>limit_side_len</code>。如果设置为 <code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>像素得分阈值。输出概率图中得分大于该阈值的像素点被认为是文本像素。可选大于0的float任意浮点数。如果设置为<code>None</code>。将使用模型默认配置。</td>
|
||||
<td><code>float|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>检测结果边框内，所有像素点的平均得分大于该阈值时，该结果会被认为是文字区域。可选大于0的float任意浮点数。如果设置为<code>None</code>, 将使用模型默认配置。</td>
|
||||
<td><code>float|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Vatti clipping算法的扩张系数，使用该方法对文字区域进行扩张。可选大于0的任意浮点数。如果设置为<code>None</code>, 将使用模型默认配置。</td>
|
||||
<td><code>float|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>模型输入图像尺寸，格式为 <code>(C, H, W)</code>。若为 <code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>tuple|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用印章文本检测模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input`、 `batch_size`、 `limit_side_len`、 `limit_type`、 `thresh`、 `box_thresh`、 `max_candidates`、`unclip_ratio`，具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-OCRv4_server_seal_det` 举例，其他模型替换对应配置文件即可。首先，您需要准备文本检测的数据集，可以参考[印章文本检测 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以印章文本检测 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
### 4.1 数据集、预训练模型准备
|
||||
|
||||
#### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset
|
||||
tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/
|
||||
```
|
||||
|
||||
#### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-OCRv4_server_seal_det 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `PP-OCRv4_server_seal_det` 模型时需要使用 `PP-OCRv4_server_seal_det` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
save_inference_dir="./PP-OCRv4_server_seal_det_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-OCRv4_server_seal_det_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-OCRv4_server_seal_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
343
docs/version3.x/module_usage/table_cells_detection.en.md
Normal file
343
docs/version3.x/module_usage/table_cells_detection.en.md
Normal file
@@ -0,0 +1,343 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
# Table Cell Detection Module Usage Tutorial
|
||||
|
||||
## I. Overview
|
||||
|
||||
The Table Cell Detection Module is a key component of the table recognition task, responsible for locating and marking each cell region in table images. The performance of this module directly affects the accuracy and efficiency of the entire table recognition process. The Table Cell Detection Module typically outputs bounding boxes for each cell region, which are then passed as input to the table recognition pipeline for further processing.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wired_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wired_table_cell_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wired_table_cell_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td rowspan="2">82.7</td>
|
||||
<td rowspan="2">33.47 / 27.02</td>
|
||||
<td rowspan="2">402.55 / 256.56</td>
|
||||
<td rowspan="2">124</td>
|
||||
<td rowspan="2">RT-DETR is a real-time end-to-end object detection model. The Baidu PaddlePaddle Vision team pre-trained on a self-built table cell detection dataset based on the RT-DETR-L as the base model, achieving good performance in detecting both wired and wireless table cells.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wireless_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wireless_table_cell_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wireless_table_cell_det_pretrained.pdparams">Training Model</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> Internal evaluation set built by PaddleX.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Explanation</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Regular Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Choose the optimal prior backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## III. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please first install the PaddleOCR wheel package. For details, please refer to the [installation tutorial](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate model inference from the table cell detection module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(model_name="RT-DETR-L_wired_table_cell_det")
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'boxes': [{'cls_id': 0, 'label': 'cell', 'score': 0.9698355197906494, 'coordinate': [2.3011515, 0, 546.29926, 30.530712]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9690820574760437, 'coordinate': [212.37508, 64.62493, 403.58868, 95.61413]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9668057560920715, 'coordinate': [212.46791, 30.311079, 403.7182, 64.62613]}, {'cls_id': 0, 'label': 'cell', 'score': 0.966505229473114, 'coordinate': [403.56082, 64.62544, 546.83215, 95.66117]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9662341475486755, 'coordinate': [109.48873, 64.66485, 212.5177, 95.631294]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9654079079627991, 'coordinate': [212.39197, 95.63037, 403.60852, 126.78792]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9653300642967224, 'coordinate': [2.2320926, 64.62229, 109.600494, 95.59732]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9639787673950195, 'coordinate': [403.5752, 30.562355, 546.98975, 64.61531]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9636150002479553, 'coordinate': [2.1537683, 30.410172, 109.568306, 64.62762]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631900191307068, 'coordinate': [2.0534437, 95.57448, 109.57601, 126.71458]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631181359291077, 'coordinate': [403.65976, 95.68139, 546.84766, 126.713394]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9614537358283997, 'coordinate': [109.56504, 30.391184, 212.65425, 64.6444]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9607433080673218, 'coordinate': [109.525795, 95.62622, 212.44917, 126.8258]}]}}
|
||||
```
|
||||
|
||||
The parameter meanings are as follows:
|
||||
|
||||
- `input_path`: Path of the input image to be predicted
|
||||
- `page_index`: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is `None`
|
||||
- `boxes`: Predicted bounding box information, a list of dictionaries. Each dictionary represents a detected object and contains the following information:
|
||||
- `cls_id`: Class ID, an integer
|
||||
- `label`: Class label, a string
|
||||
- `score`: Confidence of the bounding box, a float
|
||||
- `coordinate`: Coordinates of the bounding box, a list of floats in the format <code>[xmin, ymin, xmax, ymax]</code>
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_cells_detection/01.jpg">
|
||||
|
||||
The relevant methods, parameters, etc., are described as follows:
|
||||
|
||||
* `TableCellsDetection` instantiates the table cell detection model (taking `RT-DETR-L_wired_table_cell_det` as an example here), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>RT-DETR-L_wired_table_cell_det</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td>Input image size.<ul><li><b>int</b>: e.g. <code>640</code>, resizes input image to 640x640</li><li><b>list</b>: e.g. <code>[640, 512]</code>, resizes input image to 640 width and 512 height</li></ul></td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Threshold for filtering out low-confidence prediction results.<ul><li><b>float</b>: e.g. <code>0.2</code>, filters out all boxes with confidence below 0.2.</li><li><b>dict</b>: keys are <code>int</code> (class id), values are <code>float</code> thresholds, e.g. <code>{0: 0.45, 2: 0.48, 7: 0.4}</code>, applies thresholds to specific classes.</li><li><b>None</b>: uses the model's default configuration.</li></ul></td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the table cell detection model for inference prediction. This method will return a result list. Additionally, this module also provides a `predict_iter()` method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods according to your actual needs. The `predict()` method has parameters `input`, `batch_size`, and `threshold`, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local image file or PDF file path: <code>/root/data/img.jpg</code>; <b>URL</b>: Image or PDF file network URL: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Example</a>; <b>Directory</b>: Should contain images for prediction, e.g., <code>/root/data/</code> (currently, PDF files in directories are not supported, PDF files need to be specified by file path)</li>
|
||||
<li><b>list</b>: List elements should be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## IV. Secondary Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the table cell detection module, if you need to train a table cell detection model, you can refer to the [PaddleX Table Cell Detection Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_cells_detection.html#iv-secondary-development) section for training. The trained model can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## V. FAQ
|
||||
357
docs/version3.x/module_usage/table_cells_detection.md
Normal file
357
docs/version3.x/module_usage/table_cells_detection.md
Normal file
@@ -0,0 +1,357 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 表格单元格检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
表格单元格检测模块是表格识别任务的关键组成部分，负责在表格图像中定位和标记每个单元格区域，该模块的性能直接影响到整个表格识别过程的准确性和效率。表格单元格检测模块通常会输出各个单元格区域的边界框（Bounding Boxes），这些边界框将作为输入传递给表格识别相关产线进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wired_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wired_table_cell_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wired_table_cell_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td rowspan="2">82.7</td>
|
||||
<td rowspan="2">33.47 / 27.02</td>
|
||||
<td rowspan="2">402.55 / 256.56</td>
|
||||
<td rowspan="2">124</td>
|
||||
<td rowspan="2">RT-DETR 是一个实时的端到端目标检测模型。百度飞桨视觉团队基于 RT-DETR-L 作为基础模型，在自建表格单元格检测数据集上完成预训练，实现了对有线表格、无线表格均有较好性能的表格单元格检测。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wireless_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wireless_table_cell_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wireless_table_cell_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建的内部评测集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将表格单元格检测的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(model_name="RT-DETR-L_wired_table_cell_det")
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'boxes': [{'cls_id': 0, 'label': 'cell', 'score': 0.9698355197906494, 'coordinate': [2.3011515, 0, 546.29926, 30.530712]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9690820574760437, 'coordinate': [212.37508, 64.62493, 403.58868, 95.61413]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9668057560920715, 'coordinate': [212.46791, 30.311079, 403.7182, 64.62613]}, {'cls_id': 0, 'label': 'cell', 'score': 0.966505229473114, 'coordinate': [403.56082, 64.62544, 546.83215, 95.66117]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9662341475486755, 'coordinate': [109.48873, 64.66485, 212.5177, 95.631294]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9654079079627991, 'coordinate': [212.39197, 95.63037, 403.60852, 126.78792]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9653300642967224, 'coordinate': [2.2320926, 64.62229, 109.600494, 95.59732]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9639787673950195, 'coordinate': [403.5752, 30.562355, 546.98975, 64.61531]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9636150002479553, 'coordinate': [2.1537683, 30.410172, 109.568306, 64.62762]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631900191307068, 'coordinate': [2.0534437, 95.57448, 109.57601, 126.71458]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631181359291077, 'coordinate': [403.65976, 95.68139, 546.84766, 126.713394]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9614537358283997, 'coordinate': [109.56504, 30.391184, 212.65425, 64.6444]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9607433080673218, 'coordinate': [109.525795, 95.62622, 212.44917, 126.8258]}]}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
|
||||
- `input_path`：输入的待预测图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `boxes`：预测的目标框信息，一个字典列表。每个字典代表一个检出的目标，包含以下信息：
|
||||
- `cls_id`：类别ID，一个整数
|
||||
- `label`：类别标签，一个字符串
|
||||
- `score`：目标框置信度，一个浮点数
|
||||
- `coordinate`：目标框坐标，一个浮点数列表，格式为<code>[xmin, ymin, xmax, ymax]</code>
|
||||
|
||||
可视化图像如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_cells_detection/01.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TableCellsDetection`实例化表格单元格检测模型（此处以`RT-DETR-L_wired_table_cell_det`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>RT-DETR-L_wired_table_cell_det</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td>输入图像大小。
|
||||
<ul>
|
||||
<li><b>int</b>：如<code>640</code>，表示将输入图像resize到640x640大小。</li>
|
||||
<li><b>list</b>：如<code>[640, 512]</code>，表示将输入图像resize到宽为640，高为512大小。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>用于过滤掉低置信度预测结果的阈值。
|
||||
<ul>
|
||||
<li><b>float</b>：如 <code>0.2</code>，表示过滤掉所有阈值小于0.2的目标框。/li>
|
||||
<li><b>dict</b>：字典的键为 <code>int</code> 类型，代表类别ID；值为 <code>float</code> 类型阈值。如 <code>{0: 0.45, 2: 0.48, 7: 0.4}</code>，表示对类别ID为0的类别应用阈值0.45、类别ID为1的类别应用阈值0.48、类别ID为7的类别应用阈值0.4。</li>
|
||||
<li><b>None</b>：使用模型默认配置。
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 调用表格单元格检测模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input`、`batch_size`和`threshold`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供表格单元格检测模块的训练，因此，如果需要训练表格单元格检测模型，可以参考 [PaddleX 表格单元格检测模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_cells_detection.html#_4)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 五、FAQ
|
||||
306
docs/version3.x/module_usage/table_classification.en.md
Normal file
306
docs/version3.x/module_usage/table_classification.en.md
Normal file
@@ -0,0 +1,306 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
# Table Classification Module Usage Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The Table Classification Module is a key component in computer vision systems, responsible for classifying input table images. The performance of this module directly affects the accuracy and efficiency of the entire table recognition process. The Table Classification Module typically receives table images as input and, using deep learning algorithms, classifies them into predefined categories based on the characteristics and content of the images, such as wired and wireless tables. The classification results from the Table Classification Module serve as output for use in table recognition pipelines.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Top1 Acc(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_table_cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_table_cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_table_cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>94.2</td>
|
||||
<td>2.62 / 0.60</td>
|
||||
<td>3.17 / 1.14</td>
|
||||
<td>6.6</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> Internal evaluation dataset built by PaddleX.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Explanation</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Regular Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Choose the optimal prior backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please first install the PaddleOCR wheel package. For details, please refer to the [installation tutorial](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate model inference from the table classification module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(model_name="PP-LCNet_x1_0_table_cls")
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'class_ids': array([0, 1], dtype=int32), 'scores': array([0.84421, 0.15579], dtype=float32), 'label_names': ['wired_table', 'wireless_table']}}
|
||||
```
|
||||
|
||||
The parameter meanings are as follows:
|
||||
- `input_path`: Path of the input image
|
||||
- `page_index`: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is `None`
|
||||
- `class_ids`: Class IDs of the prediction results
|
||||
- `scores`: Confidence scores of the prediction results
|
||||
- `label_names`: Class names of the prediction results
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_classification/01.jpg">
|
||||
|
||||
The relevant methods, parameters, etc., are described as follows:
|
||||
|
||||
* `TableClassification` instantiates the table classification model (taking `PP-LCNet_x1_0_table_cls` as an example here), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-LCNet_x1_0_table_cls</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the table classification model for inference prediction. This method will return a result list. Additionally, this module also provides a `predict_iter()` method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods according to your actual needs. The `predict()` method has parameters `input` and `batch_size`, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local image file or PDF file path: <code>/root/data/img.jpg</code>; <b>URL</b>: Image or PDF file network URL: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Example</a>; <b>Directory</b>: Should contain images for prediction, e.g., <code>/root/data/</code> (currently, PDF files in directories are not supported, PDF files need to be specified by file path)</li>
|
||||
<li><b>list</b>: List elements should be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the table classification module, if you need to train a table classification model, you can refer to the [PaddleX Table Classification Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_classification.html#iv-secondary-development) section for training. The trained model can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## 5. FAQ
|
||||
309
docs/version3.x/module_usage/table_classification.md
Normal file
309
docs/version3.x/module_usage/table_classification.md
Normal file
@@ -0,0 +1,309 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 表格分类模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
表格分类模块是计算机视觉系统中的关键组成部分，负责对输入的表格图像进行分类，该模块的性能直接影响到整个表格识别过程的准确性和效率。表格分类模块通常会接收表格图像作为输入，然后通过深度学习算法，根据图像的特性和内容，将其分类到预定义的类别中，例如有线表和无线表。表格分类模块的分类结果将作为输出，供表格识别相关产线使用。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>Top1 Acc(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_table_cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_table_cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_table_cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>94.2</td>
|
||||
<td>2.62 / 0.60</td>
|
||||
<td>3.17 / 1.14</td>
|
||||
<td>6.6</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建的内部评测数据集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将表格分类的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(model_name="PP-LCNet_x1_0_table_cls")
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'class_ids': array([0, 1], dtype=int32), 'scores': array([0.84421, 0.15579], dtype=float32), 'label_names': ['wired_table', 'wireless_table']}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入图片的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `class_ids`：表示预测结果的类别id
|
||||
- `scores`：表示预测结果的置信度
|
||||
- `label_names`：表示预测结果的类别名
|
||||
|
||||
可视化图像如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_classification/01.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TableClassification`实例化表格分类模型（此处以`PP-LCNet_x1_0_table_cls`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-LCNet_x1_0_table_cls</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用表格分类模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供表格分类模块的训练，因此，如果需要训练表格分类模型，可以参考 [PaddleX 表格分类模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_classification.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 五、FAQ
|
||||
398
docs/version3.x/module_usage/table_structure_recognition.en.md
Normal file
398
docs/version3.x/module_usage/table_structure_recognition.en.md
Normal file
@@ -0,0 +1,398 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Table Structure Recognition Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Table structure recognition is an important component of table recognition systems, capable of converting non-editable table images into editable table formats (such as HTML). The goal of table structure recognition is to identify the positions of rows, columns, and cells in tables. The performance of this module directly affects the accuracy and efficiency of the entire table recognition system. The table structure recognition module usually outputs HTML code for the table area, which is then passed as input to the tabl recognition pipeline for further processing.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Accuracy (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_pretrained.pdparams">Training Model</a></td>
|
||||
<td>59.52</td>
|
||||
<td>23.96 / 21.75</td>
|
||||
<td>- / 43.12</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet is a table structure recognition model independently developed by Baidu PaddlePaddle Vision Team. By adopting a CPU-friendly lightweight backbone network PP-LCNet, high-low level feature fusion module CSP-PAN, and SLA Head, a feature decoding module aligning structure and position information, this model greatly improves the accuracy and inference speed of table structure recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet_plus</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_plus_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams">Training Model</a></td>
|
||||
<td>63.69</td>
|
||||
<td>23.43 / 22.16</td>
|
||||
<td>- / 41.80</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet_plus is an enhanced version of the table structure recognition model SLANet independently developed by the Baidu PaddlePaddle Vision Team. Compared to SLANet, SLANet_plus has greatly improved the recognition ability for wireless and complex tables, and reduced the model's sensitivity to table positioning accuracy. Even if the table positioning is offset, it can still be accurately recognized.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wired</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wired_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wired_pretrained.pdparams">Training Model</a></td>
|
||||
<td rowspan="2">69.65</td>
|
||||
<td rowspan="2">85.92 / 85.92</td>
|
||||
<td rowspan="2">- / 501.66</td>
|
||||
<td rowspan="2">351</td>
|
||||
<td rowspan="2">The SLANeXt series is a new generation of table structure recognition models independently developed by the Baidu PaddlePaddle Vision Team. Compared to SLANet and SLANet_plus, SLANeXt focuses on table structure recognition, and trains dedicated weights for wired and wireless tables separately. The recognition ability for all types of tables has been significantly improved, especially for wired tables.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wireless</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wireless_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wireless_pretrained.pdparams">Training Model</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> High-difficulty Chinese table recognition dataset.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 precision / No TRT acceleration</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High Performance Mode</td>
|
||||
<td>Optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>Selects the prior optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before getting started, please install the PaddleOCR wheel package. For details, please refer to the [Installation Tutorial](../installation.en.md).
|
||||
|
||||
Quickly experience with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference of the table structure recognition module into your own project. Before running the code below, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg) to your local machine.
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(model_name="SLANet")
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'bbox': [[42, 2, 390, 2, 388, 27, 40, 26], [11, 35, 89, 35, 87, 63, 11, 63], [113, 34, 192, 34, 186, 64, 109, 64], [219, 33, 399, 33, 393, 62, 212, 62], [413, 33, 544, 33, 544, 64, 407, 64], [12, 67, 98, 68, 96, 93, 12, 93], [115, 66, 205, 66, 200, 91, 111, 91], [234, 65, 390, 65, 385, 92, 227, 92], [414, 66, 537, 67, 537, 95, 409, 95], [7, 97, 106, 97, 104, 128, 7, 128], [113, 96, 206, 95, 201, 127, 109, 127], [236, 96, 386, 96, 381, 128, 230, 128], [413, 96, 534, 95, 533, 127, 408, 127]], 'structure': ['<html>', '<body>', '<table>', '<tr>', '<td', ' colspan="4"', '>', '</td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '</table>', '</body>', '</html>'], 'structure_score': 0.99948007}}
|
||||
```
|
||||
|
||||
Parameter meanings are as follows:
|
||||
|
||||
- `input_path`: The path of the input table image to be predicted
|
||||
- `page_index`: If the input is a PDF file, indicates the page number of the PDF; otherwise, it is `None`
|
||||
- `boxes`: Predicted table cell information, a list consisting of the coordinates of predicted table cells. Notably, table cell predictions for the SLANeXt series models are invalid
|
||||
- `structure`: Predicted table structure HTML expressions, a list consisting of predicted HTML keywords in order
|
||||
- `structure_score`: Confidence of the predicted table structure
|
||||
|
||||
Descriptions of related methods and parameters are as follows:
|
||||
|
||||
* `TableStructureRecognition` instantiates a table structure recognition model (using `SLANet` as an example). Details are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-LCNet_x1_0_table_cls</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the table structure recognition model for inference prediction, which returns a result list. In addition, this module also provides the `predict_iter()` method. The two are completely consistent in parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where you want to save memory. You can choose to use either method according to your actual needs. The `predict()` method has parameters `input` and `batch_size`, described as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
Data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local path to an image or PDF file, e.g., <code>/root/data/img.jpg</code>; <b>URL</b>: Network URL to an image or PDF file, e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Example</a>; <b>Directory</b>: A local directory containing images for prediction, e.g., <code>/root/data/</code> (Note: PDF files in directories are not supported; to predict a PDF, specify its file path directly)</li>
|
||||
<li><b>list</b>: A list of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, can be set to any positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
* For processing prediction results, the prediction result of each sample is the corresponding Result object, and supports printing and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to use <code>JSON</code> indentation formatting for the output</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> keeps the original characters. Effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save result as json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path to save the file. If it's a directory, the saved file will be named the same as the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> keeps the original characters. Effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
If the above models are still not ideal for your scenario, you can try the following steps for secondary development. Here, training `SLANet_plus` is used as an example, and for other models, just replace the corresponding configuration file. First, you need to prepare a dataset for table structure recognition, which can be prepared with reference to the format of the [table structure recognition demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar). Once ready, you can train and export the model as follows. After exporting, you can quickly integrate the model into the above API. Here, the table structure recognition demo data is used as an example. Before training the model, please make sure you have installed the dependencies required by PaddleOCR according to the [installation documentation](../installation.en.md).
|
||||
|
||||
|
||||
## 4.1 Dataset and Pretrained Model Preparation
|
||||
|
||||
### 4.1.1 Prepare Dataset
|
||||
|
||||
```shell
|
||||
# Download sample dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar
|
||||
tar -xf table_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
### 4.1.2 Download Pretrained Model
|
||||
|
||||
```shell
|
||||
# Download SLANet_plus pretrained model
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR is modularized. When training the `SLANet_plus` recognition model, you need to use the [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/table/SLANet_plus.yml) of `SLANet_plus`.
|
||||
|
||||
|
||||
The training commands are as follows:
|
||||
|
||||
```bash
|
||||
# Single card training (default training method)
|
||||
python3 tools/train.py -c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
|
||||
# Multi-card training, specify card numbers via --gpus parameter
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command:
|
||||
|
||||
```bash
|
||||
# Note to set the path of pretrained_model to the local path. If you use the model saved by your own training, please modify the path and file name to {path/to/weights}/{model_name}.
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./SLANet_plus_infer/"
|
||||
```
|
||||
|
||||
After exporting the model, the static graph model will be stored in `./SLANet_plus_infer/` in the current directory. In this directory, you will see the following files:
|
||||
```
|
||||
./SLANet_plus_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
At this point, secondary development is complete, and this static graph model can be directly integrated into the PaddleOCR API.
|
||||
|
||||
## 5. FAQ
|
||||
404
docs/version3.x/module_usage/table_structure_recognition.md
Normal file
404
docs/version3.x/module_usage/table_structure_recognition.md
Normal file
@@ -0,0 +1,404 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 表格结构识别模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
表格结构识别是表格识别系统中的重要组成部分，能够将不可编辑表格图片转换为可编辑的表格形式（例如html）。表格结构识别的目标是对表格的行、列和单元格位置进行识别，该模块的性能直接影响到整个表格识别系统的准确性和效率。表格结构识别模块会输出表格区域的html代码，这些代码将作为输入传递给表格识别产线进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>精度（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_pretrained.pdparams">训练模型</a></td>
|
||||
<td>59.52</td>
|
||||
<td>23.96 / 21.75</td>
|
||||
<td>- / 43.12</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet 是百度飞桨视觉团队自研的表格结构识别模型。该模型通过采用 CPU 友好型轻量级骨干网络 PP-LCNet、高低层特征融合模块 CSP-PAN、结构与位置信息对齐的特征解码模块 SLA Head，大幅提升了表格结构识别的精度和推理速度。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet_plus</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_plus_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams">训练模型</a></td>
|
||||
<td>63.69</td>
|
||||
<td>23.43 / 22.16</td>
|
||||
<td>- / 41.80</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet_plus 是百度飞桨视觉团队自研的表格结构识别模型 SLANet 的增强版。相较于 SLANet，SLANet_plus 对无线表、复杂表格的识别能力得到了大幅提升，并降低了模型对表格定位准确性的敏感度，即使表格定位出现偏移，也能够较准确地进行识别。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wired</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wired_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wired_pretrained.pdparams">训练模型</a></td>
|
||||
<td rowspan="2">69.65</td>
|
||||
<td rowspan="2">85.92 / 85.92</td>
|
||||
<td rowspan="2">- / 501.66</td>
|
||||
<td rowspan="2">351</td>
|
||||
<td rowspan="2">SLANeXt 系列是百度飞桨视觉团队自研的新一代表格结构识别模型。相较于 SLANet 和 SLANet_plus，SLANeXt 专注于对表格结构进行识别，并且对有线表格(wired)和无线表格(wireless)的识别分别训练了专用的权重，对各类型表格的识别能力都得到了明显提高，特别是对有线表格的识别能力得到了大幅提升。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wireless</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wireless_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wireless_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>内部自建的高难度中文表格识别数据集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将表格结构识别的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(model_name="SLANet")
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'bbox': [[42, 2, 390, 2, 388, 27, 40, 26], [11, 35, 89, 35, 87, 63, 11, 63], [113, 34, 192, 34, 186, 64, 109, 64], [219, 33, 399, 33, 393, 62, 212, 62], [413, 33, 544, 33, 544, 64, 407, 64], [12, 67, 98, 68, 96, 93, 12, 93], [115, 66, 205, 66, 200, 91, 111, 91], [234, 65, 390, 65, 385, 92, 227, 92], [414, 66, 537, 67, 537, 95, 409, 95], [7, 97, 106, 97, 104, 128, 7, 128], [113, 96, 206, 95, 201, 127, 109, 127], [236, 96, 386, 96, 381, 128, 230, 128], [413, 96, 534, 95, 533, 127, 408, 127]], 'structure': ['<html>', '<body>', '<table>', '<tr>', '<td', ' colspan="4"', '>', '</td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '</table>', '</body>', '</html>'], 'structure_score': 0.99948007}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
|
||||
- `input_path`：输入的待预测表格图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `boxes`：预测的表格单元格信息，一个列表，由预测的若干表格单元格坐标组成。特别地， SLANeXt 系列模型预测的表格单元格无效
|
||||
- `structure`：预测的表格结构Html表达式，一个列表，由预测的若干Html关键字按顺序组成
|
||||
- `structure_score`：预测表格结构的置信度
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TableStructureRecognition`实例化表格结构识别模型（此处以`SLANet`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-LCNet_x1_0_table_cls</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用表格结构识别模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `SLANet_plus` 举例，其他模型替换对应配置文件即可。首先，您需要准备表格结构识别的数据集，可以参考[表格结构识别 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以表格结构识别 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
### 4.1 数据集、预训练模型准备
|
||||
|
||||
#### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar
|
||||
tar -xf table_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 SLANet_plus 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `SLANet_plus` 识别模型时需要使用 `SLANet_plus` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/table/SLANet_plus.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./SLANet_plus_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./SLANet_plus_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./SLANet_plus_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
504
docs/version3.x/module_usage/text_detection.en.md
Normal file
504
docs/version3.x/module_usage/text_detection.en.md
Normal file
@@ -0,0 +1,504 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Text Detection Module Usage Guide
|
||||
|
||||
## 1. Overview
|
||||
The text detection module is a critical component of OCR (Optical Character Recognition) systems, responsible for locating and marking text-containing regions in images. The performance of this module directly impacts the accuracy and efficiency of the entire OCR system. The text detection module typically outputs bounding boxes for text regions, which are then passed to the text recognition module for further processing.
|
||||
|
||||
## 2. Supported Models List
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Detection Hmean (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Standard Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Standard Mode / High-Performance Mode]</th>
|
||||
<th>Model Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_server_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>83.8</td>
|
||||
<td>89.55 / 70.19</td>
|
||||
<td>383.15 / 383.15</td>
|
||||
<td>84.3</td>
|
||||
<td>PP-OCRv5 server-side text detection model with higher accuracy, suitable for deployment on high-performance servers</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_mobile_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>79.0</td>
|
||||
<td>10.67 / 6.36</td>
|
||||
<td>57.77 / 28.15</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv5 mobile-side text detection model with higher efficiency, suitable for deployment on edge devices</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>69.2</td>
|
||||
<td>127.82 / 98.87</td>
|
||||
<td>585.95 / 489.77</td>
|
||||
<td>109</td>
|
||||
<td>PP-OCRv4 server-side text detection model with higher accuracy, suitable for deployment on high-performance servers</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>63.8</td>
|
||||
<td>9.87 / 4.17</td>
|
||||
<td>56.60 / 20.79</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv4 mobile-side text detection model with higher efficiency, suitable for deployment on edge devices</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Testing Environment:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Testing Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> PaddleOCR3.0 newly constructed multilingual dataset (including Chinese, Traditional Chinese, English, Japanese), covering street scenes, web images, documents, handwriting, blur, rotation, distortion, etc., totaling 2677 images.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Techniques</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Standard Mode</td>
|
||||
<td>FP32 precision / No TRT acceleration</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of precision types and acceleration strategies</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>Optimal backend selection (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting, please install the PaddleOCR wheel package. Refer to the [Installation Guide](../installation.en.md) for details.
|
||||
|
||||
Use the following command for a quick experience:
|
||||
|
||||
```bash
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference into your project. Before running the following code, download the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection(model_name="PP-OCRv5_server_det")
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
The output will be:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_001.png', 'page_index': None, 'dt_polys': array([[[ 75, 549],
|
||||
...,
|
||||
[ 77, 586]],
|
||||
|
||||
...,
|
||||
|
||||
[[ 31, 406],
|
||||
...,
|
||||
[ 34, 455]]], dtype=int16), 'dt_scores': [0.873949039891189, 0.8948166013613552, 0.8842595305917041, 0.876953790920377]}}
|
||||
```
|
||||
|
||||
Output parameter meanings:
|
||||
- `input_path`: Path of the input image.
|
||||
- `page_index`: If the input is a PDF, this indicates the current page number; otherwise, it is `None`.
|
||||
- `dt_polys`: Predicted text detection boxes, where each box contains four vertices (x, y coordinates).
|
||||
- `dt_scores`: Confidence scores of the predicted text detection boxes.
|
||||
|
||||
Visualization example:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_det/general_ocr_001_res.png"/>
|
||||
|
||||
Method and parameter descriptions:
|
||||
|
||||
* Instantiate the text detection model (e.g., `PP-OCRv5_server_det`):
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-OCRv5_server_det</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Limit on the side length of the input image for detection. <code>int</code> specifies the value. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Type of image side length limitation. <code>"min"</code> ensures the shortest side of the image is no less than <code>det_limit_side_len</code>; <code>"max"</code> ensures the longest side is no greater than <code>limit_side_len</code>. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_side_limit</code></td>
|
||||
<td>Limit on the max length of the input image for detection. <code>int</code> limits the longest side of the image for input detection model. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>Pixel score threshold. Pixels in the output probability map with scores greater than this threshold are considered text pixels. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>If the average score of all pixels inside the bounding box is greater than this threshold, the result is considered a text region. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Expansion ratio for the Vatti clipping algorithm, used to expand the text region. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>Input image size for the model in the format <code>(C, H, W)</code>.</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The `predict()` method parameters:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python variable</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local image file or PDF file path: <code>/root/data/img.jpg</code>; <b>URL</b>: Image or PDF file network URL: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">Example</a>; <b>Directory</b>: Should contain images for prediction, e.g., <code>/root/data/</code> (currently, PDF files in directories are not supported, PDF files need to be specified by file path)</li>
|
||||
<li><b>list</b>: List elements should be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Result processing methods:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameters</th>
|
||||
<th>Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print results to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Format output as JSON</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>JSON indentation level</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Escape non-ASCII characters</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save results as JSON file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Output file path</td>
|
||||
<td>Required</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>JSON indentation level</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Escape non-ASCII characters</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save results as image</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Output file path</td>
|
||||
<td>Required</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additional attributes:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>json</code></td>
|
||||
<td>Get prediction results in JSON format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img</code></td>
|
||||
<td>Get visualization image as a dictionary</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Custom Development
|
||||
|
||||
If the above models do not meet your requirements, follow these steps for custom development (using `PP-OCRv5_server_det` as an example). First, prepare a text detection dataset (refer to the [Demo Dataset](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar) format). After preparation, proceed with model training and export. The exported model can be integrated into the API. Ensure PaddleOCR dependencies are installed as per the [Installation Guide](../installation.en.md).
|
||||
|
||||
### 4.1 Dataset and Pretrained Model Preparation
|
||||
|
||||
#### 4.1.1 Prepare Dataset
|
||||
|
||||
```shell
|
||||
# Download example dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar
|
||||
tar -xf ocr_det_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 Download Pretrained Model
|
||||
|
||||
```shell
|
||||
# Download PP-OCRv5_server_det pretrained model
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR modularizes the code. To train the `PP-OCRv5_server_det` model, use its [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv5/PP-OCRv5_server_det.yml).
|
||||
|
||||
Training command:
|
||||
|
||||
```bash
|
||||
# Single-GPU training (default)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
|
||||
# Multi-GPU training (specify GPUs with --gpus)
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate trained weights (e.g., `output/PP-OCRv5_server_det/best_accuracy.pdparams`) using the following command:
|
||||
|
||||
```bash
|
||||
# Note: Set pretrained_model to local path. For custom-trained models, modify the path and filename as {path/to/weights}/{model_name}.
|
||||
# Demo dataset evaluation
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml -o \
|
||||
Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_det_infer/"
|
||||
```
|
||||
|
||||
After export, the static graph model will be saved in `./PP-OCRv5_server_det_infer/` with the following files:
|
||||
```
|
||||
./PP-OCRv5_server_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
The custom development is now complete. This static graph model can be directly integrated into PaddleOCR's API.
|
||||
|
||||
## 5. FAQ
|
||||
|
||||
- Use parameters `limit_type` and `limit_side_len` to constrain image dimensions.
|
||||
- `limit_type` options: [`max`, `min`]
|
||||
- `limit_side_len`: Positive integer (typically multiples of 32, e.g., 960).
|
||||
- For lower-resolution images, use `limit_type=min` and `limit_side_len=960` to balance computational efficiency and detection quality.
|
||||
- For higher-resolution images requiring larger detection scales, set `limit_side_len` to desired values (e.g., 1216).
|
||||
508
docs/version3.x/module_usage/text_detection.md
Normal file
508
docs/version3.x/module_usage/text_detection.md
Normal file
@@ -0,0 +1,508 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
文本检测模块是OCR（光学字符识别）系统中的关键组成部分，负责在图像中定位和标记出包含文本的区域。该模块的性能直接影响到整个OCR系统的准确性和效率。文本检测模块通常会输出文本区域的边界框（Bounding Boxes），这些边界框将作为输入传递给文本识别模块进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>检测Hmean（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_server_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>83.8</td>
|
||||
<td>89.55 / 70.19</td>
|
||||
<td>383.15 / 383.15</td>
|
||||
<td>84.3</td>
|
||||
<td>PP-OCRv5 的服务端文本检测模型，精度更高，适合在性能较好的服务器上部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_mobile_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>79.0</td>
|
||||
<td>10.67 / 6.36</td>
|
||||
<td>57.77 / 28.15</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv5 的移动端文本检测模型，效率更高，适合在端侧设备部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>69.2</td>
|
||||
<td>127.82 / 98.87</td>
|
||||
<td>585.95 / 489.77</td>
|
||||
<td>109</td>
|
||||
<td>PP-OCRv4 的服务端文本检测模型，精度更高，适合在性能较好的服务器上部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>63.8</td>
|
||||
<td>9.87 / 4.17</td>
|
||||
<td>56.60 / 20.79</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv4 的移动端文本检测模型，效率更高，适合在端侧设备部署</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>PaddleOCR3.0 全新构建多语种（包含中、繁、英、日），覆盖街景、网图、文档、手写、模糊、旋转、扭曲等多个场景的文本检测数据集，包含2677 张图片。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将文本检测的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection(model_name="PP-OCRv5_server_det")
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_001.png', 'page_index': None, 'dt_polys': array([[[ 75, 549],
|
||||
...,
|
||||
[ 77, 586]],
|
||||
|
||||
...,
|
||||
|
||||
[[ 31, 406],
|
||||
...,
|
||||
[ 34, 455]]], dtype=int16), 'dt_scores': [0.873949039891189, 0.8948166013613552, 0.8842595305917041, 0.876953790920377]}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待预测图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `dt_polys`：表示预测的文本检测框，其中每个文本检测框包含一个四边形的四个顶点。其中每个顶点都是一个列表，分别表示该顶点的x坐标和y坐标
|
||||
- `dt_scores`：表示预测的文本检测框的置信度
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_det/general_ocr_001_res.png"/>
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TextDetection`实例化文本检测模型（此处以`PP-OCRv5_server_det`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-OCRv5_server_det</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>检测的图像边长限制：<code>int</code> 表示边长限制数值。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>检测的图像边长限制，检测的边长限制类型，<code>"min"</code> 表示保证图像最短边不小于det_limit_side_len，<code>"max"</code>表示保证图像最长边不大于limit_side_len。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_side_limit</code></td>
|
||||
<td>检测的图像边长最大值限制：<code>int</code> 限制输入检测模型的图片最长边。如果设置为 <code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>像素得分阈值。输出概率图中得分大于该阈值的像素点被认为是文本像素。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>检测结果边框内，所有像素点的平均得分大于该阈值时，该结果会被认为是文字区域。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Vatti clipping算法的扩张系数，使用该方法对文字区域进行扩张。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>模型输入图像尺寸，格式为 <code>(C, H, W)</code>。</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文本检测模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input`、 `batch_size`、 `limit_side_len`、 `limit_type`、 `thresh`、 `box_thresh`、 `max_candidates`、`unclip_ratio`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-OCRv5_server_det` 举例，其他模型替换对应配置文件即可。首先，您需要准备文本检测的数据集，可以参考[文本检测 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以文本检测 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
### 4.1 数据集、预训练模型准备
|
||||
|
||||
#### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar
|
||||
tar -xf ocr_det_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-OCRv5_server_det 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `PP-OCRv5_server_det` 识别模型时需要使用 `PP-OCRv5_server_det` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv5/PP-OCRv5_server_det.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/PP-OCRv5_server_det/best_accuracy.pdprams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml -o \
|
||||
Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_det_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-OCRv5_server_det_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-OCRv5_server_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
|
||||
- 通过参数`limit_type`和`limit_side_len`来对图片的尺寸进行限制，`limit_type`可选参数为[`max`, `min`]，`limit_side_len` 为正整数，一般设置为 32 的倍数，比如 960。
|
||||
如果输入图形分辨率不大，建议使用`limit_type=min` 和 `limit_side_len=960` 节省计算资源的同时能获得最佳检测效果。如果输入图片的分辨率比较大，而且想使用更大的分辨率预测，可以设置 `limit_side_len` 为想要的值，比如 1216。
|
||||
327
docs/version3.x/module_usage/text_image_unwarping.en.md
Normal file
327
docs/version3.x/module_usage/text_image_unwarping.en.md
Normal file
@@ -0,0 +1,327 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
# Text Image Rectification Module Usage Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The primary purpose of text image rectification is to perform geometric transformations on images to correct distortions, inclinations, perspective deformations, etc., in the document images for more accurate subsequent text recognition.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>CER</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>UVDoc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UVDoc_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UVDoc_pretrained.pdparams">Training Model</a></td>
|
||||
<td>0.179</td>
|
||||
<td>19.05 / 19.05</td>
|
||||
<td>- / 869.82</td>
|
||||
<td>30.3</td>
|
||||
<td>High-accuracy text image rectification model</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> <a href="https://www3.cs.stonybrook.edu/~cvl/docunet.html">DocUNet benchmark</a> dataset.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Explanation</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Regular Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Choose the optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Choose the optimal prior backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please first install the PaddleOCR wheel package. For details, please refer to the [installation tutorial](../installation.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the image rectification module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(model_name="UVDoc")
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'doc_test.jpg', 'page_index': None, 'doctr_img': '...'}}
|
||||
```
|
||||
|
||||
The meanings of the parameters in the result are as follows:
|
||||
- `input_path`: Indicates the path of the image to be rectified
|
||||
- `doctr_img`: Indicates the rectified image result. Due to the large amount of data, it is not convenient to print directly, so it is replaced here with `...`. You can use `res.save_to_img()` to save the prediction result as an image, and `res.save_to_json()` to save the prediction result as a json file.
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/image_unwarp/doc_test_res.jpg">
|
||||
|
||||
The relevant methods, parameters, etc., are described as follows:
|
||||
|
||||
* `TextImageUnwarping` instantiates the image rectification model (taking `UVDoc` as an example here), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Name of the model</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device(s) to use for inference.<br/>
|
||||
<b>Examples:</b> <code>cpu</code>, <code>gpu</code>, <code>npu</code>, <code>gpu:0</code>, <code>gpu:0,1</code>.<br/>
|
||||
If multiple devices are specified, inference will be performed in parallel. Note that parallel inference is not always supported.<br/>
|
||||
By default, GPU 0 will be used if available; otherwise, the CPU will be used.
|
||||
</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to use the high performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Precision for TensorRT when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>fp32</code>, <code>fp16</code>, etc.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>fp32</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the image rectification model for inference prediction. This method will return a result list. Additionally, this module also provides a `predict_iter()` method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods according to your actual needs. The `predict()` method has parameters `input` and `batch_size`, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
The current module does not support fine-tuning training and only supports inference integration. Concerning fine-tuning training for this module, there are plans to support it in the future.
|
||||
|
||||
## 5. FAQ
|
||||
325
docs/version3.x/module_usage/text_image_unwarping.md
Normal file
325
docs/version3.x/module_usage/text_image_unwarping.md
Normal file
@@ -0,0 +1,325 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本图像矫正模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文本图像矫正的主要目的是针对图像进行几何变换，以纠正图像中的文档扭曲、倾斜、透视变形等问题，以供后续的文本识别进行更加准确。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>CER </th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>UVDoc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UVDoc_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UVDoc_pretrained.pdparams">训练模型</a></td>
|
||||
<td>0.179</td>
|
||||
<td>19.05 / 19.05</td>
|
||||
<td>- / 869.82</td>
|
||||
<td>30.3</td>
|
||||
<td>高精度文本图像矫正模型</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong><a href="https://www3.cs.stonybrook.edu/~cvl/docunet.html">DocUNet benchmark</a>数据集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将图像矫正的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(model_name="UVDoc")
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'doc_test.jpg', 'page_index': None, 'doctr_img': '...'}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待矫正图像的路径
|
||||
- `doctr_img`：表示矫正后的图像结果，由于数据过多不便于直接print，所以此处用`...`替换，可以通过`res.save_to_img()`将预测结果保存为图片，通过`res.save_to_json()`将预测结果保存为json文件。
|
||||
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/image_unwarp/doc_test_res.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TextImageUnwarping`实例化图像矫正模型（此处以`UVDoc`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>UVDoc</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用图像矫正模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
当前模块暂时不支持微调训练，仅支持推理集成。关于该模块的微调训练，计划在未来支持。
|
||||
|
||||
## 五、FAQ
|
||||
758
docs/version3.x/module_usage/text_recognition.en.md
Normal file
758
docs/version3.x/module_usage/text_recognition.en.md
Normal file
@@ -0,0 +1,758 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Text Recognition Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The text recognition module is the core part of the OCR (Optical Character Recognition) system, responsible for extracting text information from text regions in images. The performance of this module directly affects the accuracy and efficiency of the entire OCR system. The text recognition module usually receives the bounding boxes of text regions output by the text detection module as input, and then converts the text in the images into editable and searchable electronic text through complex image processing and deep learning algorithms. The accuracy of text recognition results is crucial for subsequent applications such as information extraction and data mining.
|
||||
|
||||
## 2. List of Supported Models
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_server_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>86.38</td>
|
||||
<td>8.46 / 2.36</td>
|
||||
<td>31.21 / 31.21</td>
|
||||
<td>81</td>
|
||||
<td rowspan="2">PP-OCRv5_rec is a new generation text recognition model. It is designed to efficiently and accurately support the recognition of Simplified Chinese, Traditional Chinese, English, Japanese, as well as complex text scenarios such as handwriting, vertical text, pinyin, and rare characters with a single model. While maintaining recognition performance, it also balances inference speed and model robustness, providing efficient and accurate technical support for document understanding in various scenarios.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>81.29</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>16</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec_doc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv4_server_rec_doc_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_doc_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>86.58</td>
|
||||
<td>8.69 / 2.78</td>
|
||||
<td>37.93 / 37.93</td>
|
||||
<td>182</td>
|
||||
<td>PP-OCRv4_server_rec_doc is trained on a mixed dataset of more Chinese document data and PP-OCR training data, building upon PP-OCRv4_server_rec. It enhances the recognition capabilities for some Traditional Chinese characters, Japanese characters, and special symbols, supporting over 15,000 characters. In addition to improving document-related text recognition, it also enhances general text recognition capabilities.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>78.74</td>
|
||||
<td>5.26 / 1.12</td>
|
||||
<td>17.48 / 3.61</td>
|
||||
<td>10.5</td>
|
||||
<td>A lightweight recognition model of PP-OCRv4 with high inference efficiency, suitable for deployment on various hardware devices, including edge devices.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>85.19</td>
|
||||
<td>8.75 / 2.49</td>
|
||||
<td>36.93 / 36.93</td>
|
||||
<td>173</td>
|
||||
<td>The server-side model of PP-OCRv4, offering high inference accuracy and deployable on various servers.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv4_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv4_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>70.39</td>
|
||||
<td>4.81 / 1.23</td>
|
||||
<td>17.20 / 4.18</td>
|
||||
<td>7.5</td>
|
||||
<td>An ultra-lightweight English recognition model trained based on the PP-OCRv4 recognition model, supporting English and numeric character recognition.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
> ❗ The above lists the <b>4 core models</b> mainly supported by the text recognition module. The module supports a total of <b>20 full models</b>, including multiple multilingual text recognition models. The complete model list is as follows:
|
||||
|
||||
<details><summary> 👉Model List Details</summary>
|
||||
|
||||
* <b>PP-OCRv5 Multi-Scenario Models</b>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Chinese Recognition Avg Accuracy(%)</th>
|
||||
<th>English Recognition Avg Accuracy(%)</th>
|
||||
<th>Traditional Chinese Recognition Avg Accuracy(%)</th>
|
||||
<th>Japanese Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_server_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>86.38</td>
|
||||
<td>64.70</td>
|
||||
<td>93.29</td>
|
||||
<td>60.35</td>
|
||||
<td>8.46 / 2.36</td>
|
||||
<td>31.21 / 31.21</td>
|
||||
<td>81</td>
|
||||
<td rowspan="2">PP-OCRv5_rec is a new generation text recognition model. It is designed to efficiently and accurately support the recognition of Simplified Chinese, Traditional Chinese, English, Japanese, as well as complex text scenarios such as handwriting, vertical text, pinyin, and rare characters with a single model. While maintaining recognition performance, it also balances inference speed and model robustness, providing efficient and accurate technical support for document understanding in various scenarios.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>81.29</td>
|
||||
<td>66.00</td>
|
||||
<td>83.55</td>
|
||||
<td>54.65</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>16</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>Chinese Recognition Models</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec_doc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv4_server_rec_doc_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_doc_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>86.58</td>
|
||||
<td>8.69 / 2.78</td>
|
||||
<td>37.93 / 37.93</td>
|
||||
<td>182</td>
|
||||
<td>PP-OCRv4_server_rec_doc is trained on a mixed dataset of more Chinese document data and PP-OCR training data, building upon PP-OCRv4_server_rec. It enhances the recognition capabilities for some Traditional Chinese characters, Japanese characters, and special symbols, supporting over 15,000 characters. In addition to improving document-related text recognition, it also enhances general text recognition capabilities.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>78.74</td>
|
||||
<td>5.26 / 1.12</td>
|
||||
<td>17.48 / 3.61</td>
|
||||
<td>10.5</td>
|
||||
<td>A lightweight recognition model of PP-OCRv4 with high inference efficiency, suitable for deployment on various hardware devices, including edge devices.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>85.19</td>
|
||||
<td>8.75 / 2.49</td>
|
||||
<td>36.93 / 36.93</td>
|
||||
<td>173</td>
|
||||
<td>The server-side model of PP-OCRv4, offering high inference accuracy and deployable on various servers.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>72.96</td>
|
||||
<td>3.89 / 1.16</td>
|
||||
<td>8.72 / 3.56</td>
|
||||
<td>10.3</td>
|
||||
<td>A lightweight recognition model of PP-OCRv3 with high inference efficiency, suitable for deployment on various hardware devices, including edge devices.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_SVTRv2_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_SVTRv2_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_SVTRv2_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>68.81</td>
|
||||
<td>10.38 / 8.31</td>
|
||||
<td>66.52 / 30.83</td>
|
||||
<td>80.5</td>
|
||||
<td rowspan="1">SVTRv2 is a server-side text recognition model developed by the OpenOCR team of the Vision and Learning Lab (FVL) at Fudan University. It won the first prize in the PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task, with a 6% improvement in end-to-end recognition accuracy on Leaderboard A compared to PP-OCRv4.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_RepSVTR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_RepSVTR_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_RepSVTR_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>65.07</td>
|
||||
<td>6.29 / 1.57</td>
|
||||
<td>20.64 / 5.40</td>
|
||||
<td>22.1</td>
|
||||
<td rowspan="1">RepSVTR is a mobile-side text recognition model based on SVTRv2. It won the first prize in the PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task, with a 2.5% improvement in end-to-end recognition accuracy on Leaderboard B compared to PP-OCRv4, while maintaining similar inference speed.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>English Recognition Models</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv4_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv4_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td> 70.39</td>
|
||||
<td>4.81 / 1.23</td>
|
||||
<td>17.20 / 4.18</td>
|
||||
<td>7.5</td>
|
||||
<td>An ultra-lightweight English recognition model trained based on the PP-OCRv4 recognition model, supporting English and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>70.69</td>
|
||||
<td>3.56 / 0.78</td>
|
||||
<td>8.44 / 5.78</td>
|
||||
<td>17.3</td>
|
||||
<td>An ultra-lightweight English recognition model trained based on the PP-OCRv3 recognition model, supporting English and numeric character recognition.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>Multilingual Recognition Models</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>korean_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
korean_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv5_mobile_rec_pretrained.pdparams">Pre-trained Model</a></td>
|
||||
<td>90.45</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>An ultra-lightweight Korean text recognition model trained based on the PP-OCRv5 recognition framework. Supports Korean, English and numeric text recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>latin_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
latin_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv5_mobile_rec_pretrained.pdparams">Pre-trained Model</a></td>
|
||||
<td>84.7</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>A Latin-script text recognition model trained based on the PP-OCRv5 recognition framework. Supports most Latin alphabet languages and numeric text recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>eslav_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
eslav_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/eslav_PP-OCRv5_mobile_rec_pretrained.pdparams">Pre-trained Model</a></td>
|
||||
<td>85.8</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>An East Slavic language recognition model trained based on the PP-OCRv5 recognition framework. Supports East Slavic languages, English and numeric text recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>korean_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
korean_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>60.21</td>
|
||||
<td>3.73 / 0.98</td>
|
||||
<td>8.76 / 2.91</td>
|
||||
<td>9.6</td>
|
||||
<td>An ultra-lightweight Korean recognition model trained based on the PP-OCRv3 recognition model, supporting Korean and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>japan_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
japan_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/japan_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>45.69</td>
|
||||
<td>3.86 / 1.01</td>
|
||||
<td>8.62 / 2.92</td>
|
||||
<td>9.8</td>
|
||||
<td>An ultra-lightweight Japanese recognition model trained based on the PP-OCRv3 recognition model, supporting Japanese and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>chinese_cht_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
chinese_cht_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/chinese_cht_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>82.06</td>
|
||||
<td>3.90 / 1.16</td>
|
||||
<td>9.24 / 3.18</td>
|
||||
<td>10.8</td>
|
||||
<td>An ultra-lightweight Traditional Chinese recognition model trained based on the PP-OCRv3 recognition model, supporting Traditional Chinese and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>te_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
te_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/te_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>95.88</td>
|
||||
<td>3.59 / 0.81</td>
|
||||
<td>8.28 / 6.21</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Telugu recognition model trained based on the PP-OCRv3 recognition model, supporting Telugu and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ka_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ka_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ka_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>96.96</td>
|
||||
<td>3.49 / 0.89</td>
|
||||
<td>8.63 / 2.77</td>
|
||||
<td>17.4</td>
|
||||
<td>An ultra-lightweight Kannada recognition model trained based on the PP-OCRv3 recognition model, supporting Kannada and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ta_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ta_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ta_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>76.83</td>
|
||||
<td>3.49 / 0.86</td>
|
||||
<td>8.35 / 3.41</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Tamil recognition model trained based on the PP-OCRv3 recognition model, supporting Tamil and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>latin_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
latin_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>76.93</td>
|
||||
<td>3.53 / 0.78</td>
|
||||
<td>8.50 / 6.83</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Latin recognition model trained based on the PP-OCRv3 recognition model, supporting Latin and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>arabic_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
arabic_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/arabic_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>73.55</td>
|
||||
<td>3.60 / 0.83</td>
|
||||
<td>8.44 / 4.69</td>
|
||||
<td>17.3</td>
|
||||
<td>An ultra-lightweight Arabic alphabet recognition model trained based on the PP-OCRv3 recognition model, supporting Arabic alphabet and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>cyrillic_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
cyrillic_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/cyrillic_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>94.28</td>
|
||||
<td>3.56 / 0.79</td>
|
||||
<td>8.22 / 2.76</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Cyrillic alphabet recognition model trained based on the PP-OCRv3 recognition model, supporting Cyrillic alphabet and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>devanagari_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
devanagari_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/devanagari_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>96.44</td>
|
||||
<td>3.60 / 0.78</td>
|
||||
<td>6.95 / 2.87</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Devanagari alphabet recognition model trained based on the PP-OCRv3 recognition model, supporting Devanagari alphabet and numeric character recognition.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong>
|
||||
<ul>
|
||||
<li>
|
||||
Chinese Recognition Models: A self-built Chinese dataset by PaddleOCR, covering street views, online images, documents, handwriting, with 11,000 images for text recognition.
|
||||
</li>
|
||||
<li>
|
||||
ch_SVTRv2_rec: <a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task</a> Leaderboard A evaluation set.
|
||||
</li>
|
||||
<li>
|
||||
ch_RepSVTR_rec: <a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task</a> Leaderboard B evaluation set.
|
||||
</li>
|
||||
<li>
|
||||
English Recognition Models: A self-built English dataset by PaddleOCR.
|
||||
</li>
|
||||
<li>
|
||||
Multilingual Recognition Models: A self-built multilingual dataset by PaddleOCR.
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Explanation of Inference Modes</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Selection of the optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
</details>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting, please install the PaddleOCR wheel package. For details, please refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr text_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png
|
||||
```
|
||||
|
||||
<b>Note:</b> The official PaddleOCR models are downloaded from HuggingFace by default. If you cannot access HuggingFace, you can change the model source to BOS by setting the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"`. More mainstream model sources will be supported in the future.
|
||||
|
||||
You can also integrate the model inference of the text recognition module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png) to your local machine.
|
||||
|
||||
```python
|
||||
from paddleocr import TextRecognition
|
||||
model = TextRecognition(model_name="PP-OCRv5_server_rec")
|
||||
output = model.predict(input="general_ocr_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result is as follows:
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_rec_001.png', 'page_index': None, 'rec_text': '绿洲仕格维花园公寓', 'rec_score': 0.9823867082595825}}
|
||||
```
|
||||
|
||||
The meanings of the parameters in the result are as follows:
|
||||
- `input_path`: The path of the input text line image to be predicted
|
||||
- `page_index`: If the input is a PDF file, it indicates which page of the PDF the current text line is from; otherwise, it is `None`
|
||||
- `rec_text`: The predicted text of the text line image
|
||||
- `rec_score`: The confidence score of the predicted text for the text line image
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_recog/general_ocr_rec_001.png"/>
|
||||
|
||||
Descriptions of related methods and parameters are as follows:
|
||||
|
||||
* Instantiate the text recognition model using `TextRecognition` (using `PP-OCRv5_server_rec` as an example), as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>If set to <code>None</code>, <code>PP-OCRv5_server_rec</code> is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>Examples:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, inference will be performed in parallel.<br/>
|
||||
By default, GPU 0 is used; if unavailable, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to enable the TensorRT subgraph engine of Paddle Inference.<br/>
|
||||
For Paddle with CUDA 11.8, the compatible TensorRT version is 8.x (x>=6), recommended 8.6.1.6.<br/>
|
||||
For Paddle with CUDA 12.6, the compatible TensorRT version is 10.x (x>=5), recommended 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Precision for TensorRT when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>fp32</code>, <code>fp16</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>Input image size for the model in the format <code>(C, H, W)</code>.</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the text recognition model for inference. This method returns a list of results. In addition, this module also provides the `predict_iter()` method. The two methods are completely consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step. It is suitable for scenarios where large datasets need to be processed or memory savings are desired. You can choose either of these two methods according to your actual needs. The parameters of the `predict()` method include `input` and `batch_size`, with specific descriptions as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Data to be predicted, supporting multiple input types, required.
|
||||
<ul>
|
||||
<li><b>Python Var</b>: Image data represented by <code>numpy.ndarray</code></li>
|
||||
<li><b>str</b>: Local path of image file or PDF file: <code>/root/data/img.jpg</code>; <b>URL link</b>: Network URL of image file or PDF file: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">Example</a>; <b>Local directory</b>: The directory should contain the images to be predicted, such as <code>/root/data/</code> (currently, prediction of PDF files in the directory is not supported, PDF files need to be specified to a specific file path)</li>
|
||||
<li><b>list</b>: The elements of the list should be data of the above types, such as <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, can be set to any positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports operations such as printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable. Only effective when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. Only effective when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a file in <code>json</code> format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save the result. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable. Only effective when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. Only effective when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as a file in image format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save the result. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining the visualized image with results and the prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Obtain the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Obtain the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
If the above models do not perform well in your scenario, you can try the following steps for secondary development. Here, we take training `PP-OCRv5_server_rec` as an example. For other models, just replace the corresponding configuration file. First, you need to prepare a dataset for text recognition. You can refer to the format of the [Text Recognition Demo Data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_dataset_examples.tar) for preparation. After preparation, you can train and export the model as follows. After export, the model can be quickly integrated into the above API. This example uses the Text Recognition Demo Data. Before training the model, please make sure you have installed the dependencies required by PaddleOCR as described in the [Installation Guide](../installation.md).
|
||||
|
||||
### 4.1 Dataset and Pre-trained Model Preparation
|
||||
|
||||
#### 4.1.1 Prepare the Dataset
|
||||
|
||||
```shell
|
||||
# Download the example dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_dataset_examples.tar
|
||||
tar -xf ocr_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 Download the Pre-trained Model
|
||||
|
||||
```shell
|
||||
# Download the PP-OCRv5_server_rec pre-trained model
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR modularizes its code. To train the `PP-OCRv5_server_rec` recognition model, you need to use its [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml).
|
||||
|
||||
The training commands are as follows:
|
||||
|
||||
```bash
|
||||
# Single-GPU training (default training method)
|
||||
python3 tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
|
||||
# Multi-GPU training, specify GPU IDs via the --gpus parameter
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command:
|
||||
|
||||
```bash
|
||||
# Note: Set the path of pretrained_model to a local path. If you use a model you trained and saved yourself, please modify the path and file name to {path/to/weights}/{model_name}.
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_rec_infer/"
|
||||
```
|
||||
|
||||
After exporting the model, the static graph model will be stored in `./PP-OCRv5_server_rec_infer/` in the current directory. In this directory, you will see the following files:
|
||||
```
|
||||
./PP-OCRv5_server_rec_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
At this point, the secondary development is complete. This static graph model can be directly integrated into the PaddleOCR API.
|
||||
|
||||
## 5. FAQ
|
||||
770
docs/version3.x/module_usage/text_recognition.md
Normal file
770
docs/version3.x/module_usage/text_recognition.md
Normal file
@@ -0,0 +1,770 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本识别模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文本识别模块是OCR（光学字符识别）系统中的核心部分，负责从图像中的文本区域提取出文本信息。该模块的性能直接影响到整个OCR系统的准确性和效率。文本识别模块通常接收文本检测模块输出的文本区域的边界框（Bounding Boxes）作为输入，然后通过复杂的图像处理和深度学习算法，将图像中的文本转化为可编辑和可搜索的电子文本。文本识别结果的准确性，对于后续的信息提取和数据挖掘等应用至关重要。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>86.38</td>
|
||||
<td>8.46 / 2.36</td>
|
||||
<td>31.21 / 31.21</td>
|
||||
<td>81</td>
|
||||
<td rowspan="2">PP-OCRv5_rec 是新一代文本识别模型。该模型致力于以单一模型高效、精准地支持简体中文、繁体中文、英文、日文四种主要语言，以及手写、竖版、拼音、生僻字等复杂文本场景的识别。在保持识别效果的同时，兼顾推理速度和模型鲁棒性，为各种场景下的文档理解提供高效、精准的技术支撑。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>81.29</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>16</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec_doc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv4_server_rec_doc_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_doc_pretrained.pdparams">训练模型</a></td>
|
||||
<td>86.58</td>
|
||||
<td>8.69 / 2.78</td>
|
||||
<td>37.93 / 37.93</td>
|
||||
<td>182</td>
|
||||
<td>PP-OCRv4_server_rec_doc是在PP-OCRv4_server_rec的基础上，在更多中文文档数据和PP-OCR训练数据的混合数据训练而成，增加了部分繁体字、日文、特殊字符的识别能力，可支持识别的字符为1.5万+，除文档相关的文字识别能力提升外，也同时提升了通用文字的识别能力</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>78.74</td>
|
||||
<td>5.26 / 1.12</td>
|
||||
<td>17.48 / 3.61</td>
|
||||
<td>10.5</td>
|
||||
<td>PP-OCRv4的轻量级识别模型，推理效率高，可以部署在包含端侧设备的多种硬件设备中</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>85.19</td>
|
||||
<td>8.75 / 2.49</td>
|
||||
<td>36.93 / 36.93</td>
|
||||
<td>173</td>
|
||||
<td>PP-OCRv4的服务器端模型，推理精度高，可以部署在多种不同的服务器上</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv4_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>70.39</td>
|
||||
<td>4.81 / 1.23</td>
|
||||
<td>17.20 / 4.18</td>
|
||||
<td>7.5</td>
|
||||
<td>基于PP-OCRv4识别模型训练得到的超轻量英文识别模型，支持英文、数字识别</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
> ❗ 以上列出的是文本识别模块重点支持的<b>4个核心模型</b>，该模块总共支持<b>20个全量模型</b>，包含多个多语言文本识别模型，完整的模型列表如下：
|
||||
|
||||
<details><summary> 👉模型列表详情</summary>
|
||||
|
||||
* <b>PP-OCRv5 多场景模型</b>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>中文识别 Avg Accuracy(%)</th>
|
||||
<th>英文识别 Avg Accuracy(%)</th>
|
||||
<th>繁体中文识别 Avg Accuracy(%)</th>
|
||||
<th>日文识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>86.38</td>
|
||||
<td>64.70</td>
|
||||
<td>93.29</td>
|
||||
<td>60.35</td>
|
||||
<td>8.46 / 2.36</td>
|
||||
<td>31.21 / 31.21</td>
|
||||
<td>81</td>
|
||||
<td rowspan="2">PP-OCRv5_rec 是新一代文本识别模型。该模型致力于以单一模型高效、精准地支持简体中文、繁体中文、英文、日文四种主要语言，以及手写、竖版、拼音、生僻字等复杂文本场景的识别。在保持识别效果的同时，兼顾推理速度和模型鲁棒性，为各种场景下的文档理解提供高效、精准的技术支撑。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>81.29</td>
|
||||
<td>66.00</td>
|
||||
<td>83.55</td>
|
||||
<td>54.65</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>16</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>中文识别模型</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec_doc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv4_server_rec_doc_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_doc_pretrained.pdparams">训练模型</a></td>
|
||||
<td>86.58</td>
|
||||
<td>8.69 / 2.78</td>
|
||||
<td>37.93 / 37.93</td>
|
||||
<td>182</td>
|
||||
<td>PP-OCRv4_server_rec_doc是在PP-OCRv4_server_rec的基础上，在更多中文文档数据和PP-OCR训练数据的混合数据训练而成，增加了部分繁体字、日文、特殊字符的识别能力，可支持识别的字符为1.5万+，除文档相关的文字识别能力提升外，也同时提升了通用文字的识别能力</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>78.74</td>
|
||||
<td>5.26 / 1.12</td>
|
||||
<td>17.48 / 3.61</td>
|
||||
<td>10.5</td>
|
||||
<td>PP-OCRv4的轻量级识别模型，推理效率高，可以部署在包含端侧设备的多种硬件设备中</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>85.19</td>
|
||||
<td>8.75 / 2.49</td>
|
||||
<td>36.93 / 36.93</td>
|
||||
<td>173</td>
|
||||
<td>PP-OCRv4的服务器端模型，推理精度高，可以部署在多种不同的服务器上</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>72.96</td>
|
||||
<td>3.89 / 1.16</td>
|
||||
<td>8.72 / 3.56</td>
|
||||
<td>10.3</td>
|
||||
<td>PP-OCRv3的轻量级识别模型，推理效率高，可以部署在包含端侧设备的多种硬件设备中</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_SVTRv2_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_SVTRv2_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_SVTRv2_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>68.81</td>
|
||||
<td>10.38 / 8.31</td>
|
||||
<td>66.52 / 30.83</td>
|
||||
<td>80.5</td>
|
||||
<td rowspan="1">
|
||||
SVTRv2 是一种由复旦大学视觉与学习实验室（FVL）的OpenOCR团队研发的服务端文本识别模型，其在PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务中荣获一等奖，A榜端到端识别精度相比PP-OCRv4提升6%。
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_RepSVTR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_RepSVTR_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_RepSVTR_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>65.07</td>
|
||||
<td>6.29 / 1.57</td>
|
||||
<td>20.64 / 5.40</td>
|
||||
<td>48.8</td>
|
||||
<td rowspan="1"> RepSVTR 文本识别模型是一种基于SVTRv2 的移动端文本识别模型，其在PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务中荣获一等奖，B榜端到端识别精度相比PP-OCRv4提升2.5%，推理速度持平。</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>英文识别模型</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv4_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td> 70.39</td>
|
||||
<td>4.81 / 1.23</td>
|
||||
<td>17.20 / 4.18</td>
|
||||
<td>7.5</td>
|
||||
<td>基于PP-OCRv4识别模型训练得到的超轻量英文识别模型，支持英文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>70.69</td>
|
||||
<td>3.56 / 0.78</td>
|
||||
<td>8.44 / 5.78</td>
|
||||
<td>17.3</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量英文识别模型，支持英文、数字识别</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>多语言识别模型</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>korean_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
korean_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>90.45</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>基于PP-OCRv5识别模型训练得到的超轻量韩文识别模型，支持韩文、英文和数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>latin_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
latin_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>84.7</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>基于PP-OCRv5识别模型训练得到的拉丁文识别模型，支持大部分拉丁字母语言、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>eslav_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
eslav_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/eslav_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>85.8</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>基于PP-OCRv5识别模型训练得到的东斯拉夫语言识别模型， 支持东斯拉夫语言、英文和数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>korean_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
korean_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>60.21</td>
|
||||
<td>3.73 / 0.98</td>
|
||||
<td>8.76 / 2.91</td>
|
||||
<td>9.6</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量韩文识别模型，支持韩文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>japan_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
japan_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/japan_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>45.69</td>
|
||||
<td>3.86 / 1.01</td>
|
||||
<td>8.62 / 2.92</td>
|
||||
<td>9.8</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量日文识别模型，支持日文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>chinese_cht_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
chinese_cht_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/chinese_cht_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>82.06</td>
|
||||
<td>3.90 / 1.16</td>
|
||||
<td>9.24 / 3.18</td>
|
||||
<td>10.8</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量繁体中文识别模型，支持繁体中文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>te_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
te_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/te_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>95.88</td>
|
||||
<td>3.59 / 0.81</td>
|
||||
<td>8.28 / 6.21</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量泰卢固文识别模型，支持泰卢固文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ka_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ka_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ka_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>96.96</td>
|
||||
<td>3.49 / 0.89</td>
|
||||
<td>8.63 / 2.77</td>
|
||||
<td>17.4</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量卡纳达文识别模型，支持卡纳达文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ta_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ta_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ta_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>76.83</td>
|
||||
<td>3.49 / 0.86</td>
|
||||
<td>8.35 / 3.41</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量泰米尔文识别模型，支持泰米尔文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>latin_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
latin_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>76.93</td>
|
||||
<td>3.53 / 0.78</td>
|
||||
<td>8.50 / 6.83</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量拉丁文识别模型，支持拉丁文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>arabic_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
arabic_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/arabic_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>73.55</td>
|
||||
<td>3.60 / 0.83</td>
|
||||
<td>8.44 / 4.69</td>
|
||||
<td>17.3</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量阿拉伯字母识别模型，支持阿拉伯字母、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>cyrillic_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
cyrillic_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/cyrillic_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>94.28</td>
|
||||
<td>3.56 / 0.79</td>
|
||||
<td>8.22 / 2.76</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量斯拉夫字母识别模型，支持斯拉夫字母、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>devanagari_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
devanagari_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/devanagari_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>96.44</td>
|
||||
<td>3.60 / 0.78</td>
|
||||
<td>6.95 / 2.87</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量梵文字母识别模型，支持梵文字母、数字识别</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>
|
||||
<ul>
|
||||
<li>
|
||||
中文识别模型： PaddleOCR 自建的中文数据集，覆盖街景、网图、文档、手写多个场景，其中文本识别包含 1.1w 张图片。
|
||||
</li>
|
||||
<li>
|
||||
ch_SVTRv2_rec：<a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务</a>A榜评估集。
|
||||
</li>
|
||||
<li>
|
||||
ch_RepSVTR_rec：<a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务</a>B榜评估集。
|
||||
</li>
|
||||
<li>
|
||||
英文识别模型：PaddleOCR 自建的英文数据集。
|
||||
</li>
|
||||
<li>
|
||||
多语言识别模型：PaddleOCR 自建的多语种数据集。
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
</details>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr text_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将文本识别的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextRecognition
|
||||
model = TextRecognition(model_name="PP-OCRv5_server_rec")
|
||||
output = model.predict(input="general_ocr_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_rec_001.png', 'page_index': None, 'rec_text': '绿洲仕格维花园公寓', 'rec_score': 0.9823867082595825}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待预测文本行图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `rec_text`：表示文本行图像的预测文本
|
||||
- `rec_score`：表示文本行图像的预测置信度
|
||||
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_recog/general_ocr_rec_001.png"/>
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TextRecognition`实例化文本识别模型（此处以`PP-OCRv5_server_rec`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-OCRv5_server_rec</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>模型输入图像尺寸，格式为 <code>(C, H, W)</code>。</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文本识别模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-OCRv5_server_rec` 举例，其他模型替换对应配置文件即可。首先，您需要准备文本识别的数据集，可以参考[文本识别 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以文本识别 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
## 4.1 数据集、预训练模型准备
|
||||
|
||||
### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_dataset_examples.tar
|
||||
tar -xf ocr_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-OCRv5_server_rec 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `PP-OCRv5_server_rec` 识别模型时需要使用 `PP-OCRv5_server_rec` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
```
|
||||
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
#注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
#demo 测试集评估
|
||||
python3 tools/eval.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_rec_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-OCRv5_server_rec_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-OCRv5_server_rec_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
@@ -0,0 +1,372 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Text Line Orientation Classification Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
The text line orientation classification module primarily distinguishes the orientation of text lines and corrects them using post-processing. In processes such as document scanning and license/certificate photography, to capture clearer images, the capture device may be rotated, resulting in text lines in various orientations. Standard OCR pipelines cannot handle such data well. By utilizing image classification technology, the orientation of text lines can be predetermined and adjusted, thereby enhancing the accuracy of OCR processing.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Top-1 Accuracy (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x0_25_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x0_25_textline_ori_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.85</td>
|
||||
<td>2.16 / 0.41</td>
|
||||
<td>2.37 / 0.73</td>
|
||||
<td>0.96</td>
|
||||
<td>Text line classification model based on PP-LCNet_x0_25, with two classes: 0 degrees and 180 degrees</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_textline_ori_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_textline_ori_pretrained.pdparams">Training Model</a></td>
|
||||
<td>99.42</td>
|
||||
<td>- / -</td>
|
||||
<td>2.98 / 2.98</td>
|
||||
<td>6.5</td>
|
||||
<td>Text line classification model based on PP-LCNet_x1_0, with two classes: 0 degrees and 180 degrees</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ **Note**: The text line orientation classification model was upgraded on May 26, 2025, and `PP-LCNet_x1_0_textline_ori` has been added. If you need to use the pre-upgrade model weights, please click the <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.bak.tar">download link</a>.
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong> PaddleX Self-built Dataset, Covering Multiple Scenarios Such as Documents and Certificates, Containing 1000 Images.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Integration
|
||||
|
||||
> ❗ Before starting, please install the wheel package of PaddleOCR. For detailed instructions, refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience the functionality with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the text line orientation classification model into your project. Run the following code after downloading the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg) to your local machine.
|
||||
|
||||
```bash
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(model_name="PP-LCNet_x0_25_textline_ori")
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'textline_rot180_demo.jpg', 'page_index': None, 'class_ids': array([1], dtype=int32), 'scores': array([0.99864], dtype=float32), 'label_names': ['180_degree']}}
|
||||
```
|
||||
|
||||
The meanings of the running results parameters are as follows:
|
||||
|
||||
- `input_path`：Indicates the path of the input image.
|
||||
- `page_index`：If the input is a PDF file, it indicates the current page number of the PDF; otherwise, it is `None`.
|
||||
- `class_ids`：Indicates the class ID of the prediction result.
|
||||
- `scores`：Indicates the confidence score of the prediction result.
|
||||
- `label_names`：Indicates the class name of the prediction result.
|
||||
The visualization image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/textline_ori_classification/textline_rot180_demo_res.jpg">
|
||||
|
||||
The explanations for the methods, parameters, etc., are as follows:
|
||||
|
||||
* `TextLineOrientationClassification` instantiates a textline classification model (here, `PP-LCNet_x0_25_textline_ori` is used as an example), and the specific explanations are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-LCNet_x0_25_textline_ori</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Use the `predict()` method of the text line direction classification model to perform inference. This method returns a list of results. In addition, this module also provides the `predict_iter()` method. Both methods accept the same parameters and return the same result format. The difference is that `predict_iter()` returns a `generator`, which processes and retrieves prediction results step by step. It is suitable for handling large datasets or memory-efficient scenarios. You can choose either method based on your actual needs. The `predict()` method accepts the parameters `input` and `batch_size`, which are described in detail below:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the text line orientation classification model for inference. This method will return a list of results. In addition, this module also provides a `predict_iter()` method. Both methods accept the same parameters and return the same results, but `predict_iter()` returns a `generator`, which is more suitable for processing large datasets or when you want to save memory. You can choose either method according to your needs. The parameters of the `predict()` method are `input` and `batch_size`, as described below:
|
||||
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Options</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Data to be predicted, supporting multiple input types</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>Python variable</b>, such as image data represented by <code>numpy.ndarray</code></li>
|
||||
<li><b>File path</b>, such as the local path of an image file: <code>/root/data/img.jpg</code></li>
|
||||
<li><b>URL link</b>, such as the network URL of an image file: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg">Example</a></li>
|
||||
<li><b>Local directory</b>, the directory should contain data files to be predicted, such as the local path: <code>/root/data/</code></li>
|
||||
<li><b>list</b>, the elements of the list should be of the above-mentioned data types, such as <code>[numpy.ndarray, numpy.ndarray]</code>, <code>[\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"]</code>, <code>[\"/root/data1\", \"/root/data2\"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size</td>
|
||||
<td><code>int</code></td>
|
||||
<td>Any integer</td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* The prediction results are processed, and the prediction result for each sample is of type `dict`. It supports operations such as printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the results to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. If set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the results as a JSON file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. If it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. If set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the results as an image file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. If it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, it supports obtaining the visualization image with results and the prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Attribute Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visualization image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Custom Development
|
||||
|
||||
Since PaddleOCR does not natively support training for text line orientation classification, refer to [PaddleX's Custom Development Guide](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/textline_orientation_classification.html#iv-custom-development) for training. Trained models can seamlessly integrate into PaddleOCR's API for inference.
|
||||
@@ -0,0 +1,339 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本行方向分类模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文本行方向分类模块主要是将文本行的方向区分出来，并使用后处理将其矫正。在诸如文档扫描、证照拍摄等过程中，有时为了拍摄更清晰，会将拍摄设备进行旋转，导致得到的文本行也是不同方向的。此时，标准的OCR流程无法很好地应对这些数据。利用图像分类技术，可以预先判断文本行方向，并将其进行方向调整，从而提高OCR处理的准确性。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>模型下载链接</th>
|
||||
<th>Top-1 Acc（%）</th>
|
||||
<th>GPU推理耗时（ms）</th>
|
||||
<th>CPU推理耗时 (ms)</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x0_25_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x0_25_textline_ori_pretrained.pdparams">训练模型</a></td>
|
||||
<td>98.85</td>
|
||||
<td>2.16 / 0.41</td>
|
||||
<td>2.37 / 0.73</td>
|
||||
<td>0.96</td>
|
||||
<td>基于PP-LCNet_x0_25的文本行分类模型，含有两个类别，即0度，180度</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_textline_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_textline_ori_pretrained.pdparams">训练模型</a></td>
|
||||
<td>99.42</td>
|
||||
<td>- / -</td>
|
||||
<td>2.98 / 2.98</td>
|
||||
<td>6.5</td>
|
||||
<td>基于PP-LCNet_x1_0的文本行分类模型，含有两个类别，即0度，180度</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ <b>注</b>：文本行方向分类模型于 2025.5.26 升级，并增加 `PP-LCNet_x1_0_textline_ori`，如需使用升级前的模型权重，请点击<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.bak.tar">下载链接</a>。
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>PaddleOCR 自建的数据集，覆盖证件和文档等多个场景，包含 1000 张图片。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将文本行方向分类模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(model_name="PP-LCNet_x0_25_textline_ori")
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'textline_rot180_demo.jpg', 'page_index': None, 'class_ids': array([1], dtype=int32), 'scores': array([0.99864], dtype=float32), 'label_names': ['180_degree']}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入图片的路径。
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `class_ids`：表示预测结果的类别 id，含有两个类别，即0度和180度。
|
||||
- `scores`：表示预测结果的置信度。
|
||||
- `label_names`：表示预测结果的类别名。
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/textline_ori_classification/textline_rot180_demo_res.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TextLineOrientationClassification`实例化文本行方向分类模型（此处以`PP-LCNet_x0_25_textline_ori`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-LCNet_x0_25_textline_ori</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文本行方向分类模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>List</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供文本行方向分类的训练，因此，如果需要训练文档图像方向分类模型，可以参考 [PaddleX 文本行方向分类二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/textline_orientation_classification.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
@@ -0,0 +1,131 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# PaddleOCR Multi-Devices Usage Guide
|
||||
|
||||
This document focuses on the usage guide of PaddleX for Huawei Ascend NPU and Kunlun XPU hardware platforms.
|
||||
|
||||
## 1、Installation
|
||||
|
||||
### 1.1 PaddlePaddle Installation
|
||||
|
||||
First, please complete the installation of PaddlePaddle according to your hardware platform. The installation tutorials for each hardware are as follows:
|
||||
|
||||
Ascend NPU: [Ascend NPU PaddlePaddle Installation Guide](./paddlepaddle_install_NPU.en.md)
|
||||
|
||||
Kunlun XPU: [Kunlun XPU PaddlePaddle Installation Guide](./paddlepaddle_install_XPU.en.md)
|
||||
|
||||
### 1.2 PaddleOCR Installation
|
||||
|
||||
Please refer to [PaddleOCR Installation Guide](../installation.en.md) to install PaddleOCR。
|
||||
|
||||
## 2、Usage
|
||||
|
||||
The methods for training and inference of PaddleOCR on hardware platforms such as Ascend NPU and Kunlun XPU are the same as those on GPU. You only need to modify the configuration parameters according to the specific hardware platform.
|
||||
On these two hardware platforms, quick inference and model fine-tuning are supported for the three major features of PaddleOCR, including the text recognition model PP-OCRv5, the document parsing solution PP-StructureV3, and PP-ChatOCRv4.
|
||||
|
||||
### 2.1 Quick Inference
|
||||
|
||||
You can quickly experience the OCR pipeline inference with a single command:
|
||||
|
||||
* OCR pipeline inference
|
||||
|
||||
```bash
|
||||
# The default model used is PP-OCRv5
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png --device npu:0
|
||||
```
|
||||
|
||||
* PP-StructureV3 pipeline inference
|
||||
|
||||
```bash
|
||||
paddleocr pp_structurev3 -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/pp_structure_v3_demo.png --device npu:0
|
||||
```
|
||||
|
||||
To perform quick inference using the production line in your project, you can achieve this with just a few lines of code. Here’s an example of how you might set it up:
|
||||
|
||||
* OCR pipeline inference
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(device="npu:0")
|
||||
|
||||
result = ocr.predict("./general_ocr_002.png")
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
```
|
||||
|
||||
* PP-StructureV3 pipeline inference
|
||||
|
||||
```python
|
||||
from paddleocr import PPStructureV3
|
||||
|
||||
pipeline = PPStructureV3(device="npu:0")
|
||||
output = pipeline.predict("./pp_structure_v3_demo.png")
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_json(save_path="output")
|
||||
res.save_to_markdown(save_path="output")
|
||||
```
|
||||
|
||||
If you want to know more about OCR pipeline inference,please refer to [general OCR pipeline use guide](../pipeline_usage/OCR.en.md).
|
||||
If you want to know more about PPStructureV3 pipeline inference,please refer to [PP-StructureV3 pipeline use guide](../pipeline_usage/PP-StructureV3.en.md).
|
||||
|
||||
### 2.2 Model Fine-tuning
|
||||
|
||||
If you are not satisfied with the performance of the pre-trained model, you can fine-tune it.
|
||||
|
||||
* train on Ascend NPU
|
||||
```bash
|
||||
export FLAGS_npu_storage_format=0
|
||||
export FLAGS_npu_jit_compile=0
|
||||
export FLAGS_use_stride_kernel=0
|
||||
export FLAGS_allocator_strategy=auto_growth
|
||||
export FLAGS_npu_split_aclnn=True
|
||||
export FLAGS_npu_scale_aclnn=True
|
||||
export CUSTOM_DEVICE_BLACK_LIST=pad3d,pad3d_grad
|
||||
python3 -m paddle.distributed.launch --devices '0,1,2,3' \
|
||||
tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_mobile_rec.yml \
|
||||
-o Global.use_gpu=False Global.use_npu=True
|
||||
```
|
||||
|
||||
* train on Kunlun XPU
|
||||
```bash
|
||||
export FLAGS_use_stride_kernel=0
|
||||
export BKCL_FORCE_SYNC=1
|
||||
export BKCL_TIMEOUT=1800
|
||||
export XPU_BLACK_LIST=pad3d,pad3d_grad
|
||||
python3 -m paddle.distributed.launch --devices '0,1,2,3' \
|
||||
tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_mobile_rec.yml \
|
||||
-o Global.use_gpu=False Global.use_xpu=True
|
||||
```
|
||||
|
||||
### 2.3 Other Inference methods
|
||||
|
||||
On the Ascend NPU, for a small number of inference samples, using the aforementioned pipeline inference method may result in abnormal outcomes (primarily with the PP-StructureV3 pipeline). To address this issue, we support using ONNX models for inference to ensure correct results.
|
||||
|
||||
You can use the following command to convert a Paddle model to an ONNX model:
|
||||
|
||||
```bash
|
||||
paddlex --install paddle2onnx
|
||||
paddlex --paddle2onnx --paddle_model_dir /paddle_model_dir --onnx_model_dir /onnx_model_dir --opset_version 7
|
||||
```
|
||||
|
||||
Meanwhile, some models support Ascend offline OM inference, effectively optimizing inference performance and memory usage. Using models in OM + ONNX format for pipeline inference can ensure both accuracy and speed.
|
||||
|
||||
Use the ATC conversion tool to convert an ONNX model to an OM model:
|
||||
|
||||
```bash
|
||||
atc --model=inference.onnx --framework=5 --output=inference --soc_version="your_device_type" --input_shape "your_input_shape"
|
||||
```
|
||||
|
||||
We have deeply integrated ONNX and OM models into PaddleX for high-performance inference. By modifying the pipeline configuration file to set the model inference backend to ONNX or OM, you can use the PaddleX high-performance inference API for inference.
|
||||
|
||||
For specific modification methods, inference code, and more usage instructions, please refer to [Ascend NPU High-Performance Inference Tutorial](https://github.com/PaddlePaddle/PaddleX/blob/develop/docs/practical_tutorials/high_performance_npu_tutorial.en.md)。
|
||||
|
||||
## 3、 FAQ
|
||||
### 1.The inference results using PP-StructureV3 on the production line are incorrect.
|
||||
Some models on this line have precision errors in a small number of cases. You can try adjusting the model in the configuration file or use ONNX+OM models for inference.
|
||||
128
docs/version3.x/other_devices_support/multi_devices_use_guide.md
Normal file
128
docs/version3.x/other_devices_support/multi_devices_use_guide.md
Normal file
@@ -0,0 +1,128 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# PaddleOCR多硬件使用指南
|
||||
|
||||
本文档主要针对昇腾 NPU、昆仑 XPU 等硬件平台，介绍 PaddleOCR 使用指南。
|
||||
|
||||
## 1、安装
|
||||
|
||||
### 1.1 PaddlePaddle安装
|
||||
|
||||
首先请您根据所属硬件平台，完成飞桨 PaddlePaddle 的安装，各硬件的飞桨安装教程如下：
|
||||
|
||||
昇腾 NPU：[昇腾 NPU 飞桨安装教程](./paddlepaddle_install_NPU.md)
|
||||
|
||||
昆仑 XPU：[昆仑 XPU 飞桨安装教程](./paddlepaddle_install_XPU.md)
|
||||
|
||||
### 1.2 PaddleOCR安装
|
||||
|
||||
请参考[PaddleOCR安装教程](../installation.md)安装 PaddleOCR。
|
||||
|
||||
## 2、使用
|
||||
|
||||
基于昇腾 NPU、昆仑 XPU 等硬件平台的 PaddleOCR 训练、推理使用方法与 GPU 相同，只需根据所属硬件平台，修改配置设备的参数。在这两款硬件上，支持 PaddleOCR 三大特色能力的快速推理和模型微调，包括 文字识别模型 PP-OCRv5、文档解析方案 PP-StructureV3 和 PP-ChatOCRv4。
|
||||
|
||||
### 2.1 快速推理
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
* OCR 产线推理
|
||||
|
||||
```bash
|
||||
# 默认使用 PP-OCRv5 模型
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png --device npu:0 # 将设备名修改为 npu或xpu
|
||||
```
|
||||
|
||||
* PP-StructureV3 产线推理
|
||||
|
||||
```bash
|
||||
paddleocr pp_structurev3 -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/pp_structure_v3_demo.png --device npu:0
|
||||
```
|
||||
|
||||
在项目中，您可以通过几行代码即可完成产线的快速推理，推理代码如下：
|
||||
|
||||
* OCR 产线推理
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(device="npu:0")
|
||||
|
||||
result = ocr.predict("./general_ocr_002.png")
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
```
|
||||
|
||||
* PP-StructureV3 产线推理
|
||||
```python
|
||||
from paddleocr import PPStructureV3
|
||||
|
||||
pipeline = PPStructureV3(device="npu:0")
|
||||
output = pipeline.predict("./pp_structure_v3_demo.png")
|
||||
for res in output:
|
||||
res.print() ## 打印预测的结构化输出
|
||||
res.save_to_json(save_path="output") ## 保存当前图像的结构化json结果
|
||||
res.save_to_markdown(save_path="output") ## 保存当前图像的markdown格式的结果
|
||||
```
|
||||
|
||||
更多关于 OCR 产线推理的使用说明，请参考[通用OCR产线使用教程](../pipeline_usage/OCR.md)。
|
||||
更多关于 PP-StructureV3 产线推理的使用说明，请参考[PP-StructureV3产线使用教程](../pipeline_usage/PP-StructureV3.md)。
|
||||
|
||||
### 2.2 模型微调
|
||||
|
||||
如果对预训练模型的效果不满意，可以对产线上的模型进行模型微调，示例如下：
|
||||
|
||||
* 昇腾 NPU
|
||||
```bash
|
||||
export FLAGS_npu_storage_format=0
|
||||
export FLAGS_npu_jit_compile=0
|
||||
export FLAGS_use_stride_kernel=0
|
||||
export FLAGS_allocator_strategy=auto_growth
|
||||
export FLAGS_npu_split_aclnn=True
|
||||
export FLAGS_npu_scale_aclnn=True
|
||||
export CUSTOM_DEVICE_BLACK_LIST=pad3d,pad3d_grad
|
||||
python3 -m paddle.distributed.launch --devices '0,1,2,3' \
|
||||
tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_mobile_rec.yml \
|
||||
-o Global.use_gpu=False Global.use_npu=True
|
||||
```
|
||||
|
||||
* 昆仑 XPU
|
||||
```bash
|
||||
export FLAGS_use_stride_kernel=0
|
||||
export BKCL_FORCE_SYNC=1
|
||||
export BKCL_TIMEOUT=1800
|
||||
export XPU_BLACK_LIST=pad3d,pad3d_grad
|
||||
python3 -m paddle.distributed.launch --devices '0,1,2,3' \
|
||||
tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_mobile_rec.yml \
|
||||
-o Global.use_gpu=False Global.use_xpu=True
|
||||
```
|
||||
|
||||
### 2.3 其它方式推理
|
||||
|
||||
在昇腾 NPU 上，对于少量推理样本，使用上述产线推理方式存在结果异常的可能（主要是 PP-StructureV3 产线），针对这种情况，我们支持使用 ONNX 模型进行推理，保证推理结果正确。
|
||||
|
||||
使用如下命令可以将 Paddle 模型转换为 ONNX 模型：
|
||||
|
||||
```bash
|
||||
paddlex --install paddle2onnx
|
||||
paddlex --paddle2onnx --paddle_model_dir /paddle_model_dir --onnx_model_dir /onnx_model_dir --opset_version 7
|
||||
```
|
||||
|
||||
同时，部分模型支持昇腾离线 OM 推理，有效优化推理性能和内存占用。使用 OM + ONNX 格式的模型进行产线推理，能够同时保障精度和速度。
|
||||
|
||||
使用 atc 转换工具将 ONNX 模型转换为 OM 模型：
|
||||
|
||||
```bash
|
||||
atc --model=inference.onnx --framework=5 --output=inference --soc_version="your_device_type" --input_shape "your_input_shape"
|
||||
```
|
||||
|
||||
我们已将 ONNX 和 OM 模型深度集成进了 PaddleX 高性能推理，修改产线配置文件，配置模型推理后端为 ONNX 或 OM 即可使用 PaddleX 高性能推理 API 进行推理。
|
||||
|
||||
具体修改方法、推理代码以及更多的使用方法请参考[昇腾 NPU 高性能推理教程](https://github.com/PaddlePaddle/PaddleX/blob/develop/docs/practical_tutorials/high_performance_npu_tutorial.md)。
|
||||
|
||||
## 3、常见问题
|
||||
### 1. 使用 PP-StructureV3 产线推理结果不正确
|
||||
该产线上的部分模型在少量 case 上存在精度误差，可以尝试调整配置文件中的模型，或者使用 ONNX+OM 模型进行推理。
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Ascend NPU PaddlePaddle Installation Tutorial
|
||||
|
||||
Currently, PaddleOCR supports the Ascend 910B chip (more models are under support. If you have a related need for other models, please submit an issue to inform us). The Ascend driver version is 23.0.3. Considering the differences in environments, we recommend using the <b>Ascend development image provided by PaddlePaddle</b> to complete the environment preparation.
|
||||
|
||||
## 1. Docker Environment Preparation
|
||||
* Pull the image. This image is only for the development environment and does not contain a pre-compiled PaddlePaddle installation package. The image has CANN-8.0.0, the Ascend operator library, installed by default.
|
||||
```bash
|
||||
# For X86 architecture
|
||||
docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-npu:cann800-ubuntu20-npu-910b-base-x86_64-gcc84
|
||||
# For Aarch64 architecture
|
||||
docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-npu:cann800-ubuntu20-npu-910b-base-aarch64-gcc84
|
||||
```
|
||||
* Start the container with the following command. ASCEND_RT_VISIBLE_DEVICES specifies the visible NPU card numbers.
|
||||
```bash
|
||||
docker run -it --name paddle-npu-dev -v $(pwd):/work \
|
||||
--privileged --network=host --shm-size=128G -w=/work \
|
||||
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
|
||||
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
|
||||
-v /usr/local/dcmi:/usr/local/dcmi \
|
||||
-e ASCEND_RT_VISIBLE_DEVICES="0,1,2,3,4,5,6,7" \
|
||||
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-npu:cann800-ubuntu20-npu-910b-base-$(uname -m)-gcc84 /bin/bash
|
||||
```
|
||||
## 2. Install Paddle Package
|
||||
* Download and install the Python wheel installation package
|
||||
```bash
|
||||
# Note: You need to install the CPU version of PaddlePaddle first
|
||||
python -m pip install paddlepaddle==3.0.0.dev20250527 -i https://www.paddlepaddle.org.cn/packages/nightly/cpu
|
||||
python -m pip install paddle-custom-npu==3.0.0.dev20250527 -i https://www.paddlepaddle.org.cn/packages/nightly/npu
|
||||
```
|
||||
* CANN-8.0.0 does not support some versions of numpy and opencv, it is recommended to install the specified versions.
|
||||
```bash
|
||||
python -m pip install numpy==1.26.4
|
||||
python -m pip install opencv-python==3.4.18.65
|
||||
```
|
||||
* Set environment variables on the arm machine (not required for x86 environment)
|
||||
```bash
|
||||
# Solve the error reported by libgomp on the arm machine
|
||||
# "libgomp cannot allocate memory in static TLS block"
|
||||
export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1:$LD_PRELOAD
|
||||
```
|
||||
* After verifying that the installation package is installed, run the following command
|
||||
```bash
|
||||
python -c "import paddle; paddle.utils.run_check()"
|
||||
```
|
||||
The expected output is as follows
|
||||
|
||||
```
|
||||
Running verify PaddlePaddle program ...
|
||||
PaddlePaddle works well on 1 npu.
|
||||
PaddlePaddle works well on 8 npus.
|
||||
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.
|
||||
```
|
||||
@@ -0,0 +1,56 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 昇腾 NPU 飞桨安装教程
|
||||
|
||||
当前 PaddleOCR 支持昇腾 910B 芯片（更多型号还在支持中，如果您有其他型号的相关需求，请提交issue告知我们），昇腾驱动版本为 23.0.3。考虑到环境差异性，我们推荐使用<b>飞桨官方提供的昇腾开发镜像</b>完成环境准备。
|
||||
|
||||
## 1、docker环境准备
|
||||
* 拉取镜像，此镜像仅为开发环境，镜像中不包含预编译的飞桨安装包，镜像中已经默认安装了昇腾算子库 CANN-8.0.0。
|
||||
```bash
|
||||
# 适用于 X86 架构
|
||||
docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-npu:cann800-ubuntu20-npu-910b-base-x86_64-gcc84
|
||||
# 适用于 Aarch64 架构
|
||||
docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-npu:cann800-ubuntu20-npu-910b-base-aarch64-gcc84
|
||||
```
|
||||
* 参考如下命令启动容器，ASCEND_RT_VISIBLE_DEVICES 指定可见的 NPU 卡号
|
||||
```bash
|
||||
docker run -it --name paddle-npu-dev -v $(pwd):/work \
|
||||
--privileged --network=host --shm-size=128G -w=/work \
|
||||
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
|
||||
-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
|
||||
-v /usr/local/dcmi:/usr/local/dcmi \
|
||||
-e ASCEND_RT_VISIBLE_DEVICES="0,1,2,3,4,5,6,7" \
|
||||
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-npu:cann800-ubuntu20-npu-910b-base-$(uname -m)-gcc84 /bin/bash
|
||||
```
|
||||
## 2、安装paddle包
|
||||
* 下载安装 wheel 安装包
|
||||
```bash
|
||||
# 注意需要先安装飞桨 cpu 版本
|
||||
python -m pip install paddlepaddle==3.0.0.dev20250527 -i https://www.paddlepaddle.org.cn/packages/nightly/cpu
|
||||
python -m pip install paddle-custom-npu==3.0.0.dev20250527 -i https://www.paddlepaddle.org.cn/packages/nightly/npu
|
||||
```
|
||||
* CANN-8.0.0 对 numpy 和 opencv 部分版本不支持，建议安装指定版本
|
||||
```bash
|
||||
python -m pip install numpy==1.26.4
|
||||
python -m pip install opencv-python==3.4.18.65
|
||||
```
|
||||
* arm机器上需要设置环境变量（x86环境无需设置）
|
||||
```bash
|
||||
# 解决libgomp在arm机器上报错
|
||||
# "libgomp cannot allocate memory in static TLS block"
|
||||
export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1:$LD_PRELOAD
|
||||
```
|
||||
* 验证安装包安装完成之后，运行如下命令
|
||||
```bash
|
||||
python -c "import paddle; paddle.utils.run_check()"
|
||||
```
|
||||
预期得到如下输出结果
|
||||
|
||||
```
|
||||
Running verify PaddlePaddle program ...
|
||||
PaddlePaddle works well on 1 npu.
|
||||
PaddlePaddle works well on 8 npus.
|
||||
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.
|
||||
```
|
||||
@@ -0,0 +1,45 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Kunlun XPU PaddlePaddle Installation Tutorial
|
||||
|
||||
Currently, PaddleOCR supports Kunlun R200/R300 and other chips. Considering environmental differences, we recommend using the <b>Kunlun XPU development image officially released by PaddlePaddle</b>, which is pre-installed with the Kunlun basic runtime environment library (XRE).
|
||||
|
||||
## 1. Docker Environment Preparation
|
||||
Pull the image. This image is only for the development environment and does not include a pre-compiled PaddlePaddle installation package.
|
||||
|
||||
```bash
|
||||
docker pull registry.baidubce.com/device/paddle-xpu:ubuntu20-x86_64-gcc84-py310 # For X86 architecture
|
||||
docker pull registry.baidubce.com/device/paddle-xpu:kylinv10-aarch64-gcc82-py310 # For ARM architecture
|
||||
```
|
||||
Refer to the following command to start the container:
|
||||
|
||||
```bash
|
||||
docker run -it --name=xxx -m 81920M --memory-swap=81920M \
|
||||
--shm-size=128G --privileged --net=host \
|
||||
-v $(pwd):/workspace -w /workspace \
|
||||
registry.baidubce.com/device/paddle-xpu:$(uname -m)-py310 bash
|
||||
```
|
||||
|
||||
## 2. Install Paddle Package
|
||||
Currently, Python3.10 wheel installation packages are provided. If you have a need for other Python versions, you can refer to the [PaddlePaddle official documentation](https://www.paddlepaddle.org.cn/en/install/quick) to compile and install them yourself.
|
||||
|
||||
Install the Python3.10 wheel installation package:
|
||||
|
||||
```bash
|
||||
pip install https://paddle-whl.bj.bcebos.com/paddlex/xpu/paddlepaddle_xpu-2.6.1-cp310-cp310-linux_x86_64.whl # For X86 architecture
|
||||
pip install https://paddle-whl.bj.bcebos.com/paddlex/xpu/paddlepaddle_xpu-2.6.1-cp310-cp310-linux_aarch64.whl # For ARM architecture
|
||||
```
|
||||
|
||||
Verify the installation package. After installation, run the following command:
|
||||
|
||||
```bash
|
||||
python -c "import paddle; paddle.utils.run_check()"
|
||||
```
|
||||
|
||||
The expected output is:
|
||||
|
||||
```
|
||||
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.
|
||||
```
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 昆仑 XPU 飞桨安装教程
|
||||
|
||||
当前 PaddleOCR 支持昆仑 R200/R300 等芯片。考虑到环境差异性，我们推荐使用<b>飞桨官方发布的昆仑 XPU 开发镜像</b>，该镜像预装有昆仑基础运行环境库（XRE）。
|
||||
|
||||
## 1、docker环境准备
|
||||
拉取镜像，此镜像仅为开发环境，镜像中不包含预编译的飞桨安装包
|
||||
|
||||
```
|
||||
docker pull registry.baidubce.com/device/paddle-xpu:ubuntu20-x86_64-gcc84-py310 # X86 架构
|
||||
docker pull registry.baidubce.com/device/paddle-xpu:kylinv10-aarch64-gcc82-py310 # ARM 架构
|
||||
```
|
||||
参考如下命令启动容器
|
||||
|
||||
```
|
||||
docker run -it --name=xxx -m 81920M --memory-swap=81920M \
|
||||
--shm-size=128G --privileged --net=host \
|
||||
-v $(pwd):/workspace -w /workspace \
|
||||
registry.baidubce.com/device/paddle-xpu:$(uname -m)-py310 bash
|
||||
```
|
||||
## 2、安装paddle包
|
||||
当前提供 Python3.10 的 wheel 安装包。如有其他 Python 版本需求，可以参考[飞桨官方文档](https://www.paddlepaddle.org.cn/install/quick)自行编译安装。
|
||||
|
||||
安装 Python3.10 的 wheel 安装包
|
||||
|
||||
```
|
||||
pip install https://paddle-whl.bj.bcebos.com/paddlex/xpu/paddlepaddle_xpu-2.6.1-cp310-cp310-linux_x86_64.whl # X86 架构
|
||||
pip install https://paddle-whl.bj.bcebos.com/paddlex/xpu/paddlepaddle_xpu-2.6.1-cp310-cp310-linux_aarch64.whl # ARM 架构
|
||||
```
|
||||
验证安装包 安装完成之后，运行如下命令
|
||||
|
||||
```
|
||||
python -c "import paddle; paddle.utils.run_check()"
|
||||
```
|
||||
预期得到如下输出结果
|
||||
|
||||
```
|
||||
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.
|
||||
```
|
||||
89
docs/version3.x/paddleocr_and_paddlex.en.md
Normal file
89
docs/version3.x/paddleocr_and_paddlex.en.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# PaddleOCR and PaddleX
|
||||
|
||||
[PaddleX](https://github.com/PaddlePaddle/PaddleX) is a low-code development tool built on the PaddlePaddle framework. It integrates numerous out-of-the-box pre-trained models, supports the full-pipeline development from model training to inference, and is compatible with various mainstream hardware both domestically and internationally, empowering AI developers to efficiently deploy solutions in industrial practices.
|
||||
|
||||
PaddleOCR leverages PaddleX for inference deployment, enabling seamless collaboration between the two in this regard. When installing PaddleOCR, PaddleX is also installed as a dependency. Additionally, PaddleOCR and PaddleX maintain consistency in pipeline naming conventions. For quick experience, users typically do not need to understand the specific concepts of PaddleX when using basic configurations. However, knowledge of PaddleX can be beneficial in advanced configuration scenarios, service deployment, and other use cases.
|
||||
|
||||
This document introduces the relationship between PaddleOCR and PaddleX and explains how to use these two tools collaboratively.
|
||||
|
||||
## 1. Differences and Connections Between PaddleOCR and PaddleX
|
||||
|
||||
PaddleOCR and PaddleX have distinct focuses and functionalities: PaddleOCR specializes in OCR-related tasks, while PaddleX covers a wide range of task types, including time-series forecasting, face recognition, and more. Furthermore, PaddleX provides rich infrastructure with underlying capabilities for multi-model combined inference, enabling the integration of different models in a unified and flexible manner and supporting the construction of complex model pipelines.
|
||||
|
||||
PaddleOCR fully reuses the capabilities of PaddleX in the inference deployment phase, including:
|
||||
|
||||
- PaddleOCR primarily relies on PaddleX for underlying capabilities such as model inference, pre- and post-processing, and multi-model combination.
|
||||
- The high-performance inference capabilities of PaddleOCR are achieved through PaddleX's Paddle2ONNX plugin and high-performance inference plugins.
|
||||
- The service deployment solutions of PaddleOCR are based on PaddleX's implementations.
|
||||
|
||||
It is important to note that although PaddleOCR uses PaddleX at the underlying level, thanks to PaddleX’s optional dependency installation feature, **installing the PaddleOCR inference package does not include all of PaddleX’s dependencies—only those required for OCR-related tasks are installed**. Therefore, users generally do not need to worry about excessive expansion of dependency size. Tested in May 2025, in an x86-64 + Linux + Python 3.10 environment, the total size of required dependencies increased only from 717 MB to 738 MB.
|
||||
|
||||
The version correspondence between PaddleOCR, PaddleX, and the PaddlePaddle framework is as follows:
|
||||
|
||||
| PaddleOCR Version | PaddleX Version | PaddlePaddle Version |
|
||||
| --- | --- | --- |
|
||||
| `3.0.0` | `3.0.0` | `>= 3.0.0` |
|
||||
| `3.0.1` | `3.0.1` | `>= 3.0.0` |
|
||||
| `3.0.2` | `3.0.2` | `>= 3.0.0` |
|
||||
| `3.0.3` | `>= 3.0.3` | `>= 3.0.0` |
|
||||
| `3.1.0` | `>= 3.1.0` | `>= 3.0.0` |
|
||||
|
||||
## 2. Correspondence Between PaddleOCR Pipelines and PaddleX Pipeline Registration Names
|
||||
|
||||
| PaddleOCR Pipeline | PaddleX Pipeline Registration Name |
|
||||
| --- | --- |
|
||||
| General OCR | `OCR` |
|
||||
| PP-StructureV3 | `PP-StructureV3` |
|
||||
| PP-ChatOCRv4 | `PP-ChatOCRv4-doc` |
|
||||
| General Table Recognition V2 | `table_recognition_v2` |
|
||||
| Formula Recognition | `formula_recognition` |
|
||||
| Seal Text Recognition | `seal_recognition` |
|
||||
| Document Image Preprocessing | `doc_preprocessor` |
|
||||
| Document Understanding | `doc_understanding` |
|
||||
| PP-DocTranslation | `PP-DocTranslation` |
|
||||
|
||||
## 3. Using PaddleX Pipeline Configuration Files
|
||||
|
||||
During the inference deployment phase, PaddleOCR supports exporting and loading PaddleX pipeline configuration files. Users can deeply configure inference deployment-related parameters by editing these configuration files.
|
||||
|
||||
### 3.1 Exporting Pipeline Configuration Files
|
||||
|
||||
You can call the `export_paddlex_config_to_yaml` method of the PaddleOCR pipeline object to export the current pipeline configuration to a YAML file. Here is an example:
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
pipeline = PaddleOCR()
|
||||
pipeline.export_paddlex_config_to_yaml("ocr_config.yaml")
|
||||
```
|
||||
|
||||
The above code will generate a pipeline configuration file named `ocr_config.yaml` in the working directory.
|
||||
|
||||
You can also obtain the configuration file through the PaddleX CLI. Example:
|
||||
|
||||
```bash
|
||||
# Specify the pipeline registration name
|
||||
paddlex --get_pipeline_config OCR
|
||||
```
|
||||
|
||||
### 3.2 Editing Pipeline Configuration Files
|
||||
|
||||
The exported PaddleX pipeline configuration file not only includes parameters supported by PaddleOCR's CLI and Python API but also allows for more advanced configurations. Please refer to the corresponding pipeline usage tutorials in [PaddleX Pipeline Usage Overview](https://paddlepaddle.github.io/PaddleX/3.0/en/pipeline_usage/pipeline_develop_guide.html) for detailed instructions on adjusting various configurations according to your needs.
|
||||
|
||||
### 3.3 Loading Pipeline Configuration Files in CLI
|
||||
|
||||
By specifying the path to the PaddleX pipeline configuration file using the `--paddlex_config` parameter, PaddleOCR will read its contents as the default configuration for the pipeline (this takes precedence over the default values of individual initialization parameters). Here is an example:
|
||||
|
||||
```bash
|
||||
paddleocr ocr --paddlex_config ocr_config.yaml ...
|
||||
```
|
||||
|
||||
### 3.4 Loading Pipeline Configuration Files in Python API
|
||||
|
||||
When initializing the pipeline object, you can pass the path to the PaddleX pipeline configuration file or a configuration dictionary through the `paddlex_config` parameter, and PaddleOCR will use it as the default configuration (this takes precedence over the default values of individual initialization parameters). Here is an example:
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
pipeline = PaddleOCR(paddlex_config="ocr_config.yaml")
|
||||
```
|
||||
89
docs/version3.x/paddleocr_and_paddlex.md
Normal file
89
docs/version3.x/paddleocr_and_paddlex.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# PaddleOCR 与 PaddleX
|
||||
|
||||
[PaddleX](https://github.com/PaddlePaddle/PaddleX) 是一款基于飞桨框架构建的低代码开发工具，集成了众多开箱即用的预训练模型，支持模型从训练到推理的全流程开发，兼容多款国内外主流硬件，助力 AI 开发者在产业实践中高效落地。
|
||||
|
||||
PaddleOCR 在推理部署方面基于 PaddleX 构建，二者在该环节可实现无缝协同。在安装 PaddleOCR 时，PaddleX 也将作为其依赖一并安装。此外，PaddleOCR 与 PaddleX 在产线名称等方面也保持一致。对于快速体验，如果只使用基础配置，用户通常无需了解 PaddleX 的具体概念；但在涉及高级配置、服务化部署等使用场景时，了解 PaddleX 的相关知识将有所帮助。
|
||||
|
||||
本文档将介绍 PaddleOCR 与 PaddleX 之间的关系，并说明如何协同使用这两个工具。
|
||||
|
||||
## 1. PaddleOCR 与 PaddleX 的区别与联系
|
||||
|
||||
PaddleOCR 与 PaddleX 在定位和功能上各有侧重：PaddleOCR 专注于 OCR 相关任务，而 PaddleX 则覆盖了包括时序预测、人脸识别等在内的多种任务类型。此外，PaddleX 提供了丰富的基础设施，具备多模型组合推理的底层能力，能够以统一且灵活的方式接入不同模型，支持构建复杂的模型产线。
|
||||
|
||||
PaddleOCR 在推理部署环节充分复用了 PaddleX 的能力，具体包括：
|
||||
|
||||
- PaddleOCR 在模型推理、前后处理及多模型组合等底层能力上，主要依赖于 PaddleX。
|
||||
- PaddleOCR 的高性能推理能力通过 PaddleX 的 Paddle2ONNX 插件及高性能推理插件实现。
|
||||
- PaddleOCR 的服务化部署方案基于 PaddleX 的实现。
|
||||
|
||||
需要特别说明的是，尽管 PaddleOCR 在底层使用了 PaddleX，但得益于 PaddleX 的可选依赖安装功能，**安装 PaddleOCR 推理包时并不会安装 PaddleX 的全部依赖，而只会安装 OCR 类任务需要使用到的依赖**，用户通常无需关心依赖体积的过度膨胀问题。2025 年 5 月测试，在 x86-64 + Linux + Python 3.10 环境中，需要安装的依赖总体积仅仅从 717 MB 增加到 738 MB。
|
||||
|
||||
PaddleOCR、PaddleX 和飞桨框架的版本存在如下对应关系：
|
||||
|
||||
| PaddleOCR 版本 | PaddleX 版本 | 飞桨版本 |
|
||||
| --- | --- | --- |
|
||||
| `3.0.0` | `3.0.0` | `>= 3.0.0` |
|
||||
| `3.0.1` | `3.0.1` | `>= 3.0.0` |
|
||||
| `3.0.2` | `3.0.2` | `>= 3.0.0` |
|
||||
| `3.0.3` | `>= 3.0.3` | `>= 3.0.0` |
|
||||
| `3.1.0` | `>= 3.1.0` | `>= 3.0.0` |
|
||||
|
||||
## 2. PaddleOCR 产线与 PaddleX 产线注册名的对应关系
|
||||
|
||||
| PaddleOCR 产线 | PaddleX 产线注册名 |
|
||||
| --- | --- |
|
||||
| 通用 OCR | `OCR` |
|
||||
| PP-StructureV3 | `PP-StructureV3` |
|
||||
| PP-ChatOCRv4 | `PP-ChatOCRv4-doc` |
|
||||
| 通用表格识别 v2 | `table_recognition_v2` |
|
||||
| 公式识别 | `formula_recognition` |
|
||||
| 印章文本识别 | `seal_recognition` |
|
||||
| 文档图像预处理 | `doc_preprocessor` |
|
||||
| 文档理解 | `doc_understanding` |
|
||||
| PP-DocTranslation | `PP-DocTranslation` |
|
||||
|
||||
## 3. 使用 PaddleX 产线配置文件
|
||||
|
||||
在推理部署阶段，PaddleOCR 支持导出和加载 PaddleX 的产线配置文件。用户可通过编辑配置文件，对推理部署相关参数进行深度配置。
|
||||
|
||||
### 3.1 导出产线配置文件
|
||||
|
||||
可调用 PaddleOCR 产线对象的 `export_paddlex_config_to_yaml` 方法，将当前产线配置导出为 YAML 文件。示例如下：
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
pipeline = PaddleOCR()
|
||||
pipeline.export_paddlex_config_to_yaml("ocr_config.yaml")
|
||||
```
|
||||
|
||||
上述代码会在工作目录下生成名为 `ocr_config.yaml` 的产线配置文件。
|
||||
|
||||
也可以通过 PaddleX CLI 获取配置文件。示例如下：
|
||||
|
||||
```bash
|
||||
# 指定产线注册名
|
||||
paddlex --get_pipeline_config OCR
|
||||
```
|
||||
|
||||
### 3.2 编辑产线配置文件
|
||||
|
||||
导出的 PaddleX 产线配置文件不仅包含 PaddleOCR CLI 和 Python API 支持的参数，还可进行更多高级配置。请在 [PaddleX模型产线使用概览](https://paddlepaddle.github.io/PaddleX/3.0/pipeline_usage/pipeline_develop_guide.html) 中找到对应的产线使用教程，参考其中的详细说明，根据需求调整各项配置。
|
||||
|
||||
### 3.3 在 CLI 中加载产线配置文件
|
||||
|
||||
通过 `--paddlex_config` 参数指定 PaddleX 产线配置文件的路径，PaddleOCR 会读取其中的内容作为产线的默认配置（优先级高于各参数默认初始化的值）。示例如下：
|
||||
|
||||
```bash
|
||||
paddleocr ocr --paddlex_config ocr_config.yaml ...
|
||||
```
|
||||
|
||||
### 3.4 在 Python API 中加载产线配置文件
|
||||
|
||||
初始化产线对象时，可通过 `paddlex_config` 参数传入 PaddleX 产线配置文件路径或配置字典，PaddleOCR 会将其作为默认配置（优先级高于各参数默认初始化的值）。示例如下：
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
pipeline = PaddleOCR(paddlex_config="ocr_config.yaml")
|
||||
```
|
||||
209
docs/version3.x/paddlex/overview.en.md
Normal file
209
docs/version3.x/paddlex/overview.en.md
Normal file
@@ -0,0 +1,209 @@
|
||||
## 1. Introduction to All-in-One Development
|
||||
|
||||
The All-in-One development tool [PaddleX](https://github.com/PaddlePaddle/PaddleX/tree/release/3.0-beta1), based on the advanced technology of PaddleOCR, supports **low-code full-process** development capabilities in the OCR field. Through low-code development, simple and efficient model use, combination, and customization can be achieved. This will significantly **reduce the time consumption** of model development, **lower its development difficulty**, and greatly accelerate the application and promotion speed of models in the industry. Features include:
|
||||
|
||||
* 🎨 [**Rich Model One-Click Call**](https://paddlepaddle.github.io/PaddleOCR/latest/en/paddlex/quick_start.html): Integrates **48 models** related to text image intelligent analysis, general OCR, general layout parsing, table recognition, formula recognition, and seal recognition into 10 pipelines, which can be quickly experienced through a simple **Python API one-click call**. In addition, the same set of APIs also supports a total of **200+ models** in image classification, object detection, image segmentation, and time series forecasting, forming 30+ single-function modules, making it convenient for developers to use **model combinations**.
|
||||
|
||||
* 🚀 [**High Efficiency and Low barrier of entry**](https://paddlepaddle.github.io/PaddleOCR/latest/en/paddlex/overview.html): Provides two methods based on **unified commands** and **GUI** to achieve simple and efficient use, combination, and customization of models. Supports multiple deployment methods such as **high-performance inference, service-oriented deployment, and on-device deployment**. Additionally, for various mainstream hardware such as **NVIDIA GPU, Kunlunxin XPU, Ascend NPU, Cambricon MLU, and Haiguang DCU**, models can be developed with **seamless switching**.
|
||||
|
||||
> **Note**: PaddleX is committed to achieving pipeline-level model training, inference, and deployment. A model pipeline refers to a series of predefined development processes for specific AI tasks, including combinations of single models (single-function modules) that can independently complete a type of task.
|
||||
## 2. OCR-Related Capability Support
|
||||
|
||||
In PaddleX, all 6 OCR-related pipelines support **local inference**, and some pipelines support **online experience**. You can quickly experience the pre-trained model effects of each pipeline. If you are satisfied with the pre-trained model effects of a pipeline, you can directly proceed with [high-performance inference](https://github.com/PaddlePaddle/PaddleX/blob/release/3.0-beta1/docs/pipeline_deploy/high_performance_inference_en.md)/[service-oriented deployment](https://github.com/PaddlePaddle/PaddleX/blob/release/3.0-beta1/docs/pipeline_deploy/service_deploy_en.md)/[on-device deployment](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_deploy/on_device_deployment.html). If not satisfied, you can also use the **custom development** capabilities of the pipeline to improve the effects. For the complete pipeline development process, please refer to [PaddleX Pipeline Usage Overview](https://github.com/PaddlePaddle/PaddleX/blob/release/3.0-beta1/docs/pipeline_usage/pipeline_develop_guide_en.md) or the tutorials for each pipeline.
|
||||
|
||||
In addition, PaddleX provides developers with a full-process efficient model training and deployment tool based on a [cloud-based GUI](https://aistudio.baidu.com/pipeline/mine). Developers **do not need code development**, just need to prepare a dataset that meets the pipeline requirements to **quickly start model training**. For details, please refer to the tutorial ["Developing Industrial-level AI Models with Zero Barrier"](https://aistudio.baidu.com/practical/introduce/546656605663301).
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Pipeline</th>
|
||||
<th>Online Experience</th>
|
||||
<th>Local Inference</th>
|
||||
<th>High-Performance Inference</th>
|
||||
<th>Service-Oriented Deployment</th>
|
||||
<th>On-Device Deployment</th>
|
||||
<th>Custom Development</th>
|
||||
<th><a href="https://aistudio.baidu.com/pipeline/mine">No-Code Development On AI Studio</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/doc_preprocessor.html">Document Image Preprocessing</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/OCR.html">OCR</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/91660/webUI?source=appMineRecent">Link</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/table_recognition.html">Table Recognition</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/91661?source=appMineRecent">Link</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/table_recognition_v2.html">Table Recognition V2</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/formula_recognition.html">Formula Recognition</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/387976/webUI?source=appCenter">Link</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/seal_recognition.html">Seal Recognition</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/387977/webUI?source=appCenter">Link</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/layout_parsing.html">Layout Parsing</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/layout_parsing_v2.html">Layout Parsing v2</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>🚧</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction.html">PP-ChatOCRv3-doc</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/182491/webUI?source=appCenter">Link</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction.html">PP-ChatOCRv4-doc</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
> ❗Note: The above capabilities are implemented based on GPU/CPU. PaddleX can also perform local inference and custom development on mainstream hardware such as Kunlunxin, Ascend, Cambricon, and Haiguang. The table below details the support status of the pipelines. For specific supported model lists, please refer to the [Model List (Kunlunxin XPU)](https://github.com/PaddlePaddle/PaddleX/blob/release/3.0-beta1/docs/support_list/model_list_xpu_en.md)/[Model List (Ascend NPU)](https://github.com/PaddlePaddle/PaddleX/blob/release/3.0-beta1/docs/support_list/model_list_npu_en.md)/[Model List (Cambricon MLU)](https://github.com/PaddlePaddle/PaddleX/blob/release/3.0-beta1/docs/support_list/model_list_mlu_en.md)/[Model List (Haiguang DCU)](https://github.com/PaddlePaddle/PaddleX/blob/release/3.0-beta1/docs/support_list/model_list_dcu_en.md). We are continuously adapting more models and promoting the implementation of high-performance and service-oriented deployment on mainstream hardware.
|
||||
**🚀 Support for Domestic Hardware Capabilities**
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Pipeline Name</th>
|
||||
<th>Ascend 910B</th>
|
||||
<th>Kunlunxin XPU</th>
|
||||
<th>Cambricon MLU</th>
|
||||
<th>Haiguang DCU</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>General OCR</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Table Recognition</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>🚧</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 3. List and Tutorials of OCR-Related Model Pipelines
|
||||
|
||||
- **OCR Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/OCR.html)
|
||||
- **Table Recognition Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/table_recognition.html)
|
||||
- **PP-ChatOCRv3-doc Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction.html)
|
||||
- **Layout Parsing Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/layout_parsing.html)
|
||||
- **Formula Recognition Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/formula_recognition.html)
|
||||
- **Seal Recognition Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/seal_recognition.html)
|
||||
|
||||
## 4. List and Tutorials of OCR-Related Modules
|
||||
|
||||
- **Text Detection Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/text_detection.html)
|
||||
- **Seal Detection Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/seal_text_detection.html)
|
||||
- **Text Recognition Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/text_recognition.html)
|
||||
- **Formula Recognition Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/formula_recognition.html)
|
||||
- **Table Structure Recognition Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_structure_recognition.html)
|
||||
- **Text Image Unwarping Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/text_image_unwarping.html)
|
||||
- **Layout Detection Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/layout_detection.html)
|
||||
- **Document Image Orientation Classification Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html)
|
||||
|
||||
|
||||
|
||||
|
||||
## 3. List of OCR-related Pipeline Models and Tutorials
|
||||
|
||||
- **Document Image Preprocessing Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/doc_preprocessor.html)
|
||||
- **OCR Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/OCR.html)
|
||||
- **Table Recognition Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/table_recognition.html)
|
||||
- **Table Recognition v2 Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/table_recognition_v2.html)
|
||||
- **Layout Parsing Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/layout_parsing.html)
|
||||
- **Layout Parsing v2 Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/layout_parsing_v2.html)
|
||||
- **Formula Recognition**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/formula_recognition.html)
|
||||
- **Seal Recognition**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/ocr_pipelines/seal_recognition.html)
|
||||
- **PP-ChatOCRv3-doc Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.html)
|
||||
- **PP-ChatOCRv4-doc Pipeline**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.html)
|
||||
|
||||
## 4. List of OCR-related Single Function Modules and Tutorials
|
||||
|
||||
- **Text Detection Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/text_detection.html)
|
||||
- **Seal Detection Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/seal_text_detection.html)
|
||||
- **Textline Orientation Classification Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/textline_orientation_classification.html)
|
||||
- **Text Recognition Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/text_recognition.html)
|
||||
- **Formula Recognition Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/formula_recognition.html)
|
||||
- **Table Structure Recognition Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_structure_recognition.html)
|
||||
- **Text Image Unwarping Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/text_image_unwarping.html)
|
||||
- **Layout Detection Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/layout_detection.html)
|
||||
- **Document Image Orientation Classification Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html)
|
||||
- **Table Cells Detection Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_cells_detection.html)
|
||||
- **Table Classification Module**: [Tutorial](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_classification.html)
|
||||
190
docs/version3.x/paddlex/overview.md
Normal file
190
docs/version3.x/paddlex/overview.md
Normal file
@@ -0,0 +1,190 @@
|
||||
---
|
||||
comments: true
|
||||
hide:
|
||||
- toc
|
||||
---
|
||||
|
||||
## 1. 低代码全流程开发简介
|
||||
|
||||
飞桨低代码开发工具 [PaddleX](https://github.com/PaddlePaddle/PaddleX)，依托于 PaddleOCR 的先进技术，支持了 OCR 领域的**低代码全流程**开发能力。通过低代码开发，可实现简单且高效的模型使用、组合与定制。这将显著**减少模型开发的时间消耗**，**降低其开发难度**，大大加快模型在行业中的应用和推广速度。特色如下：
|
||||
|
||||
* 🎨 **模型丰富一键调用**：将文本图像智能分析、通用 OCR、通用版面解析、通用表格识别、公式识别、印章文本识别涉及的**48个模型**整合为 10 条模型产线，通过极简的**Python API一键调用**，快速体验模型效果。此外，同一套API，也支持图像分类、目标检测、图像分割、时序预测等共计**200+模型**，形成30+单功能模块，方便开发者进行**模型组合**使用。
|
||||
|
||||
* 🚀 **提高效率降低门槛**：提供基于**统一命令**和**图形界面**两种方式，实现模型简洁高效的使用、组合与定制。支持**高性能推理、服务化部署和端侧部署**等多种部署方式。此外，对于各种主流硬件如**英伟达GPU、昆仑芯、昇腾、寒武纪和海光**等，进行模型开发时，都可以**无缝切换**。
|
||||
|
||||
>**说明**：PaddleX 致力于实现产线级别的模型训练、推理与部署。模型产线是指一系列预定义好的、针对特定AI任务的开发流程，其中包含能够独立完成某类任务的单模型（单功能模块）组合。
|
||||
|
||||
## 2. OCR相关能力支持
|
||||
|
||||
PaddleX 中 OCR 相关的 10 条产线均支持本地**快速推理**，部分产线支持**在线体验**，您可以快速体验各个产线的预训练模型效果，如果您对产线的预训练模型效果满意，可以直接对产线进行[高性能推理](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/high_performance_inference.html)/[服务化部署](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/service_deploy.html)/[端侧部署](https://paddlepaddle.github.io/PaddleX/latest/pipeline_deploy/on_device_deployment.html)，如果不满意，您也可以使用产线的**二次开发**能力，提升效果。完整的产线开发流程请参考[PaddleX产线使用概览](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/pipeline_develop_guide.html)或各产线使用教程。
|
||||
|
||||
此外，PaddleX为开发者提供了基于[云端图形化开发界面](https://aistudio.baidu.com/pipeline/mine)的全流程开发工具, 详细请参考[教程《零门槛开发产业级AI模型》](https://aistudio.baidu.com/practical/introduce/546656605663301)
|
||||
|
||||
<table >
|
||||
<tr>
|
||||
<th>模型产线</th>
|
||||
<th>在线体验</th>
|
||||
<th>快速推理</th>
|
||||
<th>高性能推理</th>
|
||||
<th>服务化部署</th>
|
||||
<th>端侧部署</th>
|
||||
<th>二次开发</th>
|
||||
<th><a href = "https://aistudio.baidu.com/pipeline/mine">星河零代码产线</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/doc_preprocessor.html">文档图像预处理</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/OCR.html">通用OCR</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/91660/webUI?source=appMineRecent">链接</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/table_recognition.html">通用表格识别</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/91661?source=appMineRecent">链接</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/table_recognition_v2.html">通用表格识别v2</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/formula_recognition.html">公式识别</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/387976/webUI?source=appCenter">链接</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/seal_recognition.html">印章文本识别</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/387977/webUI?source=appCenter">链接</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/layout_parsing.html">通用版面解析</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/layout_parsing_v2.html">通用版面解析v2</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>🚧</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction.html">文档场景信息抽取v3</a></td>
|
||||
<td><a href = "https://aistudio.baidu.com/community/app/182491/webUI?source=appCenter">链接</a></td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><a href="https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction.html">文档场景信息抽取v4</a></td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
> ❗注：以上功能均基于GPU/CPU实现。PaddleX还可在昆仑芯、昇腾、寒武纪和海光等主流硬件上进行快速推理和二次开发。下表详细列出了模型产线的支持情况，具体支持的模型列表请参阅 [模型列表(NPU)](https://paddlepaddle.github.io/PaddleX/latest/support_list/model_list_npu.html) // [模型列表(XPU)](https://paddlepaddle.github.io/PaddleX/latest/support_list/model_list_xpu.html) // [模型列表(MLU)](https://paddlepaddle.github.io/PaddleX/latest/support_list/model_list_mlu.html) // [模型列表DCU](https://paddlepaddle.github.io/PaddleX/latest/support_list/model_list_dcu.html)。同时我们也在适配更多的模型，并在主流硬件上推动高性能和服务化部署的实施。
|
||||
|
||||
**🚀 国产化硬件能力支持**
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型产线</th>
|
||||
<th>昇腾 910B</th>
|
||||
<th>昆仑芯 R200/R300</th>
|
||||
<th>寒武纪 MLU370X8</th>
|
||||
<th>海光 Z100/K100AI</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>通用OCR</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
<td>✅</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>通用表格识别</td>
|
||||
<td>✅</td>
|
||||
<td>🚧</td>
|
||||
<td>🚧</td>
|
||||
<td>🚧</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 3. OCR相关模型产线列表和教程
|
||||
|
||||
* **文档图像预处理产线**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/doc_preprocessor.html)
|
||||
* **通用OCR产线**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/OCR.html)
|
||||
* **通用表格识别产线**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/table_recognition.html)
|
||||
* **通用表格识别v2产线**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/table_recognition_v2.html)
|
||||
* **通用版面解析产线**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/layout_parsing.html)
|
||||
* **通用版面解析v2产线**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/layout_parsing_v2.html)
|
||||
* **公式识别**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/formula_recognition.html)
|
||||
* **印章文本识别**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/seal_recognition.html)
|
||||
* **文档场景信息抽取v3产线**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.html)
|
||||
* **文档场景信息抽取v4产线**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.html)
|
||||
|
||||
## 4. OCR相关单功能模块列表和教程
|
||||
|
||||
* **文本检测模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/text_detection.html)
|
||||
* **印章文本检测模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/seal_text_detection.html)
|
||||
* **文本行方向分类模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/textline_orientation_classification.html)
|
||||
* **文本识别模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/text_recognition.html)
|
||||
* **公式识别模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/formula_recognition.html)
|
||||
* **表格结构识别模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_structure_recognition.html)
|
||||
* **文本图像矫正模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/text_image_unwarping.html)
|
||||
* **版面区域检测模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/layout_detection.html)
|
||||
* **文档图像方向分类模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html)
|
||||
* **表格单元格检测模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_cells_detection.html)
|
||||
* **表格分类模块**: [使用教程](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_classification.html)
|
||||
122
docs/version3.x/paddlex/quick_start.en.md
Normal file
122
docs/version3.x/paddlex/quick_start.en.md
Normal file
@@ -0,0 +1,122 @@
|
||||
# ⏭️ Quick Start
|
||||
|
||||
## 🛠️ Installation
|
||||
|
||||
> ❗Before installing PaddleX, please ensure you have a basic **Python runtime environment** (Note: Currently supports running under Python 3.8 to Python 3.10, with more Python versions under adaptation). The PaddlePaddle version required by PaddleX
|
||||
|
||||
* **Installing PaddlePaddle**
|
||||
|
||||
```bash
|
||||
# CPU
|
||||
python -m pip install paddlepaddle==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/
|
||||
|
||||
# gpu，requires GPU driver version ≥450.80.02 (Linux) or ≥452.39 (Windows)
|
||||
python -m pip install paddlepaddle-gpu==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/
|
||||
|
||||
# gpu，requires GPU driver version ≥545.23.06 (Linux) or ≥545.84 (Windows)
|
||||
python -m pip install paddlepaddle-gpu==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cu123/
|
||||
```
|
||||
> ❗No need to focus on the CUDA version on the physical machine, only the GPU driver version needs attention. For more information on PaddlePaddle Wheel versions, please refer to the [PaddlePaddle Official Website](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation./docs/en/install/pip/linux-pip.html).
|
||||
|
||||
* **Installing PaddleX**
|
||||
|
||||
```bash
|
||||
pip install https://paddle-model-ecology.bj.bcebos.com/paddlex/whl/paddlex-3.0.0rc0-py3-none-any.whl
|
||||
```
|
||||
|
||||
> ❗For more installation methods, refer to the [PaddleX Installation Guide](https://paddlepaddle.github.io/PaddleX/latest/en/installation/installation.html).
|
||||
|
||||
|
||||
## 💻 CLI Usage
|
||||
|
||||
One command can quickly experience the pipeline effect, the unified CLI format is:
|
||||
|
||||
```bash
|
||||
paddlex --pipeline [Pipeline Name] --input [Input Image] --device [Running Device]
|
||||
```
|
||||
|
||||
Each Pipeline in PaddleX corresponds to specific parameters, which you can view in the respective Pipeline documentation for detailed explanations. Each Pipeline requires specifying three necessary parameters:
|
||||
|
||||
* `pipeline`: The name of the Pipeline or the configuration file of the Pipeline
|
||||
* `input`: The local path, directory, or URL of the input file (e.g., an image) to be processed
|
||||
* `device`: The hardware device and its index to use (e.g., `gpu:0` indicates using the 0th GPU), or you can choose to use NPU (`npu:0`), XPU (`xpu:0`), CPU (`cpu`), etc.
|
||||
|
||||
For example, using the OCR pipeline:
|
||||
```bash
|
||||
paddlex --pipeline OCR \
|
||||
--input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False \
|
||||
--save_path ./output \
|
||||
--device gpu:0
|
||||
```
|
||||
<summary><b>👉 Click to view the running result</b></summary>
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_002.png', 'page_index': None, 'model_settings': {'use_doc_preprocessor': False, 'use_textline_orientation': False}, 'doc_preprocessor_res': {'input_path': None, 'model_settings': {'use_doc_orientation_classify': True, 'use_doc_unwarping': False}, 'angle': 0},'dt_polys': [array([[ 3, 10],
|
||||
[82, 10],
|
||||
[82, 33],
|
||||
[ 3, 33]], dtype=int16), ...], 'text_det_params': {'limit_side_len': 960, 'limit_type': 'max', 'thresh': 0.3, 'box_thresh': 0.6, 'unclip_ratio': 2.0}, 'text_type': 'general', 'textline_orientation_angles': [-1, ...], 'text_rec_score_thresh': 0.0, 'rec_texts': ['www.99*', ...], 'rec_scores': [0.8980069160461426, ...], 'rec_polys': [array([[ 3, 10],
|
||||
[82, 10],
|
||||
[82, 33],
|
||||
[ 3, 33]], dtype=int16), ...], 'rec_boxes': array([[ 3, 10, 82, 33], ...], dtype=int16)}}
|
||||
```
|
||||
|
||||
The visualization result is as follows:
|
||||
|
||||
|
||||
|
||||
|
||||
To use the command line for other pipelines, simply adjust the `pipeline` parameter to the name of the corresponding pipeline and modify the parameters accordingly. Below are the commands for each pipeline:
|
||||
|
||||
<summary><b>👉 More CLI usage for pipelines</b></summary>
|
||||
|
||||
| Pipeline Name | Command |
|
||||
|------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| OCR | `paddlex --pipeline OCR --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png --use_doc_orientation_classify False --use_doc_unwarping False --use_textline_orientation False --save_path ./output --device gpu:0` |
|
||||
| Document Image Preprocessor | `paddlex --pipeline doc_preprocessor --input https://paddle-model-ecology.bj.bcebos.com/paddlex/demo_image/doc_test_rotated.jpg --use_doc_orientation_classify True --use_doc_unwarping True --save_path ./output --device gpu:0` |
|
||||
| Table Recognition | `paddlex --pipeline table_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg --save_path ./output --device gpu:0` |
|
||||
| Table Recognition v2 | `paddlex --pipeline table_recognition_v2 --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg --save_path ./output --device gpu:0` |
|
||||
| Formula Recognition | `paddlex --pipeline formula_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/demo_image/general_formula_recognition.png --use_layout_detection True --use_doc_orientation_classify False --use_doc_unwarping False --layout_threshold 0.5 --layout_nms True --layout_unclip_ratio 1.0 --layout_merge_bboxes_mode large --save_path ./output --device gpu:0` |
|
||||
| Seal Recognition | `paddlex --pipeline seal_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png --use_doc_orientation_classify False --use_doc_unwarping False --device gpu:0 --save_path ./output` |
|
||||
| Layout Parsing | `paddlex --pipeline layout_parsing --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/demo_paper.png --use_doc_orientation_classify False --use_doc_unwarping False --use_textline_orientation False --save_path ./output --device gpu:0` |
|
||||
| Layout Parsing v2 | `paddlex --pipeline layout_parsing_v2 --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout_parsing_v2_demo.png --use_doc_orientation_classify False --use_doc_unwarping False --use_textline_orientation False --save_path ./output --device gpu:0` |
|
||||
|
||||
|
||||
|
||||
## 📝 Python Script Usage
|
||||
|
||||
A few lines of code can complete the quick inference of the pipeline, the unified Python script format is as follows:
|
||||
```python
|
||||
from paddlex import create_pipeline
|
||||
|
||||
pipeline = create_pipeline(pipeline=[Pipeline Name])
|
||||
output = pipeline.predict([Input Image Name])
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/")
|
||||
```
|
||||
The following steps are executed:
|
||||
|
||||
* `create_pipeline()` instantiates the pipeline object
|
||||
* Passes the image and calls the `predict()` method of the pipeline object for inference prediction
|
||||
* Processes the prediction results
|
||||
|
||||
To use the Python script for other pipelines, simply adjust the `pipeline` parameter in the `create_pipeline()` method to the name of the corresponding pipeline and modify the parameters accordingly. Below are the parameter names and detailed usage explanations for each pipeline:
|
||||
|
||||
👉 More Python script usage for pipelines
|
||||
|
||||
| pipeline Name | Corresponding Parameter | Detailed Explanation |
|
||||
|-------------------------------|-------------------------------------|---------------------------------------------------------------------------------------------------------------|
|
||||
| OCR | `OCR` | [Instructions for Using the General OCR Pipeline Python Script](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/OCR.html#222-python-script-integration) |
|
||||
| Document Image Preprocessing | `doc_preprocessor` | [Instructions for Using the Document Image Preprocessing Pipeline Python Script](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/doc_preprocessor.html#212-python-script-integration) |
|
||||
| Table Recognition | `table_recognition` | [Instructions for Using the General Table Recognition Pipeline Python Script](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/table_recognition.html#22-python-script-integration) |
|
||||
| Table Recognition v2 | `table_recognition_v2` | [Instructions for Using the General Table Recognition v2 Pipeline Python Script](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/table_recognition_v2.html#22-python-script-integration) |
|
||||
| Formula Recognition | `formula_recognition` | [Instructions for Using the Formula Recognition Pipeline Python Script](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/formula_recognition.html#22-python-script-integration) |
|
||||
| Seal Recognition | `seal_recognition` | [Instructions for Using the Seal Text Recognition Pipeline Python Script](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/seal_recognition.html#22-python-script-integration) |
|
||||
| Layout Parsing | `layout_parsing` | [Instructions for Using the General Layout Parsing Pipeline Python Script](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/layout_parsing.html#22-python-script-integration) |
|
||||
| Layout Parsing v2 | `layout_parsing_v2` | [Instructions for Using the General Layout Parsing v2 Pipeline Python Script](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/layout_parsing_v2.html#22-python-script-integration) |
|
||||
| PP-ChatOCRv3-doc | `PP-ChatOCRv3-doc` | [PP-ChatOCRv3-doc Pipeline Python Script Usage Instructions](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.html) |
|
||||
| PP-ChatOCRv4-doc | `PP-ChatOCRv4-doc` | [PP-ChatOCRv4-doc Pipeline Python Script Usage Instructions](https://paddlepaddle.github.io/PaddleX/latest/en/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.html) |
|
||||
127
docs/version3.x/paddlex/quick_start.md
Normal file
127
docs/version3.x/paddlex/quick_start.md
Normal file
@@ -0,0 +1,127 @@
|
||||
# 快速开始
|
||||
|
||||
>**说明：**
|
||||
|
||||
>* 飞桨低代码开发工具[PaddleX](https://github.com/PaddlePaddle/PaddleX)，依托于PaddleOCR的先进技术，支持了OCR领域的**低代码全流程**开发能力。通过低代码开发，可实现简单且高效的模型使用、组合与定制。
|
||||
|
||||
>* PaddleX 致力于实现产线级别的模型训练、推理与部署。模型产线是指一系列预定义好的、针对特定AI任务的开发流程，其中包含能够独立完成某类任务的单模型（单功能模块）组合。本文档提供**OCR相关产线**的快速推理使用，单功能模块的快速使用以及更多功能请参考[PaddleOCR低代码全流程开发](https://paddlepaddle.github.io/PaddleOCR/latest/paddlex/overview.html)中相关章节。
|
||||
|
||||
|
||||
### 🛠️ 安装
|
||||
|
||||
> ❗在安装 PaddleX 之前，请确保您已具备基本的 **Python 运行环境**（注：目前支持 Python 3.8 至 Python 3.12）。PaddleX 3.0-rc0 版本依赖的 PaddlePaddle 版本为 3.0.0rc0。
|
||||
|
||||
* **安装 PaddlePaddle**
|
||||
```bash
|
||||
# CPU 版本
|
||||
python -m pip install paddlepaddle==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cpu/
|
||||
|
||||
# GPU 版本，需显卡驱动程序版本 ≥450.80.02（Linux）或 ≥452.39（Windows）
|
||||
python -m pip install paddlepaddle-gpu==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cu118/
|
||||
|
||||
# GPU 版本，需显卡驱动程序版本 ≥545.23.06（Linux）或 ≥545.84（Windows）
|
||||
python -m pip install paddlepaddle-gpu==3.0.0rc0 -i https://www.paddlepaddle.org.cn/packages/stable/cu123/
|
||||
```
|
||||
> ❗无需关注物理机上的 CUDA 版本，只需关注显卡驱动程序版本。更多飞桨 Wheel 版本信息，请参考[飞桨官网](https://www.paddlepaddle.org.cn/install/quick?docurl=/documentation./docs/zh/install/pip/linux-pip.html)。
|
||||
|
||||
* **安装PaddleX**
|
||||
|
||||
```bash
|
||||
pip install https://paddle-model-ecology.bj.bcebos.com/paddlex/whl/paddlex-3.0.0rc0-py3-none-any.whl
|
||||
```
|
||||
|
||||
> ❗ 更多安装方式参考 [PaddleX 安装教程](https://paddlepaddle.github.io/PaddleX/latest/installation/installation.html)
|
||||
|
||||
### 💻 命令行使用
|
||||
|
||||
一行命令即可快速体验产线效果，统一的命令行格式为：
|
||||
|
||||
```bash
|
||||
paddlex --pipeline [产线名称] --input [输入图片] --device [运行设备]
|
||||
```
|
||||
|
||||
PaddleX的每一条产线对应特定的参数，您可以在各自的产线文档中查看具体的参数说明。每条产线需指定必要的三个参数：
|
||||
* `pipeline`：产线名称或产线配置文件
|
||||
* `input`：待处理的输入文件（如图片）的本地路径、目录或 URL
|
||||
* `device`：使用的硬件设备及序号（例如`gpu:0`表示使用第 0 块 GPU），也可选择使用 NPU(`npu:0`)、 XPU(`xpu:0`)、CPU(`cpu`)等。
|
||||
|
||||
|
||||
以通用 OCR 产线为例：
|
||||
```bash
|
||||
paddlex --pipeline OCR \
|
||||
--input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False \
|
||||
--save_path ./output \
|
||||
--device gpu:0
|
||||
```
|
||||
|
||||
<b>👉 点击查看运行结果 </b><
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_002.png', 'page_index': None, 'model_settings': {'use_doc_preprocessor': False, 'use_textline_orientation': False}, 'doc_preprocessor_res': {'input_path': None, 'model_settings': {'use_doc_orientation_classify': True, 'use_doc_unwarping': False}, 'angle': 0},'dt_polys': [array([[ 3, 10],
|
||||
[82, 10],
|
||||
[82, 33],
|
||||
[ 3, 33]], dtype=int16), ...], 'text_det_params': {'limit_side_len': 960, 'limit_type': 'max', 'thresh': 0.3, 'box_thresh': 0.6, 'unclip_ratio': 2.0}, 'text_type': 'general', 'textline_orientation_angles': [-1, ...], 'text_rec_score_thresh': 0.0, 'rec_texts': ['www.99*', ...], 'rec_scores': [0.8980069160461426, ...], 'rec_polys': [array([[ 3, 10],
|
||||
[82, 10],
|
||||
[82, 33],
|
||||
[ 3, 33]], dtype=int16), ...], 'rec_boxes': array([[ 3, 10, 82, 33], ...], dtype=int16)}}
|
||||
```
|
||||
|
||||
可视化结果如下：
|
||||
|
||||
|
||||
|
||||
|
||||
其他产线的命令行使用，只需将 `pipeline` 参数调整为相应产线的名称，参数调整为对应的产线的参数即可。下面列出了每个产线对应的命令：
|
||||
|
||||
<b>👉 更多产线的命令行使用</b>
|
||||
|
||||
| 产线名称 | 使用命令 |
|
||||
|--------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| 文档图像预处理 | `paddlex --pipeline doc_preprocessor --input https://paddle-model-ecology.bj.bcebos.com/paddlex/demo_image/doc_test_rotated.jpg --use_doc_orientation_classify True --use_doc_unwarping True --save_path ./output --device gpu:0` |
|
||||
| 通用OCR | `paddlex --pipeline OCR --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_002.png --use_doc_orientation_classify False --use_doc_unwarping False --use_textline_orientation False --save_path ./output --device gpu:0` |
|
||||
| 通用表格识别 | `paddlex --pipeline table_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg --save_path ./output --device gpu:0` |
|
||||
| 通用表格识别v2 | `paddlex --pipeline table_recognition_v2 --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg --save_path ./output --device gpu:0` |
|
||||
| 公式识别 | `paddlex --pipeline formula_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/demo_image/general_formula_recognition.png --use_layout_detection True --use_doc_orientation_classify False --use_doc_unwarping False --layout_threshold 0.5 --layout_nms True --layout_unclip_ratio 1.0 --layout_merge_bboxes_mode large --save_path ./output --device gpu:0` |
|
||||
| 印章文本识别 | `paddlex --pipeline seal_recognition --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png --use_doc_orientation_classify False --use_doc_unwarping False --device gpu:0 --save_path ./output` |
|
||||
| 通用版面解析 | `paddlex --pipeline layout_parsing --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/demo_paper.png --use_doc_orientation_classify False --use_doc_unwarping False --use_textline_orientation False --save_path ./output --device gpu:0` |
|
||||
| 通用版面解析v2 | `paddlex --pipeline layout_parsing_v2 --input https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout_parsing_v2_demo.png --use_doc_orientation_classify False --use_doc_unwarping False --use_textline_orientation False --save_path ./output --device gpu:0` |
|
||||
|
||||
|
||||
### 📝 Python 脚本使用
|
||||
|
||||
几行代码即可完成产线的快速推理，统一的 Python 脚本格式如下：
|
||||
|
||||
```python
|
||||
from paddlex import create_pipeline
|
||||
|
||||
pipeline = create_pipeline(pipeline=[产线名称])
|
||||
output = pipeline.predict([输入图片名称])
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/")
|
||||
```
|
||||
执行了如下几个步骤：
|
||||
|
||||
* `create_pipeline()` 实例化产线对象
|
||||
* 传入图片并调用产线对象的 `predict()` 方法进行推理预测
|
||||
* 对预测结果进行处理
|
||||
|
||||
其他产线的 Python 脚本使用，只需将 `create_pipeline()` 方法的 `pipeline` 参数调整为相应产线的名称，参数调整为对应的产线的参数即可。下面列出了每个产线对应的参数名称及详细的使用解释：
|
||||
<b>👉 更多产线的Python脚本使用</b>
|
||||
|
||||
| 产线名称 | 对应参数 | 详细说明 |
|
||||
|--------------------|------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
||||
| 通用OCR | `OCR` | [通用OCR产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/OCR.html#222-python脚本方式集成) |
|
||||
| 文档图像预处理 | `doc_preprocessor` | [文档图像预处理产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/doc_preprocessor.html#212-python脚本方式集成) |
|
||||
| 通用表格识别 | `table_recognition` | [通用表格识别产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/table_recognition.html#22-python脚本方式集成) |
|
||||
| 通用表格识别v2 | `table_recognition_v2` | [通用表格识别v2产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/table_recognition_v2.html#22-python脚本方式集成) |
|
||||
| 公式识别 | `formula_recognition` | [公式识别产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/formula_recognition.html#22-python脚本方式集成) |
|
||||
| 印章文本识别 | `seal_recognition` | [印章文本识别产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/seal_recognition.html#22-python脚本方式集成) |
|
||||
| 通用版面解析 | `layout_parsing` | [通用版面解析产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/layout_parsing.html#22-python脚本方式集成) |
|
||||
| 通用版面解析v2 | `layout_parsing_v2` | [通用版面解析v2产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/ocr_pipelines/layout_parsing_v2.html#22-python脚本方式集成) |
|
||||
| 文档场景信息抽取v3 | `PP-ChatOCRv3-doc` | [文档场景信息抽取v3产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v3.html#22-本地体验) |
|
||||
| 文档场景信息抽取v4 | `PP-ChatOCRv4-doc` | [文档场景信息抽取v4产线Python脚本使用说明](https://paddlepaddle.github.io/PaddleX/latest/pipeline_usage/tutorials/information_extraction_pipelines/document_scene_information_extraction_v4.html#22-本地体验) |
|
||||
2440
docs/version3.x/pipeline_usage/OCR.en.md
Normal file
2440
docs/version3.x/pipeline_usage/OCR.en.md
Normal file
File diff suppressed because it is too large
Load Diff
2442
docs/version3.x/pipeline_usage/OCR.md
Normal file
2442
docs/version3.x/pipeline_usage/OCR.md
Normal file
File diff suppressed because it is too large
Load Diff
3873
docs/version3.x/pipeline_usage/PP-ChatOCRv4.en.md
Normal file
3873
docs/version3.x/pipeline_usage/PP-ChatOCRv4.en.md
Normal file
File diff suppressed because it is too large
Load Diff
3747
docs/version3.x/pipeline_usage/PP-ChatOCRv4.md
Normal file
3747
docs/version3.x/pipeline_usage/PP-ChatOCRv4.md
Normal file
File diff suppressed because it is too large
Load Diff
2878
docs/version3.x/pipeline_usage/PP-DocTranslation.en.md
Normal file
2878
docs/version3.x/pipeline_usage/PP-DocTranslation.en.md
Normal file
File diff suppressed because it is too large
Load Diff
2998
docs/version3.x/pipeline_usage/PP-DocTranslation.md
Normal file
2998
docs/version3.x/pipeline_usage/PP-DocTranslation.md
Normal file
File diff suppressed because it is too large
Load Diff
3526
docs/version3.x/pipeline_usage/PP-StructureV3.en.md
Normal file
3526
docs/version3.x/pipeline_usage/PP-StructureV3.en.md
Normal file
File diff suppressed because one or more lines are too long
3603
docs/version3.x/pipeline_usage/PP-StructureV3.md
Normal file
3603
docs/version3.x/pipeline_usage/PP-StructureV3.md
Normal file
File diff suppressed because one or more lines are too long
1234
docs/version3.x/pipeline_usage/doc_preprocessor.en.md
Normal file
1234
docs/version3.x/pipeline_usage/doc_preprocessor.en.md
Normal file
File diff suppressed because it is too large
Load Diff
1239
docs/version3.x/pipeline_usage/doc_preprocessor.md
Normal file
1239
docs/version3.x/pipeline_usage/doc_preprocessor.md
Normal file
File diff suppressed because it is too large
Load Diff
811
docs/version3.x/pipeline_usage/doc_understanding.en.md
Normal file
811
docs/version3.x/pipeline_usage/doc_understanding.en.md
Normal file
@@ -0,0 +1,811 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Document Understanding Pipeline Usage Tutorial
|
||||
|
||||
## 1. Introduction to the Document Understanding Pipeline
|
||||
|
||||
The Document Understanding Pipeline is an advanced document processing technology based on Visual-Language Models (VLM), designed to overcome the limitations of traditional document processing. Traditional methods rely on fixed templates or predefined rules to parse documents, whereas this pipeline leverages the multimodal capabilities of VLM to accurately answer user queries by inputting document images and user questions, integrating visual and language information. This technology does not require pre-training for specific document formats, allowing it to flexibly handle diverse document content, significantly enhancing the generalization and practicality of document processing. It has broad application prospects in intelligent Q&A, information extraction, and other scenarios. Currently, the pipeline does not support secondary development of VLM models, but plans to support it in the future.
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/doc_understanding/doc_understanding.png">
|
||||
|
||||
<b>The document understanding pipeline includes the following module. Each module can be trained and inferred independently and contains multiple models. For more details, click the corresponding module to view the documentation.</b>
|
||||
|
||||
- [Document-like Vision Language Model Module](../module_usage/doc_vlm.md)
|
||||
|
||||
In this pipeline, you can choose the model to use based on the benchmark data below.
|
||||
|
||||
<details>
|
||||
<summary> <b>Document-like Vision Language Model Module:</b></summary>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Model Storage Size (GB)</th>
|
||||
<th>Total Score</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-2B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-2B_infer.tar">Inference Model</a></td>
|
||||
<td>4.2</td>
|
||||
<td>765</td>
|
||||
<td rowspan="2">PP-DocBee is a multimodal large model independently developed by the PaddlePaddle team, focusing on document understanding, with excellent performance in Chinese document understanding tasks. The model is fine-tuned and optimized using nearly 5 million multimodal datasets related to document understanding, including general VQA, OCR, chart, text-rich documents, math and complex reasoning, synthetic data, pure text data, etc., with different training data ratios set. In several authoritative English document understanding evaluation lists in academia, PP-DocBee has generally achieved SOTA for models of the same parameter scale. In internal business Chinese scenarios, PP-DocBee also outperforms current popular open and closed-source models.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-7B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-7B_infer.tar">Inference Model</a></td>
|
||||
<td>15.8</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee2-3B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee2-3B_infer.tar">Inference Model</a></td>
|
||||
<td>7.6</td>
|
||||
<td>852</td>
|
||||
<td>PP-DocBee2 is a multimodal large model independently developed by the PaddlePaddle team, focusing on document understanding. It further optimizes the basic model based on PP-DocBee and introduces new data optimization schemes to improve data quality. With only 470,000 data generated using self-developed data synthesis strategy, PP-DocBee2 performs better in Chinese document understanding tasks. In internal business Chinese scenarios, PP-DocBee2 improves by about 11.4% compared to PP-DocBee and also outperforms current popular open and closed-source models of the same scale.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<b>Note: The above total scores are the model test results of the internal evaluation set. All images in the internal evaluation set have a resolution (height, width) of (1680,1204), with a total of 1196 data, including scenarios such as financial reports, laws and regulations, science and engineering papers, manuals, humanities papers, contracts, and research reports, with no plans to make it public for now.</b>
|
||||
</details>
|
||||
|
||||
<br />
|
||||
<b>If you focus more on model accuracy, choose a model with higher accuracy; if you care more about inference speed, choose a model with faster inference speed; if you are concerned about storage size, choose a model with a smaller storage volume.</b>
|
||||
|
||||
## 2. Quick Start
|
||||
|
||||
Before using the document understanding pipeline locally, ensure that you have completed the installation of the wheel package according to the [installation tutorial](../installation.en.md). After installation, you can experience it locally using the command line or Python integration.
|
||||
|
||||
Please note: If you encounter issues such as the program becoming unresponsive, unexpected program termination, running out of memory resources, or extremely slow inference during execution, please try adjusting the configuration according to the documentation, such as disabling unnecessary features or using lighter-weight models.
|
||||
|
||||
### 2.1 Command Line Experience
|
||||
|
||||
Experience the doc_understanding pipeline with just one command line:
|
||||
|
||||
```bash
|
||||
paddleocr doc_understanding -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容，以markdown格式输出'}"
|
||||
```
|
||||
|
||||
<details><summary><b>The command line supports more parameter settings, click to expand for a detailed explanation of the command line parameters</b></summary>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Data to be predicted, required.
|
||||
"{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': 'Recognize the content of this table and output it in markdown format'}".
|
||||
</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_path</code></td>
|
||||
<td>Specify the path for saving the inference result file. If not set, the inference result will not be saved locally.</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_model_name</code></td>
|
||||
<td>The name of the document understanding model. If not set, the default model of the pipeline will be used.</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_model_dir</code></td>
|
||||
<td>The directory path of the document understanding model. If not set, the official model will be downloaded.</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_batch_size</code></td>
|
||||
<td>The batch size of the document understanding model. If not set, the default batch size will be set to <code>1</code>.</td>
|
||||
<td><code>int</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>The device used for inference. Supports specifying a specific card number:
|
||||
<ul>
|
||||
<li><b>CPU</b>: For example, <code>cpu</code> indicates using the CPU for inference;</li>
|
||||
<li><b>GPU</b>: For example, <code>gpu:0</code> indicates using the first GPU for inference;</li>
|
||||
<li><b>NPU</b>: For example, <code>npu:0</code> indicates using the first NPU for inference;</li>
|
||||
<li><b>XPU</b>: For example, <code>xpu:0</code> indicates using the first XPU for inference;</li>
|
||||
<li><b>MLU</b>: For example, <code>mlu:0</code> indicates using the first MLU for inference;</li>
|
||||
<li><b>DCU</b>: For example, <code>dcu:0</code> indicates using the first DCU for inference;</li>
|
||||
</ul>If not set, the pipeline initialized value for this parameter will be used. During initialization, the local GPU device 0 will be preferred; if unavailable, the CPU device will be used.
|
||||
</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>paddlex_config</code></td>
|
||||
<td>Path to PaddleX pipeline configuration file.</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</details>
|
||||
<br />
|
||||
|
||||
The results will be printed to the terminal, and the default configuration of the doc_understanding pipeline will produce the following output:
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容，以markdown格式输出', 'result': '| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n'}}
|
||||
```
|
||||
|
||||
### 2.2 Python Script Integration
|
||||
|
||||
The command line method is for quickly experiencing the effect. Generally, in projects, code integration is often required. You can complete quick inference of the pipeline with just a few lines of code. The inference code is as follows:
|
||||
|
||||
```python
|
||||
from paddleocr import DocUnderstanding
|
||||
|
||||
pipeline = DocUnderstanding()
|
||||
output = pipeline.predict(
|
||||
{
|
||||
"image": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png",
|
||||
"query": "识别这份表格的内容，以markdown格式输出"
|
||||
}
|
||||
)
|
||||
for res in output:
|
||||
res.print() ## Print the structured output of the prediction
|
||||
res.save_to_json("./output/")
|
||||
```
|
||||
|
||||
In the above Python script, the following steps are performed:
|
||||
|
||||
(1) Instantiate a Document Understanding Pipeline object through `DocUnderstanding()`. The specific parameter descriptions are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>doc_understanding_model_name</code></td>
|
||||
<td>The name of the document understanding model. If set to <code>None</code>, the default model of the pipeline will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_model_dir</code></td>
|
||||
<td>The directory path of the document understanding model. If set to <code>None</code>, the official model will be downloaded.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_batch_size</code></td>
|
||||
<td>The batch size of the document understanding model. If set to <code>None</code>, the default batch size will be set to <code>1</code>.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>The device used for inference. Supports specifying a specific card number:
|
||||
<ul>
|
||||
<li><b>CPU</b>: For example, <code>cpu</code> indicates using the CPU for inference;</li>
|
||||
<li><b>GPU</b>: For example, <code>gpu:0</code> indicates using the first GPU for inference;</li>
|
||||
<li><b>NPU</b>: For example, <code>npu:0</code> indicates using the first NPU for inference;</li>
|
||||
<li><b>XPU</b>: For example, <code>xpu:0</code> indicates using the first XPU for inference;</li>
|
||||
<li><b>MLU</b>: For example, <code>mlu:0</code> indicates using the first MLU for inference;</li>
|
||||
<li><b>DCU</b>: For example, <code>dcu:0</code> indicates using the first DCU for inference;</li>
|
||||
<li><b>None</b>: If set to <code>None</code>, the pipeline initialized value for this parameter will be used. During initialization, the local GPU device 0 will be preferred; if unavailable, the CPU device will be used.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>paddlex_config</code></td>
|
||||
<td>Path to PaddleX pipeline configuration file.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
(2) Call the `predict()` method of the Document Understanding Pipeline object for inference prediction, which will return a result list.
|
||||
|
||||
Additionally, the pipeline also provides a `predict_iter()` method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator` that can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either method according to your actual needs.
|
||||
|
||||
Below are the parameters and their descriptions for the `predict()` method:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Data to be predicted, currently only supports dictionary type input
|
||||
<ul>
|
||||
<li><b>Python Dict</b>: The input format for PP-DocBee is: <code>{"image":/path/to/image, "query": user question}</code>, representing the input image and corresponding user question.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Dict</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
(3) Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation.</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a JSON format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
- Calling the `print()` method will print the result to the terminal. The content printed to the terminal is explained as follows:
|
||||
|
||||
- `image`: `(str)` Input path of the image
|
||||
|
||||
- `query`: `(str)` Question regarding the input image
|
||||
|
||||
- `result`: `(str)` Output result of the model
|
||||
|
||||
- Calling the `save_to_json()` method will save the above content to the specified `save_path`. If specified as a directory, the path saved will be `save_path/{your_img_basename}_res.json`, and if specified as a file, it will be saved directly to that file.
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "2"><code>img</code></td>
|
||||
<td rowspan = "2">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
- The prediction result obtained through the `json` attribute is data of the dict type, consistent with the content saved by calling the `save_to_json()` method.
|
||||
|
||||
## 3. Development Integration/Deployment
|
||||
|
||||
If the pipeline meets your requirements for pipeline inference speed and accuracy, you can proceed with development integration/deployment directly.
|
||||
|
||||
If you need to apply the pipeline directly to your Python project, you can refer to the example code in [2.2 Python Script Integration](#22-python-script-integration).
|
||||
|
||||
In addition, PaddleOCR also provides two other deployment methods, detailed descriptions are as follows:
|
||||
|
||||
🚀 High-Performance Inference: In real production environments, many applications have strict standards for the performance indicators of deployment strategies (especially response speed) to ensure efficient system operation and smooth user experience. To this end, PaddleOCR provides high-performance inference capabilities, aiming to deeply optimize the performance of model inference and pre-and post-processing, achieving significant acceleration of the end-to-end process. For detailed high-performance inference processes, refer to [High-Performance Inference](../deployment/high_performance_inference.md).
|
||||
|
||||
☁️ Service Deployment: Service deployment is a common form of deployment in real production environments. By encapsulating inference functions as services, clients can access these services through network requests to obtain inference results. For detailed pipeline service deployment processes, refer to [Serving](../deployment/serving.md).
|
||||
|
||||
Below is the API reference for basic service deployment and examples of service invocation in multiple languages:
|
||||
|
||||
<details><summary>API Reference</summary>
|
||||
|
||||
<p>For the main operations provided by the service:</p>
|
||||
<ul>
|
||||
<li>The HTTP request method is POST.</li>
|
||||
<li>Both the request body and response body are JSON data (JSON object).</li>
|
||||
<li>When the request is processed successfully, the response status code is <code>200</code>, and the response body has the following attributes:</li>
|
||||
</ul>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>logId</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>UUID of the request.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>errorCode</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>Error code. Fixed as <code>0</code>.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>errorMsg</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Error description. Fixed as <code>"Success"</code>.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>result</code></td>
|
||||
<td><code>object</code></td>
|
||||
<td>Operation result.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<ul>
|
||||
<li>When the request is not processed successfully, the response body has the following attributes:</li>
|
||||
</ul>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>logId</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>UUID of the request.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>errorCode</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>Error code. Same as the response status code.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>errorMsg</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Error description.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>The main operations provided by the service are as follows:</p>
|
||||
<ul>
|
||||
<li><b><code>infer</code></b></li>
|
||||
</ul>
|
||||
<p>Perform inference on the input message to generate a response.</p>
|
||||
<p><code>POST /document-understanding</code></p>
|
||||
<p>Note: The above interface is also known as /chat/completion, compatible with OpenAI interfaces.</p>
|
||||
|
||||
<ul>
|
||||
<li>The attributes of the request body are as follows:</li>
|
||||
</ul>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
<th>Required</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>The name of the model to use</td>
|
||||
<td>Yes</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>messages</code></td>
|
||||
<td><code>array</code></td>
|
||||
<td>List of dialogue messages</td>
|
||||
<td>Yes</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_tokens</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>Maximum number of tokens to generate</td>
|
||||
<td>No</td>
|
||||
<td>1024</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>temperature</code></td>
|
||||
<td><code>float</code></td>
|
||||
<td>Sampling temperature</td>
|
||||
<td>No</td>
|
||||
<td>0.1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>top_p</code></td>
|
||||
<td><code>float</code></td>
|
||||
<td>Core sampling probability</td>
|
||||
<td>No</td>
|
||||
<td>0.95</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>stream</code></td>
|
||||
<td><code>boolean</code></td>
|
||||
<td>Whether to output in streaming mode</td>
|
||||
<td>No</td>
|
||||
<td>false</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_image_tokens</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Maximum number of input tokens for images</td>
|
||||
<td>No</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>Each element in <code>messages</code> is an <code>object</code> with the following attributes:</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
<th>Required</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>role</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Message role (user/assistant/system)</td>
|
||||
<td>Yes</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>content</code></td>
|
||||
<td><code>string</code> or <code>array</code></td>
|
||||
<td>Message content (text or mixed media)</td>
|
||||
<td>Yes</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>When <code>content</code> is an array, each element is an <code>object</code> with the following attributes:</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
<th>Required</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>type</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Content type (text/image_url)</td>
|
||||
<td>Yes</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>text</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Text content (when type is text)</td>
|
||||
<td>Conditionally required</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>image_url</code></td>
|
||||
<td><code>string</code> or <code>object</code></td>
|
||||
<td>Image URL or object (when type is image_url)</td>
|
||||
<td>Conditionally required</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>When <code>image_url</code> is an object, it has the following attributes:</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
<th>Required</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>url</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Image URL</td>
|
||||
<td>Yes</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>detail</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Image detail processing method (low/high/auto)</td>
|
||||
<td>No</td>
|
||||
<td>auto</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>When the request is processed successfully, the <code>result</code> in the response body has the following attributes:</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>id</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Request ID</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>object</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Object type (chat.completion)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>created</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>Creation timestamp</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>choices</code></td>
|
||||
<td><code>array</code></td>
|
||||
<td>Generated result options</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>usage</code></td>
|
||||
<td><code>object</code></td>
|
||||
<td>Token usage</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>Each element in <code>choices</code> is a <code>Choice</code> object with the following attributes:</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
<th>Optional Values</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>finish_reason</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Reason for the model to stop generating tokens</td>
|
||||
<td><code>stop</code> (natural stop)<br><code>length</code> (reached max token count)<br><code>tool_calls</code> (called a tool)<br><code>content_filter</code> (content filtering)<br><code>function_call</code> (called a function, deprecated)</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>index</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>Index of the option in the list</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>logprobs</code></td>
|
||||
<td><code>object</code> | <code>null</code></td>
|
||||
<td>Log probability information of the option</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>message</code></td>
|
||||
<td><code>ChatCompletionMessage</code></td>
|
||||
<td>Chat message generated by the model</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>The <code>message</code> object has the following attributes:</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
<th>Remarks</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>content</code></td>
|
||||
<td><code>string</code> | <code>null</code></td>
|
||||
<td>Message content</td>
|
||||
<td>May be empty</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>refusal</code></td>
|
||||
<td><code>string</code> | <code>null</code></td>
|
||||
<td>Refusal message generated by the model</td>
|
||||
<td>Provided when content is refused</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>role</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>Role of the message author</td>
|
||||
<td>Fixed as <code>"assistant"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>audio</code></td>
|
||||
<td><code>object</code> | <code>null</code></td>
|
||||
<td>Audio output data</td>
|
||||
<td>Provided when audio output is requested<br><a href="https://platform.openai.com/docs/guides/audio">Learn more</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>function_call</code></td>
|
||||
<td><code>object</code> | <code>null</code></td>
|
||||
<td>Name and parameters of the function to be called</td>
|
||||
<td>Deprecated, recommended to use <code>tool_calls</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>tool_calls</code></td>
|
||||
<td><code>array</code> | <code>null</code></td>
|
||||
<td>Tool calls generated by the model</td>
|
||||
<td>Such as function calls</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>The <code>usage</code> object has the following attributes:</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Name</th>
|
||||
<th>Type</th>
|
||||
<th>Meaning</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>prompt_tokens</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>Number of prompt tokens</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>completion_tokens</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>Number of generated tokens</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>total_tokens</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>Total number of tokens</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>An example of a <code>result</code> is as follows:</p>
|
||||
<pre><code class="language-json">{
|
||||
"id": "ed960013-eb19-43fa-b826-3c1b59657e35",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"message": {
|
||||
"content": "| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n",
|
||||
"role": "assistant"
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1745218041,
|
||||
"model": "pp-docbee",
|
||||
"object": "chat.completion"
|
||||
}
|
||||
</code></pre></details>
|
||||
|
||||
<details><summary>Multi-language Service Invocation Examples</summary>
|
||||
|
||||
<details>
|
||||
<summary>Python</summary>
|
||||
OpenAI interface invocation example
|
||||
|
||||
<pre><code class="language-python">import base64
|
||||
from openai import OpenAI
|
||||
|
||||
API_BASE_URL = "http://127.0.0.1:8080"
|
||||
|
||||
# Initialize OpenAI client
|
||||
client = OpenAI(
|
||||
api_key='xxxxxxxxx',
|
||||
base_url=f'{API_BASE_URL}'
|
||||
)
|
||||
|
||||
# Function to convert image to base64
|
||||
def encode_image(image_path):
|
||||
with open(image_path, "rb") as image_file:
|
||||
return base64.b64encode(image_file.read()).decode('utf-8')
|
||||
|
||||
# Input image path
|
||||
image_path = "medal_table.png"
|
||||
|
||||
# Convert original image to base64
|
||||
base64_image = encode_image(image_path)
|
||||
|
||||
# Submit information to PP-DocBee model
|
||||
response = client.chat.completions.create(
|
||||
model="pp-docbee",# Choose Model
|
||||
messages=[
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a helpful assistant."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content":[
|
||||
{
|
||||
"type": "text",
|
||||
"text": "识别这份表格的内容,输出html格式的内容"
|
||||
},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}
|
||||
},
|
||||
]
|
||||
},
|
||||
],
|
||||
)
|
||||
content = response.choices[0].message.content
|
||||
print('Reply:', content)
|
||||
</code></pre></details>
|
||||
</details>
|
||||
<br/>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
The current pipeline does not support fine-tuning training and only supports inference integration. Concerning fine-tuning training for this pipeline, there are plans to support it in the future.
|
||||
809
docs/version3.x/pipeline_usage/doc_understanding.md
Normal file
809
docs/version3.x/pipeline_usage/doc_understanding.md
Normal file
@@ -0,0 +1,809 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文档理解产线使用教程
|
||||
|
||||
## 1. 文档理解产线介绍
|
||||
|
||||
文档理解产线是基于视觉-语言模型（VLM）打造的先进文档处理技术，旨在突破传统文档处理的局限。传统方法依赖固定模板或预定义规则解析文档，而该产线借助VLM的多模态能力，仅需输入文档图片和用户问题，即可通过融合视觉与语言信息，精准回答用户提问。这种技术无需针对特定文档格式预训练，能够灵活应对多样化文档内容，显著提升文档处理的泛化性与实用性，在智能问答、信息提取等场景中具有广阔应用前景。本产线目前暂不支持对VLM模型的二次开发，后续计划支持。
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/doc_understanding/doc_understanding.png">
|
||||
|
||||
<b>文档理解产线中包含以下1个模块。每个模块均可独立进行训练和推理，并包含多个模型。有关详细信息，请点击相应模块以查看文档。</b>
|
||||
|
||||
- [文档类视觉语言模型模块](../module_usage/doc_vlm.md)
|
||||
|
||||
在本产线中，您可以根据下方的基准测试数据选择使用的模型。
|
||||
|
||||
<details>
|
||||
<summary> <b>文档类视觉语言模型模块：</b></summary>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>模型存储大小（GB）</th>
|
||||
<th>模型总分</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-2B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-2B_infer.tar">推理模型</a></td>
|
||||
<td>4.2</td>
|
||||
<td>765</td>
|
||||
<td rowspan="2">PP-DocBee 是飞桨团队自研的一款专注于文档理解的多模态大模型，在中文文档理解任务上具有卓越表现。该模型通过近 500 万条文档理解类多模态数据集进行微调优化，各种数据集包括了通用VQA类、OCR类、图表类、text-rich文档类、数学和复杂推理类、合成数据类、纯文本数据等，并设置了不同训练数据配比。在学术界权威的几个英文文档理解评测榜单上，PP-DocBee基本都达到了同参数量级别模型的SOTA。在内部业务中文场景类的指标上，PP-DocBee也高于目前的热门开源和闭源模型。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-7B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-7B_infer.tar">推理模型</a></td>
|
||||
<td>15.8</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee2-3B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee2-3B_infer.tar">推理模型</a></td>
|
||||
<td>7.6</td>
|
||||
<td>852</td>
|
||||
<td>PP-DocBee2 是飞桨团队自研的一款专注于文档理解的多模态大模型，在PP-DocBee的基础上进一步优化了基础模型，并引入了新的数据优化方案，提高了数据质量，使用自研数据合成策略生成的少量的47万数据便使得PP-DocBee2在中文文档理解任务上表现更佳。在内部业务中文场景类的指标上，PP-DocBee2相较于PP-DocBee提升了约11.4%，同时也高于目前的同规模热门开源和闭源模型。</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<b>注：以上模型总分为内部评估集模型测试结果，内部评估集所有图像分辨率 (height，width) 为 (1680,1204)，共1196条数据，包括了财报、法律法规、理工科论文、说明书、文科论文、合同、研报等场景，暂时未有计划公开。</b>
|
||||
</details>
|
||||
|
||||
<br />
|
||||
<b>如果您更注重模型的精度，请选择精度较高的模型；如果您更在意模型的推理速度，请选择推理速度较快的模型；如果您关注模型的存储大小，请选择存储体积较小的模型。</b>
|
||||
|
||||
## 2. 快速开始
|
||||
|
||||
在本地使用文档理解产线前，请确保您已经按照[安装教程](../installation.md)完成了wheel包安装。安装完成后，可以在本地使用命令行体验或 Python 集成。
|
||||
|
||||
**请注意，如果在执行过程中遇到程序失去响应、程序异常退出、内存资源耗尽、推理速度极慢等问题，请尝试参考文档调整配置，例如关闭不需要使用的功能或使用更轻量的模型。**
|
||||
|
||||
### 2.1 命令行方式体验
|
||||
|
||||
一行命令即可快速体验 doc_understanding 产线效果：
|
||||
|
||||
```bash
|
||||
paddleocr doc_understanding -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容，以markdown格式输出'}"
|
||||
```
|
||||
|
||||
<details><summary><b>命令行支持更多参数设置，点击展开以查看命令行参数的详细说明</b></summary>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，必填。如"{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容，以markdown格式输出'}"。
|
||||
</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_path</code></td>
|
||||
<td>指定推理结果文件保存的路径。如果不设置，推理结果将不会保存到本地。</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_model_name</code></td>
|
||||
<td>文档理解模型的名称。如果不设置，将会使用产线默认模型。</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_model_dir</code></td>
|
||||
<td>文档理解模型的目录路径。如果不设置，将会下载官方模型。</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_batch_size</code></td>
|
||||
<td>文档理解模型的batch size。如果设置为<code>None</code>，将默认设置batch size为<code>1</code>。</td>
|
||||
<td><code>int</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。支持指定具体卡号：
|
||||
<ul>
|
||||
<li><b>CPU</b>：如 <code>cpu</code> 表示使用 CPU 进行推理；</li>
|
||||
<li><b>GPU</b>：如 <code>gpu:0</code> 表示使用第 1 块 GPU 进行推理；</li>
|
||||
<li><b>NPU</b>：如 <code>npu:0</code> 表示使用第 1 块 NPU 进行推理；</li>
|
||||
<li><b>XPU</b>：如 <code>xpu:0</code> 表示使用第 1 块 XPU 进行推理；</li>
|
||||
<li><b>MLU</b>：如 <code>mlu:0</code> 表示使用第 1 块 MLU 进行推理；</li>
|
||||
<li><b>DCU</b>：如 <code>dcu:0</code> 表示使用第 1 块 DCU 进行推理；</li>
|
||||
</ul>如果不设置，将默认使用产线初始化的该参数值，初始化时，会优先使用本地的 GPU 0号设备，如果没有，则使用 CPU 设备。
|
||||
</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>paddlex_config</code></td>
|
||||
<td>PaddleX产线配置文件路径。</td>
|
||||
<td><code>str</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</details>
|
||||
<br />
|
||||
|
||||
运行结果会被打印到终端上，默认配置的 doc_understanding 产线的运行结果如下：
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容，以markdown格式输出', 'result': '| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n'}}
|
||||
```
|
||||
|
||||
### 2.2 Python脚本方式集成
|
||||
|
||||
命令行方式是为了快速体验查看效果，一般来说，在项目中，往往需要通过代码集成，您可以通过几行代码即可完成产线的快速推理，推理代码如下：
|
||||
|
||||
```python
|
||||
from paddleocr import DocUnderstanding
|
||||
|
||||
pipeline = DocUnderstanding()
|
||||
output = pipeline.predict(
|
||||
{
|
||||
"image": "https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png",
|
||||
"query": "识别这份表格的内容，以markdown格式输出"
|
||||
}
|
||||
)
|
||||
for res in output:
|
||||
res.print() ## 打印预测的结构化输出
|
||||
res.save_to_json("./output/")
|
||||
```
|
||||
|
||||
在上述 Python 脚本中，执行了如下几个步骤：
|
||||
|
||||
（1）通过 `DocUnderstanding()` 实例化文档理解产线产线对象，具体参数说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>doc_understanding_model_name</code></td>
|
||||
<td>文档理解模型的名称。如果设置为<code>None</code>，将会使用产线默认模型。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_model_dir</code></td>
|
||||
<td>文档理解模型的目录路径。如果设置为<code>None</code>，将会下载官方模型。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>doc_understanding_batch_size</code></td>
|
||||
<td>文档理解模型的batch size。如果设置为<code>None</code>，将默认设置batch size为<code>1</code>。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。支持指定具体卡号：
|
||||
<ul>
|
||||
<li><b>CPU</b>：如 <code>cpu</code> 表示使用 CPU 进行推理；</li>
|
||||
<li><b>GPU</b>：如 <code>gpu:0</code> 表示使用第 1 块 GPU 进行推理；</li>
|
||||
<li><b>NPU</b>：如 <code>npu:0</code> 表示使用第 1 块 NPU 进行推理；</li>
|
||||
<li><b>XPU</b>：如 <code>xpu:0</code> 表示使用第 1 块 XPU 进行推理；</li>
|
||||
<li><b>MLU</b>：如 <code>mlu:0</code> 表示使用第 1 块 MLU 进行推理；</li>
|
||||
<li><b>DCU</b>：如 <code>dcu:0</code> 表示使用第 1 块 DCU 进行推理；</li>
|
||||
<li><b>None</b>：如果设置为<code>None</code>，将默认使用产线初始化的该参数值，初始化时，会优先使用本地的 GPU 0号设备，如果没有，则使用 CPU 设备。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>paddlex_config</code></td>
|
||||
<td>PaddleX产线配置文件路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
（2）调用 文档理解产线 产线对象的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。
|
||||
|
||||
另外，产线还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。
|
||||
|
||||
以下是 `predict()` 方法的参数及其说明：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，目前仅支持dict类型的输入
|
||||
<ul>
|
||||
<li><b>Python Dict</b>：如PP-DocBee的输入形式为: <code>{"image":/path/to/image, "query": user question}</code> ,分别表示输入的图像和对应的用户问题。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Dict</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
（3）对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化。</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效。</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效。</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致。</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效。</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效。</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
- 调用`print()` 方法会将结果打印到终端，打印到终端的内容解释如下：
|
||||
|
||||
- `image`: `(str)` 图像的输入路径
|
||||
|
||||
- `query`: `(str)` 针对输入图像的问题
|
||||
|
||||
- `result`: `(str)` 模型的输出结果
|
||||
|
||||
- 调用`save_to_json()` 方法会将上述内容保存到指定的`save_path`中，如果指定为目录，则保存的路径为`save_path/{your_img_basename}_res.json`，如果指定为文件，则直接保存到该文件中。
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的 <code>json</code> 格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "2"><code>img</code></td>
|
||||
<td rowspan = "2">获取格式为 <code>dict</code> 的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
- `json` 属性获取的预测结果为dict类型的数据，相关内容与调用 `save_to_json()` 方法保存的内容一致。
|
||||
|
||||
## 3. 开发集成/部署
|
||||
|
||||
如果产线可以达到您对产线推理速度和精度的要求，您可以直接进行开发集成/部署。
|
||||
|
||||
若您需要将产线直接应用在您的Python项目中，可以参考 [2.2 Python脚本方式](#22-python脚本方式集成) 中的示例代码。
|
||||
|
||||
此外，PaddleOCR 也提供了其他两种部署方式，详细说明如下：
|
||||
|
||||
🚀 高性能推理：在实际生产环境中，许多应用对部署策略的性能指标（尤其是响应速度）有着较严苛的标准，以确保系统的高效运行与用户体验的流畅性。为此，PaddleOCR 提供高性能推理功能，旨在对模型推理及前后处理进行深度性能优化，实现端到端流程的显著提速，详细的高性能推理流程请参考[高性能推理](../deployment/high_performance_inference.md)。
|
||||
|
||||
☁️ 服务化部署：服务化部署是实际生产环境中常见的一种部署形式。通过将推理功能封装为服务，客户端可以通过网络请求来访问这些服务，以获取推理结果。详细的产线服务化部署流程请参考[服务化部署](../deployment/serving.md)。
|
||||
|
||||
以下是基础服务化部署的API参考与多语言服务调用示例：
|
||||
|
||||
<details><summary>API参考</summary>
|
||||
|
||||
<p>对于服务提供的主要操作：</p>
|
||||
<ul>
|
||||
<li>HTTP请求方法为POST。</li>
|
||||
<li>请求体和响应体均为JSON数据（JSON对象）。</li>
|
||||
<li>当请求处理成功时，响应状态码为<code>200</code>，响应体的属性如下：</li>
|
||||
</ul>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>logId</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>请求的UUID。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>errorCode</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>错误码。固定为<code>0</code>。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>errorMsg</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>错误说明。固定为<code>"Success"</code>。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>result</code></td>
|
||||
<td><code>object</code></td>
|
||||
<td>操作结果。</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<ul>
|
||||
<li>当请求处理未成功时，响应体的属性如下：</li>
|
||||
</ul>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>logId</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>请求的UUID。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>errorCode</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>错误码。与响应状态码相同。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>errorMsg</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>错误说明。</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p>服务提供的主要操作如下：</p>
|
||||
<ul>
|
||||
<li><b><code>infer</code></b></li>
|
||||
</ul>
|
||||
<p>对输入消息进行推理生成响应。</p>
|
||||
<p><code>POST /document-understanding</code></p>
|
||||
<p>说明 以上接口别名/chat/completion，openai兼容的接口</p>
|
||||
|
||||
<ul>
|
||||
<li>请求体的属性如下：</li>
|
||||
</ul>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
<th>是否必填</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>要使用的模型名称</td>
|
||||
<td>是</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>messages</code></td>
|
||||
<td><code>array</code></td>
|
||||
<td>对话消息列表</td>
|
||||
<td>是</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_tokens</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>生成的最大token数</td>
|
||||
<td>否</td>
|
||||
<td>1024</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>temperature</code></td>
|
||||
<td><code>float</code></td>
|
||||
<td>采样温度</td>
|
||||
<td>否</td>
|
||||
<td>0.1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>top_p</code></td>
|
||||
<td><code>float</code></td>
|
||||
<td>核心采样概率</td>
|
||||
<td>否</td>
|
||||
<td>0.95</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>stream</code></td>
|
||||
<td><code>boolean</code></td>
|
||||
<td>是否流式输出</td>
|
||||
<td>否</td>
|
||||
<td>false</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_image_tokens</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>图像的最大输入token数</td>
|
||||
<td>否</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p><code>messages</code>中的每个元素为一个<code>object</code>，具有如下属性：</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
<th>是否必填</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>role</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>消息角色（user/assistant/system）</td>
|
||||
<td>是</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>content</code></td>
|
||||
<td><code>string</code>或<code>array</code></td>
|
||||
<td>消息内容（文本或图文混合）</td>
|
||||
<td>是</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>当<code>content</code>为数组时，每个元素为一个<code>object</code>，具有如下属性：</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
<th>是否必填</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>type</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>内容类型（text/image_url）</td>
|
||||
<td>是</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>text</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>文本内容（当type为text时）</td>
|
||||
<td>条件必填</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>image_url</code></td>
|
||||
<td><code>string</code>或<code>object</code></td>
|
||||
<td>图片URL或对象（当type为image_url时）</td>
|
||||
<td>条件必填</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>当<code>image_url</code>为对象时，具有如下属性：</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
<th>是否必填</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>url</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>图片URL</td>
|
||||
<td>是</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>detail</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>图片细节处理方式（low/high/auto）</td>
|
||||
<td>否</td>
|
||||
<td>auto</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p>请求处理成功时，响应体的<code>result</code>具有如下属性：</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>id</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>请求ID</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>object</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>对象类型（chat.completion）</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>created</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>创建时间戳</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>choices</code></td>
|
||||
<td><code>array</code></td>
|
||||
<td>生成结果选项</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>usage</code></td>
|
||||
<td><code>object</code></td>
|
||||
<td>token使用情况</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p><code>choices</code>中的每个元素为一个<code>Choice</code>对象，具有如下属性：</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
<th>可选值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>finish_reason</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>模型停止生成token的原因</td>
|
||||
<td><code>stop</code>（自然停止）<br><code>length</code>（达到最大token数）<br><code>tool_calls</code>（调用了工具）<br><code>content_filter</code>（内容过滤）<br><code>function_call</code>（调用了函数，已弃用）</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>index</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>选项在列表中的索引</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>logprobs</code></td>
|
||||
<td><code>object</code> | <code>null</code></td>
|
||||
<td>选项的log概率信息</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>message</code></td>
|
||||
<td><code>ChatCompletionMessage</code></td>
|
||||
<td>模型生成的聊天消息</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p><code>message</code>对象具有如下属性：</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
<th>备注</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>content</code></td>
|
||||
<td><code>string</code> | <code>null</code></td>
|
||||
<td>消息内容</td>
|
||||
<td>可能为空</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>refusal</code></td>
|
||||
<td><code>string</code> | <code>null</code></td>
|
||||
<td>模型生成的拒绝消息</td>
|
||||
<td>当内容被拒绝时提供</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>role</code></td>
|
||||
<td><code>string</code></td>
|
||||
<td>消息作者角色</td>
|
||||
<td>固定为<code>"assistant"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>audio</code></td>
|
||||
<td><code>object</code> | <code>null</code></td>
|
||||
<td>音频输出数据</td>
|
||||
<td>当请求音频输出时提供<br><a href="https://platform.openai.com/docs/guides/audio">了解更多</a></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>function_call</code></td>
|
||||
<td><code>object</code> | <code>null</code></td>
|
||||
<td>应调用的函数名称和参数</td>
|
||||
<td>已弃用，推荐使用<code>tool_calls</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>tool_calls</code></td>
|
||||
<td><code>array</code> | <code>null</code></td>
|
||||
<td>模型生成的工具调用</td>
|
||||
<td>如函数调用等</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<p><code>usage</code>对象具有如下属性：</p>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>名称</th>
|
||||
<th>类型</th>
|
||||
<th>含义</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>prompt_tokens</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>提示token数</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>completion_tokens</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>生成token数</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>total_tokens</code></td>
|
||||
<td><code>integer</code></td>
|
||||
<td>总token数</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<p><code>result</code>示例如下：</p>
|
||||
<pre><code class="language-json">{
|
||||
"id": "ed960013-eb19-43fa-b826-3c1b59657e35",
|
||||
"choices": [
|
||||
{
|
||||
"finish_reason": "stop",
|
||||
"index": 0,
|
||||
"message": {
|
||||
"content": "| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n",
|
||||
"role": "assistant"
|
||||
}
|
||||
}
|
||||
],
|
||||
"created": 1745218041,
|
||||
"model": "pp-docbee",
|
||||
"object": "chat.completion"
|
||||
}
|
||||
</code></pre></details>
|
||||
|
||||
<details><summary>多语言调用服务示例</summary>
|
||||
|
||||
<details>
|
||||
<summary>Python</summary>
|
||||
openai接口调用示例
|
||||
|
||||
<pre><code class="language-python">import base64
|
||||
from openai import OpenAI
|
||||
|
||||
API_BASE_URL = "http://127.0.0.1:8080"
|
||||
|
||||
# 初始化OpenAI客户端
|
||||
client = OpenAI(
|
||||
api_key='xxxxxxxxx',
|
||||
base_url=f'{API_BASE_URL}'
|
||||
)
|
||||
|
||||
#图片转base64函数
|
||||
def encode_image(image_path):
|
||||
with open(image_path, "rb") as image_file:
|
||||
return base64.b64encode(image_file.read()).decode('utf-8')
|
||||
|
||||
#输入图片路径
|
||||
image_path = "medal_table.png"
|
||||
|
||||
#原图片转base64
|
||||
base64_image = encode_image(image_path)
|
||||
|
||||
#提交信息至PP-DocBee模型
|
||||
response = client.chat.completions.create(
|
||||
model="pp-docbee",#选择模型
|
||||
messages=[
|
||||
{
|
||||
"role": "system",
|
||||
"content": "You are a helpful assistant."
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
"content":[
|
||||
{
|
||||
"type": "text",
|
||||
"text": "识别这份表格的内容,输出html格式的内容"
|
||||
},
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": f"data:image/jpeg;base64,{base64_image}"}
|
||||
},
|
||||
]
|
||||
},
|
||||
],
|
||||
)
|
||||
content = response.choices[0].message.content
|
||||
print('Reply:', content)
|
||||
</code></pre></details>
|
||||
</details>
|
||||
<br/>
|
||||
|
||||
## 4. 二次开发
|
||||
|
||||
当前产线暂时不支持微调训练，仅支持推理集成。关于该产线的微调训练，计划在未来支持。
|
||||
1939
docs/version3.x/pipeline_usage/formula_recognition.en.md
Normal file
1939
docs/version3.x/pipeline_usage/formula_recognition.en.md
Normal file
File diff suppressed because one or more lines are too long
1915
docs/version3.x/pipeline_usage/formula_recognition.md
Normal file
1915
docs/version3.x/pipeline_usage/formula_recognition.md
Normal file
File diff suppressed because one or more lines are too long
@@ -0,0 +1,215 @@
|
||||
# Parallel Inference in Pipeline
|
||||
|
||||
## Specifying Multiple Inference Devices
|
||||
|
||||
For some pipelines, both the CLI and Python API of PaddleOCR support specifying multiple inference devices simultaneously. If multiple devices are specified, during pipeline initialization, an instance of the underlying pipeline class will be created on each device, and the received inputs will be processed using parallel inference. For example, for the document image preprocessing pipeline:
|
||||
|
||||
```bash
|
||||
paddleocr doc_preprocessor \
|
||||
--input input_images/ \
|
||||
--device 'gpu:0,1,2,3' \
|
||||
--use_doc_orientation_classify True \
|
||||
--use_doc_unwarping True
|
||||
--save_path ./output \
|
||||
|
||||
```
|
||||
|
||||
```python
|
||||
from paddleocr import DocPreprocessor
|
||||
|
||||
|
||||
pipeline = DocPreprocessor(device="gpu:0,1,2,3")
|
||||
output = pipeline.predict(
|
||||
input="input_images/",
|
||||
use_doc_orientation_classify=True,
|
||||
use_doc_unwarping=True)
|
||||
|
||||
```
|
||||
|
||||
Both examples above use 4 GPUs (numbered 0, 1, 2, 3) to perform parallel inference on the `doc_test_rotated.jpg` image.
|
||||
|
||||
When specifying multiple devices, the inference interface remains consistent with that of single-device usage. Please refer to the production line usage tutorial to check whether a specific production line supports multiple inference devices.
|
||||
|
||||
## Example of Multi-Process Parallel Inference
|
||||
|
||||
Beyond PaddleOCR's built-in multi-device parallel inference capability, users can also implement parallelism by wrapping PaddleOCR pipeline API calls themselves according to their specific scenario, with a view to achieving a better speedup. Below is an example of using Python multiprocessing to perform multi-GPU, multi-instance parallel processing on files in an input directory.
|
||||
|
||||
|
||||
|
||||
```python
|
||||
import argparse
|
||||
import sys
|
||||
from multiprocessing import Manager, Process
|
||||
from pathlib import Path
|
||||
from queue import Empty
|
||||
|
||||
import paddleocr
|
||||
|
||||
|
||||
def load_pipeline(class_name: str, device: str):
|
||||
if not hasattr(paddleocr, class_name):
|
||||
raise ValueError(f"Class {class_name} not found in paddleocr module.")
|
||||
cls = getattr(paddleocr, class_name)
|
||||
return cls(device=device)
|
||||
|
||||
|
||||
def worker(pipeline_class_path, device, task_queue, batch_size, output_dir):
|
||||
pipeline = load_pipeline(pipeline_class_path, device)
|
||||
|
||||
should_end = False
|
||||
batch = []
|
||||
|
||||
while not should_end:
|
||||
try:
|
||||
input_path = task_queue.get_nowait()
|
||||
except Empty:
|
||||
should_end = True
|
||||
else:
|
||||
batch.append(input_path)
|
||||
|
||||
if batch and (len(batch) == batch_size or should_end):
|
||||
try:
|
||||
for result in pipeline.predict(batch):
|
||||
input_path = Path(result["input_path"])
|
||||
if result.get("page_index") is not None:
|
||||
output_path = f"{input_path.stem}_{result['page_index']}.json"
|
||||
else:
|
||||
output_path = f"{input_path.stem}.json"
|
||||
output_path = str(Path(output_dir, output_path))
|
||||
result.save_to_json(output_path)
|
||||
print(f"Processed {repr(str(input_path))}")
|
||||
except Exception as e:
|
||||
print(
|
||||
f"Error processing {batch} on {repr(device)}: {e}",
|
||||
file=sys.stderr
|
||||
)
|
||||
batch.clear()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
|
||||
parser.add_argument(
|
||||
"--pipeline",
|
||||
type=str,
|
||||
required=True,
|
||||
help="PaddleOCR pipeline, e.g. 'DocPreprocessor'.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--input_dir", type=str, required=True, help="Input directory."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--device",
|
||||
type=str,
|
||||
required=True,
|
||||
help="Specifies the devices for performing parallel inference.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output_dir", type=str, default="output", help="Output directory."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--instances_per_device",
|
||||
type=int,
|
||||
default=1,
|
||||
help="Number of pipeline instances per device.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--batch_size",
|
||||
type=int,
|
||||
default=1,
|
||||
help="Inference batch size for each pipeline instance.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--input_glob_pattern",
|
||||
type=str,
|
||||
default="*",
|
||||
help="Pattern to find the input files.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
input_dir = Path(args.input_dir)
|
||||
if not input_dir.exists():
|
||||
print(f"The input directory does not exist: {input_dir}", file=sys.stderr)
|
||||
return 2
|
||||
if not input_dir.is_dir():
|
||||
print(f"{repr(str(input_dir))} is not a directory.", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
output_dir = Path(args.output_dir)
|
||||
if output_dir.exists() and not output_dir.is_dir():
|
||||
print(f"{repr(str(output_dir))} is not a directory.", file=sys.stderr)
|
||||
return 2
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
from paddlex.utils.device import constr_device, parse_device
|
||||
|
||||
device_type, device_ids = parse_device(args.device)
|
||||
if device_ids is None or len(device_ids) == 1:
|
||||
print(
|
||||
"Please specify at least two devices for performing parallel inference.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
|
||||
if args.batch_size <= 0:
|
||||
print("Batch size must be greater than 0.", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
with Manager() as manager:
|
||||
task_queue = manager.Queue()
|
||||
for img_path in input_dir.glob(args.input_glob_pattern):
|
||||
task_queue.put(str(img_path))
|
||||
|
||||
processes = []
|
||||
for device_id in device_ids:
|
||||
for _ in range(args.instances_per_device):
|
||||
device = constr_device(device_type, [device_id])
|
||||
p = Process(
|
||||
target=worker,
|
||||
args=(
|
||||
args.pipeline,
|
||||
device,
|
||||
task_queue,
|
||||
args.batch_size,
|
||||
str(output_dir),
|
||||
),
|
||||
)
|
||||
p.start()
|
||||
processes.append(p)
|
||||
|
||||
for p in processes:
|
||||
p.join()
|
||||
|
||||
print("All done")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
```
|
||||
Assuming the script is saved as `infer_mp.py`, here are some example commands for running it:
|
||||
```bash
|
||||
# For the exact value of the `--pipeline` parameter, please refer to the **script** import name of the pipeline
|
||||
# This is for the general layout analysis V3 pipeline, corresponding to `PPStructureV3`
|
||||
# Process all files in the `input_images` directory
|
||||
# Use GPUs 0, 1, 2, and 3, with 1 pipeline instance per GPU, and each instance processes 1 input file at a time
|
||||
python infer_mp.py \
|
||||
--pipeline PPStructureV3 \
|
||||
--input_dir input_images/ \
|
||||
--device 'gpu:0,1,2,3' \
|
||||
--output_dir output
|
||||
|
||||
# General layout analysis V3 pipeline
|
||||
# Process all files with the `.jpg` suffix in the `input_images` directory
|
||||
# Use GPUs 0 and 2, with 2 pipeline instances per GPU, and each instance processes 4 input files at a time
|
||||
python infer_mp.py \
|
||||
--pipeline PPStructureV3 \
|
||||
--input_dir input_images/ \
|
||||
--device 'gpu:0,2' \
|
||||
--output_dir output \
|
||||
--instances_per_device 2 \
|
||||
--batch_size 4 \
|
||||
--input_glob_pattern '*.jpg'
|
||||
|
||||
```
|
||||
@@ -0,0 +1,214 @@
|
||||
# 产线并行推理
|
||||
|
||||
## 指定多个推理设备
|
||||
|
||||
对于部分产线的 CLI 和 Python API，PaddleOCR 支持同时指定多个推理设备。如果指定了多个设备，产线初始化时将在每个设备上创建一个底层产线类对象的实例，并对接收到的输入进行并行推理。例如，对于文档图像预处理产线：
|
||||
|
||||
```bash
|
||||
paddleocr doc_preprocessor \
|
||||
--input input_images/ \
|
||||
--device 'gpu:0,1,2,3' \
|
||||
--use_doc_orientation_classify True \
|
||||
--use_doc_unwarping True
|
||||
--save_path ./output \
|
||||
|
||||
```
|
||||
|
||||
```python
|
||||
from paddleocr import DocPreprocessor
|
||||
|
||||
|
||||
pipeline = DocPreprocessor(device="gpu:0,1,2,3")
|
||||
output = pipeline.predict(
|
||||
input="input_images/",
|
||||
use_doc_orientation_classify=True,
|
||||
use_doc_unwarping=True)
|
||||
|
||||
```
|
||||
|
||||
以上两个例子均使用 4 块 GPU（编号为 0、1、2、3）对 `doc_test_rotated.jpg` 图片进行并行推理。
|
||||
|
||||
指定多个设备时，推理接口仍然与指定单设备时保持一致。请查看产线使用教程以了解某一产线是否支持指定多个推理设备。
|
||||
|
||||
## 多进程并行推理示例
|
||||
|
||||
除了使用 PaddleOCR 内置的多设备并行推理功能外，用户也可以结合实际场景，通过封装 PaddleOCR 产线 API 调用来实现并行处理，从而获得更优的加速效果。如下是使用 Python 多进程实现多卡、多实例并行处理输入目录中的文件的示例代码：
|
||||
|
||||
```python
|
||||
import argparse
|
||||
import sys
|
||||
from multiprocessing import Manager, Process
|
||||
from pathlib import Path
|
||||
from queue import Empty
|
||||
|
||||
import paddleocr
|
||||
|
||||
|
||||
def load_pipeline(class_name: str, device: str):
|
||||
if not hasattr(paddleocr, class_name):
|
||||
raise ValueError(f"Class {class_name} not found in paddleocr module.")
|
||||
cls = getattr(paddleocr, class_name)
|
||||
return cls(device=device)
|
||||
|
||||
|
||||
def worker(pipeline_class_path, device, task_queue, batch_size, output_dir):
|
||||
pipeline = load_pipeline(pipeline_class_path, device)
|
||||
|
||||
should_end = False
|
||||
batch = []
|
||||
|
||||
while not should_end:
|
||||
try:
|
||||
input_path = task_queue.get_nowait()
|
||||
except Empty:
|
||||
should_end = True
|
||||
else:
|
||||
batch.append(input_path)
|
||||
|
||||
if batch and (len(batch) == batch_size or should_end):
|
||||
try:
|
||||
for result in pipeline.predict(batch):
|
||||
input_path = Path(result["input_path"])
|
||||
if result.get("page_index") is not None:
|
||||
output_path = f"{input_path.stem}_{result['page_index']}.json"
|
||||
else:
|
||||
output_path = f"{input_path.stem}.json"
|
||||
output_path = str(Path(output_dir, output_path))
|
||||
result.save_to_json(output_path)
|
||||
print(f"Processed {repr(str(input_path))}")
|
||||
except Exception as e:
|
||||
print(
|
||||
f"Error processing {batch} on {repr(device)}: {e}",
|
||||
file=sys.stderr
|
||||
)
|
||||
batch.clear()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
|
||||
parser.add_argument(
|
||||
"--pipeline",
|
||||
type=str,
|
||||
required=True,
|
||||
help="PaddleOCR pipeline, e.g. 'DocPreprocessor'.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--input_dir", type=str, required=True, help="Input directory."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--device",
|
||||
type=str,
|
||||
required=True,
|
||||
help="Specifies the devices for performing parallel inference.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output_dir", type=str, default="output", help="Output directory."
|
||||
)
|
||||
parser.add_argument(
|
||||
"--instances_per_device",
|
||||
type=int,
|
||||
default=1,
|
||||
help="Number of pipeline instances per device.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--batch_size",
|
||||
type=int,
|
||||
default=1,
|
||||
help="Inference batch size for each pipeline instance.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--input_glob_pattern",
|
||||
type=str,
|
||||
default="*",
|
||||
help="Pattern to find the input files.",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
input_dir = Path(args.input_dir)
|
||||
if not input_dir.exists():
|
||||
print(f"The input directory does not exist: {input_dir}", file=sys.stderr)
|
||||
return 2
|
||||
if not input_dir.is_dir():
|
||||
print(f"{repr(str(input_dir))} is not a directory.", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
output_dir = Path(args.output_dir)
|
||||
if output_dir.exists() and not output_dir.is_dir():
|
||||
print(f"{repr(str(output_dir))} is not a directory.", file=sys.stderr)
|
||||
return 2
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
from paddlex.utils.device import constr_device, parse_device
|
||||
|
||||
device_type, device_ids = parse_device(args.device)
|
||||
if device_ids is None or len(device_ids) == 1:
|
||||
print(
|
||||
"Please specify at least two devices for performing parallel inference.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
|
||||
if args.batch_size <= 0:
|
||||
print("Batch size must be greater than 0.", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
with Manager() as manager:
|
||||
task_queue = manager.Queue()
|
||||
for img_path in input_dir.glob(args.input_glob_pattern):
|
||||
task_queue.put(str(img_path))
|
||||
|
||||
processes = []
|
||||
for device_id in device_ids:
|
||||
for _ in range(args.instances_per_device):
|
||||
device = constr_device(device_type, [device_id])
|
||||
p = Process(
|
||||
target=worker,
|
||||
args=(
|
||||
args.pipeline,
|
||||
device,
|
||||
task_queue,
|
||||
args.batch_size,
|
||||
str(output_dir),
|
||||
),
|
||||
)
|
||||
p.start()
|
||||
processes.append(p)
|
||||
|
||||
for p in processes:
|
||||
p.join()
|
||||
|
||||
print("All done")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
|
||||
```
|
||||
|
||||
假设将上述脚本存储为 `infer_mp.py`，以下是一些调用示例：
|
||||
|
||||
```bash
|
||||
# 确定 `--pipeline` 参数需查看其产线 **脚本方式** 导入类名称
|
||||
# 此处为通用版面解析 v3 产线，对应PPStructureV3
|
||||
# 处理 `input_images` 目录中所有文件
|
||||
# 使用 GPU 0、1、2、3，每块 GPU 上 1 个产线实例，每个实例一次处理 1 个输入文件
|
||||
python infer_mp.py \
|
||||
--pipeline PPStructureV3 \
|
||||
--input_dir input_images/ \
|
||||
--device 'gpu:0,1,2,3' \
|
||||
--output_dir output
|
||||
|
||||
# 通用版面解析 v3 产线
|
||||
# 处理 `input_images` 目录中所有后缀为 `.jpg` 的文件
|
||||
# 使用 GPU 0、2，每块 GPU 上 2 个产线实例，每个实例一次处理 4 个输入文件
|
||||
python infer_mp.py \
|
||||
--pipeline PPStructureV3 \
|
||||
--input_dir input_images/ \
|
||||
--device 'gpu:0,2' \
|
||||
--output_dir output \
|
||||
--instances_per_device 2 \
|
||||
--batch_size 4 \
|
||||
--input_glob_pattern '*.jpg'
|
||||
```
|
||||
3
docs/version3.x/pipeline_usage/pipeline_overview.en.md
Normal file
3
docs/version3.x/pipeline_usage/pipeline_overview.en.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# Pipeline Overview
|
||||
|
||||
A pipeline is a practical functional implementation composed of one or more modules. Through reasonable module combination and configuration, pipelines can meet the needs of complex application scenarios, such as technological applications like Optical Character Recognition (OCR). Pipelines not only demonstrate the integrated application of basic modules but also support capabilities such as high-performance inference and service-oriented deployment, providing users with higher development efficiency and broader application possibilities.
|
||||
3
docs/version3.x/pipeline_usage/pipeline_overview.md
Normal file
3
docs/version3.x/pipeline_usage/pipeline_overview.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# 产线概述
|
||||
|
||||
产线是由一个或多个模块组合而成的实际功能实现。通过合理的模块组合与配置，产线能够满足复杂应用场景的需求，比如光学字符识别（OCR）等技术应用。产线不仅体现了基础模块的整合应用，还支持高性能推理和服务化部署等能力，赋予用户更高的开发效率和更广泛的应用可能性。
|
||||
2382
docs/version3.x/pipeline_usage/seal_recognition.en.md
Normal file
2382
docs/version3.x/pipeline_usage/seal_recognition.en.md
Normal file
File diff suppressed because it is too large
Load Diff
2373
docs/version3.x/pipeline_usage/seal_recognition.md
Normal file
2373
docs/version3.x/pipeline_usage/seal_recognition.md
Normal file
File diff suppressed because it is too large
Load Diff
2703
docs/version3.x/pipeline_usage/table_recognition_v2.en.md
Normal file
2703
docs/version3.x/pipeline_usage/table_recognition_v2.en.md
Normal file
File diff suppressed because it is too large
Load Diff
2702
docs/version3.x/pipeline_usage/table_recognition_v2.md
Normal file
2702
docs/version3.x/pipeline_usage/table_recognition_v2.md
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user