This commit is contained in:
64
docs/version3.x/algorithm/PP-ChatOCRv4/PP-ChatOCRv4.en.md
Normal file
64
docs/version3.x/algorithm/PP-ChatOCRv4/PP-ChatOCRv4.en.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# Introduction to PP-ChatOCRV4
|
||||
**PP-ChatOCRv4** is a unique document and image intelligent analysis solution from PaddlePaddle, combining LLM, MLLM, and OCR technologies to address complex document information extraction challenges such as layout analysis, rare characters, multi-page PDFs, tables, and seal recognition. Integrated with ERNIE Bot, it fuses massive data and knowledge, achieving high accuracy and wide applicability. This pipeline also provides flexible service deployment options, supporting deployment on various hardware. Furthermore, it offers custom development capabilities, allowing you to train and fine-tune models on your own datasets, with seamless integration of trained models.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# Key Metrics
|
||||
|
||||
<div align="center">
|
||||
<table>
|
||||
<thead>
|
||||
<tr >
|
||||
<th class>Solution</td>
|
||||
<th class>Avg Recall</td>
|
||||
</tr>
|
||||
<thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>GPT-4o</td>
|
||||
<td>63.47%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-ChatOCRv3</td>
|
||||
<td class>70.08%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>80.26%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-ChatOCRv4</b></td>
|
||||
<td><b>85.55%</b></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
# Demo
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo1.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo2.png" width="350"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo3.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo4.png" width="350"/>
|
||||
</div>
|
||||
|
||||
|
||||
# FAQ
|
||||
|
||||
1. Does support other multimodal models?
|
||||
|
||||
Yes, only set on pipeline configuration.
|
||||
|
||||
2. How to reduce latency and improve throughput?
|
||||
|
||||
Use the High-performance inference plugin, and deploy multi instances.
|
||||
|
||||
3. How to further improve accuracy?
|
||||
|
||||
Firstly, it is necessary to check whether the extracted visual information is correct. If the visual information is incorrect, it is necessary to visualize the visual prediction results to determine which model performs poorly, and then fine-tune train the model with more data. If the visual information is correct but cannot extract the correct information, the prompt needs to be adjusted according to the analysing about the question and answer.
|
||||
65
docs/version3.x/algorithm/PP-ChatOCRv4/PP-ChatOCRv4.md
Normal file
65
docs/version3.x/algorithm/PP-ChatOCRv4/PP-ChatOCRv4.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# 一、PP-ChatOCRV4简介
|
||||
**PP-ChatOCRv4**是飞桨特色的文档图像智能分析解决方案，结合了 LLM、MLLM 和 OCR 等技术，一站式解决版面分析、生僻字识别、多页 PDF 文件批量解析、复杂表格识别、印章识别等常见的复杂文档信息抽取难点问题，结合文心大模型将海量数据和知识相融合，信息抽取准确率高且应用广泛。本产线同时提供了灵活的服务化部署方式，支持在多种硬件上部署。不仅如此，本产线也提供了二次开发的能力，您可以基于本产线在您自己的数据集上训练调优，训练后的模型也可以无缝集成。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# 二、关键指标
|
||||
|
||||
<div align="center">
|
||||
<table>
|
||||
<thead>
|
||||
<tr >
|
||||
<th class>Solution</td>
|
||||
<th class>Avg Recall</td>
|
||||
</tr>
|
||||
<thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>GPT-4o</td>
|
||||
<td>63.47%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-ChatOCRv3</td>
|
||||
<td class>70.08%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>80.26%</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-ChatOCRv4</b></td>
|
||||
<td><b>85.55%</b></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
# 三、PP-ChatOCRv4 Demo示例
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo1.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo2.png" width="350"/>
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo3.png" width="350"/>
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-ChatOCRv4/algorithm_ppchatocrv4_demo4.png" width="350"/>
|
||||
</div>
|
||||
|
||||
# 四、使用方法和常见问题
|
||||
|
||||
1. 多模态大模型除了DocBee，是否支持其他多模态模型？
|
||||
|
||||
支持，只需在配置文件中进行设置即可。
|
||||
|
||||
2. 如何降低时延、提升吞吐？
|
||||
|
||||
无论使用哪一种服务化部署方案，都可以通过启用高性能推理插件提升模型推理速度，从而降低处理时延。
|
||||
|
||||
此外，对于高稳定性服务化部署方案，通过调整服务配置，设置多个实例，也可以充分利用部署机器的资源，有效提升吞吐。
|
||||
|
||||
3. 如何进一步提升精度？
|
||||
|
||||
首先需要检查提取的视觉信息是否正确，如果视觉信息有误，则需要通过可视化视觉预测结果，判断哪个模型效果较差，从而针对性地训练微调较差的模型；如果视觉信息无误，但无法抽取正确信息，则需要根据问答的具体情况调整Prompt。
|
||||
337
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5.en.md
Normal file
337
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5.en.md
Normal file
@@ -0,0 +1,337 @@
|
||||
# Introduction to PP-OCRv5
|
||||
|
||||
**PP-OCRv5** is the new generation text recognition solution of PP-OCR, focusing on multi-scenario and multi-text type recognition. In terms of text types, PP-OCRv5 supports 5 major mainstream text types: Simplified Chinese, Chinese Pinyin, Traditional Chinese, English, and Japanese. For scenarios, PP-OCRv5 has upgraded recognition capabilities for challenging scenarios such as complex Chinese and English handwriting, vertical text, and uncommon characters. On internal complex evaluation sets across multiple scenarios, PP-OCRv5 achieved a 13 percentage point end-to-end improvement over PP-OCRv4.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/algorithm_ppocrv5.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# Key Metrics
|
||||
|
||||
### 1. Text Detection Metrics
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th>
|
||||
<th>Handwritten Chinese</th>
|
||||
<th>Handwritten English</th>
|
||||
<th>Printed Chinese</th>
|
||||
<th>Printed English</th>
|
||||
<th>Traditional Chinese</th>
|
||||
<th>Ancient Text</th>
|
||||
<th>Japanese</th>
|
||||
<th>General Scenario</th>
|
||||
<th>Pinyin</th>
|
||||
<th>Rotation</th>
|
||||
<th>Distortion</th>
|
||||
<th>Artistic Text</th>
|
||||
<th>Average</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><b>PP-OCRv5_server_det</b></td>
|
||||
<td><b>0.803</b></td>
|
||||
<td><b>0.841</b></td>
|
||||
<td><b>0.945</b></td>
|
||||
<td><b>0.917</b></td>
|
||||
<td><b>0.815</b></td>
|
||||
<td><b>0.676</b></td>
|
||||
<td><b>0.772</b></td>
|
||||
<td><b>0.797</b></td>
|
||||
<td><b>0.671</b></td>
|
||||
<td><b>0.8</b></td>
|
||||
<td><b>0.876</b></td>
|
||||
<td><b>0.673</b></td>
|
||||
<td><b>0.827</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td>0.706</td>
|
||||
<td>0.249</td>
|
||||
<td>0.888</td>
|
||||
<td>0.690</td>
|
||||
<td>0.759</td>
|
||||
<td>0.473</td>
|
||||
<td>0.685</td>
|
||||
<td>0.715</td>
|
||||
<td>0.542</td>
|
||||
<td>0.366</td>
|
||||
<td>0.775</td>
|
||||
<td>0.583</td>
|
||||
<td>0.662</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-OCRv5_mobile_det</b></td>
|
||||
<td><b>0.744</b></td>
|
||||
<td><b>0.777</b></td>
|
||||
<td><b>0.905</b></td>
|
||||
<td><b>0.910</b></td>
|
||||
<td><b>0.823</b></td>
|
||||
<td><b>0.581</b></td>
|
||||
<td><b>0.727</b></td>
|
||||
<td><b>0.721</b></td>
|
||||
<td><b>0.575</b></td>
|
||||
<td><b>0.647</b></td>
|
||||
<td><b>0.827</b></td>
|
||||
<td>0.525</td>
|
||||
<td><b>0.770</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td>0.583</td>
|
||||
<td>0.369</td>
|
||||
<td>0.872</td>
|
||||
<td>0.773</td>
|
||||
<td>0.663</td>
|
||||
<td>0.231</td>
|
||||
<td>0.634</td>
|
||||
<td>0.710</td>
|
||||
<td>0.430</td>
|
||||
<td>0.299</td>
|
||||
<td>0.715</td>
|
||||
<td><b>0.549</b></td>
|
||||
<td>0.624</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
Compared to PP-OCRv4, PP-OCRv5 shows significant improvement in all detection scenarios, especially in handwriting, ancient texts, and Japanese detection capabilities.
|
||||
|
||||
### 2. Text Recognition Metrics
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/ocrv5_rec_acc.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Evaluation Set Category</th>
|
||||
<th>Handwritten Chinese</th>
|
||||
<th>Handwritten English</th>
|
||||
<th>Printed Chinese</th>
|
||||
<th>Printed English</th>
|
||||
<th>Traditional Chinese</th>
|
||||
<th>Ancient Text</th>
|
||||
<th>Japanese</th>
|
||||
<th>Confusable Characters</th>
|
||||
<th>General Scenario</th>
|
||||
<th>Pinyin</th>
|
||||
<th>Vertical Text</th>
|
||||
<th>Artistic Text</th>
|
||||
<th>Weighted Average</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><b>0.5807</b></td>
|
||||
<td><b>0.5806</b></td>
|
||||
<td><b>0.9013</b></td>
|
||||
<td><b>0.8679</b></td>
|
||||
<td><b>0.7472</b></td>
|
||||
<td><b>0.6039</b></td>
|
||||
<td><b>0.7372</b></td>
|
||||
<td><b>0.5946</b></td>
|
||||
<td><b>0.8384</b></td>
|
||||
<td><b>0.7435</b></td>
|
||||
<td><b>0.9314</b></td>
|
||||
<td><b>0.6397</b></td>
|
||||
<td><b>0.8401</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td>0.3626</td>
|
||||
<td>0.2661</td>
|
||||
<td>0.8486</td>
|
||||
<td>0.6677</td>
|
||||
<td>0.4097</td>
|
||||
<td>0.3080</td>
|
||||
<td>0.4623</td>
|
||||
<td>0.5028</td>
|
||||
<td>0.8362</td>
|
||||
<td>0.2694</td>
|
||||
<td>0.5455</td>
|
||||
<td>0.5892</td>
|
||||
<td>0.5735</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><b>0.4166</b></td>
|
||||
<td><b>0.4944</b></td>
|
||||
<td><b>0.8605</b></td>
|
||||
<td><b>0.8753</b></td>
|
||||
<td><b>0.7199</b></td>
|
||||
<td><b>0.5786</b></td>
|
||||
<td><b>0.7577</b></td>
|
||||
<td><b>0.5570</b></td>
|
||||
<td>0.7703</td>
|
||||
<td><b>0.7248</b></td>
|
||||
<td><b>0.8089</b></td>
|
||||
<td>0.5398</td>
|
||||
<td><b>0.8015</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td>0.2980</td>
|
||||
<td>0.2550</td>
|
||||
<td>0.8398</td>
|
||||
<td>0.6598</td>
|
||||
<td>0.3218</td>
|
||||
<td>0.2593</td>
|
||||
<td>0.4724</td>
|
||||
<td>0.4599</td>
|
||||
<td><b>0.8106</b></td>
|
||||
<td>0.2593</td>
|
||||
<td>0.5924</td>
|
||||
<td><b>0.5555</b></td>
|
||||
<td>0.5301</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
A single model can cover multiple languages and text types, with recognition accuracy significantly ahead of previous generation products and mainstream open-source solutions.
|
||||
|
||||
# PP-OCRv5 Demo Examples
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/algorithm_ppocrv5_demo1.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/doc_images/PP-OCRv5/algorithm_ppocrv5_demo.pdf">More Demos</a>
|
||||
|
||||
## Reference Data for Inference Performance
|
||||
|
||||
Test Environment:
|
||||
|
||||
- NVIDIA Tesla V100
|
||||
- Intel Xeon Gold 6271C
|
||||
- PaddlePaddle 3.0.0
|
||||
|
||||
Tested on 200 images (including both general and document images). During testing, images are read from disk, so the image reading time and other associated overhead are also included in the total time consumption. If the images are preloaded into memory, the average time per image can be further reduced by approximately 25 ms.
|
||||
|
||||
Unless otherwise specified:
|
||||
|
||||
- PP-OCRv4_mobile_det and PP-OCRv4_mobile_rec models are used.
|
||||
- Document orientation classification, image correction, and text line orientation classification are not used.
|
||||
- `text_det_limit_type` is set to `"min"` and `text_det_limit_side_len` to `732`.
|
||||
|
||||
### 1. Comparison of Inference Performance Between PP-OCRv5 and PP-OCRv4
|
||||
|
||||
| Config | Description |
|
||||
| --------------- | ------------------------------------------------------------ |
|
||||
| v5_mobile | Uses PP-OCRv5_mobile_det and PP-OCRv5_mobile_rec models. |
|
||||
| v4_mobile | Uses PP-OCRv4_mobile_det and PP-OCRv4_mobile_rec models. |
|
||||
| v5_server | Uses PP-OCRv5_server_det and PP-OCRv5_server_rec models. |
|
||||
| v4_server | Uses PP-OCRv4_server_det and PP-OCRv4_server_rec models. |
|
||||
|
||||
**GPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) | Avg. GPU Utilization (%) | Peak VRAM Usage (MB) | Avg. VRAM Usage (MB) |
|
||||
| ------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- | ------------------------ | -------------------- | -------------------- |
|
||||
| v5_mobile | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| v4_mobile | 0.27 | 2246 | 111.20 | 1392.22 | 1318.76 | 28.90 | 1304.00 | 1166.46 |
|
||||
| v5_server | 0.70 | 929 | 105.31 | 1634.85 | 1428.55 | 36.21 | 5402.00 | 4685.13 |
|
||||
| v4_server | 0.44 | 1418 | 106.96 | 1455.34 | 1346.95 | 58.82 | 6760.00 | 5817.46 |
|
||||
|
||||
**GPU, with high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) | Avg. GPU Utilization (%) | Peak VRAM Usage (MB) | Avg. VRAM Usage (MB) |
|
||||
| ------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- | ------------------------ | -------------------- | -------------------- |
|
||||
| v5_mobile | 0.50 | 1301 | 106.50 | 1338.12 | 1155.86 | 11.97 | 4112.00 | 3536.36 |
|
||||
| v4_mobile | 0.21 | 2887 | 114.09 | 1113.27 | 1054.46 | 15.22 | 2072.00 | 1840.59 |
|
||||
| v5_server | 0.60 | 1084 | 105.73 | 1980.73 | 1776.20 | 22.10 | 12150.00 | 11849.40 |
|
||||
| v4_server | 0.36 | 1687 | 104.15 | 1186.42 | 1065.67 | 38.12 | 13058.00 | 12679.00 |
|
||||
|
||||
**CPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) |
|
||||
| ------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- |
|
||||
| v5_mobile | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| v4_mobile | 1.09 | 556 | 813.16 | 11996.30 | 6834.25 |
|
||||
| v5_server | 3.79 | 172 | 799.24 | 50216.00 | 27902.40 |
|
||||
| v4_server | 4.22 | 148 | 803.74 | 51428.70 | 28593.60 |
|
||||
|
||||
**CPU, with high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) |
|
||||
| ------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- |
|
||||
| v5_mobile | 1.14 | 571 | 339.68 | 3245.17 | 2560.55 |
|
||||
| v4_mobile | 0.68 | 892 | 443.00 | 3057.38 | 2329.44 |
|
||||
| v5_server | 3.56 | 183 | 797.03 | 45664.70 | 26905.90 |
|
||||
| v4_server | 4.22 | 148 | 803.74 | 51428.70 | 28593.60 |
|
||||
|
||||
> Note: PP-OCRv5 uses a larger dictionary in the recognition model, which increases inference time and causes slower performance compared to PP-OCRv4.
|
||||
|
||||
### 2. Impact of Auxiliary Features on PP-OCRv5 Inference Performance
|
||||
|
||||
| Config | Description |
|
||||
| --------------- | --------------------------------------------------------------------------------------------------------- |
|
||||
| base | No document orientation classification, no image correction, no text line orientation classification. |
|
||||
| with_textline | Includes text line orientation classification only. |
|
||||
| with_all | Includes document orientation classification, image correction, and text line orientation classification. |
|
||||
|
||||
**GPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) | Avg. GPU Utilization (%) | Peak VRAM Usage (MB) | Avg. VRAM Usage (MB) |
|
||||
| -------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- | ------------------------ | -------------------- | -------------------- |
|
||||
| base | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| with_textline | 0.59 | 1104 | 105.58 | 1765.64 | 1478.53 | 19.48 | 4350.00 | 3267.77 |
|
||||
| with_all | 1.02 | 600 | 104.92 | 1924.23 | 1628.50 | 10.96 | 2632.00 | 2217.01 |
|
||||
|
||||
**CPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) |
|
||||
| -------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- |
|
||||
| base | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| with_textline | 1.50 | 434 | 799.47 | 12007.20 | 6882.22 |
|
||||
| with_all | 1.93 | 316 | 646.49 | 11759.60 | 6940.54 |
|
||||
|
||||
> Note: Auxiliary features such as image unwarping can impact inference accuracy. More features do not necessarily yield better results and may increase resource usage.
|
||||
|
||||
### 3. Impact of Input Scaling Strategy in Text Detection Module on PP-OCRv5 Inference Performance
|
||||
|
||||
| Config | Description |
|
||||
| ----------------- | -------------------------------------------------------------------------------------- |
|
||||
| mobile_min_1280 | Uses `min` limit type and `text_det_limit_side_len=1280` with PP-OCRv5_mobile models. |
|
||||
| mobile_min_736 | Same as default, `min`, `side_len=736`. |
|
||||
| mobile_max_960 | Uses `max` limit type and `side_len=960`. |
|
||||
| mobile_max_640 | Uses `max` limit type and `side_len=640`. |
|
||||
| server_min_1280 | Uses `min`, `side_len=1280` with PP-OCRv5_server models. |
|
||||
| server_min_736 | Same as default, `min`, `side_len=736`. |
|
||||
| server_max_960 | Uses `max`, `side_len=960`. |
|
||||
| server_max_640 | Uses `max`, `side_len=640`. |
|
||||
|
||||
**GPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) | Avg. GPU Utilization (%) | Peak VRAM Usage (MB) | Avg. VRAM Usage (MB) |
|
||||
| ----------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- | ------------------------ | -------------------- | -------------------- |
|
||||
| mobile_min_1280 | 0.61 | 1071 | 109.12 | 1663.71 | 1439.72 | 19.27 | 4202.00 | 3550.32 |
|
||||
| mobile_min_736 | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| mobile_max_960 | 0.48 | 1313 | 103.49 | 1587.25 | 1395.48 | 19.37 | 2642.00 | 2319.03 |
|
||||
| mobile_max_640 | 0.42 | 1436 | 103.07 | 1651.14 | 1422.62 | 18.95 | 2530.00 | 2149.11 |
|
||||
| server_min_1280 | 0.82 | 795 | 107.17 | 1678.16 | 1428.94 | 40.43 | 10368.00 | 8320.43 |
|
||||
| server_min_736 | 0.70 | 929 | 105.31 | 1634.85 | 1428.55 | 36.21 | 5402.00 | 4685.13 |
|
||||
| server_max_960 | 0.59 | 1073 | 103.03 | 1590.19 | 1383.62 | 33.42 | 2928.00 | 2079.47 |
|
||||
| server_max_640 | 0.54 | 1099 | 102.63 | 1602.09 | 1416.49 | 30.77 | 3152.00 | 2737.81 |
|
||||
|
||||
**CPU, without high-performance inference:**
|
||||
|
||||
| Configuration | Avg. Time per Image (s) | Avg. Characters Predicted per Second | Avg. CPU Utilization (%) | Peak RAM Usage (MB) | Avg. RAM Usage (MB) |
|
||||
| ----------------- | ----------------------- | ------------------------------------ | ------------------------ | ------------------- | ------------------- |
|
||||
| mobile_min_1280 | 1.64 | 398 | 799.45 | 12344.10 | 7100.60 |
|
||||
| mobile_min_736 | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| mobile_max_960 | 1.21 | 521 | 800.13 | 11099.10 | 6369.49 |
|
||||
| mobile_max_640 | 1.01 | 597 | 802.52 | 9585.48 | 5573.52 |
|
||||
| server_min_1280 | 4.48 | 145 | 800.49 | 50683.10 | 28273.30 |
|
||||
| server_min_736 | 3.79 | 172 | 799.24 | 50216.00 | 27902.40 |
|
||||
| server_max_960 | 2.67 | 237 | 797.63 | 49362.50 | 26075.60 |
|
||||
| server_max_640 | 2.36 | 251 | 795.18 | 45656.10 | 24900.80 |
|
||||
|
||||
# Deployment and Secondary Development
|
||||
* **Multiple System Support**: Compatible with mainstream operating systems including Windows, Linux, and Mac.
|
||||
* **Multiple Hardware Support**: Besides NVIDIA GPUs, it also supports inference and deployment on Intel CPU, Kunlun chips, Ascend, and other new hardware.
|
||||
* **High-Performance Inference Plugin**: Recommended to combine with high-performance inference plugins to further improve inference speed. See [High-Performance Inference Guide](../../deployment/high_performance_inference.md) for details.
|
||||
* **Service Deployment**: Supports highly stable service deployment solutions. See [Service Deployment Guide](../../deployment/serving.md) for details.
|
||||
* **Secondary Development Capability**: Supports custom dataset training, dictionary extension, and model fine-tuning. Example: To add Korean recognition, you can extend the dictionary and fine-tune the model, seamlessly integrating into existing pipelines. See [Text Detection Module Usage Tutorial](../../module_usage/text_detection.en.md) and [Text Recognition Module Usage Tutorial](../../module_usage/text_recognition.en.md) for details.
|
||||
340
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5.md
Normal file
340
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5.md
Normal file
@@ -0,0 +1,340 @@
|
||||
# 一、PP-OCRv5简介
|
||||
**PP-OCRv5** 是PP-OCR新一代文字识别解决方案，该方案聚焦于多场景、多文字类型的文字识别。在文字类型方面，PP-OCRv5支持简体中文、中文拼音、繁体中文、英文、日文5大主流文字类型，在场景方面，PP-OCRv5升级了中英复杂手写体、竖排文本、生僻字等多种挑战性场景的识别能力。在内部多场景复杂评估集上，PP-OCRv5较PP-OCRv4端到端提升13个百分点。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/algorithm_ppocrv5.png" width="600"/>
|
||||
</div>
|
||||
|
||||
|
||||
# 二、关键指标
|
||||
### 1. 文本检测指标
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>手写中文</th>
|
||||
<th>手写英文</th>
|
||||
<th>印刷中文</th>
|
||||
<th>印刷英文</th>
|
||||
<th>繁体中文</th>
|
||||
<th>古籍文本</th>
|
||||
<th>日文</th>
|
||||
<th>通用场景</th>
|
||||
<th>拼音</th>
|
||||
<th>旋转</th>
|
||||
<th>扭曲</th>
|
||||
<th>艺术字</th>
|
||||
<th>平均</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><b>PP-OCRv5_server_det</b></td>
|
||||
<td><b>0.803</b></td>
|
||||
<td><b>0.841</b></td>
|
||||
<td><b>0.945</b></td>
|
||||
<td><b>0.917</b></td>
|
||||
<td><b>0.815</b></td>
|
||||
<td><b>0.676</b></td>
|
||||
<td><b>0.772</b></td>
|
||||
<td><b>0.797</b></td>
|
||||
<td><b>0.671</b></td>
|
||||
<td><b>0.8</b></td>
|
||||
<td><b>0.876</b></td>
|
||||
<td><b>0.673</b></td>
|
||||
<td><b>0.827</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td>0.706</td>
|
||||
<td>0.249</td>
|
||||
<td>0.888</td>
|
||||
<td>0.690</td>
|
||||
<td>0.759</td>
|
||||
<td>0.473</td>
|
||||
<td>0.685</td>
|
||||
<td>0.715</td>
|
||||
<td>0.542</td>
|
||||
<td>0.366</td>
|
||||
<td>0.775</td>
|
||||
<td>0.583</td>
|
||||
<td>0.662</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><b>PP-OCRv5_mobile_det</b></td>
|
||||
<td><b>0.744</b></td>
|
||||
<td><b>0.777</b></td>
|
||||
<td><b>0.905</b></td>
|
||||
<td><b>0.910</b></td>
|
||||
<td><b>0.823</b></td>
|
||||
<td><b>0.581</b></td>
|
||||
<td><b>0.727</b></td>
|
||||
<td><b>0.721</b></td>
|
||||
<td><b>0.575</b></td>
|
||||
<td><b>0.647</b></td>
|
||||
<td><b>0.827</b></td>
|
||||
<td>0.525</td>
|
||||
<td><b>0.770</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td>0.583</td>
|
||||
<td>0.369</td>
|
||||
<td>0.872</td>
|
||||
<td>0.773</td>
|
||||
<td>0.663</td>
|
||||
<td>0.231</td>
|
||||
<td>0.634</td>
|
||||
<td>0.710</td>
|
||||
<td>0.430</td>
|
||||
<td>0.299</td>
|
||||
<td>0.715</td>
|
||||
<td><b>0.549</b></td>
|
||||
<td>0.624</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
对比PP-OCRv4，PP-OCRv5在所有检测场景下均有明显提升，尤其在手写、古籍、日文检测能力上表现更优。
|
||||
|
||||
### 2. 文本识别指标
|
||||
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/ocrv5_rec_acc.png" width="600"/>
|
||||
</div>
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>评估集类别</th>
|
||||
<th>手写中文</th>
|
||||
<th>手写英文</th>
|
||||
<th>印刷中文</th>
|
||||
<th>印刷英文</th>
|
||||
<th>繁体中文</th>
|
||||
<th>古籍文本</th>
|
||||
<th>日文</th>
|
||||
<th>易混淆字符</th>
|
||||
<th>通用场景</th>
|
||||
<th>拼音</th>
|
||||
<th>竖直文本</th>
|
||||
<th>艺术字</th>
|
||||
<th>加权平均</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><b>0.5807</b></td>
|
||||
<td><b>0.5806</b></td>
|
||||
<td><b>0.9013</b></td>
|
||||
<td><b>0.8679</b></td>
|
||||
<td><b>0.7472</b></td>
|
||||
<td><b>0.6039</b></td>
|
||||
<td><b>0.7372</b></td>
|
||||
<td><b>0.5946</b></td>
|
||||
<td><b>0.8384</b></td>
|
||||
<td><b>0.7435</b></td>
|
||||
<td><b>0.9314</b></td>
|
||||
<td><b>0.6397</b></td>
|
||||
<td><b>0.8401</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td>0.3626</td>
|
||||
<td>0.2661</td>
|
||||
<td>0.8486</td>
|
||||
<td>0.6677</td>
|
||||
<td>0.4097</td>
|
||||
<td>0.3080</td>
|
||||
<td>0.4623</td>
|
||||
<td>0.5028</td>
|
||||
<td>0.8362</td>
|
||||
<td>0.2694</td>
|
||||
<td>0.5455</td>
|
||||
<td>0.5892</td>
|
||||
<td>0.5735</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><b>0.4166</b></td>
|
||||
<td><b>0.4944</b></td>
|
||||
<td><b>0.8605</b></td>
|
||||
<td><b>0.8753</b></td>
|
||||
<td><b>0.7199</b></td>
|
||||
<td><b>0.5786</b></td>
|
||||
<td><b>0.7577</b></td>
|
||||
<td><b>0.5570</b></td>
|
||||
<td>0.7703</td>
|
||||
<td><b>0.7248</b></td>
|
||||
<td><b>0.8089</b></td>
|
||||
<td>0.5398</td>
|
||||
<td><b>0.8015</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td>0.2980</td>
|
||||
<td>0.2550</td>
|
||||
<td>0.8398</td>
|
||||
<td>0.6598</td>
|
||||
<td>0.3218</td>
|
||||
<td>0.2593</td>
|
||||
<td>0.4724</td>
|
||||
<td>0.4599</td>
|
||||
<td><b>0.8106</b></td>
|
||||
<td>0.2593</td>
|
||||
<td>0.5924</td>
|
||||
<td><b>0.5555</b></td>
|
||||
<td>0.5301</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
单模型即可覆盖多语言和多类型文本，识别精度大幅领先前代产品和主流开源方案。
|
||||
|
||||
|
||||
# 三、PP-OCRv5 Demo示例
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-OCRv5/algorithm_ppocrv5_demo1.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/PaddleX3.0/doc_images/PP-OCRv5/algorithm_ppocrv5_demo.pdf">更多示例</a>
|
||||
|
||||
## 四、推理性能参考数据
|
||||
|
||||
测试环境：
|
||||
|
||||
- NVIDIA Tesla V100
|
||||
- Intel Xeon Gold 6271C
|
||||
- PaddlePaddle 3.0.0
|
||||
|
||||
在 200 张图像（包括通用图像与文档图像）上测试。测试时从磁盘读取图像，因此读图时间及其他额外开销也被包含在总耗时内。如果将图像提前载入到内存，可进一步减少平均每图约 25 ms 的时间开销。
|
||||
|
||||
如果不特别说明，则：
|
||||
|
||||
- 使用 PP-OCRv4_mobile_det 和 PP-OCRv4_mobile_rec 模型。
|
||||
- 不使用文档图像方向分类、文本图像矫正、文本行方向分类。
|
||||
- 将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `732`。
|
||||
|
||||
### 1. PP-OCRv5 与 PP-OCRv4 推理性能对比
|
||||
|
||||
| 配置 | 说明 |
|
||||
| --- | --- |
|
||||
| v5_mobile | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型。 |
|
||||
| v4_mobile | 使用 PP-OCRv4_mobile_det 和 PP-OCRv4_mobile_rec 模型。 |
|
||||
| v5_server | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型。 |
|
||||
| v4_server | 使用 PP-OCRv4_server_det 和 PP-OCRv4_server_rec 模型。 |
|
||||
|
||||
**GPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） | 平均GPU利用率（%） | 峰值 VRAM 用量（MB） | 平均 VRAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| v5_mobile | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| v4_mobile | 0.27 | 2246 | 111.20 | 1392.22 | 1318.76 | 28.90 | 1304.00 | 1166.46 |
|
||||
| v5_server | 0.70 | 929 | 105.31 | 1634.85 | 1428.55 | 36.21 | 5402.00 | 4685.13 |
|
||||
| v4_server | 0.44 | 1418 | 106.96 | 1455.34 | 1346.95 | 58.82 | 6760.00 | 5817.46 |
|
||||
|
||||
**GPU，使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） | 平均GPU利用率（%） | 峰值 VRAM 用量（MB） | 平均 VRAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| v5_mobile | 0.50 | 1301 | 106.50 | 1338.12 | 1155.86 | 11.97 | 4112.00 | 3536.36 |
|
||||
| v4_mobile | 0.21 | 2887 | 114.09 | 1113.27 | 1054.46 | 15.22 | 2072.00 | 1840.59 |
|
||||
| v5_server | 0.60 | 1084 | 105.73 | 1980.73 | 1776.20 | 22.10 | 12150.00 | 11849.40 |
|
||||
| v4_server | 0.36 | 1687 | 104.15 | 1186.42 | 1065.67 | 38.12 | 13058.00 | 12679.00 |
|
||||
|
||||
**CPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| v5_mobile | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| v4_mobile | 1.09 | 556 | 813.16 | 11996.30 | 6834.25 |
|
||||
| v5_server | 3.79 | 172 | 799.24 | 50216.00 | 27902.40 |
|
||||
| v4_server | 4.22 | 148 | 803.74 | 51428.70 | 28593.60 |
|
||||
|
||||
**CPU，使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| v5_mobile | 1.14 | 571 | 339.68 | 3245.17 | 2560.55 |
|
||||
| v4_mobile | 0.68 | 892 | 443.00 | 3057.38 | 2329.44 |
|
||||
| v5_server | 3.56 | 183 | 797.03 | 45664.70 | 26905.90 |
|
||||
| v4_server | 4.22 | 148 | 803.74 | 51428.70 | 28593.60 |
|
||||
|
||||
> 说明：PP-OCRv5 的识别模型使用了更大的字典，需要更长的推理时间，导致 PP-OCRv5 的推理速度慢于 PP-OCRv4。
|
||||
|
||||
### 2. 使用辅助功能对 PP-OCRv5 推理性能的影响
|
||||
|
||||
| 配置 | 说明 |
|
||||
| --- | --- |
|
||||
| base | 不使用文档图像方向分类、文本图像矫正、文本行方向分类。 |
|
||||
| with_textline | 使用文本行方向分类，不使用文档图像方向分类、文本图像矫正。 |
|
||||
| with_all | 使用文档图像方向分类、文本图像矫正、文本行方向分类。 |
|
||||
|
||||
**GPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） | 平均GPU利用率（%） | 峰值 VRAM 用量（MB） | 平均 VRAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| base | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| with_textline | 0.59 | 1104 | 105.58 | 1765.64 | 1478.53 | 19.48 | 4350.00 | 3267.77 |
|
||||
| with_all | 1.02 | 600 | 104.92 | 1924.23 | 1628.50 | 10.96 | 2632.00 | 2217.01 |
|
||||
|
||||
**CPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| base | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| with_textline | 1.50 | 434 | 799.47 | 12007.20 | 6882.22 |
|
||||
| with_all | 1.93 | 316 | 646.49 | 11759.60 | 6940.54 |
|
||||
|
||||
> 说明：文本图像矫正等辅助功能会对端到端推理精度造成影响，因此并不一定使用的辅助功能越多、资源用量越大。
|
||||
|
||||
### 3. 文本检测模块输入缩放尺寸策略对 PP-OCRv5 推理性能的影响
|
||||
|
||||
| 配置 | 说明 |
|
||||
| --- | --- |
|
||||
| mobile_min_1280 | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型，将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `1280`。 |
|
||||
| mobile_min_736 | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型，将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `1280`。 |
|
||||
| mobile_max_960 | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型，将 `text_det_limit_type` 设置为 `"max"`、`text_det_limit_side_len` 设置为 `960`。 |
|
||||
| mobile_max_640 | 使用 PP-OCRv5_mobile_det 和 PP-OCRv5_mobile_rec 模型，将 `text_det_limit_type` 设置为 `"max"`、`text_det_limit_side_len` 设置为 `640`。 |
|
||||
| server_min_1280 | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型，将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `1280`。 |
|
||||
| server_min_736 | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型，将 `text_det_limit_type` 设置为 `"min"`、`text_det_limit_side_len` 设置为 `1280`。 |
|
||||
| server_max_960 | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型，将 `text_det_limit_type` 设置为 `"max"`、`text_det_limit_side_len` 设置为 `960`。 |
|
||||
| server_max_640 | 使用 PP-OCRv5_server_det 和 PP-OCRv5_server_rec 模型，将 `text_det_limit_type` 设置为 `"max"`、`text_det_limit_side_len` 设置为 `640`。 |
|
||||
|
||||
**GPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） | 平均GPU利用率（%） | 峰值 VRAM 用量（MB） | 平均 VRAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|
||||
| mobile_min_1280 | 0.61 | 1071 | 109.12 | 1663.71 | 1439.72 | 19.27 | 4202.00 | 3550.32 |
|
||||
| mobile_min_736 | 0.56 | 1162 | 106.02 | 1576.43 | 1420.83 | 18.95 | 4342.00 | 3258.95 |
|
||||
| mobile_max_960 | 0.48 | 1313 | 103.49 | 1587.25 | 1395.48 | 19.37 | 2642.00 | 2319.03 |
|
||||
| mobile_max_640 | 0.42 | 1436 | 103.07 | 1651.14 | 1422.62 | 18.95 | 2530.00 | 2149.11 |
|
||||
| server_min_1280 | 0.82 | 795 | 107.17 | 1678.16 | 1428.94 | 40.43 | 10368.00 | 8320.43 |
|
||||
| server_min_736 | 0.70 | 929 | 105.31 | 1634.85 | 1428.55 | 36.21 | 5402.00 | 4685.13 |
|
||||
| server_max_960 | 0.59 | 1073 | 103.03 | 1590.19 | 1383.62 | 33.42 | 2928.00 | 2079.47 |
|
||||
| server_max_640 | 0.54 | 1099 | 102.63 | 1602.09 | 1416.49 | 30.77 | 3152.00 | 2737.81 |
|
||||
|
||||
**CPU，不使用高性能推理：**
|
||||
|
||||
| 配置 | 平均每图耗时（s） | 平均每秒预测字符数量 | 平均 CPU 利用率（%） | 峰值 RAM 用量（MB） | 平均 RAM 用量（MB） |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| mobile_min_1280 | 1.64 | 398 | 799.45 | 12344.10 | 7100.60 |
|
||||
| mobile_min_736 | 1.43 | 455 | 798.93 | 11695.40 | 6829.09 |
|
||||
| mobile_max_960 | 1.21 | 521 | 800.13 | 11099.10 | 6369.49 |
|
||||
| mobile_max_640 | 1.01 | 597 | 802.52 | 9585.48 | 5573.52 |
|
||||
| server_min_1280 | 4.48 | 145 | 800.49 | 50683.10 | 28273.30 |
|
||||
| server_min_736 | 3.79 | 172 | 799.24 | 50216.00 | 27902.40 |
|
||||
| server_max_960 | 2.67 | 237 | 797.63 | 49362.50 | 26075.60 |
|
||||
| server_max_640 | 2.36 | 251 | 795.18 | 45656.10 | 24900.80 |
|
||||
|
||||
|
||||
# 五、部署与二次开发
|
||||
* **多系统支持**：兼容Windows、Linux、Mac等主流操作系统。
|
||||
* **多硬件支持**：除了英伟达GPU外，还支持Intel CPU、昆仑芯、昇腾等新硬件推理和部署。
|
||||
* **高性能推理插件**：推荐结合高性能推理插件进一步提升推理速度，详见[高性能推理指南](../../deployment/high_performance_inference.md)。
|
||||
* **服务化部署**：支持高稳定性服务化部署方案，详见[服务化部署指南](../../deployment/serving.md)。
|
||||
* **二次开发能力**：支持自定义数据集训练、字典扩展、模型微调。举例：如需增加韩文识别，可扩展字典并微调模型，无缝集成到现有产线，详见[文本检测模块使用教程](../../module_usage/text_detection.md)及[文本识别模块使用教程](../../module_usage/text_recognition.md)
|
||||
@@ -0,0 +1,116 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 1. Introduction to PP-OCRv5 Multilingual Text Recognition
|
||||
|
||||
PP-OCRv5 is the latest generation of the PP-OCR series text recognition solutions, focusing on text recognition tasks across multiple scenarios and languages. By default, the recognition model supports accurate recognition of five mainstream text types: Simplified Chinese, Chinese Pinyin, Traditional Chinese, English, and Japanese. In addition, PP-OCRv5 provides multilingual recognition capabilities covering 37 languages, including Korean, Spanish, French, Portuguese, German, Italian, Russian, and more (see [Section 4](#4-supported-languages-and-abbreviations) for the full list of supported languages and abbreviations). Compared to the previous PP-OCRv3 version, PP-OCRv5 achieves more than a 30% improvement in recognition accuracy for multilingual tasks.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## 2. Quick Start
|
||||
|
||||
You can use the `--lang` parameter in the command line to specify the text recognition model for your target language when running the general OCR pipeline:
|
||||
|
||||
```bash
|
||||
# Use the `--lang` parameter to specify the French recognition model
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png \
|
||||
--lang fr \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False \
|
||||
--save_path ./output \
|
||||
--device gpu:0
|
||||
```
|
||||
For explanations of other command line parameters, please refer to the [command line usage](../../pipeline_usage/OCR.en.md#21-command-line) of the general OCR pipeline. After execution, results will be printed to the terminal:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_ocr_french01.png', 'page_index': None, 'model_settings': {'use_doc_preprocessor': True, 'use_textline_orientation': False}, 'doc_preprocessor_res': {'input_path': None, 'page_index': None, 'model_settings': {'use_doc_orientation_classify': False, 'use_doc_unwarping': False}, 'angle': -1}, 'dt_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'text_det_params': {'limit_side_len': 64, 'limit_type': 'min', 'thresh': 0.3, 'max_side_limit': 4000, 'box_thresh': 0.6, 'unclip_ratio': 1.5}, 'text_type': 'general', 'textline_orientation_angles': array([-1, ..., -1]), 'text_rec_score_thresh': 0.0, 'rec_texts': ['mifere; la profpérité & les fuccès ac-', 'compagnent l’homme induftrieux.', 'Quel eft celui qui a acquis des ri-', 'cheffes, qui eft devenu puiffant, qui', 's’eft couvert de gloire, dont l’éloge', 'retentit par-tout, qui fiege au confeil', "du Roi? C'eft celui qui bannit la pa-", "reffe de fa maifon, & qui a dit à l'oifi-", 'veté : tu es mon ennemie.'], 'rec_scores': array([0.98409832, ..., 0.98091048]), 'rec_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'rec_boxes': array([[118, ..., 81],
|
||||
...,
|
||||
[108, ..., 562]], dtype=int16)}}
|
||||
```
|
||||
|
||||
If you specify `save_path`, the visualization results will be saved in the `save_path` directory. An example visualization is shown below:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/general_ocr_french01_res.png"/>
|
||||
|
||||
You can also use Python code to specify the recognition model for your target language using the `lang` parameter when initializing the general OCR pipeline:
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(
|
||||
lang="fr", # Specify the French recognition model via the lang parameter
|
||||
use_doc_orientation_classify=False, # Disable document orientation classification
|
||||
use_doc_unwarping=False, # Disable text image unwarping
|
||||
use_textline_orientation=False, # Disable textline orientation classification
|
||||
)
|
||||
result = ocr.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png")
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
```
|
||||
For more details on the `PaddleOCR` class parameters, refer to the [Python script integration](../../pipeline_usage/OCR.en.md#22-python-script-integration) of the general OCR pipeline.
|
||||
|
||||
## 3. Benchmark Comparison
|
||||
|
||||
| Model | Korean Dataset Accuracy (%) | | Model | Latin Script Languages Dataset Accuracy (%) | | Model | East Slavic Languages Dataset Accuracy (%) |
|
||||
|--|--|--|--|--|--|--|--|
|
||||
| korean_PP-OCRv5_mobile_rec | 88.0 | | latin_PP-OCRv5_mobile_rec | 84.7 | | eslav_PP-OCRv5_mobile_rec | 85.8 |
|
||||
| korean_PP-OCRv3_mobile_rec | 23.0 | | latin_PP-OCRv3_mobile_rec | 37.9 | | cyrillic_PP-OCRv3_mobile_rec| 50.2 |
|
||||
|
||||
**Notes:**
|
||||
- Korean Dataset: PP-OCRv5's latest dataset containing 5,007 Korean text images.
|
||||
- Latin Script Languages Dataset: The latest PP-OCRv5 recognition dataset, containing 3,111 text images in Latin script languages.
|
||||
- East Slavic Languages Dataset: PP-OCRv5's latest dataset containing a total of 7,031 Russian, Belarusian, and Ukrainian text images.
|
||||
|
||||
## 4. Supported Languages and Abbreviations
|
||||
|
||||
| Language | Description | Abbreviation | | Language | Description | Abbreviation |
|
||||
| --- | --- | --- | ---|--- | --- | --- |
|
||||
| Chinese | Chinese & English | ch | | Hungarian | Hungarian | hu |
|
||||
| English | English | en | | Serbian (Latin) | Serbian(latin) | rslatin |
|
||||
| French | French | fr | | Indonesian | Indonesian | id |
|
||||
| German | German | de | | Occitan | Occitan | oc |
|
||||
| Japanese | Japanese | japan | | Icelandic | Icelandic | is |
|
||||
| Korean | Korean | korean | | Lithuanian | Lithuanian | lt |
|
||||
| Chinese Traditional | Chinese Traditional | chinese_cht | | Maori | Maori | mi |
|
||||
| Afrikaans | Afrikaans | af | | Malay | Malay | ms |
|
||||
| Italian | Italian | it | | Dutch | Dutch | nl |
|
||||
| Spanish | Spanish | es | | Norwegian | Norwegian | no |
|
||||
| Bosnian | Bosnian | bs | | Polish | Polish | pl |
|
||||
| Portuguese | Portuguese | pt | | Slovak | Slovak | sk |
|
||||
| Czech | Czech | cs | | Slovenian | Slovenian | sl |
|
||||
| Welsh | Welsh | cy | | Albanian | Albanian | sq |
|
||||
| Danish | Danish | da | | Swedish | Swedish | sv |
|
||||
| Estonian | Estonian | et | | Swahili | Swahili | sw |
|
||||
| Irish | Irish | ga | | Tagalog | Tagalog | tl |
|
||||
| Croatian | Croatian | hr | | Turkish | Turkish | tr |
|
||||
| Uzbek | Uzbek | uz | | Latin | Latin | la |
|
||||
| Russian | Russian | ru | | Belarusian | Belarusian | be |
|
||||
| Ukrainian | Ukranian | uk | | | | |
|
||||
120
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5_multi_languages.md
Normal file
120
docs/version3.x/algorithm/PP-OCRv5/PP-OCRv5_multi_languages.md
Normal file
@@ -0,0 +1,120 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 一、PP-OCRv5多语种文本识别介绍
|
||||
|
||||
|
||||
PP-OCRv5 是 PP-OCR 系列的最新一代文字识别解决方案，专注于多场景、多语种的文字识别任务。在文字类型支持方面，默认配置的识别模型可准确识别简体中文、中文拼音、繁体中文、英文和日文这五大主流文字类型。同时，PP-OCRv5还提供了覆盖37种语言的多语种识别能力，包括韩文、西班牙文、法文、葡萄牙文、德文、意大利文、俄罗斯文等（具体支持语种及缩写详见[第四节](#四-支持语种及缩写)）。相较于前代 PP-OCRv3 版本，PP-OCRv5 在多语言识别准确率上实现了超过30%的提升。
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## 二、快速使用
|
||||
|
||||
您可以通过在命令行中使用 `--lang` 参数，来使用指定语种的文本识别模型进行通用 OCR 产线的推理：
|
||||
|
||||
```bash
|
||||
# 通过 `--lang` 参数指定使用法语的识别模型
|
||||
paddleocr ocr -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png \
|
||||
--lang fr \
|
||||
--use_doc_orientation_classify False \
|
||||
--use_doc_unwarping False \
|
||||
--use_textline_orientation False \
|
||||
--save_path ./output \
|
||||
--device gpu:0
|
||||
```
|
||||
上述命令行的其他参数说明请参考通用 OCR 产线的[命令行使用方式](../../pipeline_usage/OCR.md#21-命令行方式), 运行后结果会被打印到终端上：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_ocr_french01.png', 'page_index': None, 'model_settings': {'use_doc_preprocessor': True, 'use_textline_orientation': False}, 'doc_preprocessor_res': {'input_path': None, 'page_index': None, 'model_settings': {'use_doc_orientation_classify': False, 'use_doc_unwarping': False}, 'angle': -1}, 'dt_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'text_det_params': {'limit_side_len': 64, 'limit_type': 'min', 'thresh': 0.3, 'max_side_limit': 4000, 'box_thresh': 0.6, 'unclip_ratio': 1.5}, 'text_type': 'general', 'textline_orientation_angles': array([-1, ..., -1]), 'text_rec_score_thresh': 0.0, 'rec_texts': ['mifere; la profpérité & les fuccès ac-', 'compagnent l’homme induftrieux.', 'Quel eft celui qui a acquis des ri-', 'cheffes, qui eft devenu puiffant, qui', 's’eft couvert de gloire, dont l’éloge', 'retentit par-tout, qui fiege au confeil', "du Roi? C'eft celui qui bannit la pa-", "reffe de fa maifon, & qui a dit à l'oifi-", 'veté : tu es mon ennemie.'], 'rec_scores': array([0.98409832, ..., 0.98091048]), 'rec_polys': array([[[119, 23],
|
||||
...,
|
||||
[118, 75]],
|
||||
|
||||
...,
|
||||
|
||||
[[109, 506],
|
||||
...,
|
||||
[108, 556]]], dtype=int16), 'rec_boxes': array([[118, ..., 81],
|
||||
...,
|
||||
[108, ..., 562]], dtype=int16)}}
|
||||
```
|
||||
|
||||
若指定了`save_path`，则会保存可视化结果在`save_path`下。可视化结果如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/pipelines/ocr/general_ocr_french01_res.png"/>
|
||||
|
||||
|
||||
您也可以使用 Python 代码，在通用 OCR 产线初始化时，通过 `lang` 参数来使用指定语种的识别模型：
|
||||
|
||||
```python
|
||||
from paddleocr import PaddleOCR
|
||||
|
||||
ocr = PaddleOCR(
|
||||
lang="fr" # 通过 lang 参数指定使用法语的识别模型
|
||||
use_doc_orientation_classify=False, # 通过 use_doc_orientation_classify 参数指定不使用文档方向分类模型
|
||||
use_doc_unwarping=False, # 通过 use_doc_unwarping 参数指定不使用文本图像矫正模型
|
||||
use_textline_orientation=False, # 通过 use_textline_orientation 参数指定不使用文本行方向分类模型
|
||||
)
|
||||
result = ocr.predict("https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_french01.png")
|
||||
for res in result:
|
||||
res.print()
|
||||
res.save_to_img("output")
|
||||
res.save_to_json("output")
|
||||
```
|
||||
更过关于 `PaddleOCR` 类参数的说明参考通用 OCR 产线的[脚本方式集成](../../pipeline_usage/OCR.md#22-python脚本方式集成)。
|
||||
|
||||
|
||||
## 三、指标对比
|
||||
|
||||
| 模型 |韩语数据集 精度 (%)| | 模型 | 拉丁字母语言数据集 精度 (%)| | 模型| 东斯拉夫语言数据集 精度 (%) |
|
||||
|--|--|--|--|--|--|--|--|
|
||||
| korean_PP-OCRv5_mobile_rec | 88.0 | | latin_PP-OCRv5_mobile_rec | 84.7 | | eslav_PP-OCRv5_mobile_rec | 85.8 |
|
||||
| korean_PP-OCRv3_mobile_rec | 23.0 | | latin_PP-OCRv3_mobile_rec | 37.9 | | cyrillic_PP-OCRv3_mobile_rec| 50.2 |
|
||||
|
||||
**注：**
|
||||
- 韩语数据集：PP-OCRv5 最新构建的包含了 5007 张韩语文本图片的识别数据集。
|
||||
- 拉丁字母语言数据集：PP-OCRv5 最新构建的包含了 3111 张拉丁字母语言的文本图片识别数据集。
|
||||
- 东斯拉夫语言数据集：PP-OCRv5 最新构建的包含了俄语、 白俄罗斯语和乌克兰语共计 7031 张文本图片的识别数据集。
|
||||
|
||||
## 四、 支持语种及缩写
|
||||
|
||||
| 语种 | 描述 | 缩写 | | 语种 | 描述 | 缩写 |
|
||||
| --- | --- | --- | ---|--- | --- | --- |
|
||||
| 中文 | Chinese & English | ch | | 匈牙利文 | Hungarian | hu |
|
||||
| 英文 | English | en | | 塞尔维亚文（latin） | Serbian(latin) | rslatin |
|
||||
| 法文 | French | fr | | 印度尼西亚文 | Indonesian | id |
|
||||
| 德文 | German | de | | 欧西坦文 | Occitan | oc |
|
||||
| 日文 | Japanese | japan | | 冰岛文 | Icelandic | is |
|
||||
| 韩文 | Korean | korean | | 立陶宛文 | Lithuanian | lt |
|
||||
| 中文繁体 | Chinese Traditional | chinese_cht | | 毛利文 | Maori | mi |
|
||||
| 南非荷兰文 | Afrikaans | af | | 马来文 | Malay | ms |
|
||||
| 意大利文 | Italian | it | | 荷兰文 | Dutch | nl |
|
||||
| 西班牙文 | Spanish | es | | 挪威文 | Norwegian | no |
|
||||
| 波斯尼亚文 | Bosnian | bs | | 波兰文 | Polish | pl |
|
||||
| 葡萄牙文 | Portuguese | pt | | 斯洛伐克文 | Slovak | sk |
|
||||
| 捷克文 | Czech | cs | | 斯洛文尼亚文 | Slovenian | sl |
|
||||
| 威尔士文 | Welsh | cy | | 阿尔巴尼亚文 | Albanian | sq |
|
||||
| 丹麦文 | Danish | da | | 瑞典文 | Swedish | sv |
|
||||
| 爱沙尼亚文 | Estonian | et | | 西瓦希里文 | Swahili | sw |
|
||||
| 爱尔兰文 | Irish | ga | | 塔加洛文 | Tagalog | tl |
|
||||
| 克罗地亚文 | Croatian | hr | | 土耳其文 | Turkish | tr |
|
||||
| 乌兹别克文 | Uzbek | uz | | 拉丁文 | Latin | la |
|
||||
| 俄罗斯文 | Russian | ru | | 白俄罗斯文 | Belarusian | be |
|
||||
| 乌克兰文 | Ukranian | uk | | | | |
|
||||
981
docs/version3.x/algorithm/PP-StructureV3/PP-StructureV3.en.md
Normal file
981
docs/version3.x/algorithm/PP-StructureV3/PP-StructureV3.en.md
Normal file
@@ -0,0 +1,981 @@
|
||||
# Introduction to PP-StructureV3
|
||||
|
||||
**PP-StructureV3** pipeline, based on the Layout Parsing v1 pipeline, has strengthened the ability of layout detection, table recognition, and formula recognition. It has also added the ability to understand charts and restore reading order, as well as the ability to convert results into Markdown files. In various document data, it performs excellently and can handle more complex document data. This pipeline also provides flexible service-oriented deployment methods, supporting the use of multiple programming languages on various hardware. Moreover, it also provides the ability for secondary development. You can train and optimize on your own dataset based on this pipeline, and the trained model can be seamlessly integrated.
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-StructureV3/algorithm_ppstructurev3.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# Key Metrics
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th rowspan="2">Method Type</th>
|
||||
<th rowspan="2">Methods</th>
|
||||
<th colspan="2">Overall<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Text<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Formula<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Table<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Read Order<sup>Edit</sup>↓</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="9">Pipeline Tools</td>
|
||||
<td><b>PP-structureV3</b></td>
|
||||
<td><b>0.147</b></td>
|
||||
<td><b>0.212</b></td>
|
||||
<td>0.059</td>
|
||||
<td><b>0.09</b></td>
|
||||
<td>0.295</td>
|
||||
<td>0.535</td>
|
||||
<td>0.159</td>
|
||||
<td><b>0.109</b></td>
|
||||
<td>0.075</td>
|
||||
<td><b>0.114</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MinerU-0.9.3</td>
|
||||
<td>0.15</td>
|
||||
<td>0.357</td>
|
||||
<td>0.061</td>
|
||||
<td>0.215</td>
|
||||
<td>0.278</td>
|
||||
<td>0.577</td>
|
||||
<td>0.18</td>
|
||||
<td>0.344</td>
|
||||
<td>0.079</td>
|
||||
<td>0.292</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MinerU-1.3.11</td>
|
||||
<td>0.166</td>
|
||||
<td>0.310</td>
|
||||
<td>0.0826</td>
|
||||
<td>0.2000</td>
|
||||
<td>0.3368</td>
|
||||
<td>0.6236</td>
|
||||
<td>0.1613</td>
|
||||
<td>0.1833</td>
|
||||
<td>0.0834</td>
|
||||
<td>0.2316</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Marker-1.2.3</td>
|
||||
<td>0.336</td>
|
||||
<td>0.556</td>
|
||||
<td>0.08</td>
|
||||
<td>0.315</td>
|
||||
<td>0.53</td>
|
||||
<td>0.883</td>
|
||||
<td>0.619</td>
|
||||
<td>0.685</td>
|
||||
<td>0.114</td>
|
||||
<td>0.34</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mathpix</td>
|
||||
<td>0.191</td>
|
||||
<td>0.365</td>
|
||||
<td>0.105</td>
|
||||
<td>0.384</td>
|
||||
<td>0.306</td>
|
||||
<td>0.454</td>
|
||||
<td>0.243</td>
|
||||
<td>0.32</td>
|
||||
<td>0.108</td>
|
||||
<td>0.304</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Docling-2.14.0</td>
|
||||
<td>0.589</td>
|
||||
<td>0.909</td>
|
||||
<td>0.416</td>
|
||||
<td>0.987</td>
|
||||
<td>0.999</td>
|
||||
<td>1</td>
|
||||
<td>0.627</td>
|
||||
<td>0.81</td>
|
||||
<td>0.313</td>
|
||||
<td>0.837</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Pix2Text-1.1.2.3</td>
|
||||
<td>0.32</td>
|
||||
<td>0.528</td>
|
||||
<td>0.138</td>
|
||||
<td>0.356</td>
|
||||
<td><b>0.276</b></td>
|
||||
<td>0.611</td>
|
||||
<td>0.584</td>
|
||||
<td>0.645</td>
|
||||
<td>0.281</td>
|
||||
<td>0.499</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Unstructured-0.17.2</td>
|
||||
<td>0.586</td>
|
||||
<td>0.716</td>
|
||||
<td>0.198</td>
|
||||
<td>0.481</td>
|
||||
<td>0.999</td>
|
||||
<td>1</td>
|
||||
<td>1</td>
|
||||
<td>0.998</td>
|
||||
<td>0.145</td>
|
||||
<td>0.387</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenParse-0.7.0</td>
|
||||
<td>0.646</td>
|
||||
<td>0.814</td>
|
||||
<td>0.681</td>
|
||||
<td>0.974</td>
|
||||
<td>0.996</td>
|
||||
<td>1</td>
|
||||
<td>0.284</td>
|
||||
<td>0.639</td>
|
||||
<td>0.595</td>
|
||||
<td>0.641</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="5">Expert VLMs</td>
|
||||
<td>GOT-OCR</td>
|
||||
<td>0.287</td>
|
||||
<td>0.411</td>
|
||||
<td>0.189</td>
|
||||
<td>0.315</td>
|
||||
<td>0.36</td>
|
||||
<td>0.528</td>
|
||||
<td>0.459</td>
|
||||
<td>0.52</td>
|
||||
<td>0.141</td>
|
||||
<td>0.28</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Nougat</td>
|
||||
<td>0.452</td>
|
||||
<td>0.973</td>
|
||||
<td>0.365</td>
|
||||
<td>0.998</td>
|
||||
<td>0.488</td>
|
||||
<td>0.941</td>
|
||||
<td>0.572</td>
|
||||
<td>1</td>
|
||||
<td>0.382</td>
|
||||
<td>0.954</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mistral OCR</td>
|
||||
<td>0.268</td>
|
||||
<td>0.439</td>
|
||||
<td>0.072</td>
|
||||
<td>0.325</td>
|
||||
<td>0.318</td>
|
||||
<td>0.495</td>
|
||||
<td>0.6</td>
|
||||
<td>0.65</td>
|
||||
<td>0.083</td>
|
||||
<td>0.284</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OLMOCR-sglang</td>
|
||||
<td>0.326</td>
|
||||
<td>0.469</td>
|
||||
<td>0.097</td>
|
||||
<td>0.293</td>
|
||||
<td>0.455</td>
|
||||
<td>0.655</td>
|
||||
<td>0.608</td>
|
||||
<td>0.652</td>
|
||||
<td>0.145</td>
|
||||
<td>0.277</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SmolDocling-256M_transformer</td>
|
||||
<td>0.493</td>
|
||||
<td>0.816</td>
|
||||
<td>0.262</td>
|
||||
<td>0.838</td>
|
||||
<td>0.753</td>
|
||||
<td>0.997</td>
|
||||
<td>0.729</td>
|
||||
<td>0.907</td>
|
||||
<td>0.227</td>
|
||||
<td>0.522</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="6">General VLMs</td>
|
||||
<td>Gemini2.0-flash</td>
|
||||
<td>0.191</td>
|
||||
<td>0.264</td>
|
||||
<td>0.091</td>
|
||||
<td>0.139</td>
|
||||
<td>0.389</td>
|
||||
<td>0.584</td>
|
||||
<td>0.193</td>
|
||||
<td>0.206</td>
|
||||
<td>0.092</td>
|
||||
<td>0.128</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini2.5-Pro</td>
|
||||
<td>0.148</td>
|
||||
<td><b>0.212</b></td>
|
||||
<td><b>0.055</b></td>
|
||||
<td>0.168</td>
|
||||
<td>0.356</td>
|
||||
<td>0.439</td>
|
||||
<td><b>0.13</b></td>
|
||||
<td>0.119</td>
|
||||
<td><b>0.049</b></td>
|
||||
<td>0.121</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GPT4o</td>
|
||||
<td>0.233</td>
|
||||
<td>0.399</td>
|
||||
<td>0.144</td>
|
||||
<td>0.409</td>
|
||||
<td>0.425</td>
|
||||
<td>0.606</td>
|
||||
<td>0.234</td>
|
||||
<td>0.329</td>
|
||||
<td>0.128</td>
|
||||
<td>0.251</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2-VL-72B</td>
|
||||
<td>0.252</td>
|
||||
<td>0.327</td>
|
||||
<td>0.096</td>
|
||||
<td>0.218</td>
|
||||
<td>0.404</td>
|
||||
<td>0.487</td>
|
||||
<td>0.387</td>
|
||||
<td>0.408</td>
|
||||
<td>0.119</td>
|
||||
<td>0.193</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>0.214</td>
|
||||
<td>0.261</td>
|
||||
<td>0.092</td>
|
||||
<td>0.18</td>
|
||||
<td>0.315</td>
|
||||
<td><b>0.434</b></td>
|
||||
<td>0.341</td>
|
||||
<td>0.262</td>
|
||||
<td>0.106</td>
|
||||
<td>0.168</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>InternVL2-76B</td>
|
||||
<td>0.44</td>
|
||||
<td>0.443</td>
|
||||
<td>0.353</td>
|
||||
<td>0.29</td>
|
||||
<td>0.543</td>
|
||||
<td>0.701</td>
|
||||
<td>0.547</td>
|
||||
<td>0.555</td>
|
||||
<td>0.317</td>
|
||||
<td>0.228</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
The above data is from:
|
||||
* <a href="https://github.com/opendatalab/OmniDocBench">OmniDocBench</a>
|
||||
* <a href="https://arxiv.org/abs/2412.07626">OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations</a>
|
||||
|
||||
# End to End Benchmark
|
||||
|
||||
The performance of PP-StructureV3 and MinerU with different configurations under different GPU environments are as follows.
|
||||
|
||||
Requirements:
|
||||
* Paddle 3.0
|
||||
* PaddleOCR 3.0.0
|
||||
* MinerU 1.3.10
|
||||
* CUDA 11.8
|
||||
* cuDNN 8.9
|
||||
|
||||
## Local inference
|
||||
|
||||
Local inference was tested with both V100 and A100 GPU, evaluating the performance of PP-StructureV3 under 6 different configurations. The test data consists of 15 PDF files, totaling 925 pages, including elements such as tables, formulas, seals, and charts.
|
||||
|
||||
In the following PP-StructureV3 configuration, please refer to [PP-OCRv5](../PP-OCRv5/PP-OCRv5.en.md) for OCR model details, see [Formula Recognition](../../module_usage/formula_recognition.en.md) for formula recognition model details, and refer to [Text Detection](../../module_usage/text_detection.en.md) for the max_side_limit setting of the text detection module.
|
||||
|
||||
### Env: NVIDIA Tesla V100 + Intel Xeon Gold 6271C
|
||||
|
||||
<table border="1">
|
||||
<tr>
|
||||
<td>
|
||||
Methods
|
||||
</td>
|
||||
<td colspan="4">
|
||||
Configurations
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average time per
|
||||
page
|
||||
(s)
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average CPU
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Peak RAM Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average RAM
|
||||
Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average GPU
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Peak VRAM Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average VRAM
|
||||
Usage
|
||||
（GB）
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="7">
|
||||
PP-StructureV3
|
||||
</td>
|
||||
<td>
|
||||
OCR Models
|
||||
</td>
|
||||
<td>
|
||||
Formula Recognition Model
|
||||
</td>
|
||||
<td>
|
||||
Chart Recognition Model
|
||||
</td>
|
||||
<td>
|
||||
text detection module max_side_limit
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.77
|
||||
</td>
|
||||
<td>
|
||||
111.4
|
||||
</td>
|
||||
<td>
|
||||
6.7
|
||||
</td>
|
||||
<td>
|
||||
5.2
|
||||
</td>
|
||||
<td>
|
||||
38.9
|
||||
</td>
|
||||
<td>
|
||||
17.0
|
||||
</td>
|
||||
<td>
|
||||
16.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✔
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
4.09
|
||||
</td>
|
||||
<td>
|
||||
105.3
|
||||
</td>
|
||||
<td>
|
||||
5.5
|
||||
</td>
|
||||
<td>
|
||||
4.0
|
||||
</td>
|
||||
<td>
|
||||
24.7
|
||||
</td>
|
||||
<td>
|
||||
17.0
|
||||
</td>
|
||||
<td>
|
||||
16.6
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.56
|
||||
</td>
|
||||
<td>
|
||||
113.7
|
||||
</td>
|
||||
<td>
|
||||
6.6
|
||||
</td>
|
||||
<td>
|
||||
4.9
|
||||
</td>
|
||||
<td>
|
||||
29.1
|
||||
</td>
|
||||
<td>
|
||||
10.7
|
||||
</td>
|
||||
<td>
|
||||
10.6
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.42
|
||||
</td>
|
||||
<td>
|
||||
112.9
|
||||
</td>
|
||||
<td>
|
||||
6.8
|
||||
</td>
|
||||
<td>
|
||||
5.1
|
||||
</td>
|
||||
<td>
|
||||
38
|
||||
</td>
|
||||
<td>
|
||||
16.0
|
||||
</td>
|
||||
<td>
|
||||
15.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.15
|
||||
</td>
|
||||
<td>
|
||||
114.8
|
||||
</td>
|
||||
<td>
|
||||
6.5
|
||||
</td>
|
||||
<td>
|
||||
5.0
|
||||
</td>
|
||||
<td>
|
||||
26.1
|
||||
</td>
|
||||
<td>
|
||||
8.4
|
||||
</td>
|
||||
<td>
|
||||
8.3
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
1200
|
||||
</td>
|
||||
<td>
|
||||
0.99
|
||||
</td>
|
||||
<td>
|
||||
113
|
||||
</td>
|
||||
<td>
|
||||
7.0
|
||||
</td>
|
||||
<td>
|
||||
5.6
|
||||
</td>
|
||||
<td>
|
||||
29.2
|
||||
</td>
|
||||
<td>
|
||||
8.6
|
||||
</td>
|
||||
<td>
|
||||
8.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
MinerU
|
||||
</td>
|
||||
<td colspan="4">
|
||||
-
|
||||
</td>
|
||||
<td>
|
||||
1.57
|
||||
</td>
|
||||
<td>
|
||||
142.9
|
||||
</td>
|
||||
<td>
|
||||
13.3
|
||||
</td>
|
||||
<td>
|
||||
11.8
|
||||
</td>
|
||||
<td>
|
||||
43.3
|
||||
</td>
|
||||
<td>
|
||||
31.6
|
||||
</td>
|
||||
<td>
|
||||
9.7
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
### NVIDIA A100 + Intel Xeon Platinum 8350C
|
||||
|
||||
<table border="1">
|
||||
<tr>
|
||||
<td>
|
||||
Methods
|
||||
</td>
|
||||
<td colspan="4">
|
||||
Configurations
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average time per
|
||||
page
|
||||
(s)
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average CPU
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Peak RAM Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average RAM
|
||||
Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average GPU
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Peak VRAM Usage
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
Average VRAM
|
||||
Usage
|
||||
（GB）
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="7">
|
||||
PP-StructureV3
|
||||
</td>
|
||||
<td>
|
||||
OCR Models
|
||||
</td>
|
||||
<td>
|
||||
Formula Recognition Model
|
||||
</td>
|
||||
<td>
|
||||
Chart Recognition Model
|
||||
</td>
|
||||
<td>
|
||||
text detection module max_side_limit
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.12
|
||||
</td>
|
||||
<td>
|
||||
109.8
|
||||
</td>
|
||||
<td>
|
||||
9.2
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
29.8
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✔
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
2.76
|
||||
</td>
|
||||
<td>
|
||||
103.7
|
||||
</td>
|
||||
<td>
|
||||
9.0
|
||||
</td>
|
||||
<td>
|
||||
7.7
|
||||
</td>
|
||||
<td>
|
||||
24
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.04
|
||||
</td>
|
||||
<td>
|
||||
110.7
|
||||
</td>
|
||||
<td>
|
||||
9.3
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
22
|
||||
</td>
|
||||
<td>
|
||||
12.2
|
||||
</td>
|
||||
<td>
|
||||
12.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
0.95
|
||||
</td>
|
||||
<td>
|
||||
111.4
|
||||
</td>
|
||||
<td>
|
||||
9.1
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
28.1
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.0
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
0.89
|
||||
</td>
|
||||
<td>
|
||||
112.1
|
||||
</td>
|
||||
<td>
|
||||
9.2
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
18.5
|
||||
</td>
|
||||
<td>
|
||||
11.4
|
||||
</td>
|
||||
<td>
|
||||
11.2
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
1200
|
||||
</td>
|
||||
<td>
|
||||
0.64
|
||||
</td>
|
||||
<td>
|
||||
113.5
|
||||
</td>
|
||||
<td>
|
||||
10.2
|
||||
</td>
|
||||
<td>
|
||||
8.5
|
||||
</td>
|
||||
<td>
|
||||
23.7
|
||||
</td>
|
||||
<td>
|
||||
11.4
|
||||
</td>
|
||||
<td>
|
||||
11.2
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
MinerU
|
||||
</td>
|
||||
<td colspan="4">
|
||||
-
|
||||
</td>
|
||||
<td>
|
||||
1.06
|
||||
</td>
|
||||
<td>
|
||||
168.3
|
||||
</td>
|
||||
<td>
|
||||
18.3
|
||||
</td>
|
||||
<td>
|
||||
16.8
|
||||
</td>
|
||||
<td>
|
||||
27.5
|
||||
</td>
|
||||
<td>
|
||||
76.9
|
||||
</td>
|
||||
<td>
|
||||
14.8
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## Serving Inference
|
||||
|
||||
The serving inference test is based on the NVIDIA A100 + Intel Xeon Platinum 8350C environment, with test data consisting of 1500 images, including tables, formulas, seals, charts, and other elements.
|
||||
|
||||
<table>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Instances Number</td>
|
||||
<td>Concurrent Requests Number</td>
|
||||
<td>Throughput</td>
|
||||
<td>Average Latency (s)</td>
|
||||
<td>Success Number/Total Number</td>
|
||||
</tr>
|
||||
<tr">
|
||||
<td>4 GPUs ✖️ 1 instance/gpu</td>
|
||||
<td>4</td>
|
||||
<td>1.69</td>
|
||||
<td>2.36</td>
|
||||
<td>100%</td>
|
||||
</tr>
|
||||
<tr">
|
||||
<td>4 GPUs ✖️ 4 instances/gpu</td>
|
||||
<td>16</td>
|
||||
<td>4.05</td>
|
||||
<td>3.87</td>
|
||||
<td>100%</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
# PP-StructureV3 Demo
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-StructureV3/algorithm_ppstructurev3_demo.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex%2FPaddleX3.0%2Fdoc_images%2FPP-StructureV3%2Falgorithm_ppstructurev3_demo.pdf">More Demos</a>
|
||||
|
||||
|
||||
# FAQ
|
||||
|
||||
1. What is the default configuration? How to get higher accuracy, faster speed, or smaller GPU memory?
|
||||
|
||||
When using mobile OCR models + PP-FormulaNet_plus-M, and max length of text detection set to 1200, if set use_chart_recognition to False and dont not load the chart recognition model, the GPU memory would be reduced.
|
||||
|
||||
On the V100, the peak and average GPU memory would be reduced from 8776.0 MB and 8680.8 MB to 6118.0 MB and 6016.7 MB, respectively; On the A100, the peak and average GPU memory would be reduced from 11716.0 MB and 11453.9 MB to 9850.0 MB and 9593.5 MB, respectively.
|
||||
|
||||
You can using multi-gpus by setting `device` to `gpu:<no.>,<no.>`, such as `gpu:0,1,2,3`. And about multi-process parallel inference, you can refer: [Multi-Process Parallel Inference](https://github.com/PaddlePaddle/PaddleX/blob/develop/docs/pipeline_usage/instructions/parallel_inference.en.md#example-of-multi-process-parallel-inference).
|
||||
|
||||
2. About serving deployment
|
||||
|
||||
(1) Can the service handle requests concurrently?
|
||||
|
||||
For the basic serving deployment solution, the service processes only one request at a time. This plan is mainly used for rapid verification, to establish the development chain, or for scenarios where concurrent requests are not required.
|
||||
|
||||
For high-stability serving deployment solution, the service process only one request at a time by default, but you can refer to the related docs to adjust achieve scaling.
|
||||
|
||||
（2）How to reduce latency and improve throughput?
|
||||
|
||||
Use the High-performance inference plugin, and deploy multi instances.
|
||||
971
docs/version3.x/algorithm/PP-StructureV3/PP-StructureV3.md
Normal file
971
docs/version3.x/algorithm/PP-StructureV3/PP-StructureV3.md
Normal file
@@ -0,0 +1,971 @@
|
||||
# 一、PP-StructureV3 简介
|
||||
**PP-StructureV3** 产线在通用版面解析v1产线的基础上，强化了版面区域检测、表格识别、公式识别的能力，增加了图表理解和多栏阅读顺序的恢复能力、结果转换 Markdown 文件的能力，在多种文档数据中，表现优异，可以处理较复杂的文档数据。本产线同时提供了灵活的服务化部署方式，支持在多种硬件上使用多种编程语言调用。不仅如此，本产线也提供了二次开发的能力，您可以基于本产线在您自己的数据集上训练调优，训练后的模型也可以无缝集成。
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-StructureV3/algorithm_ppstructurev3.png" width="600"/>
|
||||
</div>
|
||||
|
||||
# 二、关键指标
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th rowspan="2">Method Type</th>
|
||||
<th rowspan="2">Methods</th>
|
||||
<th colspan="2">Overall<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Text<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Formula<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Table<sup>Edit</sup>↓</th>
|
||||
<th colspan="2">Read Order<sup>Edit</sup>↓</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
<th>EN</th>
|
||||
<th>ZH</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td rowspan="9">Pipeline Tools</td>
|
||||
<td><b>PP-structureV3</b></td>
|
||||
<td><b>0.147</b></td>
|
||||
<td><b>0.212</b></td>
|
||||
<td>0.059</td>
|
||||
<td><b>0.09</b></td>
|
||||
<td>0.295</td>
|
||||
<td>0.535</td>
|
||||
<td>0.159</td>
|
||||
<td><b>0.109</b></td>
|
||||
<td>0.075</td>
|
||||
<td><b>0.114</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MinerU-0.9.3</td>
|
||||
<td>0.15</td>
|
||||
<td>0.357</td>
|
||||
<td>0.061</td>
|
||||
<td>0.215</td>
|
||||
<td>0.278</td>
|
||||
<td>0.577</td>
|
||||
<td>0.18</td>
|
||||
<td>0.344</td>
|
||||
<td>0.079</td>
|
||||
<td>0.292</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>MinerU-1.3.11</td>
|
||||
<td>0.166</td>
|
||||
<td>0.310</td>
|
||||
<td>0.0826</td>
|
||||
<td>0.2000</td>
|
||||
<td>0.3368</td>
|
||||
<td>0.6236</td>
|
||||
<td>0.1613</td>
|
||||
<td>0.1833</td>
|
||||
<td>0.0834</td>
|
||||
<td>0.2316</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Marker-1.2.3</td>
|
||||
<td>0.336</td>
|
||||
<td>0.556</td>
|
||||
<td>0.08</td>
|
||||
<td>0.315</td>
|
||||
<td>0.53</td>
|
||||
<td>0.883</td>
|
||||
<td>0.619</td>
|
||||
<td>0.685</td>
|
||||
<td>0.114</td>
|
||||
<td>0.34</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mathpix</td>
|
||||
<td>0.191</td>
|
||||
<td>0.365</td>
|
||||
<td>0.105</td>
|
||||
<td>0.384</td>
|
||||
<td>0.306</td>
|
||||
<td>0.454</td>
|
||||
<td>0.243</td>
|
||||
<td>0.32</td>
|
||||
<td>0.108</td>
|
||||
<td>0.304</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Docling-2.14.0</td>
|
||||
<td>0.589</td>
|
||||
<td>0.909</td>
|
||||
<td>0.416</td>
|
||||
<td>0.987</td>
|
||||
<td>0.999</td>
|
||||
<td>1</td>
|
||||
<td>0.627</td>
|
||||
<td>0.81</td>
|
||||
<td>0.313</td>
|
||||
<td>0.837</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Pix2Text-1.1.2.3</td>
|
||||
<td>0.32</td>
|
||||
<td>0.528</td>
|
||||
<td>0.138</td>
|
||||
<td>0.356</td>
|
||||
<td><b>0.276</b></td>
|
||||
<td>0.611</td>
|
||||
<td>0.584</td>
|
||||
<td>0.645</td>
|
||||
<td>0.281</td>
|
||||
<td>0.499</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Unstructured-0.17.2</td>
|
||||
<td>0.586</td>
|
||||
<td>0.716</td>
|
||||
<td>0.198</td>
|
||||
<td>0.481</td>
|
||||
<td>0.999</td>
|
||||
<td>1</td>
|
||||
<td>1</td>
|
||||
<td>0.998</td>
|
||||
<td>0.145</td>
|
||||
<td>0.387</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OpenParse-0.7.0</td>
|
||||
<td>0.646</td>
|
||||
<td>0.814</td>
|
||||
<td>0.681</td>
|
||||
<td>0.974</td>
|
||||
<td>0.996</td>
|
||||
<td>1</td>
|
||||
<td>0.284</td>
|
||||
<td>0.639</td>
|
||||
<td>0.595</td>
|
||||
<td>0.641</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="5">Expert VLMs</td>
|
||||
<td>GOT-OCR</td>
|
||||
<td>0.287</td>
|
||||
<td>0.411</td>
|
||||
<td>0.189</td>
|
||||
<td>0.315</td>
|
||||
<td>0.36</td>
|
||||
<td>0.528</td>
|
||||
<td>0.459</td>
|
||||
<td>0.52</td>
|
||||
<td>0.141</td>
|
||||
<td>0.28</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Nougat</td>
|
||||
<td>0.452</td>
|
||||
<td>0.973</td>
|
||||
<td>0.365</td>
|
||||
<td>0.998</td>
|
||||
<td>0.488</td>
|
||||
<td>0.941</td>
|
||||
<td>0.572</td>
|
||||
<td>1</td>
|
||||
<td>0.382</td>
|
||||
<td>0.954</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Mistral OCR</td>
|
||||
<td>0.268</td>
|
||||
<td>0.439</td>
|
||||
<td>0.072</td>
|
||||
<td>0.325</td>
|
||||
<td>0.318</td>
|
||||
<td>0.495</td>
|
||||
<td>0.6</td>
|
||||
<td>0.65</td>
|
||||
<td>0.083</td>
|
||||
<td>0.284</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>OLMOCR-sglang</td>
|
||||
<td>0.326</td>
|
||||
<td>0.469</td>
|
||||
<td>0.097</td>
|
||||
<td>0.293</td>
|
||||
<td>0.455</td>
|
||||
<td>0.655</td>
|
||||
<td>0.608</td>
|
||||
<td>0.652</td>
|
||||
<td>0.145</td>
|
||||
<td>0.277</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SmolDocling-256M_transformer</td>
|
||||
<td>0.493</td>
|
||||
<td>0.816</td>
|
||||
<td>0.262</td>
|
||||
<td>0.838</td>
|
||||
<td>0.753</td>
|
||||
<td>0.997</td>
|
||||
<td>0.729</td>
|
||||
<td>0.907</td>
|
||||
<td>0.227</td>
|
||||
<td>0.522</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="6">General VLMs</td>
|
||||
<td>Gemini2.0-flash</td>
|
||||
<td>0.191</td>
|
||||
<td>0.264</td>
|
||||
<td>0.091</td>
|
||||
<td>0.139</td>
|
||||
<td>0.389</td>
|
||||
<td>0.584</td>
|
||||
<td>0.193</td>
|
||||
<td>0.206</td>
|
||||
<td>0.092</td>
|
||||
<td>0.128</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gemini2.5-Pro</td>
|
||||
<td>0.148</td>
|
||||
<td><b>0.212</b></td>
|
||||
<td><b>0.055</b></td>
|
||||
<td>0.168</td>
|
||||
<td>0.356</td>
|
||||
<td>0.439</td>
|
||||
<td><b>0.13</b></td>
|
||||
<td>0.119</td>
|
||||
<td><b>0.049</b></td>
|
||||
<td>0.121</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>GPT4o</td>
|
||||
<td>0.233</td>
|
||||
<td>0.399</td>
|
||||
<td>0.144</td>
|
||||
<td>0.409</td>
|
||||
<td>0.425</td>
|
||||
<td>0.606</td>
|
||||
<td>0.234</td>
|
||||
<td>0.329</td>
|
||||
<td>0.128</td>
|
||||
<td>0.251</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2-VL-72B</td>
|
||||
<td>0.252</td>
|
||||
<td>0.327</td>
|
||||
<td>0.096</td>
|
||||
<td>0.218</td>
|
||||
<td>0.404</td>
|
||||
<td>0.487</td>
|
||||
<td>0.387</td>
|
||||
<td>0.408</td>
|
||||
<td>0.119</td>
|
||||
<td>0.193</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Qwen2.5-VL-72B</td>
|
||||
<td>0.214</td>
|
||||
<td>0.261</td>
|
||||
<td>0.092</td>
|
||||
<td>0.18</td>
|
||||
<td>0.315</td>
|
||||
<td><b>0.434</b></td>
|
||||
<td>0.341</td>
|
||||
<td>0.262</td>
|
||||
<td>0.106</td>
|
||||
<td>0.168</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>InternVL2-76B</td>
|
||||
<td>0.44</td>
|
||||
<td>0.443</td>
|
||||
<td>0.353</td>
|
||||
<td>0.29</td>
|
||||
<td>0.543</td>
|
||||
<td>0.701</td>
|
||||
<td>0.547</td>
|
||||
<td>0.555</td>
|
||||
<td>0.317</td>
|
||||
<td>0.228</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
以上部分数据出自：
|
||||
* <a href="https://github.com/opendatalab/OmniDocBench">OmniDocBench</a>
|
||||
* <a href="https://arxiv.org/abs/2412.07626">OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations</a>
|
||||
|
||||
|
||||
# 三、推理 Benchmark
|
||||
|
||||
在不同GPU环境下，不同配置的 PP-StructureV3 和 MinerU 对比的性能指标如下。
|
||||
|
||||
基本测试环境：
|
||||
* Paddle 3.0正式版
|
||||
* PaddleOCR 3.0.0正式版
|
||||
* MinerU 1.3.10
|
||||
* CUDA 11.8
|
||||
* cuDNN 8.9
|
||||
|
||||
## 3.1 本地推理
|
||||
|
||||
本地推理分别在 V100 和 A100 两种 GPU机器上，测试了 6 种不同配置下 PP-StructureV3 的性能，测试数据为15个PDF文件，共925页，包含表格、公式、印章、图表等元素。
|
||||
|
||||
下述 PP-StructureV3 配置中，OCR 模型详情请见[PP-OCRv5](../PP-OCRv5/PP-OCRv5.md)，公式识别模型详情请见[公式识别](../../module_usage/formula_recognition.md)，文本检测模块 max_side_limit 设置请见[文本检测](../../module_usage/text_detection.md)。
|
||||
|
||||
### NVIDIA Tesla V100 + Intel Xeon Gold 6271C
|
||||
<table border="1">
|
||||
<tr>
|
||||
<td>
|
||||
方案
|
||||
</td>
|
||||
<td colspan="4">
|
||||
配置
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均每页耗时
|
||||
（s）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均CPU利用率
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
峰值RAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均RAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均GPU利用率
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
峰值VRAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均VRAM用量
|
||||
（GB）
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="7">
|
||||
PP-StructureV3
|
||||
</td>
|
||||
<td>
|
||||
OCR模型
|
||||
</td>
|
||||
<td>
|
||||
公式识别模型
|
||||
</td>
|
||||
<td>
|
||||
是否启用图表识别模块
|
||||
</td>
|
||||
<td>
|
||||
文本检测max_side_limit
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.77
|
||||
</td>
|
||||
<td>
|
||||
111.4
|
||||
</td>
|
||||
<td>
|
||||
6.7
|
||||
</td>
|
||||
<td>
|
||||
5.2
|
||||
</td>
|
||||
<td>
|
||||
38.9
|
||||
</td>
|
||||
<td>
|
||||
17.0
|
||||
</td>
|
||||
<td>
|
||||
16.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✔
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
4.09
|
||||
</td>
|
||||
<td>
|
||||
105.3
|
||||
</td>
|
||||
<td>
|
||||
5.5
|
||||
</td>
|
||||
<td>
|
||||
4.0
|
||||
</td>
|
||||
<td>
|
||||
24.7
|
||||
</td>
|
||||
<td>
|
||||
17.0
|
||||
</td>
|
||||
<td>
|
||||
16.6
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.56
|
||||
</td>
|
||||
<td>
|
||||
113.7
|
||||
</td>
|
||||
<td>
|
||||
6.6
|
||||
</td>
|
||||
<td>
|
||||
4.9
|
||||
</td>
|
||||
<td>
|
||||
29.1
|
||||
</td>
|
||||
<td>
|
||||
10.7
|
||||
</td>
|
||||
<td>
|
||||
10.6
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.42
|
||||
</td>
|
||||
<td>
|
||||
112.9
|
||||
</td>
|
||||
<td>
|
||||
6.8
|
||||
</td>
|
||||
<td>
|
||||
5.1
|
||||
</td>
|
||||
<td>
|
||||
38
|
||||
</td>
|
||||
<td>
|
||||
16.0
|
||||
</td>
|
||||
<td>
|
||||
15.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.15
|
||||
</td>
|
||||
<td>
|
||||
114.8
|
||||
</td>
|
||||
<td>
|
||||
6.5
|
||||
</td>
|
||||
<td>
|
||||
5.0
|
||||
</td>
|
||||
<td>
|
||||
26.1
|
||||
</td>
|
||||
<td>
|
||||
8.4
|
||||
</td>
|
||||
<td>
|
||||
8.3
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
1200
|
||||
</td>
|
||||
<td>
|
||||
0.99
|
||||
</td>
|
||||
<td>
|
||||
113
|
||||
</td>
|
||||
<td>
|
||||
7.0
|
||||
</td>
|
||||
<td>
|
||||
5.6
|
||||
</td>
|
||||
<td>
|
||||
29.2
|
||||
</td>
|
||||
<td>
|
||||
8.6
|
||||
</td>
|
||||
<td>
|
||||
8.5
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
MinerU
|
||||
</td>
|
||||
<td colspan="4">
|
||||
-
|
||||
</td>
|
||||
<td>
|
||||
1.57
|
||||
</td>
|
||||
<td>
|
||||
142.9
|
||||
</td>
|
||||
<td>
|
||||
13.3
|
||||
</td>
|
||||
<td>
|
||||
11.8
|
||||
</td>
|
||||
<td>
|
||||
43.3
|
||||
</td>
|
||||
<td>
|
||||
31.6
|
||||
</td>
|
||||
<td>
|
||||
9.7
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
### NVIDIA A100 + Intel Xeon Platinum 8350C
|
||||
<table border="1">
|
||||
<tr>
|
||||
<td>
|
||||
方案
|
||||
</td>
|
||||
<td colspan="4">
|
||||
配置
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均每页耗时
|
||||
（s）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均CPU利用率
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
峰值RAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均RAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均GPU利用率
|
||||
（%）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
峰值VRAM用量
|
||||
（GB）
|
||||
</td>
|
||||
<td rowspan="2">
|
||||
平均VRAM用量
|
||||
（GB）
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="7">
|
||||
PP-StructureV3
|
||||
</td>
|
||||
<td>
|
||||
OCR模型
|
||||
</td>
|
||||
<td>
|
||||
公式识别模型
|
||||
</td>
|
||||
<td>
|
||||
是否启用图表识别模块
|
||||
</td>
|
||||
<td>
|
||||
文本检测max_side_limit
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.12
|
||||
</td>
|
||||
<td>
|
||||
109.8
|
||||
</td>
|
||||
<td>
|
||||
9.2
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
29.8
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✔
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
2.76
|
||||
</td>
|
||||
<td>
|
||||
103.7
|
||||
</td>
|
||||
<td>
|
||||
9.0
|
||||
</td>
|
||||
<td>
|
||||
7.7
|
||||
</td>
|
||||
<td>
|
||||
24
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-L
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
1.04
|
||||
</td>
|
||||
<td>
|
||||
110.7
|
||||
</td>
|
||||
<td>
|
||||
9.3
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
22
|
||||
</td>
|
||||
<td>
|
||||
12.2
|
||||
</td>
|
||||
<td>
|
||||
12.1
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Server系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
0.95
|
||||
</td>
|
||||
<td>
|
||||
111.4
|
||||
</td>
|
||||
<td>
|
||||
9.1
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
28.1
|
||||
</td>
|
||||
<td>
|
||||
21.8
|
||||
</td>
|
||||
<td>
|
||||
21.0
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
4096
|
||||
</td>
|
||||
<td>
|
||||
0.89
|
||||
</td>
|
||||
<td>
|
||||
112.1
|
||||
</td>
|
||||
<td>
|
||||
9.2
|
||||
</td>
|
||||
<td>
|
||||
7.8
|
||||
</td>
|
||||
<td>
|
||||
18.5
|
||||
</td>
|
||||
<td>
|
||||
11.4
|
||||
</td>
|
||||
<td>
|
||||
11.2
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
Mobile系列
|
||||
</td>
|
||||
<td>
|
||||
PP-FormulaNet-M
|
||||
</td>
|
||||
<td>
|
||||
✗
|
||||
</td>
|
||||
<td>
|
||||
1200
|
||||
</td>
|
||||
<td>
|
||||
0.64
|
||||
</td>
|
||||
<td>
|
||||
113.5
|
||||
</td>
|
||||
<td>
|
||||
10.2
|
||||
</td>
|
||||
<td>
|
||||
8.5
|
||||
</td>
|
||||
<td>
|
||||
23.7
|
||||
</td>
|
||||
<td>
|
||||
11.4
|
||||
</td>
|
||||
<td>
|
||||
11.2
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>
|
||||
MinerU
|
||||
</td>
|
||||
<td colspan="4">
|
||||
-
|
||||
</td>
|
||||
<td>
|
||||
1.06
|
||||
</td>
|
||||
<td>
|
||||
168.3
|
||||
</td>
|
||||
<td>
|
||||
18.3
|
||||
</td>
|
||||
<td>
|
||||
16.8
|
||||
</td>
|
||||
<td>
|
||||
27.5
|
||||
</td>
|
||||
<td>
|
||||
76.9
|
||||
</td>
|
||||
<td>
|
||||
14.8
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 3.2 服务化部署
|
||||
|
||||
服务化部署测试基于 NVIDIA A100 + Intel Xeon Platinum 8350C 环境，测试数据为 1500 张图像，包含表格、公式、印章、图表等元素。
|
||||
|
||||
<table>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>实例数</td>
|
||||
<td>并发请求数</td>
|
||||
<td>吞吐</td>
|
||||
<td>平均时延（s）</td>
|
||||
<td>成功请求数/总请求数</td>
|
||||
</tr>
|
||||
<tr">
|
||||
<td>4卡 ✖️ 1实例/卡</td>
|
||||
<td>4</td>
|
||||
<td>1.69</td>
|
||||
<td>2.36</td>
|
||||
<td>100%</td>
|
||||
</tr>
|
||||
<tr">
|
||||
<td>4卡 ✖️ 4实例/卡</td>
|
||||
<td>16</td>
|
||||
<td>4.05</td>
|
||||
<td>3.87</td>
|
||||
<td>100%</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
# 四、PP-StructureV3 Demo示例
|
||||
|
||||
<div align="center">
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/paddleocr/PP-StructureV3/algorithm_ppstructurev3_demo.png" width="600"/>
|
||||
</div>
|
||||
|
||||
<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex%2FPaddleX3.0%2Fdoc_images%2FPP-StructureV3%2Falgorithm_ppstructurev3_demo.pdf">更多示例</a>
|
||||
|
||||
# 五、使用方法和常见问题
|
||||
|
||||
1. 默认模型是什么配置，如果需要更高精度、更快速度、或者更小显存，应该调哪些参数或者更换哪些模型，对结果影响大概有多大？
|
||||
|
||||
在“使用轻量OCR模型+轻量公式模型，文本检测max 1200”的基础上，将产线配置文件中的use_chart_recognition设置为False，不加载图表识别模型，可以进一步减少显存用量。在V100测试环境中，峰值和平均显存用量分别从8776.0 MB和8680.8 MB降低到6118.0 MB和6016.7 MB；在A100测试环境中，峰值和平均显存用量分别从11716.0 MB和11453.9 MB降低到9850.0 MB和9593.5 MB。
|
||||
在Python API或CLI设置device为<设备类型>:<设备编号1>,<设备编号2>...（例如gpu:0,1,2,3）可实现多卡并行推理。如果内置的多卡并行推理功能提速效果仍不满足预期，可参考多进程并行推理示例代码，结合具体场景进行进一步优化：[多进程并行推理](https://github.com/PaddlePaddle/PaddleX/blob/develop/docs/pipeline_usage/instructions/parallel_inference.md#%E5%A4%9A%E8%BF%9B%E7%A8%8B%E5%B9%B6%E8%A1%8C%E6%8E%A8%E7%90%86%E7%A4%BA%E4%BE%8B)。
|
||||
|
||||
2. 服务化部署的常见问题
|
||||
|
||||
（1）服务可以并发处理请求吗？
|
||||
|
||||
对于基础服务化部署方案，服务同一时间只处理一个请求，该方案主要用于快速验证、打通开发链路，或者用在不需要并发请求的场景；
|
||||
|
||||
对于高稳定性服务化部署方案，服务默认在同一时间只处理一个请求，但用户可以参考服务化部署指南，通过调整配置实现水平扩展，以使服务同时处理多个请求。
|
||||
|
||||
（2）如何降低时延、提升吞吐？
|
||||
|
||||
无论使用哪一种服务化部署方案，都可以通过启用高性能推理插件提升模型推理速度，从而降低处理时延。
|
||||
|
||||
此外，对于高稳定性服务化部署方案，通过调整服务配置，设置多个实例，也可以充分利用部署机器的资源，有效提升吞吐。
|
||||
Reference in New Issue
Block a user