This commit is contained in:
@@ -0,0 +1,327 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Document Image Orientation Classification Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The Document Image Orientation Classification Module is primarily designed to distinguish the orientation of document images and correct them through post-processing. During processes such as document scanning or ID photo capturing, the device might be rotated to achieve clearer images, resulting in images with various orientations. Standard OCR pipelines may not handle these images effectively. By leveraging image classification techniques, the orientation of documents or IDs containing text regions can be pre-determined and adjusted, thereby improving the accuracy of OCR processing.
|
||||
|
||||
## 2. Supported Models List
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Top-1 Acc (%)</th>
|
||||
<th>GPU Inference Time (ms)<br>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_doc_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_doc_ori_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_doc_ori_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>99.06</td>
|
||||
<td>2.62 / 0.59</td>
|
||||
<td>3.24 / 1.19</td>
|
||||
<td>7</td>
|
||||
<td>A document image classification model based on PP-LCNet_x1_0, with four categories: 0°, 90°, 180°, and 270°.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> Self-built multi-scenario dataset (1000 images, including ID/document scenarios)</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Optimal backend selected (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting, please install the PaddleOCR wheel package. For details, refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference of the Document Image Orientation Classification Module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg) to your local machine.
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(model_name="PP-LCNet_x1_0_doc_ori")
|
||||
output = model.predict("img_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result will be:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'img_rot180_demo.jpg', 'page_index': None, 'class_ids': array([2], dtype=int32), 'scores': array([0.88164], dtype=float32), 'label_names': ['180']}}
|
||||
```
|
||||
|
||||
The meaning of the output parameters is as follows:
|
||||
- `input_path`: Represents the path of the input image.
|
||||
- `class_ids`: Represents the predicted class ID, with four categories: 0°, 90°, 180°, and 270°.
|
||||
```- `scores`: Represents the confidence level of the prediction result.
|
||||
- `label_names`: Represents the category names of the prediction results.
|
||||
|
||||
Here is the visualization of the image:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/doc_img_ori_classification/img_rot180_demo_res.jpg">
|
||||
|
||||
The explanations of relevant methods and parameters are as follows:
|
||||
|
||||
* Instantiate the document image orientation classification model with `DocImgOrientationClassification` (taking `PP-LCNet_x1_0_doc_ori` as an example here). The specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-LCNet_x1_0_doc_ori</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b><code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the document image orientation classification model for inference prediction. This method will return a list of results. In addition, this module also provides the `predict_iter()` method. The two methods are completely consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for scenarios where large datasets need to be processed or memory needs to be saved. You can choose either of these two methods according to your actual needs. The parameters of the `predict()` method are `input` and `batch_size`, and the specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is the corresponding Result object, and it supports operations such as printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data and make it more readable. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; when set to <code>False</code>, the original characters will be retained. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a file in <code>json</code> format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data and make it more readable. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; when set to <code>False</code>, the original characters will be retained. It is only valid when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as a file in image format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining the visualization image with results and the prediction results through attributes. The specifics are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualization image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## IV. Secondary Development
|
||||
|
||||
Since PaddleOCR does not directly provide training functionality for document image orientation classification, if you need to train a document image orientation classification model, you can refer to the [PaddleX Secondary Development for Document Image Orientation Classification](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html#iv-custom-development) section for training guidance. The trained model can be seamlessly integrated into PaddleOCR's API for inference purposes.
|
||||
|
||||
## V. FAQ
|
||||
@@ -0,0 +1,326 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文档图像方向分类模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文档图像方向分类模块主要是将文档图像的方向区分出来，并使用后处理将其矫正。在诸如文档扫描、证照拍摄等过程中，有时为了拍摄更清晰，会将拍摄设备进行旋转，导致得到的图片也是不同方向的。此时，标准的OCR流程无法很好地应对这些数据。利用图像分类技术，可以预先判断含文字区域的文档或证件的方向，并将其进行方向调整，从而提高OCR处理的准确性。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>Top-1 Acc（%）</th>
|
||||
<th>GPU推理耗时（ms）<br>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_doc_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_doc_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_doc_ori_pretrained.pdparams">训练模型</a></td>
|
||||
<td>99.06</td>
|
||||
<td>2.62 / 0.59</td>
|
||||
<td>3.24 / 1.19</td>
|
||||
<td>7</td>
|
||||
<td>基于PP-LCNet_x1_0的文档图像分类模型，含有四个类别，即0度，90度，180度，270度</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建多场景数据集（1000张图片，含证件/文档等场景）</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr doc_img_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg
|
||||
```
|
||||
|
||||
您也可以将文档图像方向分类模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import DocImgOrientationClassification
|
||||
|
||||
model = DocImgOrientationClassification(model_name="PP-LCNet_x1_0_doc_ori")
|
||||
output = model.predict("img_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'img_rot180_demo.jpg', 'page_index': None, 'class_ids': array([2], dtype=int32), 'scores': array([0.88164], dtype=float32), 'label_names': ['180']}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入图片的路径。
|
||||
- `class_ids`：表示预测结果的类别id，含有四个类别，即0度，90度，180度和270度。
|
||||
- `scores`：表示预测结果的置信度。
|
||||
- `label_names`：表示预测结果的类别名。
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/doc_img_ori_classification/img_rot180_demo_res.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `DocImgOrientationClassification`实例化文档图像方向分类模型（此处以`PP-LCNet_x1_0_doc_ori`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>说明</th>
|
||||
<th>类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-LCNet_x1_0_doc_ori</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文档图像方向分类模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供文档图像方向分类的训练，因此，如果需要训练文档图像方向分类模型，可以参考 [PaddleX 文档图像方向分类二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/doc_img_orientation_classification.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 五、FAQ
|
||||
238
docs/version3.x/module_usage/doc_vlm.en.md
Normal file
238
docs/version3.x/module_usage/doc_vlm.en.md
Normal file
@@ -0,0 +1,238 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Document Visual Language Model Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
|
||||
Document visual language models are a cutting-edge multimodal processing technology aimed at addressing the limitations of traditional document processing methods. Traditional methods are often limited to processing document information in specific formats or predefined categories, whereas document visual language models can integrate visual and linguistic information to understand and handle diverse document content. By combining computer vision and natural language processing technologies, these models can recognize images, text, and their relationships within documents, and even understand semantic information within complex layout structures. This makes document processing more intelligent and flexible, with stronger generalization capabilities, showing broad application prospects in automated office work, information extraction, and other fields.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Model Storage Size (GB)</th>
|
||||
<th>Total Score</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-2B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-2B_infer.tar">Inference Model</a></td>
|
||||
<td>4.2</td>
|
||||
<td>765</td>
|
||||
<td rowspan="2">PP-DocBee is a self-developed multimodal large model by the PaddlePaddle team, focusing on document understanding, and it performs excellently in Chinese document understanding tasks. The model is fine-tuned and optimized using nearly 5 million multimodal datasets for document understanding, including general VQA, OCR, charts, text-rich documents, mathematics and complex reasoning, synthetic data, and pure text data, with different training data ratios set. On several authoritative English document understanding evaluation lists in academia, PP-DocBee has basically achieved SOTA for models of the same parameter scale. In terms of internal business Chinese scenario indicators, PP-DocBee also outperforms the current popular open-source and closed-source models.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-7B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-7B_infer.tar">Inference Model</a></td>
|
||||
<td>15.8</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee2-3B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee2-3B_infer.tar">Inference Model</a></td>
|
||||
<td>7.6</td>
|
||||
<td>852</td>
|
||||
<td>PP-DocBee2 is a self-developed multimodal large model by the PaddlePaddle team, further optimizing the base model on the foundation of PP-DocBee and introducing a new data optimization scheme to improve data quality. Using a small amount of 470,000 data generated by a self-developed data synthesis strategy, PP-DocBee2 performs better in Chinese document understanding tasks. In terms of internal business Chinese scenario indicators, PP-DocBee2 improves by about 11.4% compared to PP-DocBee, and also outperforms the current popular open-source and closed-source models of the same scale.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<b>Note: The total scores of the above models are test results from an internal evaluation set, where all images have a resolution (height, width) of (1680, 1204), with a total of 1196 data entries, covering scenarios such as financial reports, laws and regulations, scientific and technical papers, manuals, humanities papers, contracts, research reports, etc. There are no plans for public release at the moment.</b>
|
||||
|
||||
## III. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please install the PaddleOCR wheel package. For details, please refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one line of command:
|
||||
|
||||
```bash
|
||||
paddleocr doc_vlm -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出'}"
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the open document visual language model module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import DocVLM
|
||||
model = DocVLM(model_name="PP-DocBee2-3B")
|
||||
results = model.predict(
|
||||
input={"image": "medal_table.png", "query": "识别这份表格的内容, 以markdown格式输出"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出', 'result': '| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n'}}
|
||||
```
|
||||
|
||||
The meaning of the result parameters is as follows:
|
||||
- `image`: Indicates the path of the input image to be predicted
|
||||
- `query`: Represents the input text information to be predicted
|
||||
- `result`: Information of the model's prediction result
|
||||
|
||||
The visualization of the prediction result is as follows:
|
||||
|
||||
```bash
|
||||
| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |
|
||||
| 2 | 美国（USA） | 36 | 39 | 37 | 112 |
|
||||
| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |
|
||||
| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |
|
||||
| 5 | 德国（GER） | 16 | 11 | 14 | 41 |
|
||||
| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |
|
||||
| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |
|
||||
| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |
|
||||
| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |
|
||||
| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |
|
||||
| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |
|
||||
| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |
|
||||
| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |
|
||||
| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |
|
||||
| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |
|
||||
```
|
||||
|
||||
Explanations of related methods, parameters, etc., are as follows:
|
||||
|
||||
* `DocVLM` instantiates the document visual language model (taking `PP-DocBee-2B` as an example), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-DocBee-2B</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the document visual language model for inference prediction. This method will return a result list. Additionally, this module also provides the `predict_iter()` method. Both are completely consistent in terms of parameter acceptance and result return, the difference being that `predict_iter()` returns a `generator`, capable of gradually processing and obtaining prediction results, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods based on actual needs. The `predict()` method parameters include `input`, `batch_size`, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data. Required. Since multimodal models have different input requirements, please refer to the specific model for the correct format.<br/>
|
||||
For example, for the PP-DocBee series models, the input format should be: <code>{'image': image_path, 'query': query_text}</code>
|
||||
</td>
|
||||
<td><code>dict</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is the corresponding Result object, and it supports operations such as printing and saving as `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print results to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path of the file to be saved. When it is a directory, the naming of the saved file is consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, it also supports obtaining prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## IV. Secondary Development
|
||||
|
||||
The current module does not support fine-tuning training temporarily, only inference integration is supported. The fine-tuning training of this module is planned to be supported in the future.
|
||||
|
||||
## V. FAQ
|
||||
241
docs/version3.x/module_usage/doc_vlm.md
Normal file
241
docs/version3.x/module_usage/doc_vlm.md
Normal file
@@ -0,0 +1,241 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文档类视觉语言模型模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文档类视觉语言模型是当前一种前沿的多模态处理技术，旨在解决传统文档处理方法的局限性。传统方法往往局限于处理特定格式或预定义类别的文档信息，而文档类视觉语言模型能够融合视觉与语言信息，理解并处理多样化的文档内容。通过结合计算机视觉与自然语言处理技术，模型可以识别文档中的图像、文本及其相互关系，甚至能理解复杂版面结构中的语义信息。这使得文档处理更加智能化、灵活化，具备更强的泛化能力，在自动化办公、信息提取等领域展现出广阔的应用前景。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>模型存储大小（GB）</th>
|
||||
<th>模型总分</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-2B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-2B_infer.tar">推理模型</a></td>
|
||||
<td>4.2</td>
|
||||
<td>765</td>
|
||||
<td rowspan="2">PP-DocBee 是飞桨团队自研的一款专注于文档理解的多模态大模型，在中文文档理解任务上具有卓越表现。该模型通过近 500 万条文档理解类多模态数据集进行微调优化，各种数据集包括了通用VQA类、OCR类、图表类、text-rich文档类、数学和复杂推理类、合成数据类、纯文本数据等，并设置了不同训练数据配比。在学术界权威的几个英文文档理解评测榜单上，PP-DocBee基本都达到了同参数量级别模型的SOTA。在内部业务中文场景类的指标上，PP-DocBee也高于目前的热门开源和闭源模型。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee-7B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee-7B_infer.tar">推理模型</a></td>
|
||||
<td>15.8</td>
|
||||
<td>-</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocBee2-3B</td><td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBee2-3B_infer.tar">推理模型</a></td>
|
||||
<td>7.6</td>
|
||||
<td>852</td>
|
||||
<td>PP-DocBee2 是飞桨团队自研的一款专注于文档理解的多模态大模型，在PP-DocBee的基础上进一步优化了基础模型，并引入了新的数据优化方案，提高了数据质量，使用自研数据合成策略生成的少量的47万数据便使得PP-DocBee2在中文文档理解任务上表现更佳。在内部业务中文场景类的指标上，PP-DocBee2相较于PP-DocBee提升了约11.4%，同时也高于目前的同规模热门开源和闭源模型。</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<b>注：以上模型总分为内部评估集模型测试结果，内部评估集所有图像分辨率 (height, width) 为 (1680,1204)，共1196条数据，包括了财报、法律法规、理工科论文、说明书、文科论文、合同、研报等场景，暂时未有计划公开。</b>
|
||||
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr doc_vlm -i "{'image': 'https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出'}"
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将开放文档类视觉语言模型模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/medal_table.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import DocVLM
|
||||
model = DocVLM(model_name="PP-DocBee2-3B")
|
||||
results = model.predict(
|
||||
input={"image": "medal_table.png", "query": "识别这份表格的内容, 以markdown格式输出"},
|
||||
batch_size=1
|
||||
)
|
||||
for res in results:
|
||||
res.print()
|
||||
res.save_to_json(f"./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'image': 'medal_table.png', 'query': '识别这份表格的内容, 以markdown格式输出', 'result': '| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |\n| --- | --- | --- | --- | --- | --- |\n| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |\n| 2 | 美国（USA） | 36 | 39 | 37 | 112 |\n| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |\n| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |\n| 5 | 德国（GER） | 16 | 11 | 14 | 41 |\n| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |\n| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |\n| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |\n| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |\n| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |\n| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |\n| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |\n| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |\n| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |\n| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |\n'}}
|
||||
```
|
||||
运行结果参数含义如下：
|
||||
- `image`: 表示输入待预测图像的路径
|
||||
- `query`: 表述输入待预测的文本信息
|
||||
- `result`: 模型预测的结果信息
|
||||
|
||||
预测结果打印可视化如下：
|
||||
|
||||
```bash
|
||||
| 名次 | 国家/地区 | 金牌 | 银牌 | 铜牌 | 奖牌总数 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| 1 | 中国（CHN） | 48 | 22 | 30 | 100 |
|
||||
| 2 | 美国（USA） | 36 | 39 | 37 | 112 |
|
||||
| 3 | 俄罗斯（RUS） | 24 | 13 | 23 | 60 |
|
||||
| 4 | 英国（GBR） | 19 | 13 | 19 | 51 |
|
||||
| 5 | 德国（GER） | 16 | 11 | 14 | 41 |
|
||||
| 6 | 澳大利亚（AUS） | 14 | 15 | 17 | 46 |
|
||||
| 7 | 韩国（KOR） | 13 | 11 | 8 | 32 |
|
||||
| 8 | 日本（JPN） | 9 | 8 | 8 | 25 |
|
||||
| 9 | 意大利（ITA） | 8 | 9 | 10 | 27 |
|
||||
| 10 | 法国（FRA） | 7 | 16 | 20 | 43 |
|
||||
| 11 | 荷兰（NED） | 7 | 5 | 4 | 16 |
|
||||
| 12 | 乌克兰（UKR） | 7 | 4 | 11 | 22 |
|
||||
| 13 | 肯尼亚（KEN） | 6 | 4 | 6 | 16 |
|
||||
| 14 | 西班牙（ESP） | 5 | 11 | 3 | 19 |
|
||||
| 15 | 牙买加（JAM） | 5 | 4 | 2 | 11 |
|
||||
```
|
||||
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `DocVLM`实例化文档类视觉语言模型（此处以`PP-DocBee-2B`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>>模型名称。如果设置为<code>None</code>，则使用<code>PP-DocBee-2B</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code></code>。
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文档类视觉语言模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 、 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，必填。由于多模态模型对输入要求不同，请根据具体模型设定输入格式。<br/>
|
||||
例如：对于 PP-DocBee 系列模型，输入形式应为：<code>{'image': image_path, 'query': query_text}</code>
|
||||
</td>
|
||||
<td><code>dict</code></td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
当前模块暂时不支持微调训练，仅支持推理集成。关于该模块的微调训练，计划在未来支持。
|
||||
|
||||
## 五、FAQ
|
||||
473
docs/version3.x/module_usage/formula_recognition.en.md
Normal file
473
docs/version3.x/module_usage/formula_recognition.en.md
Normal file
@@ -0,0 +1,473 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Formula Recognition Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
|
||||
The formula recognition module is a key component of an OCR (Optical Character Recognition) system, responsible for converting mathematical formulas in images into editable text or computer-readable formats. The performance of this module directly affects the accuracy and efficiency of the entire OCR system. The formula recognition module typically outputs LaTeX or MathML code of the mathematical formulas, which will be passed as input to the text understanding module for further processing.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>En-BLEU(%)</th>
|
||||
<th>Zh-BLEU(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UniMERNet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UniMERNet_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UniMERNet_pretrained.pdparams">Training Model</a></td>
|
||||
<td>85.91</td>
|
||||
<td>43.50</td>
|
||||
<td>1311.84 / 1311.84</td>
|
||||
<td>- / 8288.07</td>
|
||||
<td>1530</td>
|
||||
<td>UniMERNet is a formula recognition model developed by Shanghai AI Lab. It uses Donut Swin as the encoder and MBartDecoder as the decoder. The model is trained on a dataset of one million samples, including simple formulas, complex formulas, scanned formulas, and handwritten formulas, significantly improving the recognition accuracy of real-world formulas.</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>87.00</td>
|
||||
<td>45.71</td>
|
||||
<td>182.25 / 182.25</td>
|
||||
<td>- / 254.39</td>
|
||||
<td>224</td>
|
||||
<td rowspan="2">PP-FormulaNet is an advanced formula recognition model developed by the Baidu PaddlePaddle Vision Team. The PP-FormulaNet-S version uses PP-HGNetV2-B4 as its backbone network. Through parallel masking and model distillation techniques, it significantly improves inference speed while maintaining high recognition accuracy, making it suitable for applications requiring fast inference. The PP-FormulaNet-L version, on the other hand, uses Vary_VIT_B as its backbone network and is trained on a large-scale formula dataset, showing significant improvements in recognizing complex formulas compared to PP-FormulaNet-S.</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>90.36</td>
|
||||
<td>45.78</td>
|
||||
<td>1482.03 / 1482.03</td>
|
||||
<td>- / 3131.54</td>
|
||||
<td>695</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet_plus-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>88.71</td>
|
||||
<td>53.32</td>
|
||||
<td>179.20 / 179.20</td>
|
||||
<td>- / 260.99</td>
|
||||
<td>248</td>
|
||||
<td rowspan="3">PP-FormulaNet_plus is an enhanced version of the formula recognition model developed by the Baidu PaddlePaddle Vision Team, building upon the original PP-FormulaNet. Compared to the original version, PP-FormulaNet_plus utilizes a more diverse formula dataset during training, including sources such as Chinese dissertations, professional books, textbooks, exam papers, and mathematics journals. This expansion significantly improves the model’s recognition capabilities. Among the models, PP-FormulaNet_plus-M and PP-FormulaNet_plus-L have added support for Chinese formulas and increased the maximum number of predicted tokens for formulas from 1,024 to 2,560, greatly enhancing the recognition performance for complex formulas. Meanwhile, the PP-FormulaNet_plus-S model focuses on improving the recognition of English formulas. With these improvements, the PP-FormulaNet_plus series models perform exceptionally well in handling complex and diverse formula recognition tasks. </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-M_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-M_pretrained.pdparams">Training Model</a></td>
|
||||
<td>91.45</td>
|
||||
<td>89.76</td>
|
||||
<td>1040.27 / 1040.27</td>
|
||||
<td>- / 1615.80</td>
|
||||
<td>592</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>92.22</td>
|
||||
<td>90.64</td>
|
||||
<td>1476.07 / 1476.07</td>
|
||||
<td>- / 3125.58</td>
|
||||
<td>698</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LaTeX_OCR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/LaTeX_OCR_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/LaTeX_OCR_rec_pretrained.pdparams">Training Model</a></td>
|
||||
<td>74.55</td>
|
||||
<td>39.96</td>
|
||||
<td>1088.89 / 1088.89</td>
|
||||
<td>- / -</td>
|
||||
<td>99</td>
|
||||
<td>LaTeX-OCR is a formula recognition algorithm based on an autoregressive large model. It uses Hybrid ViT as the backbone network and a transformer as the decoder, significantly improving the accuracy of formula recognition.</td>
|
||||
</tr>
|
||||
</table>
|
||||
<strong>Test Environment Description:</strong>
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> PaddleOCR internal custom formula recognition test set</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technique Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Standard Mode</td>
|
||||
<td>FP32 precision / No TRT acceleration</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of predefined precision type and acceleration strategy</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>Optimal predefined backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## III. Quick Start
|
||||
|
||||
> ❗ Before getting started, please install the PaddleOCR wheel package. For details, refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly try it out with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr formula_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the formula recognition module into your own project.Before running the code below, please download the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import FormulaRecognition
|
||||
model = FormulaRecognition(model_name="PP-FormulaNet_plus-M")
|
||||
output = model.predict(input="general_formula_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
After running, the output is:
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_formula_rec_001.png', 'page_index': None, 'rec_formula': '\\zeta_{0}(\\nu)=-\\frac{\\nu\\varrho^{-2\\nu}}{\\pi}\\int_{\\mu}^{\\infty}d\\omega\\int_{C_{+}}d z\\frac{2z^{2}}{(z^{2}+\\omega^{2})^{\\nu+1}}\\breve{\\Psi}(\\omega;z)e^{i\\epsilon z}\\quad,'}}
|
||||
```
|
||||
Explanation of the result parameters:
|
||||
|
||||
- `input_path`： Indicates the path to the input formula image to be predicted
|
||||
- `page_index`： If the input is a PDF file, this represents the page number; otherwise, it is None
|
||||
- `rec_formula`：Indicates the predicted LaTeX source code of the formula image
|
||||
The visualization image is as follows. The left side is the input formula image, and the right side is the rendered formula from the prediction:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/formula_recog/general_formula_rec_001_res_paddleocr3.png">
|
||||
|
||||
<b>Note: If you need to visualize the formula recognition module, you must install the LaTeX rendering environment by running the following command. Currently, visualization is only supported on Ubuntu. Other environments are not supported for now. For complex formulas, the LaTeX result may contain advanced representations that may not render successfully in Markdown or similar environments:</b>
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install texlive texlive-latex-base texlive-xetex latex-cjk-all texlive-latex-extra -y
|
||||
```
|
||||
|
||||
Related methods and parameter descriptions are as follows:
|
||||
|
||||
* `FormulaRecognition` instantiates the formula recognition model (here using `PP-FormulaNet_plus-M` as an example), with detailed description as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-FormulaNet_plus-M</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b><code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the formula recognition model to perform inference, which returns a result list.
|
||||
Additionally, this module provides the `predict_iter()` method. Both accept the same parameters and return the same result format.
|
||||
The difference is that `predict_iter()` returns a `generator`, which can process and retrieve results step-by-step, suitable for large datasets or memory-efficient scenarios.
|
||||
|
||||
You can choose either method based on your actual needs. The `predict()` method takes parameters `input` and `batch_size`, described as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* The prediction results can be processed. Each result corresponds to a `Result` object, which supports printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Details</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the <code>JSON</code> output; only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. If set to <code>True</code>, all non-ASCII characters are escaped; if <code>False</code>, original characters are kept. Only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json-formatted file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path to save the file. If it is a directory, the saved file name will match the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the <code>JSON</code> output; only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether non-<code>ASCII</code> characters are escaped to <code>Unicode</code>. If set to <code>True</code>, all non-ASCII characters are escaped; if <code>False</code>, original characters are kept. Only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as an image file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path to save the file. If it is a directory, the saved file name will match the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, you can also access the visualized image and prediction result via attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
If the models above do not perform well in your scenario, you can try the following steps for custom development.
|
||||
Here we take training `PP-FormulaNet_plus-M` as an example. For other models, just replace the corresponding config file. First, you need to prepare a formula recognition dataset. You can follow the format of the [formula recognition demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar). Once the data is ready, follow the steps below to train and export the model. After export, the model can be quickly integrated into the API described above. This example uses the demo dataset. Before training the model, please ensure you have installed all PaddleOCR dependencies as described in the [installation documentation](../installation.en.md).
|
||||
|
||||
### 4.1 Environment Setup
|
||||
|
||||
To train the formula recognition model, you need to install additional Python and Linux dependencies. Run the following commands:
|
||||
|
||||
```shell
|
||||
sudo apt-get update
|
||||
sudo apt-get install libmagickwand-dev
|
||||
pip install tokenizers==0.19.1 imagesize ftfy Wand
|
||||
```
|
||||
|
||||
### 4.2 Dataset and Pretrained Model Preparation
|
||||
|
||||
#### 4.2.1 Prepare the Dataset
|
||||
```shell
|
||||
# Download the demo dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar
|
||||
tar -xf ocr_rec_latexocr_dataset_example.tar
|
||||
```
|
||||
|
||||
#### 4.2.2 Download the Pretrained Model
|
||||
```shell
|
||||
# Download the PP-FormulaNet_plus-M pre-trained model
|
||||
wget https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_plus_m_train.tar
|
||||
tar -xf rec_ppformulanet_plus_m_train.tar
|
||||
```
|
||||
|
||||
### 4.3 Model Training
|
||||
PaddleOCR is modularized. To train the `PP-FormulaNet_plus-M` model, you need to use its [config file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml).
|
||||
|
||||
Training commands are as follows:
|
||||
```bash
|
||||
# Single GPU training (default)
|
||||
python3 tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
|
||||
# Multi-GPU training, specify GPU IDs with --gpus
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
|
||||
```
|
||||
**Note:**
|
||||
|
||||
- By default, evaluation is performed every 1 epoch.If you change the batch size or dataset, modify the following accordingly:
|
||||
```bash
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.eval_batch_step=[0,{length_of_dataset//batch_size//4}] \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.4 Model Evaluation
|
||||
You can evaluate trained weights, e.g., output/xxx/xxx.pdparams, or use the downloaded [model](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_plus_m_train.tar ) with the following command:
|
||||
|
||||
```bash
|
||||
# Make sure pretrained_model is set to the local path.
|
||||
# For custom-trained models, modify the path and file name as {path/to/weights}/{model_name}
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.5 Model Export
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-FormulaNet_plus-M_infer/"
|
||||
```
|
||||
|
||||
After exporting, the static graph model will be saved in `./PP-FormulaNet_plus-M_infer/`, and you will see the following files:
|
||||
```
|
||||
./PP-FormulaNet_plus-M_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
At this point, the secondary development is complete. This static graph model can be directly integrated into the PaddleOCR API.
|
||||
|
||||
## V. FAQ
|
||||
|
||||
**Q1: Which formula recognition model does PaddleOCR recommend?**
|
||||
|
||||
A1: It is recommended to use the PP-FormulaNet series.
|
||||
If your scenario is mainly in English and inference speed is not a concern, use PP-FormulaNet-L or PP-FormulaNet_plus-L.
|
||||
For mainly Chinese use cases, use PP-FormulaNet_plus-L or PP-FormulaNet_plus-M.
|
||||
If your device has limited computing power and you are working with English formulas, use PP-FormulaNet-S.
|
||||
|
||||
**Q2: Why does the inference report an error?**
|
||||
A2: The formula recognition model depends heavily on Paddle 3.0 official release.
|
||||
Please ensure the correct version is installed.
|
||||
|
||||
**Q3: Why is there no visualization image after prediction?**
|
||||
A3: This may be due to LaTeX not being installed.You need to refer to Section III and install the LaTeX rendering tools.
|
||||
476
docs/version3.x/module_usage/formula_recognition.md
Normal file
476
docs/version3.x/module_usage/formula_recognition.md
Normal file
@@ -0,0 +1,476 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 公式识别模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
公式识别模块是OCR（光学字符识别）系统中的关键组成部分，负责将图像中的数学公式转换为可编辑的文本或计算机可识别的格式。该模块的性能直接影响到整个OCR系统的准确性和效率。公式识别模块通常会输出数学公式的 LaTeX 或 MathML 代码，这些代码将作为输入传递给文本理解模块进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>En-BLEU(%)</th>
|
||||
<th>Zh-BLEU(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>UniMERNet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UniMERNet_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UniMERNet_pretrained.pdparams">训练模型</a></td>
|
||||
<td>85.91</td>
|
||||
<td>43.50</td>
|
||||
<td>1311.84 / 1311.84</td>
|
||||
<td>- / 8288.07</td>
|
||||
<td>1530</td>
|
||||
<td>UniMERNet是由上海AI Lab研发的一款公式识别模型。该模型采用Donut Swin作为编码器，MBartDecoder作为解码器，并通过在包含简单公式、复杂公式、扫描捕捉公式和手写公式在内的一百万数据集上进行训练，大幅提升了模型对真实场景公式的识别准确率</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-S_pretrained.pdparams">训练模型</a></td>
|
||||
<td>87.00</td>
|
||||
<td>45.71</td>
|
||||
<td>182.25 / 182.25</td>
|
||||
<td>- / 254.39</td>
|
||||
<td>224</td>
|
||||
<td rowspan="2">PP-FormulaNet 是由百度飞桨视觉团队开发的一款先进的公式识别模型，支持5万个常见LateX源码词汇的识别。PP-FormulaNet-S 版本采用了 PP-HGNetV2-B4 作为其骨干网络，通过并行掩码和模型蒸馏等技术，大幅提升了模型的推理速度，同时保持了较高的识别精度，适用于简单印刷公式、跨行简单印刷公式等场景。而 PP-FormulaNet-L 版本则基于 Vary_VIT_B 作为骨干网络，并在大规模公式数据集上进行了深入训练，在复杂公式的识别方面，相较于PP-FormulaNet-S表现出显著的提升，适用于简单印刷公式、复杂印刷公式、手写公式等场景。 </td>
|
||||
|
||||
</tr>
|
||||
<td>PP-FormulaNet-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>90.36</td>
|
||||
<td>45.78</td>
|
||||
<td>1482.03 / 1482.03</td>
|
||||
<td>- / 3131.54</td>
|
||||
<td>695</td>
|
||||
</tr>
|
||||
<td>PP-FormulaNet_plus-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-S_pretrained.pdparams">训练模型</a></td>
|
||||
<td>88.71</td>
|
||||
<td>53.32</td>
|
||||
<td>179.20 / 179.20</td>
|
||||
<td>- / 260.99</td>
|
||||
<td>248</td>
|
||||
<td rowspan="3">PP-FormulaNet_plus 是百度飞桨视觉团队在 PP-FormulaNet 的基础上开发的增强版公式识别模型。与原版相比，PP-FormulaNet_plus 在训练中使用了更为丰富的公式数据集，包括中文学位论文、专业书籍、教材试卷以及数学期刊等多种来源。这一扩展显著提升了模型的识别能力。
|
||||
|
||||
其中，PP-FormulaNet_plus-M 和 PP-FormulaNet_plus-L 模型新增了对中文公式的支持，并将公式的最大预测 token 数从 1024 扩大至 2560，大幅提升了对复杂公式的识别性能。同时，PP-FormulaNet_plus-S 模型则专注于增强英文公式的识别能力。通过这些改进，PP-FormulaNet_plus 系列模型在处理复杂多样的公式识别任务时表现更加出色。 </td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-M_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-M_pretrained.pdparams">训练模型</a></td>
|
||||
<td>91.45</td>
|
||||
<td>89.76</td>
|
||||
<td>1040.27 / 1040.27</td>
|
||||
<td>- / 1615.80</td>
|
||||
<td>592</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-FormulaNet_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-FormulaNet_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-FormulaNet_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>92.22</td>
|
||||
<td>90.64</td>
|
||||
<td>1476.07 / 1476.07</td>
|
||||
<td>- / 3125.58</td>
|
||||
<td>698</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>LaTeX_OCR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/LaTeX_OCR_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/LaTeX_OCR_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>74.55</td>
|
||||
<td>39.96</td>
|
||||
<td>1088.89 / 1088.89</td>
|
||||
<td>- / -</td>
|
||||
<td>99</td>
|
||||
<td>LaTeX-OCR是一种基于自回归大模型的公式识别算法，通过采用 Hybrid ViT 作为骨干网络，transformer作为解码器，显著提升了公式识别的准确性。</td>
|
||||
</tr>
|
||||
</table>
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>PaddleOCR 内部自建公式识别测试集</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr formula_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将公式识别的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_formula_rec_001.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import FormulaRecognition
|
||||
model = FormulaRecognition(model_name="PP-FormulaNet_plus-M")
|
||||
output = model.predict(input="general_formula_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': '/root/.paddlex/predict_input/general_formula_rec_001.png', 'page_index': None, 'rec_formula': '\\zeta_{0}(\\nu)=-\\frac{\\nu\\varrho^{-2\\nu}}{\\pi}\\int_{\\mu}^{\\infty}d\\omega\\int_{C_{+}}d z\\frac{2z^{2}}{(z^{2}+\\omega^{2})^{\\nu+1}}\\breve{\\Psi}(\\omega;z)e^{i\\epsilon z}\\quad,'}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待预测公式图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `rec_formula`：表示公式图像的预测LaTeX源码
|
||||
|
||||
|
||||
可视化图片如下，左侧是待预测的公式图像，右边是预测的结果渲染后的公式图像：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/formula_recog/general_formula_rec_001_res_paddleocr3.png">
|
||||
|
||||
<b> 注：如果您需要对公式识别模块进行可视化，需要运行如下命令来对LaTeX渲染环境进行安装。目前公式识别模块可视化只支持Ubuntu环境，其他环境暂不支持。对于复杂公式，LaTeX 结果可能包含部分高级的表示，Markdown等环境中未必可以成功显示：</b>
|
||||
```bash
|
||||
sudo apt-get update
|
||||
sudo apt-get install texlive texlive-latex-base texlive-xetex latex-cjk-all texlive-latex-extra -y
|
||||
```
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `FormulaRecognition`实例化公式识别模型（此处以`PP-FormulaNet_plus-M`为例），具体说明如下：
|
||||
<table><thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr></thead><tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-FormulaNet_plus-M</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用公式识别模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
如果以上模型在您的场景下效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-FormulaNet-S` 举例，其他模型替换对应配置文件即可。首先，您需要准备公式识别的数据集，可以参考[公式识别 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述API中。此处以公式识别 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
### 4.1 环境配置
|
||||
|
||||
训练公式识别模型需要安装额外的Python依赖和linux依赖，执行如下命令安装：
|
||||
```shell
|
||||
sudo apt-get update
|
||||
sudo apt-get install libmagickwand-dev
|
||||
pip install tokenizers==0.19.1 imagesize ftfy Wand
|
||||
```
|
||||
|
||||
### 4.2 数据集、预训练模型准备
|
||||
|
||||
#### 4.2.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_latexocr_dataset_example.tar
|
||||
tar -xf ocr_rec_latexocr_dataset_example.tar
|
||||
```
|
||||
|
||||
#### 4.2.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-FormulaNet_plus-M 预训练模型
|
||||
wget https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_plus_m_train.tar
|
||||
tar -xf rec_ppformulanet_plus_m_train.tar
|
||||
```
|
||||
|
||||
### 4.3 模型训练
|
||||
|
||||
PaddleOCR对代码进行了模块化，训练 `PP-FormulaNet_plus-M` 识别模型时需要使用 `PP-FormulaNet_plus-M` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
**注意：**
|
||||
|
||||
- 默认每训练 1个 epoch 进行 1 次评估，若您更改训练的 batch_size，或更换数据集，请在训练时作出如下修改
|
||||
```bash
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml \
|
||||
-o Global.eval_batch_step=[0,{length_of_dataset//batch_size//4}] \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.4 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，也可以使用已经下载的[模型文件](https://paddleocr.bj.bcebos.com/contribution/rec_ppformulanet_s_train.tar)，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
|
||||
#注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
#demo 测试集评估
|
||||
python3 tools/eval.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams
|
||||
```
|
||||
|
||||
### 4.5 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-FormuaNet/PP-FormulaNet_plus-M.yaml -o \
|
||||
Global.pretrained_model=./rec_ppformulanet_plus_m_train/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-FormulaNet_plus-M_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-FormulaNet_plus-M_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-FormulaNet_plus-M_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
|
||||
**Q1: PaddleOCR 更推荐哪个公式识别模型？**
|
||||
|
||||
A1: 更推荐使用 PP-FormulaNet 系列模型，如果是英文场景居多且不考虑推理耗时，则可以使用 PP-FormulaNet-L 或者 PP-FormulaNet_plus-L 模型，如果中文场景居多，则可以使用 PP-FormulaNet_plus-L 或者 PP-FormulaNet_plus-M，如果推理设备算力有限且是英文场景，则可以使用 PP-FormulaNet-S。
|
||||
|
||||
**Q2: 为什么推理报错？**
|
||||
|
||||
A2: 公式识别模型的推理强依赖于 Paddle 框架 3.0 正式版，请确保版本一致。
|
||||
|
||||
|
||||
**Q3: 为什么预测后没有可视化图像？**
|
||||
|
||||
A3: 可能是因为没有安装LaTeX导致，您需要参考第三节安装LaTeX渲染工具。
|
||||
638
docs/version3.x/module_usage/layout_detection.en.md
Normal file
638
docs/version3.x/module_usage/layout_detection.en.md
Normal file
@@ -0,0 +1,638 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Layout Detection Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
The core task of structure analysis is to parse and segment the content of input document images. By identifying different elements in the image (such as text, charts, images, etc.), they are classified into predefined categories (e.g., pure text area, title area, table area, image area, list area, etc.), and the position and size of these regions in the document are determined.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
* <b>The layout detection model includes 20 common categories: document title, paragraph title, text, page number, abstract, table, references, footnotes, header, footer, algorithm, formula, formula number, image, table, seal, figure_table title, chart, and sidebar text and lists of references</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>83.2</td>
|
||||
<td>53.03 / 17.23</td>
|
||||
<td>634.62 / 378.32</td>
|
||||
<td>126.01</td>
|
||||
<td>A higher-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>The layout detection model includes 1 category: Block:</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocBlockLayout</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBlockLayout_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocBlockLayout_pretrained.pdparams">Training Model</a></td>
|
||||
<td>95.9</td>
|
||||
<td>34.60 / 28.54</td>
|
||||
<td>506.43 / 256.83</td>
|
||||
<td>123.92</td>
|
||||
<td>A layout block localization model trained on a self-built dataset containing Chinese and English papers, PPT, multi-layout magazines, contracts, books, exams, ancient books and research reports using RT-DETR-L</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>The layout detection model includes 23 common categories: document title, paragraph title, text, page number, abstract, table of contents, references, footnotes, header, footer, algorithm, formula, formula number, image, figure caption, table, table caption, seal, figure title, figure, header image, footer image, and sidebar text</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-L_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-L_pretrained.pdparams">Training Model</a></td>
|
||||
<td>90.4</td>
|
||||
<td>33.59 / 33.59</td>
|
||||
<td>503.01 / 251.08</td>
|
||||
<td>123.76</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using RT-DETR-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-M_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-M_pretrained.pdparams">Training Model</a></td>
|
||||
<td>75.2</td>
|
||||
<td>13.03 / 4.72</td>
|
||||
<td>43.39 / 24.44</td>
|
||||
<td>22.578</td>
|
||||
<td>A layout area localization model with balanced precision and efficiency, trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-S_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-S_pretrained.pdparams">Training Model</a></td>
|
||||
<td>70.9</td>
|
||||
<td>11.54 / 3.86</td>
|
||||
<td>18.53 / 6.29</td>
|
||||
<td>4.834</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset containing Chinese and English papers, magazines, contracts, books, exams, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ The above list includes the <b>4 core models</b> that are key supported by the text recognition module. The module actually supports a total of <b>12 full models</b>, including several predefined models with different categories. The complete model list is as follows:
|
||||
|
||||
<details><summary> 👉 Details of Model List</summary>
|
||||
|
||||
* <b>Table Layout Detection Model</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x_table</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_table_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_table_pretrained.pdparams">Training Model</a></td>
|
||||
<td>97.5</td>
|
||||
<td>9.57 / 6.63</td>
|
||||
<td>27.66 / 16.75</td>
|
||||
<td>7.4</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset using PicoDet-1x, capable of detecting table regions.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>3-Class Layout Detection Model, including Table, Image, and Stamp</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>88.2</td>
|
||||
<td>8.43 / 3.44</td>
|
||||
<td>17.60 / 6.51</td>
|
||||
<td>4.8</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.80 / 9.57</td>
|
||||
<td>45.04 / 23.86</td>
|
||||
<td>22.6</td>
|
||||
<td>A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_3cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_3cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>95.8</td>
|
||||
<td>114.80 / 25.65</td>
|
||||
<td>924.38 / 924.38</td>
|
||||
<td>470.1</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>5-Class English Document Area Detection Model, including Text, Title, Table, Image, and List</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_pretrained.pdparams">Training Model</a></td>
|
||||
<td>97.8</td>
|
||||
<td>9.62 / 6.75</td>
|
||||
<td>26.96 / 12.77</td>
|
||||
<td>7.4</td>
|
||||
<td>A high-efficiency English document layout area localization model trained on the PubLayNet dataset using PicoDet-1x.</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>17-Class Area Detection Model, including 17 common layout categories: Paragraph Title, Image, Text, Number, Abstract, Content, Figure Caption, Formula, Table, Table Caption, References, Document Title, Footnote, Header, Algorithm, Footer, and Stamp</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(0.5) (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>87.4</td>
|
||||
<td>8.80 / 3.62</td>
|
||||
<td>17.51 / 6.35</td>
|
||||
<td>4.8</td>
|
||||
<td>A high-efficiency layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-S.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.60 / 10.27</td>
|
||||
<td>43.70 / 24.42</td>
|
||||
<td>22.6</td>
|
||||
<td>A balanced efficiency and precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using PicoDet-L.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_17cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_17cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.3</td>
|
||||
<td>115.29 / 101.18</td>
|
||||
<td>964.75 / 964.75</td>
|
||||
<td>470.2</td>
|
||||
<td>A high-precision layout area localization model trained on a self-built dataset of Chinese and English papers, magazines, and research reports using RT-DETR-H.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong>
|
||||
<ul>
|
||||
<li>20 types of layout detection models: PaddleOCR's self built layout area detection dataset, including Chinese and English papers, magazines, newspapers, research papers PPT、 1300 images of document types such as test papers and textbooks. </li>
|
||||
<li>Type 1 version face region detection model: PaddleOCR's self built version face region detection dataset, including Chinese and English papers, magazines, newspapers, research reports PPT、 1000 document type images such as test papers and textbooks. </li>
|
||||
<li>23 categories Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, containing 500 common document type images such as Chinese and English papers, magazines, contracts, books, exam papers, and research reports.</li>
|
||||
<li>Table Layout Detection Model: A self-built table area detection dataset by PaddleOCR, including 7,835 Chinese and English paper document type images with tables.</li>
|
||||
<li> 3-Class Layout Detection Model: A self-built layout area detection dataset by PaddleOCR, comprising 1,154 common document type images such as Chinese and English papers, magazines, and research reports.</li>
|
||||
<li>5-Class English Document Area Detection Model: The evaluation dataset of <a href="https://developer.ibm.com/exchanges/data/all/publaynet">PubLayNet</a>, containing 11,245 images of English documents.</li>
|
||||
<li>17-Class Area Detection Model: A self-built layout area detection dataset by PaddleOCR, including 892 common document type images such as Chinese and English papers, magazines, and research reports.</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
## III. Quick Integration <a id="quick"> </a>
|
||||
|
||||
> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../installation.en.md)。
|
||||
|
||||
Quickly experience with just one command:
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg) Go to the local area.
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayout_plus-L")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}}
|
||||
```
|
||||
|
||||
The meanings of the parameters are as follows:
|
||||
- `input_path`: The path to the input image for prediction.
|
||||
- `page_index`: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is `None`.
|
||||
- `boxes`: Information about the predicted bounding boxes, a list of dictionaries. Each dictionary represents a detected object and contains the following information:
|
||||
- `cls_id`: Class ID, an integer.
|
||||
- `label`: Class label, a string.
|
||||
- `score`: Confidence score of the bounding box, a float.
|
||||
- `coordinate`: Coordinates of the bounding box, a list of floats in the format <code>[xmin, ymin, xmax, ymax]</code>.
|
||||
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/layout_det/layout_res_plus.jpg"/>
|
||||
|
||||
Relevant methods, parameters, and explanations are as follows:
|
||||
|
||||
* `LayoutDetection` instantiates a target detection model (here, `PP-DocLayout_plus-L` is used as an example). The detailed explanation is as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-DocLayout-L</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td>Input image size.<ul>
|
||||
<li><b>int</b>: e.g. <code>640</code>, resizes input image to 640x640.</li>
|
||||
<li><b>list</b>: e.g. <code>[640, 512]</code>, resizes input image to width 640 and height 512.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Threshold for filtering low-confidence predictions.<ul>
|
||||
<li><b>float</b>: e.g. <code>0.2</code>, filters out all boxes with confidence below 0.2.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>float</code> (threshold). For example, <code>{0: 0.45, 2: 0.48, 7: 0.4}</code> means class 0 uses threshold 0.45, class 2 uses 0.48, class 7 uses 0.4.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>Whether to use NMS post-processing to filter overlapping boxes.
|
||||
<ul>
|
||||
<li><b>bool</b>: whether to use NMS for post-processing to filter overlapping boxes.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>Scaling factor for the side length of the detection box.<ul>
|
||||
<li><b>float</b>: A float greater than 0, e.g. <code>1.1</code>, expands width and height by 1.1 times.</li>
|
||||
<li><b>list</b>: e.g. <code>[1.2, 1.5]</code>, expands width by 1.2x and height by 1.5x.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>tuple</code> of two floats (width ratio, height ratio). For example, <code>{0: (1.1, 2.0)}</code> means for class 0, width is expanded by 1.1x and height by 2.0x.</li>
|
||||
<li><b>None</b>: uses the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>Merge mode for model output bounding boxes.<ul>
|
||||
<li><b>"large"</b>: Only keep the largest outer box among overlapping boxes, remove inner boxes.</li>
|
||||
<li><b>"small"</b>: Only keep the smallest inner box among overlapping boxes, remove outer boxes.</li>
|
||||
<li><b>"union"</b>: Keep all boxes, no filtering.</li>
|
||||
<li><b>dict</b>: The key is <code>int</code> (class id), the value is <code>str</code> (mode). For example, <code>{0: "large", 2: "small"}</code> means class 0 uses "large" mode, class 2 uses "small" mode.</li>
|
||||
<li><b>None</b>: Use the model's default configuration.</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The `predict()` method of the target detection model is called for inference prediction. The parameters of the `predict()` method are `input`, `batch_size`, and `threshold`, which are explained as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Process the prediction results, with each sample's prediction result being the corresponding Result object, and supporting operations such as printing, saving as an image, and saving as a 'json' file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameters</th>
|
||||
<th>Parameter type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Do you want to use <code>JSON</code> indentation formatting for the output content</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non ASCII </code>characters will be escaped; <code>False</code> preserves the original characters and is only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a JSON format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to enhance the readability of the <code>JSON</code> data output, only valid when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non ASCII characters to Unicode characters. When set to <code>True</code>, all non <code>ASCII</code> characters will be escaped; <code>False</code> preserves the original characters and is only valid when<code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the results as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The saved file path, when it is a directory, the name of the saved file is consistent with the name of the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
* Additionally, it also supports obtaining the visualized image with results and the prediction results via attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the layout detection module, if you need to train the layout area detection model, you can refer to [PaddleX Layout Detection Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/layout_detection.html#iv-custom-development)Partially conduct training. The trained model can be seamlessly integrated into PaddleOCR's API for inference.
|
||||
|
||||
## V. FAQ
|
||||
641
docs/version3.x/module_usage/layout_detection.md
Normal file
641
docs/version3.x/module_usage/layout_detection.md
Normal file
@@ -0,0 +1,641 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 版面区域检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
版面区域检测任务的核心是对输入的文档图像进行内容解析和区域划分。通过识别图像中的不同元素（如文字、图表、图像、公式、段落、摘要、参考文献等），将其归类为预定义的类别，并确定这些区域在文档中的位置。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
* <b>版面检测模型，包含20个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、表格、图和表标题（图标题、表格标题和图表标题）、印章、图表、侧栏文本和参考文献内容</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout_plus-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout_plus-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout_plus-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>83.2</td>
|
||||
<td>53.03 / 17.23</td>
|
||||
<td>634.62 / 378.32</td>
|
||||
<td>126.01</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的更高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。</b>
|
||||
|
||||
* <b>文档图像版面子模块检测，包含1个 版面区域 类别，能检测多栏的报纸、杂志的每个子文章的文本区域：</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocBlockLayout</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocBlockLayout_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocBlockLayout_pretrained.pdparams">训练模型</a></td>
|
||||
<td>95.9</td>
|
||||
<td>34.60 / 28.54</td>
|
||||
<td>506.43 / 256.83</td>
|
||||
<td>123.92</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、多栏杂志、报纸、PPT、合同、书本、试卷、研报、古籍、日文文档、竖版文字文档等场景的自建数据集训练的文档图像版面子模块检测模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。</b>
|
||||
|
||||
* <b>版面检测模型，包含23个常见的类别：文档标题、段落标题、文本、页码、摘要、目录、参考文献、脚注、页眉、页脚、算法、公式、公式编号、图像、图表标题、表格、表格标题、印章、图表标题、图表、页眉图像、页脚图像、侧栏文本</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-DocLayout-L</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-L_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-L_pretrained.pdparams">训练模型</a></td>
|
||||
<td>90.4</td>
|
||||
<td>33.59 / 33.59</td>
|
||||
<td>503.01 / 251.08</td>
|
||||
<td>123.76</td>
|
||||
<td>基于RT-DETR-L在包含中英文论文、杂志、合同、书本、试卷和研报等场景的自建数据集训练的高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-M</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-M_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-M_pretrained.pdparams">训练模型</a></td>
|
||||
<td>75.2</td>
|
||||
<td>13.03 / 4.72</td>
|
||||
<td>43.39 / 24.44</td>
|
||||
<td>22.578</td>
|
||||
<td>基于PicoDet-L在包含中英文论文、杂志、合同、书本、试卷和研报等场景的自建数据集训练的精度效率平衡的版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-DocLayout-S</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-DocLayout-S_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-DocLayout-S_pretrained.pdparams">训练模型</a></td>
|
||||
<td>70.9</td>
|
||||
<td>11.54 / 3.86</td>
|
||||
<td>18.53 / 6.29</td>
|
||||
<td>4.834</td>
|
||||
<td>基于PicoDet-S在中英文论文、杂志、合同、书本、试卷和研报等场景上自建数据集训练的高效率版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<b>注：以上精度指标的评估集是自建的版面区域检测数据集，包含中英文论文、报纸、研报和试卷等 500 张文档类型图片。</b>
|
||||
|
||||
> ❗ 以上列出的是版面检测模块重点支持的<b>5个核心模型</b>，该模块总共支持<b>13个全量模型</b>，包含多个预定义了不同类别的模型，完整的模型列表如下：
|
||||
|
||||
<details><summary> 👉模型列表详情</summary>
|
||||
|
||||
* <b>表格版面检测模型</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x_table</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_table_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_table_pretrained.pdparams">训练模型</a></td>
|
||||
<td>97.5</td>
|
||||
<td>9.57 / 6.63</td>
|
||||
<td>27.66 / 16.75</td>
|
||||
<td>7.4</td>
|
||||
<td>基于PicoDet-1x在自建数据集训练的高效率版面区域定位模型，可定位表格这1类区域</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>3类版面检测模型，包含表格、图像、印章</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>88.2</td>
|
||||
<td>8.43 / 3.44</td>
|
||||
<td>17.60 / 6.51</td>
|
||||
<td>4.8</td>
|
||||
<td>基于PicoDet-S轻量模型在中英文论文、杂志和研报等场景上自建数据集训练的高效率版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.80 / 9.57</td>
|
||||
<td>45.04 / 23.86</td>
|
||||
<td>22.6</td>
|
||||
<td>基于PicoDet-L在中英文论文、杂志和研报等场景上自建数据集训练的效率精度均衡版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_3cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_3cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_3cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>95.8</td>
|
||||
<td>114.80 / 25.65</td>
|
||||
<td>924.38 / 924.38</td>
|
||||
<td>470.1</td>
|
||||
<td>基于RT-DETR-H在中英文论文、杂志和研报等场景上自建数据集训练的高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>5类英文文档区域检测模型，包含文字、标题、表格、图片以及列表</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet_layout_1x</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet_layout_1x_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet_layout_1x_pretrained.pdparams">训练模型</a></td>
|
||||
<td>97.8</td>
|
||||
<td>9.62 / 6.75</td>
|
||||
<td>26.96 / 12.77</td>
|
||||
<td>7.4</td>
|
||||
<td>基于PicoDet-1x在PubLayNet数据集训练的高效率英文文档版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody></table>
|
||||
|
||||
* <b>17类区域检测模型，包含17个版面常见类别，分别是：段落标题、图片、文本、数字、摘要、内容、图表标题、公式、表格、表格标题、参考文献、文档标题、脚注、页眉、算法、页脚、印章</b>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(0.5)（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PicoDet-S_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-S_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-S_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>87.4</td>
|
||||
<td>8.80 / 3.62</td>
|
||||
<td>17.51 / 6.35</td>
|
||||
<td>4.8</td>
|
||||
<td>基于PicoDet-S轻量模型在中英文论文、杂志和研报等场景上自建数据集训练的高效率版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PicoDet-L_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PicoDet-L_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PicoDet-L_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>89.0</td>
|
||||
<td>12.60 / 10.27</td>
|
||||
<td>43.70 / 24.42</td>
|
||||
<td>22.6</td>
|
||||
<td>基于PicoDet-L在中英文论文、杂志和研报等场景上自建数据集训练的效率精度均衡版面区域定位模型</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-H_layout_17cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-H_layout_17cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-H_layout_17cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>98.3</td>
|
||||
<td>115.29 / 101.18</td>
|
||||
<td>964.75 / 964.75</td>
|
||||
<td>470.2</td>
|
||||
<td>基于RT-DETR-H在中英文论文、杂志和研报等场景上自建数据集训练的高精度版面区域定位模型</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>
|
||||
<ul>
|
||||
<li>20类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1300 张文档类型图片。</li>
|
||||
<li>1类版面子区域检测模型： PaddleOCR 自建的版面子区域检测数据集，包含中英文论文、杂志、报纸、研报、PPT、试卷、课本等 1000 张文档类型图片。</li>
|
||||
<li>23类版面检测模型： PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志、合同、书本、试卷和研报等常见的 500 张文档类型图片。</li>
|
||||
<li>3类版面检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 1154 张文档类型图片。</li>
|
||||
<li>5类英文文档区域检测模型： <a href="https://developer.ibm.com/exchanges/data/all/publaynet" target="_blank">PubLayNet</a> 的评估数据集，包含英文文档的 11245 张图片。</li>
|
||||
<li>17类区域检测模型：PaddleOCR 自建的版面区域检测数据集，包含中英文论文、杂志和研报等常见的 892 张文档类型图片。</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</details>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr layout_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将版面区域检测模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/layout.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import LayoutDetection
|
||||
|
||||
model = LayoutDetection(model_name="PP-DocLayout_plus-L")
|
||||
output = model.predict("layout.jpg", batch_size=1, layout_nms=True)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'layout.jpg', 'page_index': None, 'boxes': [{'cls_id': 2, 'label': 'text', 'score': 0.9870226979255676, 'coordinate': [34.101906, 349.85275, 358.59213, 611.0772]}, {'cls_id': 2, 'label': 'text', 'score': 0.9866003394126892, 'coordinate': [34.500324, 647.1585, 358.29367, 848.66797]}, {'cls_id': 2, 'label': 'text', 'score': 0.9846674203872681, 'coordinate': [385.71445, 497.40973, 711.2261, 697.84265]}, {'cls_id': 8, 'label': 'table', 'score': 0.984126091003418, 'coordinate': [73.76879, 105.94899, 321.95303, 298.84888]}, {'cls_id': 8, 'label': 'table', 'score': 0.9834211468696594, 'coordinate': [436.95642, 105.81531, 662.7168, 313.48462]}, {'cls_id': 2, 'label': 'text', 'score': 0.9832247495651245, 'coordinate': [385.62787, 346.2288, 710.10095, 458.77127]}, {'cls_id': 2, 'label': 'text', 'score': 0.9816061854362488, 'coordinate': [385.7802, 735.1931, 710.56134, 849.9764]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9577341079711914, 'coordinate': [34.421448, 20.055151, 358.71283, 76.53663]}, {'cls_id': 6, 'label': 'figure_title', 'score': 0.9505634307861328, 'coordinate': [385.72278, 20.053688, 711.29333, 74.92744]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.9001723527908325, 'coordinate': [386.46344, 477.03488, 699.4023, 490.07474]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8845751285552979, 'coordinate': [35.413048, 627.73596, 185.58383, 640.52264]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8837394118309021, 'coordinate': [387.17603, 716.3423, 524.7841, 729.258]}, {'cls_id': 0, 'label': 'paragraph_title', 'score': 0.8508939743041992, 'coordinate': [35.50064, 331.18445, 141.6444, 344.81097]}]}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
- `input_path`：输入的待预测图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `boxes`：预测的目标框信息，一个字典列表。每个字典代表一个检出的目标，包含以下信息：
|
||||
- `cls_id`：类别ID，一个整数
|
||||
- `label`：类别标签，一个字符串
|
||||
- `score`：目标框置信度，一个浮点数
|
||||
- `coordinate`：目标框坐标，一个浮点数列表，格式为<code>[xmin, ymin, xmax, ymax]</code>
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/layout_det/layout_res_plus.jpg"/>
|
||||
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `LayoutDetection`实例化目标检测模型（此处以`PP-DocLayout_plus-L`为例），具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-DocLayout-L</code></td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td>输入图像大小。
|
||||
<ul>
|
||||
<li><b>int</b>：如<code>640</code>，表示将输入图像resize到640x640大小。</li>
|
||||
<li><b>list</b>：如<code>[640, 512]</code>，表示将输入图像resize到宽为640、高为512。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>用于过滤掉低置信度预测结果的阈值。
|
||||
<ul>
|
||||
<li><b>float</b>：如<code>0.2</code>，表示过滤掉所有阈值小于0.2的目标框。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>float</code>类型阈值。如<code>{0: 0.45, 2: 0.48, 7: 0.4}</code>，表示对ID为0的类别应用阈值0.45、ID为1的类别应用阈值0.48、ID为7的类别应用阈值0.4。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>是否使用NMS后处理，过滤重叠框。
|
||||
<ul>
|
||||
<li><b>bool</b>表示使用/不使用NMS进行检测框的后处理过滤重叠框。</li>
|
||||
<li><b>None</b>使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>检测框的边长缩放倍数。</b>
|
||||
<ul>
|
||||
<li><b>float</b>：大于0的浮点数，如<code>1.1</code>，表示将模型输出的检测框中心不变，宽和高都扩张1.1倍。</li>
|
||||
<li><b>list</b>：如<code>[1.2, 1.5]</code>，表示将模型输出的检测框中心不变，宽度扩张1.2倍，高度扩张1.5倍。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>tuple</code>类型，如<code>{0: (1.1, 2.0)}</code>，表示将模型输出的第0类别检测框中心不变，宽度扩张1.1倍，高度扩张2.0倍。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>模型输出的检测框的合并处理模式。
|
||||
<ul>
|
||||
<li><b>"large"</b>：设置为<code>"large"</code>，表示在模型输出的检测框中，对于互相重叠包含的检测框，只保留外部最大的框，删除重叠的内部框。</li>
|
||||
<li><b>"small"</b>：设置为<code>"small"</code>，表示在模型输出的检测框中，对于互相重叠包含的检测框，只保留内部被包含的小框，删除重叠的外部框。</li>
|
||||
<li><b>"union"</b>：不进行框的过滤处理，内外框都保留。</li>
|
||||
<li><b>dict</b>：字典的键为<code>int</code>类型，代表类别ID；值为<code>str</code>类型, 如<code>{0: "large", 2: "small"}</code>, 表示对第0类别检测框使用<code>large</code>模式，对第2类别检测框使用<code>small</code>。</li>
|
||||
<li><b>None</b>：使用模型默认的配置。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* 调用目标检测模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input`、`batch_size`和`threshold`，具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_nms</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</b>
|
||||
<td><code>bool|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_unclip_ratio</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。
|
||||
<td><code>float|list|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>layout_merge_bboxes_mode</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。
|
||||
</td>
|
||||
<td><code>str|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供版面区域检测模块的训练，因此，如果需要训练版面区域测模型，可以参考 [PaddleX 版面区域检测模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/layout_detection.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 五、FAQ
|
||||
3
docs/version3.x/module_usage/module_overview.en.md
Normal file
3
docs/version3.x/module_usage/module_overview.en.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# Module Overview
|
||||
|
||||
A module is the smallest unit that implements basic functionality. Modules typically use a single model to accomplish specific tasks, such as text detection, image classification, and other basic functions. As fundamental building blocks, modules provide the necessary functional support for more complex application scenarios. This design approach allows users to flexibly select and combine different modules according to their needs, thereby simplifying the development process and enhancing development flexibility and efficiency.
|
||||
3
docs/version3.x/module_usage/module_overview.md
Normal file
3
docs/version3.x/module_usage/module_overview.md
Normal file
@@ -0,0 +1,3 @@
|
||||
# 模块概述
|
||||
|
||||
模块是实现基本功能的最小单位。模块通常使用单个模型去完成特定的任务，比如文本检测、图像分类等基本功能。模块作为基础构建单元，为更复杂的应用场景提供了必要的功能支持。这种设计方式使得用户可以根据需要灵活选择和组合不同的模块，从而简化了开发流程，并提高了开发的灵活性和效率。
|
||||
469
docs/version3.x/module_usage/seal_text_detection.en.md
Normal file
469
docs/version3.x/module_usage/seal_text_detection.en.md
Normal file
@@ -0,0 +1,469 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Seal Text Detection Module Tutorial
|
||||
|
||||
## I. Overview
|
||||
The seal text detection module typically outputs multi-point bounding boxes around text regions, which are then passed as inputs to the distortion correction and text recognition modules for subsequent processing to identify the textual content of the seal. Recognizing seal text is an integral part of document processing and finds applications in various scenarios such as contract comparison, inventory access auditing, and invoice reimbursement verification. The seal text detection module serves as a subtask within OCR (Optical Character Recognition), responsible for locating and marking the regions containing seal text within an image. The performance of this module directly impacts the accuracy and efficiency of the entire seal text OCR system.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model Name</th><th>Model Download Link</th>
|
||||
<th>Hmean（%）</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_seal_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.40</td>
|
||||
<td>124.64 / 91.57</td>
|
||||
<td>545.68 / 439.86</td>
|
||||
<td>109</td>
|
||||
<td>The server-side seal text detection model of PP-OCRv4 boasts higher accuracy and is suitable for deployment on better-equipped servers.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_seal_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_seal_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>96.36</td>
|
||||
<td>9.70 / 3.56</td>
|
||||
<td>50.38 / 19.64</td>
|
||||
<td>4.6</td>
|
||||
<td>The mobile-side seal text detection model of PP-OCRv4, on the other hand, offers greater efficiency and is suitable for deployment on end devices.</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong> A Self-built Internal Dataset, Containing 500 Images of Circular Stamps.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## III. Quick Integration <a id="quick"> </a>
|
||||
|
||||
> ❗ Before quick integration, please install the PaddleOCR wheel package. For detailed instructions, refer to [PaddleOCR Local Installation Tutorial](../installation.en.md)。
|
||||
|
||||
Quickly experience with just one command:
|
||||
|
||||
```bash
|
||||
paddleocr seal_text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the layout area detection module into your project. Before running the following code, please download [Example Image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png) Go to the local area.
|
||||
|
||||
```python
|
||||
from paddleocr import SealTextDetection
|
||||
model = SealTextDetection(model_name="PP-OCRv4_server_seal_det")
|
||||
output = model.predict("seal_text_det.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'seal_text_det.png', 'page_index': None, 'dt_polys': [array([[463, 477],
|
||||
...,
|
||||
[428, 505]]), array([[297, 444],
|
||||
...,
|
||||
[230, 443]]), array([[457, 346],
|
||||
...,
|
||||
[267, 345]]), array([[325, 38],
|
||||
...,
|
||||
[322, 37]])], 'dt_scores': [0.9912680344777314, 0.9906849624837963, 0.9847219455533163, 0.9914791724153904]}}
|
||||
```
|
||||
|
||||
The meanings of the parameters are as follows:
|
||||
- `input_path`: represents the path of the input image to be predicted
|
||||
- `dt_polys`: represents the predicted text detection boxes, where each text detection box contains multiple vertices of a polygon. Each vertex is a list of two elements, representing the x and y coordinates of the vertex respectively
|
||||
- `dt_scores`: represents the confidence scores of the predicted text detection boxes
|
||||
|
||||
The visualization image is as follows:
|
||||
|
||||
<img alt="Visualization Image" src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/seal_text_det/seal_text_det_res.png"/>
|
||||
|
||||
The explanations of related methods and parameters are as follows:
|
||||
|
||||
* `SealTextDetection` instantiates a text detection model (here we take `PP-OCRv4_server_seal_det` as an example), and the specific explanations are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-OCRv4_mobile_seal_det</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Limit on the side length of the input image for detection. <code>int</code> specifies the value. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Type of image side length limitation. <code>"min"</code> ensures the shortest side of the image is no less than <code>det_limit_side_len</code>; <code>"max"</code> ensures the longest side is no greater than <code>limit_side_len</code>. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>Pixel score threshold. Pixels in the output probability map with scores greater than this threshold are considered text pixels. Accepts any float value greater than 0. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>If the average score of all pixels inside the bounding box is greater than this threshold, the result is considered a text region. Accepts any float value greater than 0. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Expansion ratio for the Vatti clipping algorithm, used to expand the text region. Accepts any float value greater than 0. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>Input image size for the model in the format <code>(C, H, W)</code>. If set to <code>None</code>, the model's default size will be used.</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The `predict()` method of the seal text detection model is called for inference prediction. The parameters of the `predict()` method include `input`, `batch_size`, `limit_side_len`, `limit_type`, `thresh`, `box_thresh`, `max_candidates`, `unclip_ratio`. The specific descriptions are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, can be set to any positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Process the prediction results. Each sample's prediction result is a corresponding Result object, and it supports operations such as printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a file in JSON format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path for saving. When it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. This is only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as a file in image format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path for saving. When it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining visual images with results and prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Attribute Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visual image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## IV. Custom Development
|
||||
|
||||
If the above model is still not performing well in your scenario, you can try the following steps for secondary development. Here, we'll use training `PP-OCRv4_server_seal_det` as an example; you can replace it with the corresponding configuration files for other models. First, you need to prepare a text detection dataset. You can refer to the format of the [seal text detection demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar) for preparation. Once prepared, you can follow the steps below for model training and export. After export, you can quickly integrate the model into the above API. This example uses a seal text detection demo dataset. Before training the model, please ensure that you have installed the dependencies required by PaddleOCR as per the [installation documentation](../installation.en.md).
|
||||
|
||||
### 4.1 Dataset and Pre-trained Model Preparation
|
||||
|
||||
#### 4.1.1 Preparing the Dataset
|
||||
|
||||
```shell
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset
|
||||
tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/
|
||||
```
|
||||
|
||||
#### 4.1.1 Preparing the pre-trained model
|
||||
|
||||
|
||||
```shell
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR has modularized the code, and when training the `PP-OCRv4_server_seal_det` model, you need to use the [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml) for `PP-OCRv4_server_seal_det`.
|
||||
|
||||
The training commands are as follows:
|
||||
|
||||
```bash
|
||||
# Single GPU training (default training method)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
|
||||
# Multi-GPU training, specify GPU ids using the --gpus parameter
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command:
|
||||
|
||||
```bash
|
||||
# Make sure to set the pretrained_model path to the local path. If using a model that was trained and saved by yourself, be sure to modify the path and filename to {path/to/weights}/{model_name}.
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
save_inference_dir="./PP-OCRv4_server_seal_det_infer/"
|
||||
```
|
||||
|
||||
After exporting the model, the static graph model will be stored in the `./PP-OCRv4_server_seal_det_infer/` directory. In this directory, you will see the following files:
|
||||
```
|
||||
./PP-OCRv4_server_seal_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
With this, the secondary development is complete, and the static graph model can be directly integrated into PaddleOCR's API.
|
||||
|
||||
## 5. FAQ
|
||||
468
docs/version3.x/module_usage/seal_text_detection.md
Normal file
468
docs/version3.x/module_usage/seal_text_detection.md
Normal file
@@ -0,0 +1,468 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 印章文本检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
印章文本检测模块通常会输出文本区域的多点边界框（Bounding Boxes），这些边界框将作为输入传递给弯曲矫正和文本检测模块进行后续处理，识别出印章的文字内容。印章文本的识别是文档处理的一部分，在很多场景都有用途，例如合同比对，出入库审核以及发票报销审核等场景。印章文本检测模块是OCR（光学字符识别）中的子任务，负责在图像中定位和标记出包含印章文本的区域。该模块的性能直接影响到整个印章文本OCR系统的准确性和效率。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>检测Hmean（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_seal_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>98.40</td>
|
||||
<td>124.64 / 91.57</td>
|
||||
<td>545.68 / 439.86</td>
|
||||
<td>109</td>
|
||||
<td>PP-OCRv4的服务端印章文本检测模型，精度更高，适合在较好的服务器上部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_seal_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_seal_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_seal_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>96.36</td>
|
||||
<td>9.70 / 3.56</td>
|
||||
<td>50.38 / 19.64</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv4的移动端印章文本检测模型，效率更高，适合在端侧部署</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建的内部数据集，包含500张圆形印章图像。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr seal_text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将印章文本检测的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/seal_text_det.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import SealTextDetection
|
||||
model = SealTextDetection(model_name="PP-OCRv4_server_seal_det")
|
||||
output = model.predict("seal_text_det.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'seal_text_det.png', 'page_index': None, 'dt_polys': [array([[463, 477],
|
||||
...,
|
||||
[428, 505]]), array([[297, 444],
|
||||
...,
|
||||
[230, 443]]), array([[457, 346],
|
||||
...,
|
||||
[267, 345]]), array([[325, 38],
|
||||
...,
|
||||
[322, 37]])], 'dt_scores': [0.9912680344777314, 0.9906849624837963, 0.9847219455533163, 0.9914791724153904]}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待预测图像的路径
|
||||
- `dt_polys`：表示预测的文本检测框，其中每个文本检测框包含一个多边形的多个顶点。其中每个顶点都是一个列表，分别表示该顶点的x坐标和y坐标
|
||||
- `dt_scores`：表示预测的文本检测框的置信度
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/seal_text_det/seal_text_det_res.png"/>
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `SealTextDetection`实例化文本检测模型（此处以`PP-OCRv4_server_seal_det`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。模型名称。如果设置为<code>None</code>，则使用<code>PP-OCRv4_mobile_seal_det</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>检测的图像边长限制：<code>int</code> 表示边长限制数值，如果设置为<code>None</code>, 将使用模型默认配置。</td>
|
||||
<td><code>int|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>检测的图像边长限制，检测的边长限制类型，<code>"min"</code> 表示保证图像最短边不小于 det_limit_side_len，<code>"max"</code> 表示保证图像最长边不大于 <code>limit_side_len</code>。如果设置为 <code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>像素得分阈值。输出概率图中得分大于该阈值的像素点被认为是文本像素。可选大于0的float任意浮点数。如果设置为<code>None</code>。将使用模型默认配置。</td>
|
||||
<td><code>float|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>检测结果边框内，所有像素点的平均得分大于该阈值时，该结果会被认为是文字区域。可选大于0的float任意浮点数。如果设置为<code>None</code>, 将使用模型默认配置。</td>
|
||||
<td><code>float|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Vatti clipping算法的扩张系数，使用该方法对文字区域进行扩张。可选大于0的任意浮点数。如果设置为<code>None</code>, 将使用模型默认配置。</td>
|
||||
<td><code>float|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>模型输入图像尺寸，格式为 <code>(C, H, W)</code>。若为 <code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>tuple|None</td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用印章文本检测模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input`、 `batch_size`、 `limit_side_len`、 `limit_type`、 `thresh`、 `box_thresh`、 `max_candidates`、`unclip_ratio`，具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-OCRv4_server_seal_det` 举例，其他模型替换对应配置文件即可。首先，您需要准备文本检测的数据集，可以参考[印章文本检测 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以印章文本检测 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
### 4.1 数据集、预训练模型准备
|
||||
|
||||
#### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_curve_det_dataset_examples.tar -P ./dataset
|
||||
tar -xf ./dataset/ocr_curve_det_dataset_examples.tar -C ./dataset/
|
||||
```
|
||||
|
||||
#### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-OCRv4_server_seal_det 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `PP-OCRv4_server_seal_det` 模型时需要使用 `PP-OCRv4_server_seal_det` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv4_server_seal_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv4/PP-OCRv4_server_seal_det.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
save_inference_dir="./PP-OCRv4_server_seal_det_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-OCRv4_server_seal_det_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-OCRv4_server_seal_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
343
docs/version3.x/module_usage/table_cells_detection.en.md
Normal file
343
docs/version3.x/module_usage/table_cells_detection.en.md
Normal file
@@ -0,0 +1,343 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
# Table Cell Detection Module Usage Tutorial
|
||||
|
||||
## I. Overview
|
||||
|
||||
The Table Cell Detection Module is a key component of the table recognition task, responsible for locating and marking each cell region in table images. The performance of this module directly affects the accuracy and efficiency of the entire table recognition process. The Table Cell Detection Module typically outputs bounding boxes for each cell region, which are then passed as input to the table recognition pipeline for further processing.
|
||||
|
||||
## II. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>mAP(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wired_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wired_table_cell_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wired_table_cell_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td rowspan="2">82.7</td>
|
||||
<td rowspan="2">33.47 / 27.02</td>
|
||||
<td rowspan="2">402.55 / 256.56</td>
|
||||
<td rowspan="2">124</td>
|
||||
<td rowspan="2">RT-DETR is a real-time end-to-end object detection model. The Baidu PaddlePaddle Vision team pre-trained on a self-built table cell detection dataset based on the RT-DETR-L as the base model, achieving good performance in detecting both wired and wireless table cells.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wireless_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wireless_table_cell_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wireless_table_cell_det_pretrained.pdparams">Training Model</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> Internal evaluation set built by PaddleX.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Explanation</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Regular Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Choose the optimal prior backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## III. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please first install the PaddleOCR wheel package. For details, please refer to the [installation tutorial](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate model inference from the table cell detection module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(model_name="RT-DETR-L_wired_table_cell_det")
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'boxes': [{'cls_id': 0, 'label': 'cell', 'score': 0.9698355197906494, 'coordinate': [2.3011515, 0, 546.29926, 30.530712]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9690820574760437, 'coordinate': [212.37508, 64.62493, 403.58868, 95.61413]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9668057560920715, 'coordinate': [212.46791, 30.311079, 403.7182, 64.62613]}, {'cls_id': 0, 'label': 'cell', 'score': 0.966505229473114, 'coordinate': [403.56082, 64.62544, 546.83215, 95.66117]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9662341475486755, 'coordinate': [109.48873, 64.66485, 212.5177, 95.631294]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9654079079627991, 'coordinate': [212.39197, 95.63037, 403.60852, 126.78792]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9653300642967224, 'coordinate': [2.2320926, 64.62229, 109.600494, 95.59732]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9639787673950195, 'coordinate': [403.5752, 30.562355, 546.98975, 64.61531]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9636150002479553, 'coordinate': [2.1537683, 30.410172, 109.568306, 64.62762]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631900191307068, 'coordinate': [2.0534437, 95.57448, 109.57601, 126.71458]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631181359291077, 'coordinate': [403.65976, 95.68139, 546.84766, 126.713394]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9614537358283997, 'coordinate': [109.56504, 30.391184, 212.65425, 64.6444]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9607433080673218, 'coordinate': [109.525795, 95.62622, 212.44917, 126.8258]}]}}
|
||||
```
|
||||
|
||||
The parameter meanings are as follows:
|
||||
|
||||
- `input_path`: Path of the input image to be predicted
|
||||
- `page_index`: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is `None`
|
||||
- `boxes`: Predicted bounding box information, a list of dictionaries. Each dictionary represents a detected object and contains the following information:
|
||||
- `cls_id`: Class ID, an integer
|
||||
- `label`: Class label, a string
|
||||
- `score`: Confidence of the bounding box, a float
|
||||
- `coordinate`: Coordinates of the bounding box, a list of floats in the format <code>[xmin, ymin, xmax, ymax]</code>
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_cells_detection/01.jpg">
|
||||
|
||||
The relevant methods, parameters, etc., are described as follows:
|
||||
|
||||
* `TableCellsDetection` instantiates the table cell detection model (taking `RT-DETR-L_wired_table_cell_det` as an example here), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>RT-DETR-L_wired_table_cell_det</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td>Input image size.<ul><li><b>int</b>: e.g. <code>640</code>, resizes input image to 640x640</li><li><b>list</b>: e.g. <code>[640, 512]</code>, resizes input image to 640 width and 512 height</li></ul></td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Threshold for filtering out low-confidence prediction results.<ul><li><b>float</b>: e.g. <code>0.2</code>, filters out all boxes with confidence below 0.2.</li><li><b>dict</b>: keys are <code>int</code> (class id), values are <code>float</code> thresholds, e.g. <code>{0: 0.45, 2: 0.48, 7: 0.4}</code>, applies thresholds to specific classes.</li><li><b>None</b>: uses the model's default configuration.</li></ul></td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the table cell detection model for inference prediction. This method will return a result list. Additionally, this module also provides a `predict_iter()` method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods according to your actual needs. The `predict()` method has parameters `input`, `batch_size`, and `threshold`, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local image file or PDF file path: <code>/root/data/img.jpg</code>; <b>URL</b>: Image or PDF file network URL: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Example</a>; <b>Directory</b>: Should contain images for prediction, e.g., <code>/root/data/</code> (currently, PDF files in directories are not supported, PDF files need to be specified by file path)</li>
|
||||
<li><b>list</b>: List elements should be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## IV. Secondary Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the table cell detection module, if you need to train a table cell detection model, you can refer to the [PaddleX Table Cell Detection Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_cells_detection.html#iv-secondary-development) section for training. The trained model can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## V. FAQ
|
||||
357
docs/version3.x/module_usage/table_cells_detection.md
Normal file
357
docs/version3.x/module_usage/table_cells_detection.md
Normal file
@@ -0,0 +1,357 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 表格单元格检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
表格单元格检测模块是表格识别任务的关键组成部分，负责在表格图像中定位和标记每个单元格区域，该模块的性能直接影响到整个表格识别过程的准确性和效率。表格单元格检测模块通常会输出各个单元格区域的边界框（Bounding Boxes），这些边界框将作为输入传递给表格识别相关产线进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>mAP(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wired_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wired_table_cell_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wired_table_cell_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td rowspan="2">82.7</td>
|
||||
<td rowspan="2">33.47 / 27.02</td>
|
||||
<td rowspan="2">402.55 / 256.56</td>
|
||||
<td rowspan="2">124</td>
|
||||
<td rowspan="2">RT-DETR 是一个实时的端到端目标检测模型。百度飞桨视觉团队基于 RT-DETR-L 作为基础模型，在自建表格单元格检测数据集上完成预训练，实现了对有线表格、无线表格均有较好性能的表格单元格检测。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>RT-DETR-L_wireless_table_cell_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/RT-DETR-L_wireless_table_cell_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/RT-DETR-L_wireless_table_cell_det_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建的内部评测集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr table_cells_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将表格单元格检测的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TableCellsDetection
|
||||
model = TableCellsDetection(model_name="RT-DETR-L_wired_table_cell_det")
|
||||
output = model.predict("table_recognition.jpg", threshold=0.3, batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'boxes': [{'cls_id': 0, 'label': 'cell', 'score': 0.9698355197906494, 'coordinate': [2.3011515, 0, 546.29926, 30.530712]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9690820574760437, 'coordinate': [212.37508, 64.62493, 403.58868, 95.61413]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9668057560920715, 'coordinate': [212.46791, 30.311079, 403.7182, 64.62613]}, {'cls_id': 0, 'label': 'cell', 'score': 0.966505229473114, 'coordinate': [403.56082, 64.62544, 546.83215, 95.66117]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9662341475486755, 'coordinate': [109.48873, 64.66485, 212.5177, 95.631294]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9654079079627991, 'coordinate': [212.39197, 95.63037, 403.60852, 126.78792]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9653300642967224, 'coordinate': [2.2320926, 64.62229, 109.600494, 95.59732]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9639787673950195, 'coordinate': [403.5752, 30.562355, 546.98975, 64.61531]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9636150002479553, 'coordinate': [2.1537683, 30.410172, 109.568306, 64.62762]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631900191307068, 'coordinate': [2.0534437, 95.57448, 109.57601, 126.71458]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9631181359291077, 'coordinate': [403.65976, 95.68139, 546.84766, 126.713394]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9614537358283997, 'coordinate': [109.56504, 30.391184, 212.65425, 64.6444]}, {'cls_id': 0, 'label': 'cell', 'score': 0.9607433080673218, 'coordinate': [109.525795, 95.62622, 212.44917, 126.8258]}]}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
|
||||
- `input_path`：输入的待预测图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `boxes`：预测的目标框信息，一个字典列表。每个字典代表一个检出的目标，包含以下信息：
|
||||
- `cls_id`：类别ID，一个整数
|
||||
- `label`：类别标签，一个字符串
|
||||
- `score`：目标框置信度，一个浮点数
|
||||
- `coordinate`：目标框坐标，一个浮点数列表，格式为<code>[xmin, ymin, xmax, ymax]</code>
|
||||
|
||||
可视化图像如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_cells_detection/01.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TableCellsDetection`实例化表格单元格检测模型（此处以`RT-DETR-L_wired_table_cell_det`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>RT-DETR-L_wired_table_cell_det</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img_size</code></td>
|
||||
<td>输入图像大小。
|
||||
<ul>
|
||||
<li><b>int</b>：如<code>640</code>，表示将输入图像resize到640x640大小。</li>
|
||||
<li><b>list</b>：如<code>[640, 512]</code>，表示将输入图像resize到宽为640，高为512大小。</li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>int|list|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>用于过滤掉低置信度预测结果的阈值。
|
||||
<ul>
|
||||
<li><b>float</b>：如 <code>0.2</code>，表示过滤掉所有阈值小于0.2的目标框。/li>
|
||||
<li><b>dict</b>：字典的键为 <code>int</code> 类型，代表类别ID；值为 <code>float</code> 类型阈值。如 <code>{0: 0.45, 2: 0.48, 7: 0.4}</code>，表示对类别ID为0的类别应用阈值0.45、类别ID为1的类别应用阈值0.48、类别ID为7的类别应用阈值0.4。</li>
|
||||
<li><b>None</b>：使用模型默认配置。
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 调用表格单元格检测模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input`、`batch_size`和`threshold`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>threshold</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|dict|None</code></td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供表格单元格检测模块的训练，因此，如果需要训练表格单元格检测模型，可以参考 [PaddleX 表格单元格检测模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_cells_detection.html#_4)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 五、FAQ
|
||||
306
docs/version3.x/module_usage/table_classification.en.md
Normal file
306
docs/version3.x/module_usage/table_classification.en.md
Normal file
@@ -0,0 +1,306 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
# Table Classification Module Usage Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The Table Classification Module is a key component in computer vision systems, responsible for classifying input table images. The performance of this module directly affects the accuracy and efficiency of the entire table recognition process. The Table Classification Module typically receives table images as input and, using deep learning algorithms, classifies them into predefined categories based on the characteristics and content of the images, such as wired and wireless tables. The classification results from the Table Classification Module serve as output for use in table recognition pipelines.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Top1 Acc(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Regular Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_table_cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_table_cls_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_table_cls_pretrained.pdparams">Training Model</a></td>
|
||||
<td>94.2</td>
|
||||
<td>2.62 / 0.60</td>
|
||||
<td>3.17 / 1.14</td>
|
||||
<td>6.6</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> Internal evaluation dataset built by PaddleX.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Explanation</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Regular Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Choose the optimal prior backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please first install the PaddleOCR wheel package. For details, please refer to the [installation tutorial](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate model inference from the table classification module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(model_name="PP-LCNet_x1_0_table_cls")
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'class_ids': array([0, 1], dtype=int32), 'scores': array([0.84421, 0.15579], dtype=float32), 'label_names': ['wired_table', 'wireless_table']}}
|
||||
```
|
||||
|
||||
The parameter meanings are as follows:
|
||||
- `input_path`: Path of the input image
|
||||
- `page_index`: If the input is a PDF file, it indicates which page of the PDF it is; otherwise, it is `None`
|
||||
- `class_ids`: Class IDs of the prediction results
|
||||
- `scores`: Confidence scores of the prediction results
|
||||
- `label_names`: Class names of the prediction results
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_classification/01.jpg">
|
||||
|
||||
The relevant methods, parameters, etc., are described as follows:
|
||||
|
||||
* `TableClassification` instantiates the table classification model (taking `PP-LCNet_x1_0_table_cls` as an example here), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-LCNet_x1_0_table_cls</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the table classification model for inference prediction. This method will return a result list. Additionally, this module also provides a `predict_iter()` method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods according to your actual needs. The `predict()` method has parameters `input` and `batch_size`, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local image file or PDF file path: <code>/root/data/img.jpg</code>; <b>URL</b>: Image or PDF file network URL: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Example</a>; <b>Directory</b>: Should contain images for prediction, e.g., <code>/root/data/</code> (currently, PDF files in directories are not supported, PDF files need to be specified by file path)</li>
|
||||
<li><b>list</b>: List elements should be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
Since PaddleOCR does not directly provide training for the table classification module, if you need to train a table classification model, you can refer to the [PaddleX Table Classification Module Secondary Development](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/table_classification.html#iv-secondary-development) section for training. The trained model can be seamlessly integrated into the PaddleOCR API for inference.
|
||||
|
||||
## 5. FAQ
|
||||
309
docs/version3.x/module_usage/table_classification.md
Normal file
309
docs/version3.x/module_usage/table_classification.md
Normal file
@@ -0,0 +1,309 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 表格分类模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
表格分类模块是计算机视觉系统中的关键组成部分，负责对输入的表格图像进行分类，该模块的性能直接影响到整个表格识别过程的准确性和效率。表格分类模块通常会接收表格图像作为输入，然后通过深度学习算法，根据图像的特性和内容，将其分类到预定义的类别中，例如有线表和无线表。表格分类模块的分类结果将作为输出，供表格识别相关产线使用。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>Top1 Acc(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_table_cls</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_table_cls_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_table_cls_pretrained.pdparams">训练模型</a></td>
|
||||
<td>94.2</td>
|
||||
<td>2.62 / 0.60</td>
|
||||
<td>3.17 / 1.14</td>
|
||||
<td>6.6</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>自建的内部评测数据集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr table_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将表格分类的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TableClassification
|
||||
model = TableClassification(model_name="PP-LCNet_x1_0_table_cls")
|
||||
output = model.predict("table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'class_ids': array([0, 1], dtype=int32), 'scores': array([0.84421, 0.15579], dtype=float32), 'label_names': ['wired_table', 'wireless_table']}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入图片的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `class_ids`：表示预测结果的类别id
|
||||
- `scores`：表示预测结果的置信度
|
||||
- `label_names`：表示预测结果的类别名
|
||||
|
||||
可视化图像如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/table_classification/01.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TableClassification`实例化表格分类模型（此处以`PP-LCNet_x1_0_table_cls`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-LCNet_x1_0_table_cls</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用表格分类模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供表格分类模块的训练，因此，如果需要训练表格分类模型，可以参考 [PaddleX 表格分类模块二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/table_classification.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
|
||||
## 五、FAQ
|
||||
398
docs/version3.x/module_usage/table_structure_recognition.en.md
Normal file
398
docs/version3.x/module_usage/table_structure_recognition.en.md
Normal file
@@ -0,0 +1,398 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Table Structure Recognition Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
Table structure recognition is an important component of table recognition systems, capable of converting non-editable table images into editable table formats (such as HTML). The goal of table structure recognition is to identify the positions of rows, columns, and cells in tables. The performance of this module directly affects the accuracy and efficiency of the entire table recognition system. The table structure recognition module usually outputs HTML code for the table area, which is then passed as input to the tabl recognition pipeline for further processing.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Accuracy (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_pretrained.pdparams">Training Model</a></td>
|
||||
<td>59.52</td>
|
||||
<td>23.96 / 21.75</td>
|
||||
<td>- / 43.12</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet is a table structure recognition model independently developed by Baidu PaddlePaddle Vision Team. By adopting a CPU-friendly lightweight backbone network PP-LCNet, high-low level feature fusion module CSP-PAN, and SLA Head, a feature decoding module aligning structure and position information, this model greatly improves the accuracy and inference speed of table structure recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet_plus</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_plus_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams">Training Model</a></td>
|
||||
<td>63.69</td>
|
||||
<td>23.43 / 22.16</td>
|
||||
<td>- / 41.80</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet_plus is an enhanced version of the table structure recognition model SLANet independently developed by the Baidu PaddlePaddle Vision Team. Compared to SLANet, SLANet_plus has greatly improved the recognition ability for wireless and complex tables, and reduced the model's sensitivity to table positioning accuracy. Even if the table positioning is offset, it can still be accurately recognized.
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wired</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wired_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wired_pretrained.pdparams">Training Model</a></td>
|
||||
<td rowspan="2">69.65</td>
|
||||
<td rowspan="2">85.92 / 85.92</td>
|
||||
<td rowspan="2">- / 501.66</td>
|
||||
<td rowspan="2">351</td>
|
||||
<td rowspan="2">The SLANeXt series is a new generation of table structure recognition models independently developed by the Baidu PaddlePaddle Vision Team. Compared to SLANet and SLANet_plus, SLANeXt focuses on table structure recognition, and trains dedicated weights for wired and wireless tables separately. The recognition ability for all types of tables has been significantly improved, especially for wired tables.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wireless</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wireless_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wireless_pretrained.pdparams">Training Model</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> High-difficulty Chinese table recognition dataset.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 precision / No TRT acceleration</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High Performance Mode</td>
|
||||
<td>Optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>Selects the prior optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before getting started, please install the PaddleOCR wheel package. For details, please refer to the [Installation Tutorial](../installation.en.md).
|
||||
|
||||
Quickly experience with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference of the table structure recognition module into your own project. Before running the code below, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg) to your local machine.
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(model_name="SLANet")
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result is:
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'bbox': [[42, 2, 390, 2, 388, 27, 40, 26], [11, 35, 89, 35, 87, 63, 11, 63], [113, 34, 192, 34, 186, 64, 109, 64], [219, 33, 399, 33, 393, 62, 212, 62], [413, 33, 544, 33, 544, 64, 407, 64], [12, 67, 98, 68, 96, 93, 12, 93], [115, 66, 205, 66, 200, 91, 111, 91], [234, 65, 390, 65, 385, 92, 227, 92], [414, 66, 537, 67, 537, 95, 409, 95], [7, 97, 106, 97, 104, 128, 7, 128], [113, 96, 206, 95, 201, 127, 109, 127], [236, 96, 386, 96, 381, 128, 230, 128], [413, 96, 534, 95, 533, 127, 408, 127]], 'structure': ['<html>', '<body>', '<table>', '<tr>', '<td', ' colspan="4"', '>', '</td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '</table>', '</body>', '</html>'], 'structure_score': 0.99948007}}
|
||||
```
|
||||
|
||||
Parameter meanings are as follows:
|
||||
|
||||
- `input_path`: The path of the input table image to be predicted
|
||||
- `page_index`: If the input is a PDF file, indicates the page number of the PDF; otherwise, it is `None`
|
||||
- `boxes`: Predicted table cell information, a list consisting of the coordinates of predicted table cells. Notably, table cell predictions for the SLANeXt series models are invalid
|
||||
- `structure`: Predicted table structure HTML expressions, a list consisting of predicted HTML keywords in order
|
||||
- `structure_score`: Confidence of the predicted table structure
|
||||
|
||||
Descriptions of related methods and parameters are as follows:
|
||||
|
||||
* `TableStructureRecognition` instantiates a table structure recognition model (using `SLANet` as an example). Details are as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-LCNet_x1_0_table_cls</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the table structure recognition model for inference prediction, which returns a result list. In addition, this module also provides the `predict_iter()` method. The two are completely consistent in parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where you want to save memory. You can choose to use either method according to your actual needs. The `predict()` method has parameters `input` and `batch_size`, described as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
Data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local path to an image or PDF file, e.g., <code>/root/data/img.jpg</code>; <b>URL</b>: Network URL to an image or PDF file, e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">Example</a>; <b>Directory</b>: A local directory containing images for prediction, e.g., <code>/root/data/</code> (Note: PDF files in directories are not supported; to predict a PDF, specify its file path directly)</li>
|
||||
<li><b>list</b>: A list of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, can be set to any positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
* For processing prediction results, the prediction result of each sample is the corresponding Result object, and supports printing and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to use <code>JSON</code> indentation formatting for the output</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> keeps the original characters. Effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save result as json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Path to save the file. If it's a directory, the saved file will be named the same as the input file type</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> keeps the original characters. Effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
If the above models are still not ideal for your scenario, you can try the following steps for secondary development. Here, training `SLANet_plus` is used as an example, and for other models, just replace the corresponding configuration file. First, you need to prepare a dataset for table structure recognition, which can be prepared with reference to the format of the [table structure recognition demo data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar). Once ready, you can train and export the model as follows. After exporting, you can quickly integrate the model into the above API. Here, the table structure recognition demo data is used as an example. Before training the model, please make sure you have installed the dependencies required by PaddleOCR according to the [installation documentation](../installation.en.md).
|
||||
|
||||
|
||||
## 4.1 Dataset and Pretrained Model Preparation
|
||||
|
||||
### 4.1.1 Prepare Dataset
|
||||
|
||||
```shell
|
||||
# Download sample dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar
|
||||
tar -xf table_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
### 4.1.2 Download Pretrained Model
|
||||
|
||||
```shell
|
||||
# Download SLANet_plus pretrained model
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR is modularized. When training the `SLANet_plus` recognition model, you need to use the [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/table/SLANet_plus.yml) of `SLANet_plus`.
|
||||
|
||||
|
||||
The training commands are as follows:
|
||||
|
||||
```bash
|
||||
# Single card training (default training method)
|
||||
python3 tools/train.py -c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
|
||||
# Multi-card training, specify card numbers via --gpus parameter
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command:
|
||||
|
||||
```bash
|
||||
# Note to set the path of pretrained_model to the local path. If you use the model saved by your own training, please modify the path and file name to {path/to/weights}/{model_name}.
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./SLANet_plus_infer/"
|
||||
```
|
||||
|
||||
After exporting the model, the static graph model will be stored in `./SLANet_plus_infer/` in the current directory. In this directory, you will see the following files:
|
||||
```
|
||||
./SLANet_plus_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
At this point, secondary development is complete, and this static graph model can be directly integrated into the PaddleOCR API.
|
||||
|
||||
## 5. FAQ
|
||||
404
docs/version3.x/module_usage/table_structure_recognition.md
Normal file
404
docs/version3.x/module_usage/table_structure_recognition.md
Normal file
@@ -0,0 +1,404 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 表格结构识别模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
表格结构识别是表格识别系统中的重要组成部分，能够将不可编辑表格图片转换为可编辑的表格形式（例如html）。表格结构识别的目标是对表格的行、列和单元格位置进行识别，该模块的性能直接影响到整个表格识别系统的准确性和效率。表格结构识别模块会输出表格区域的html代码，这些代码将作为输入传递给表格识别产线进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>精度（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_pretrained.pdparams">训练模型</a></td>
|
||||
<td>59.52</td>
|
||||
<td>23.96 / 21.75</td>
|
||||
<td>- / 43.12</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet 是百度飞桨视觉团队自研的表格结构识别模型。该模型通过采用 CPU 友好型轻量级骨干网络 PP-LCNet、高低层特征融合模块 CSP-PAN、结构与位置信息对齐的特征解码模块 SLA Head，大幅提升了表格结构识别的精度和推理速度。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANet_plus</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANet_plus_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams">训练模型</a></td>
|
||||
<td>63.69</td>
|
||||
<td>23.43 / 22.16</td>
|
||||
<td>- / 41.80</td>
|
||||
<td>6.9</td>
|
||||
<td rowspan="1">SLANet_plus 是百度飞桨视觉团队自研的表格结构识别模型 SLANet 的增强版。相较于 SLANet，SLANet_plus 对无线表、复杂表格的识别能力得到了大幅提升，并降低了模型对表格定位准确性的敏感度，即使表格定位出现偏移，也能够较准确地进行识别。
|
||||
</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wired</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wired_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wired_pretrained.pdparams">训练模型</a></td>
|
||||
<td rowspan="2">69.65</td>
|
||||
<td rowspan="2">85.92 / 85.92</td>
|
||||
<td rowspan="2">- / 501.66</td>
|
||||
<td rowspan="2">351</td>
|
||||
<td rowspan="2">SLANeXt 系列是百度飞桨视觉团队自研的新一代表格结构识别模型。相较于 SLANet 和 SLANet_plus，SLANeXt 专注于对表格结构进行识别，并且对有线表格(wired)和无线表格(wireless)的识别分别训练了专用的权重，对各类型表格的识别能力都得到了明显提高，特别是对有线表格的识别能力得到了大幅提升。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>SLANeXt_wireless</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/SLANeXt_wireless_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANeXt_wireless_pretrained.pdparams">训练模型</a></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>内部自建的高难度中文表格识别数据集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr table_structure_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将表格结构识别的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/table_recognition.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TableStructureRecognition
|
||||
model = TableStructureRecognition(model_name="SLANet")
|
||||
output = model.predict(input="table_recognition.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```
|
||||
{'res': {'input_path': 'table_recognition.jpg', 'page_index': None, 'bbox': [[42, 2, 390, 2, 388, 27, 40, 26], [11, 35, 89, 35, 87, 63, 11, 63], [113, 34, 192, 34, 186, 64, 109, 64], [219, 33, 399, 33, 393, 62, 212, 62], [413, 33, 544, 33, 544, 64, 407, 64], [12, 67, 98, 68, 96, 93, 12, 93], [115, 66, 205, 66, 200, 91, 111, 91], [234, 65, 390, 65, 385, 92, 227, 92], [414, 66, 537, 67, 537, 95, 409, 95], [7, 97, 106, 97, 104, 128, 7, 128], [113, 96, 206, 95, 201, 127, 109, 127], [236, 96, 386, 96, 381, 128, 230, 128], [413, 96, 534, 95, 533, 127, 408, 127]], 'structure': ['<html>', '<body>', '<table>', '<tr>', '<td', ' colspan="4"', '>', '</td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '<tr>', '<td></td>', '<td></td>', '<td></td>', '<td></td>', '</tr>', '</table>', '</body>', '</html>'], 'structure_score': 0.99948007}}
|
||||
```
|
||||
|
||||
参数含义如下：
|
||||
|
||||
- `input_path`：输入的待预测表格图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `boxes`：预测的表格单元格信息，一个列表，由预测的若干表格单元格坐标组成。特别地， SLANeXt 系列模型预测的表格单元格无效
|
||||
- `structure`：预测的表格结构Html表达式，一个列表，由预测的若干Html关键字按顺序组成
|
||||
- `structure_score`：预测表格结构的置信度
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TableStructureRecognition`实例化表格结构识别模型（此处以`SLANet`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-LCNet_x1_0_table_cls</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用表格结构识别模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `SLANet_plus` 举例，其他模型替换对应配置文件即可。首先，您需要准备表格结构识别的数据集，可以参考[表格结构识别 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以表格结构识别 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
### 4.1 数据集、预训练模型准备
|
||||
|
||||
#### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/table_rec_dataset_examples.tar
|
||||
tar -xf table_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 SLANet_plus 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/SLANet_plus_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `SLANet_plus` 识别模型时需要使用 `SLANet_plus` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/table/SLANet_plus.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/table/SLANet_plus.yml \
|
||||
-o Global.pretrained_model=./SLANet_plus_pretrained.pdparams
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Train.dataset.label_file_list='[./table_rec_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
Eval.dataset.data_dir=./table_rec_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./table_rec_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/table/SLANet_plus.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./SLANet_plus_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./SLANet_plus_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./SLANet_plus_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
504
docs/version3.x/module_usage/text_detection.en.md
Normal file
504
docs/version3.x/module_usage/text_detection.en.md
Normal file
@@ -0,0 +1,504 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Text Detection Module Usage Guide
|
||||
|
||||
## 1. Overview
|
||||
The text detection module is a critical component of OCR (Optical Character Recognition) systems, responsible for locating and marking text-containing regions in images. The performance of this module directly impacts the accuracy and efficiency of the entire OCR system. The text detection module typically outputs bounding boxes for text regions, which are then passed to the text recognition module for further processing.
|
||||
|
||||
## 2. Supported Models List
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Detection Hmean (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Standard Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Standard Mode / High-Performance Mode]</th>
|
||||
<th>Model Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_server_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>83.8</td>
|
||||
<td>89.55 / 70.19</td>
|
||||
<td>383.15 / 383.15</td>
|
||||
<td>84.3</td>
|
||||
<td>PP-OCRv5 server-side text detection model with higher accuracy, suitable for deployment on high-performance servers</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_mobile_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>79.0</td>
|
||||
<td>10.67 / 6.36</td>
|
||||
<td>57.77 / 28.15</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv5 mobile-side text detection model with higher efficiency, suitable for deployment on edge devices</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>69.2</td>
|
||||
<td>127.82 / 98.87</td>
|
||||
<td>585.95 / 489.77</td>
|
||||
<td>109</td>
|
||||
<td>PP-OCRv4 server-side text detection model with higher accuracy, suitable for deployment on high-performance servers</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_det_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_det_pretrained.pdparams">Training Model</a></td>
|
||||
<td>63.8</td>
|
||||
<td>9.87 / 4.17</td>
|
||||
<td>56.60 / 20.79</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv4 mobile-side text detection model with higher efficiency, suitable for deployment on edge devices</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Testing Environment:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Testing Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> PaddleOCR3.0 newly constructed multilingual dataset (including Chinese, Traditional Chinese, English, Japanese), covering street scenes, web images, documents, handwriting, blur, rotation, distortion, etc., totaling 2677 images.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Techniques</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Standard Mode</td>
|
||||
<td>FP32 precision / No TRT acceleration</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of precision types and acceleration strategies</td>
|
||||
<td>FP32 precision / 8 threads</td>
|
||||
<td>Optimal backend selection (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting, please install the PaddleOCR wheel package. Refer to the [Installation Guide](../installation.en.md) for details.
|
||||
|
||||
Use the following command for a quick experience:
|
||||
|
||||
```bash
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference into your project. Before running the following code, download the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection(model_name="PP-OCRv5_server_det")
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
The output will be:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_001.png', 'page_index': None, 'dt_polys': array([[[ 75, 549],
|
||||
...,
|
||||
[ 77, 586]],
|
||||
|
||||
...,
|
||||
|
||||
[[ 31, 406],
|
||||
...,
|
||||
[ 34, 455]]], dtype=int16), 'dt_scores': [0.873949039891189, 0.8948166013613552, 0.8842595305917041, 0.876953790920377]}}
|
||||
```
|
||||
|
||||
Output parameter meanings:
|
||||
- `input_path`: Path of the input image.
|
||||
- `page_index`: If the input is a PDF, this indicates the current page number; otherwise, it is `None`.
|
||||
- `dt_polys`: Predicted text detection boxes, where each box contains four vertices (x, y coordinates).
|
||||
- `dt_scores`: Confidence scores of the predicted text detection boxes.
|
||||
|
||||
Visualization example:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_det/general_ocr_001_res.png"/>
|
||||
|
||||
Method and parameter descriptions:
|
||||
|
||||
* Instantiate the text detection model (e.g., `PP-OCRv5_server_det`):
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-OCRv5_server_det</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b><code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Limit on the side length of the input image for detection. <code>int</code> specifies the value. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Type of image side length limitation. <code>"min"</code> ensures the shortest side of the image is no less than <code>det_limit_side_len</code>; <code>"max"</code> ensures the longest side is no greater than <code>limit_side_len</code>. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_side_limit</code></td>
|
||||
<td>Limit on the max length of the input image for detection. <code>int</code> limits the longest side of the image for input detection model. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>Pixel score threshold. Pixels in the output probability map with scores greater than this threshold are considered text pixels. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>If the average score of all pixels inside the bounding box is greater than this threshold, the result is considered a text region. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Expansion ratio for the Vatti clipping algorithm, used to expand the text region. If set to <code>None</code>, the model's default configuration will be used.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>Input image size for the model in the format <code>(C, H, W)</code>.</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* The `predict()` method parameters:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>
|
||||
Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python variable</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>: Local image file or PDF file path: <code>/root/data/img.jpg</code>; <b>URL</b>: Image or PDF file network URL: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">Example</a>; <b>Directory</b>: Should contain images for prediction, e.g., <code>/root/data/</code> (currently, PDF files in directories are not supported, PDF files need to be specified by file path)</li>
|
||||
<li><b>list</b>: List elements should be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Same meaning as the instantiation parameters. If set to <code>None</code>, the instantiation value is used; otherwise, this parameter takes precedence.</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
* Result processing methods:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameters</th>
|
||||
<th>Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print results to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Format output as JSON</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>JSON indentation level</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Escape non-ASCII characters</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save results as JSON file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Output file path</td>
|
||||
<td>Required</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>JSON indentation level</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Escape non-ASCII characters</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save results as image</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>Output file path</td>
|
||||
<td>Required</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additional attributes:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>json</code></td>
|
||||
<td>Get prediction results in JSON format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>img</code></td>
|
||||
<td>Get visualization image as a dictionary</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Custom Development
|
||||
|
||||
If the above models do not meet your requirements, follow these steps for custom development (using `PP-OCRv5_server_det` as an example). First, prepare a text detection dataset (refer to the [Demo Dataset](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar) format). After preparation, proceed with model training and export. The exported model can be integrated into the API. Ensure PaddleOCR dependencies are installed as per the [Installation Guide](../installation.en.md).
|
||||
|
||||
### 4.1 Dataset and Pretrained Model Preparation
|
||||
|
||||
#### 4.1.1 Prepare Dataset
|
||||
|
||||
```shell
|
||||
# Download example dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar
|
||||
tar -xf ocr_det_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 Download Pretrained Model
|
||||
|
||||
```shell
|
||||
# Download PP-OCRv5_server_det pretrained model
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR modularizes the code. To train the `PP-OCRv5_server_det` model, use its [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv5/PP-OCRv5_server_det.yml).
|
||||
|
||||
Training command:
|
||||
|
||||
```bash
|
||||
# Single-GPU training (default)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
|
||||
# Multi-GPU training (specify GPUs with --gpus)
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate trained weights (e.g., `output/PP-OCRv5_server_det/best_accuracy.pdparams`) using the following command:
|
||||
|
||||
```bash
|
||||
# Note: Set pretrained_model to local path. For custom-trained models, modify the path and filename as {path/to/weights}/{model_name}.
|
||||
# Demo dataset evaluation
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml -o \
|
||||
Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_det_infer/"
|
||||
```
|
||||
|
||||
After export, the static graph model will be saved in `./PP-OCRv5_server_det_infer/` with the following files:
|
||||
```
|
||||
./PP-OCRv5_server_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
The custom development is now complete. This static graph model can be directly integrated into PaddleOCR's API.
|
||||
|
||||
## 5. FAQ
|
||||
|
||||
- Use parameters `limit_type` and `limit_side_len` to constrain image dimensions.
|
||||
- `limit_type` options: [`max`, `min`]
|
||||
- `limit_side_len`: Positive integer (typically multiples of 32, e.g., 960).
|
||||
- For lower-resolution images, use `limit_type=min` and `limit_side_len=960` to balance computational efficiency and detection quality.
|
||||
- For higher-resolution images requiring larger detection scales, set `limit_side_len` to desired values (e.g., 1216).
|
||||
508
docs/version3.x/module_usage/text_detection.md
Normal file
508
docs/version3.x/module_usage/text_detection.md
Normal file
@@ -0,0 +1,508 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本检测模块使用教程
|
||||
|
||||
## 一、概述
|
||||
文本检测模块是OCR（光学字符识别）系统中的关键组成部分，负责在图像中定位和标记出包含文本的区域。该模块的性能直接影响到整个OCR系统的准确性和效率。文本检测模块通常会输出文本区域的边界框（Bounding Boxes），这些边界框将作为输入传递给文本识别模块进行后续处理。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>检测Hmean（%）</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_server_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>83.8</td>
|
||||
<td>89.55 / 70.19</td>
|
||||
<td>383.15 / 383.15</td>
|
||||
<td>84.3</td>
|
||||
<td>PP-OCRv5 的服务端文本检测模型，精度更高，适合在性能较好的服务器上部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv5_mobile_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>79.0</td>
|
||||
<td>10.67 / 6.36</td>
|
||||
<td>57.77 / 28.15</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv5 的移动端文本检测模型，效率更高，适合在端侧设备部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>69.2</td>
|
||||
<td>127.82 / 98.87</td>
|
||||
<td>585.95 / 489.77</td>
|
||||
<td>109</td>
|
||||
<td>PP-OCRv4 的服务端文本检测模型，精度更高，适合在性能较好的服务器上部署</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_det</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_det_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_det_pretrained.pdparams">训练模型</a></td>
|
||||
<td>63.8</td>
|
||||
<td>9.87 / 4.17</td>
|
||||
<td>56.60 / 20.79</td>
|
||||
<td>4.7</td>
|
||||
<td>PP-OCRv4 的移动端文本检测模型，效率更高，适合在端侧设备部署</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>PaddleOCR3.0 全新构建多语种（包含中、繁、英、日），覆盖街景、网图、文档、手写、模糊、旋转、扭曲等多个场景的文本检测数据集，包含2677 张图片。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr text_detection -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将文本检测的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_001.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextDetection
|
||||
model = TextDetection(model_name="PP-OCRv5_server_det")
|
||||
output = model.predict("general_ocr_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_001.png', 'page_index': None, 'dt_polys': array([[[ 75, 549],
|
||||
...,
|
||||
[ 77, 586]],
|
||||
|
||||
...,
|
||||
|
||||
[[ 31, 406],
|
||||
...,
|
||||
[ 34, 455]]], dtype=int16), 'dt_scores': [0.873949039891189, 0.8948166013613552, 0.8842595305917041, 0.876953790920377]}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待预测图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `dt_polys`：表示预测的文本检测框，其中每个文本检测框包含一个四边形的四个顶点。其中每个顶点都是一个列表，分别表示该顶点的x坐标和y坐标
|
||||
- `dt_scores`：表示预测的文本检测框的置信度
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_det/general_ocr_001_res.png"/>
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TextDetection`实例化文本检测模型（此处以`PP-OCRv5_server_det`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-OCRv5_server_det</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>检测的图像边长限制：<code>int</code> 表示边长限制数值。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>检测的图像边长限制，检测的边长限制类型，<code>"min"</code> 表示保证图像最短边不小于det_limit_side_len，<code>"max"</code>表示保证图像最长边不大于limit_side_len。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>max_side_limit</code></td>
|
||||
<td>检测的图像边长最大值限制：<code>int</code> 限制输入检测模型的图片最长边。如果设置为 <code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>像素得分阈值。输出概率图中得分大于该阈值的像素点被认为是文本像素。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>检测结果边框内，所有像素点的平均得分大于该阈值时，该结果会被认为是文字区域。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>Vatti clipping算法的扩张系数，使用该方法对文字区域进行扩张。如果设置为<code>None</code>，将使用模型默认配置。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>模型输入图像尺寸，格式为 <code>(C, H, W)</code>。</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文本检测模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input`、 `batch_size`、 `limit_side_len`、 `limit_type`、 `thresh`、 `box_thresh`、 `max_candidates`、`unclip_ratio`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_side_len</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>int|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>limit_type</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>thresh</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>box_thresh</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>unclip_ratio</code></td>
|
||||
<td>参数含义与实例化参数基本相同。设置为<code>None</code>表示使用实例化参数，否则该参数优先级更高。</td>
|
||||
<td><code>float|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-OCRv5_server_det` 举例，其他模型替换对应配置文件即可。首先，您需要准备文本检测的数据集，可以参考[文本检测 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以文本检测 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
### 4.1 数据集、预训练模型准备
|
||||
|
||||
#### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_det_dataset_examples.tar
|
||||
tar -xf ocr_det_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-OCRv5_server_det 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_det_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `PP-OCRv5_server_det` 识别模型时需要使用 `PP-OCRv5_server_det` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/det/PP-OCRv5/PP-OCRv5_server_det.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py \
|
||||
-c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_det_pretrained.pdparams \
|
||||
Train.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Train.dataset.label_file_list='[./ocr_det_dataset_examples/train.txt]' \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/PP-OCRv5_server_det/best_accuracy.pdprams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
# 注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
# demo 测试集评估
|
||||
python3 tools/eval.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml \
|
||||
-o Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Eval.dataset.data_dir=./ocr_det_dataset_examples \
|
||||
Eval.dataset.label_file_list='[./ocr_det_dataset_examples/val.txt]'
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/det/PP-OCRv5/PP-OCRv5_server_det.yml -o \
|
||||
Global.pretrained_model=output/PP-OCRv5_server_det/best_accuracy.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_det_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-OCRv5_server_det_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-OCRv5_server_det_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
|
||||
- 通过参数`limit_type`和`limit_side_len`来对图片的尺寸进行限制，`limit_type`可选参数为[`max`, `min`]，`limit_side_len` 为正整数，一般设置为 32 的倍数，比如 960。
|
||||
如果输入图形分辨率不大，建议使用`limit_type=min` 和 `limit_side_len=960` 节省计算资源的同时能获得最佳检测效果。如果输入图片的分辨率比较大，而且想使用更大的分辨率预测，可以设置 `limit_side_len` 为想要的值，比如 1216。
|
||||
327
docs/version3.x/module_usage/text_image_unwarping.en.md
Normal file
327
docs/version3.x/module_usage/text_image_unwarping.en.md
Normal file
@@ -0,0 +1,327 @@
|
||||
---
|
||||
|
||||
comments: true
|
||||
|
||||
---
|
||||
|
||||
# Text Image Rectification Module Usage Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The primary purpose of text image rectification is to perform geometric transformations on images to correct distortions, inclinations, perspective deformations, etc., in the document images for more accurate subsequent text recognition.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>CER</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>UVDoc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UVDoc_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UVDoc_pretrained.pdparams">Training Model</a></td>
|
||||
<td>0.179</td>
|
||||
<td>19.05 / 19.05</td>
|
||||
<td>- / 869.82</td>
|
||||
<td>30.3</td>
|
||||
<td>High-accuracy text image rectification model</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong> <a href="https://www3.cs.stonybrook.edu/~cvl/docunet.html">DocUNet benchmark</a> dataset.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Explanation</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Regular Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Choose the optimal combination of prior precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Choose the optimal prior backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting quickly, please first install the PaddleOCR wheel package. For details, please refer to the [installation tutorial](../installation.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the model inference from the image rectification module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg) locally.
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(model_name="UVDoc")
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'doc_test.jpg', 'page_index': None, 'doctr_img': '...'}}
|
||||
```
|
||||
|
||||
The meanings of the parameters in the result are as follows:
|
||||
- `input_path`: Indicates the path of the image to be rectified
|
||||
- `doctr_img`: Indicates the rectified image result. Due to the large amount of data, it is not convenient to print directly, so it is replaced here with `...`. You can use `res.save_to_img()` to save the prediction result as an image, and `res.save_to_json()` to save the prediction result as a json file.
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/image_unwarp/doc_test_res.jpg">
|
||||
|
||||
The relevant methods, parameters, etc., are described as follows:
|
||||
|
||||
* `TextImageUnwarping` instantiates the image rectification model (taking `UVDoc` as an example here), with specific explanations as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Name of the model</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device(s) to use for inference.<br/>
|
||||
<b>Examples:</b> <code>cpu</code>, <code>gpu</code>, <code>npu</code>, <code>gpu:0</code>, <code>gpu:0,1</code>.<br/>
|
||||
If multiple devices are specified, inference will be performed in parallel. Note that parallel inference is not always supported.<br/>
|
||||
By default, GPU 0 will be used if available; otherwise, the CPU will be used.
|
||||
</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to use the high performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Precision for TensorRT when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>fp32</code>, <code>fp16</code>, etc.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>fp32</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the image rectification model for inference prediction. This method will return a result list. Additionally, this module also provides a `predict_iter()` method. Both methods are consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step, suitable for handling large datasets or scenarios where memory saving is desired. You can choose to use either of these methods according to your actual needs. The `predict()` method has parameters `input` and `batch_size`, with specific explanations as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:
|
||||
<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">Print result to terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">Save the result as a json format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters into <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> will retain the original characters, effective only when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as an image format file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. When specified as a directory, the saved file is named consistent with the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, the result can be obtained through attributes that provide the visualized images with results and the prediction results, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">Get the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
The current module does not support fine-tuning training and only supports inference integration. Concerning fine-tuning training for this module, there are plans to support it in the future.
|
||||
|
||||
## 5. FAQ
|
||||
325
docs/version3.x/module_usage/text_image_unwarping.md
Normal file
325
docs/version3.x/module_usage/text_image_unwarping.md
Normal file
@@ -0,0 +1,325 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本图像矫正模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文本图像矫正的主要目的是针对图像进行几何变换，以纠正图像中的文档扭曲、倾斜、透视变形等问题，以供后续的文本识别进行更加准确。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>CER </th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>UVDoc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/UVDoc_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/UVDoc_pretrained.pdparams">训练模型</a></td>
|
||||
<td>0.179</td>
|
||||
<td>19.05 / 19.05</td>
|
||||
<td>- / 869.82</td>
|
||||
<td>30.3</td>
|
||||
<td>高精度文本图像矫正模型</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong><a href="https://www3.cs.stonybrook.edu/~cvl/docunet.html">DocUNet benchmark</a>数据集。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr text_image_unwarping -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将图像矫正的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/doc_test.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextImageUnwarping
|
||||
model = TextImageUnwarping(model_name="UVDoc")
|
||||
output = model.predict("doc_test.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'doc_test.jpg', 'page_index': None, 'doctr_img': '...'}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待矫正图像的路径
|
||||
- `doctr_img`：表示矫正后的图像结果，由于数据过多不便于直接print，所以此处用`...`替换，可以通过`res.save_to_img()`将预测结果保存为图片，通过`res.save_to_json()`将预测结果保存为json文件。
|
||||
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/image_unwarp/doc_test_res.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TextImageUnwarping`实例化图像矫正模型（此处以`UVDoc`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>UVDoc</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用图像矫正模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/img_rot180_demo.jpg">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
当前模块暂时不支持微调训练，仅支持推理集成。关于该模块的微调训练，计划在未来支持。
|
||||
|
||||
## 五、FAQ
|
||||
758
docs/version3.x/module_usage/text_recognition.en.md
Normal file
758
docs/version3.x/module_usage/text_recognition.en.md
Normal file
@@ -0,0 +1,758 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Text Recognition Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
|
||||
The text recognition module is the core part of the OCR (Optical Character Recognition) system, responsible for extracting text information from text regions in images. The performance of this module directly affects the accuracy and efficiency of the entire OCR system. The text recognition module usually receives the bounding boxes of text regions output by the text detection module as input, and then converts the text in the images into editable and searchable electronic text through complex image processing and deep learning algorithms. The accuracy of text recognition results is crucial for subsequent applications such as information extraction and data mining.
|
||||
|
||||
## 2. List of Supported Models
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_server_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>86.38</td>
|
||||
<td>8.46 / 2.36</td>
|
||||
<td>31.21 / 31.21</td>
|
||||
<td>81</td>
|
||||
<td rowspan="2">PP-OCRv5_rec is a new generation text recognition model. It is designed to efficiently and accurately support the recognition of Simplified Chinese, Traditional Chinese, English, Japanese, as well as complex text scenarios such as handwriting, vertical text, pinyin, and rare characters with a single model. While maintaining recognition performance, it also balances inference speed and model robustness, providing efficient and accurate technical support for document understanding in various scenarios.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>81.29</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>16</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec_doc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv4_server_rec_doc_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_doc_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>86.58</td>
|
||||
<td>8.69 / 2.78</td>
|
||||
<td>37.93 / 37.93</td>
|
||||
<td>182</td>
|
||||
<td>PP-OCRv4_server_rec_doc is trained on a mixed dataset of more Chinese document data and PP-OCR training data, building upon PP-OCRv4_server_rec. It enhances the recognition capabilities for some Traditional Chinese characters, Japanese characters, and special symbols, supporting over 15,000 characters. In addition to improving document-related text recognition, it also enhances general text recognition capabilities.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>78.74</td>
|
||||
<td>5.26 / 1.12</td>
|
||||
<td>17.48 / 3.61</td>
|
||||
<td>10.5</td>
|
||||
<td>A lightweight recognition model of PP-OCRv4 with high inference efficiency, suitable for deployment on various hardware devices, including edge devices.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>85.19</td>
|
||||
<td>8.75 / 2.49</td>
|
||||
<td>36.93 / 36.93</td>
|
||||
<td>173</td>
|
||||
<td>The server-side model of PP-OCRv4, offering high inference accuracy and deployable on various servers.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv4_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv4_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>70.39</td>
|
||||
<td>4.81 / 1.23</td>
|
||||
<td>17.20 / 4.18</td>
|
||||
<td>7.5</td>
|
||||
<td>An ultra-lightweight English recognition model trained based on the PP-OCRv4 recognition model, supporting English and numeric character recognition.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
> ❗ The above lists the <b>4 core models</b> mainly supported by the text recognition module. The module supports a total of <b>20 full models</b>, including multiple multilingual text recognition models. The complete model list is as follows:
|
||||
|
||||
<details><summary> 👉Model List Details</summary>
|
||||
|
||||
* <b>PP-OCRv5 Multi-Scenario Models</b>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Chinese Recognition Avg Accuracy(%)</th>
|
||||
<th>English Recognition Avg Accuracy(%)</th>
|
||||
<th>Traditional Chinese Recognition Avg Accuracy(%)</th>
|
||||
<th>Japanese Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_server_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>86.38</td>
|
||||
<td>64.70</td>
|
||||
<td>93.29</td>
|
||||
<td>60.35</td>
|
||||
<td>8.46 / 2.36</td>
|
||||
<td>31.21 / 31.21</td>
|
||||
<td>81</td>
|
||||
<td rowspan="2">PP-OCRv5_rec is a new generation text recognition model. It is designed to efficiently and accurately support the recognition of Simplified Chinese, Traditional Chinese, English, Japanese, as well as complex text scenarios such as handwriting, vertical text, pinyin, and rare characters with a single model. While maintaining recognition performance, it also balances inference speed and model robustness, providing efficient and accurate technical support for document understanding in various scenarios.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>81.29</td>
|
||||
<td>66.00</td>
|
||||
<td>83.55</td>
|
||||
<td>54.65</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>16</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>Chinese Recognition Models</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec_doc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv4_server_rec_doc_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_doc_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>86.58</td>
|
||||
<td>8.69 / 2.78</td>
|
||||
<td>37.93 / 37.93</td>
|
||||
<td>182</td>
|
||||
<td>PP-OCRv4_server_rec_doc is trained on a mixed dataset of more Chinese document data and PP-OCR training data, building upon PP-OCRv4_server_rec. It enhances the recognition capabilities for some Traditional Chinese characters, Japanese characters, and special symbols, supporting over 15,000 characters. In addition to improving document-related text recognition, it also enhances general text recognition capabilities.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>78.74</td>
|
||||
<td>5.26 / 1.12</td>
|
||||
<td>17.48 / 3.61</td>
|
||||
<td>10.5</td>
|
||||
<td>A lightweight recognition model of PP-OCRv4 with high inference efficiency, suitable for deployment on various hardware devices, including edge devices.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>85.19</td>
|
||||
<td>8.75 / 2.49</td>
|
||||
<td>36.93 / 36.93</td>
|
||||
<td>173</td>
|
||||
<td>The server-side model of PP-OCRv4, offering high inference accuracy and deployable on various servers.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>72.96</td>
|
||||
<td>3.89 / 1.16</td>
|
||||
<td>8.72 / 3.56</td>
|
||||
<td>10.3</td>
|
||||
<td>A lightweight recognition model of PP-OCRv3 with high inference efficiency, suitable for deployment on various hardware devices, including edge devices.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_SVTRv2_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_SVTRv2_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_SVTRv2_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>68.81</td>
|
||||
<td>10.38 / 8.31</td>
|
||||
<td>66.52 / 30.83</td>
|
||||
<td>80.5</td>
|
||||
<td rowspan="1">SVTRv2 is a server-side text recognition model developed by the OpenOCR team of the Vision and Learning Lab (FVL) at Fudan University. It won the first prize in the PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task, with a 6% improvement in end-to-end recognition accuracy on Leaderboard A compared to PP-OCRv4.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_RepSVTR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_RepSVTR_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_RepSVTR_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>65.07</td>
|
||||
<td>6.29 / 1.57</td>
|
||||
<td>20.64 / 5.40</td>
|
||||
<td>22.1</td>
|
||||
<td rowspan="1">RepSVTR is a mobile-side text recognition model based on SVTRv2. It won the first prize in the PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task, with a 2.5% improvement in end-to-end recognition accuracy on Leaderboard B compared to PP-OCRv4, while maintaining similar inference speed.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>English Recognition Models</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv4_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv4_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td> 70.39</td>
|
||||
<td>4.81 / 1.23</td>
|
||||
<td>17.20 / 4.18</td>
|
||||
<td>7.5</td>
|
||||
<td>An ultra-lightweight English recognition model trained based on the PP-OCRv4 recognition model, supporting English and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>70.69</td>
|
||||
<td>3.56 / 0.78</td>
|
||||
<td>8.44 / 5.78</td>
|
||||
<td>17.3</td>
|
||||
<td>An ultra-lightweight English recognition model trained based on the PP-OCRv3 recognition model, supporting English and numeric character recognition.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>Multilingual Recognition Models</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Links</th>
|
||||
<th>Recognition Avg Accuracy(%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Introduction</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>korean_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
korean_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv5_mobile_rec_pretrained.pdparams">Pre-trained Model</a></td>
|
||||
<td>90.45</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>An ultra-lightweight Korean text recognition model trained based on the PP-OCRv5 recognition framework. Supports Korean, English and numeric text recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>latin_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
latin_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv5_mobile_rec_pretrained.pdparams">Pre-trained Model</a></td>
|
||||
<td>84.7</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>A Latin-script text recognition model trained based on the PP-OCRv5 recognition framework. Supports most Latin alphabet languages and numeric text recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>eslav_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
eslav_PP-OCRv5_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/eslav_PP-OCRv5_mobile_rec_pretrained.pdparams">Pre-trained Model</a></td>
|
||||
<td>85.8</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>An East Slavic language recognition model trained based on the PP-OCRv5 recognition framework. Supports East Slavic languages, English and numeric text recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>korean_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
korean_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>60.21</td>
|
||||
<td>3.73 / 0.98</td>
|
||||
<td>8.76 / 2.91</td>
|
||||
<td>9.6</td>
|
||||
<td>An ultra-lightweight Korean recognition model trained based on the PP-OCRv3 recognition model, supporting Korean and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>japan_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
japan_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/japan_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>45.69</td>
|
||||
<td>3.86 / 1.01</td>
|
||||
<td>8.62 / 2.92</td>
|
||||
<td>9.8</td>
|
||||
<td>An ultra-lightweight Japanese recognition model trained based on the PP-OCRv3 recognition model, supporting Japanese and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>chinese_cht_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
chinese_cht_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/chinese_cht_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>82.06</td>
|
||||
<td>3.90 / 1.16</td>
|
||||
<td>9.24 / 3.18</td>
|
||||
<td>10.8</td>
|
||||
<td>An ultra-lightweight Traditional Chinese recognition model trained based on the PP-OCRv3 recognition model, supporting Traditional Chinese and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>te_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
te_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/te_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>95.88</td>
|
||||
<td>3.59 / 0.81</td>
|
||||
<td>8.28 / 6.21</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Telugu recognition model trained based on the PP-OCRv3 recognition model, supporting Telugu and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ka_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ka_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ka_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>96.96</td>
|
||||
<td>3.49 / 0.89</td>
|
||||
<td>8.63 / 2.77</td>
|
||||
<td>17.4</td>
|
||||
<td>An ultra-lightweight Kannada recognition model trained based on the PP-OCRv3 recognition model, supporting Kannada and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ta_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ta_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ta_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>76.83</td>
|
||||
<td>3.49 / 0.86</td>
|
||||
<td>8.35 / 3.41</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Tamil recognition model trained based on the PP-OCRv3 recognition model, supporting Tamil and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>latin_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
latin_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>76.93</td>
|
||||
<td>3.53 / 0.78</td>
|
||||
<td>8.50 / 6.83</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Latin recognition model trained based on the PP-OCRv3 recognition model, supporting Latin and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>arabic_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
arabic_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/arabic_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>73.55</td>
|
||||
<td>3.60 / 0.83</td>
|
||||
<td>8.44 / 4.69</td>
|
||||
<td>17.3</td>
|
||||
<td>An ultra-lightweight Arabic alphabet recognition model trained based on the PP-OCRv3 recognition model, supporting Arabic alphabet and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>cyrillic_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
cyrillic_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/cyrillic_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>94.28</td>
|
||||
<td>3.56 / 0.79</td>
|
||||
<td>8.22 / 2.76</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Cyrillic alphabet recognition model trained based on the PP-OCRv3 recognition model, supporting Cyrillic alphabet and numeric character recognition.</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>devanagari_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
devanagari_PP-OCRv3_mobile_rec_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/devanagari_PP-OCRv3_mobile_rec_pretrained.pdparams">Pretrained Model</a></td>
|
||||
<td>96.44</td>
|
||||
<td>3.60 / 0.78</td>
|
||||
<td>6.95 / 2.87</td>
|
||||
<td>8.7</td>
|
||||
<td>An ultra-lightweight Devanagari alphabet recognition model trained based on the PP-OCRv3 recognition model, supporting Devanagari alphabet and numeric character recognition.</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset:</strong>
|
||||
<ul>
|
||||
<li>
|
||||
Chinese Recognition Models: A self-built Chinese dataset by PaddleOCR, covering street views, online images, documents, handwriting, with 11,000 images for text recognition.
|
||||
</li>
|
||||
<li>
|
||||
ch_SVTRv2_rec: <a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task</a> Leaderboard A evaluation set.
|
||||
</li>
|
||||
<li>
|
||||
ch_RepSVTR_rec: <a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR Algorithm Model Challenge - Task 1: OCR End-to-End Recognition Task</a> Leaderboard B evaluation set.
|
||||
</li>
|
||||
<li>
|
||||
English Recognition Models: A self-built English dataset by PaddleOCR.
|
||||
</li>
|
||||
<li>
|
||||
Multilingual Recognition Models: A self-built multilingual dataset by PaddleOCR.
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Explanation of Inference Modes</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration</th>
|
||||
<th>CPU Configuration</th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of precision type and acceleration strategy</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Selection of the optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
</details>
|
||||
|
||||
## 3. Quick Start
|
||||
|
||||
> ❗ Before starting, please install the PaddleOCR wheel package. For details, please refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience it with one command:
|
||||
|
||||
```bash
|
||||
paddleocr text_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png
|
||||
```
|
||||
|
||||
<b>Note:</b> The official PaddleOCR models are downloaded from HuggingFace by default. If you cannot access HuggingFace, you can change the model source to BOS by setting the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"`. More mainstream model sources will be supported in the future.
|
||||
|
||||
You can also integrate the model inference of the text recognition module into your project. Before running the following code, please download the [sample image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png) to your local machine.
|
||||
|
||||
```python
|
||||
from paddleocr import TextRecognition
|
||||
model = TextRecognition(model_name="PP-OCRv5_server_rec")
|
||||
output = model.predict(input="general_ocr_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result is as follows:
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_rec_001.png', 'page_index': None, 'rec_text': '绿洲仕格维花园公寓', 'rec_score': 0.9823867082595825}}
|
||||
```
|
||||
|
||||
The meanings of the parameters in the result are as follows:
|
||||
- `input_path`: The path of the input text line image to be predicted
|
||||
- `page_index`: If the input is a PDF file, it indicates which page of the PDF the current text line is from; otherwise, it is `None`
|
||||
- `rec_text`: The predicted text of the text line image
|
||||
- `rec_score`: The confidence score of the predicted text for the text line image
|
||||
|
||||
The visualized image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_recog/general_ocr_rec_001.png"/>
|
||||
|
||||
Descriptions of related methods and parameters are as follows:
|
||||
|
||||
* Instantiate the text recognition model using `TextRecognition` (using `PP-OCRv5_server_rec` as an example), as follows:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>If set to <code>None</code>, <code>PP-OCRv5_server_rec</code> is used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>Examples:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, inference will be performed in parallel.<br/>
|
||||
By default, GPU 0 is used; if unavailable, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to enable the TensorRT subgraph engine of Paddle Inference.<br/>
|
||||
For Paddle with CUDA 11.8, the compatible TensorRT version is 8.x (x>=6), recommended 8.6.1.6.<br/>
|
||||
For Paddle with CUDA 12.6, the compatible TensorRT version is 10.x (x>=5), recommended 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Precision for TensorRT when using the Paddle Inference TensorRT subgraph engine.<br/><b>Options:</b> <code>fp32</code>, <code>fp16</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>MKL-DNN cache capacity.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>Input image size for the model in the format <code>(C, H, W)</code>.</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the text recognition model for inference. This method returns a list of results. In addition, this module also provides the `predict_iter()` method. The two methods are completely consistent in terms of parameter acceptance and result return. The difference is that `predict_iter()` returns a `generator`, which can process and obtain prediction results step by step. It is suitable for scenarios where large datasets need to be processed or memory savings are desired. You can choose either of these two methods according to your actual needs. The parameters of the `predict()` method include `input` and `batch_size`, with specific descriptions as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Data to be predicted, supporting multiple input types, required.
|
||||
<ul>
|
||||
<li><b>Python Var</b>: Image data represented by <code>numpy.ndarray</code></li>
|
||||
<li><b>str</b>: Local path of image file or PDF file: <code>/root/data/img.jpg</code>; <b>URL link</b>: Network URL of image file or PDF file: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">Example</a>; <b>Local directory</b>: The directory should contain the images to be predicted, such as <code>/root/data/</code> (currently, prediction of PDF files in the directory is not supported, PDF files need to be specified to a specific file path)</li>
|
||||
<li><b>list</b>: The elements of the list should be data of the above types, such as <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, can be set to any positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Process the prediction results. The prediction result for each sample is a corresponding Result object, which supports operations such as printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Type</th>
|
||||
<th>Description</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the result to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable. Only effective when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. Only effective when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the result as a file in <code>json</code> format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save the result. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specifies the indentation level to beautify the output <code>JSON</code> data, making it more readable. Only effective when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Controls whether to escape non-<code>ASCII</code> characters as <code>Unicode</code>. When set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters. Only effective when <code>format_json</code> is <code>True</code>.</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the result as a file in image format</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The file path to save the result. When it is a directory, the saved file name is consistent with the naming of the input file type.</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* In addition, it also supports obtaining the visualized image with results and the prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Obtain the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Obtain the visualized image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Secondary Development
|
||||
|
||||
If the above models do not perform well in your scenario, you can try the following steps for secondary development. Here, we take training `PP-OCRv5_server_rec` as an example. For other models, just replace the corresponding configuration file. First, you need to prepare a dataset for text recognition. You can refer to the format of the [Text Recognition Demo Data](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_dataset_examples.tar) for preparation. After preparation, you can train and export the model as follows. After export, the model can be quickly integrated into the above API. This example uses the Text Recognition Demo Data. Before training the model, please make sure you have installed the dependencies required by PaddleOCR as described in the [Installation Guide](../installation.md).
|
||||
|
||||
### 4.1 Dataset and Pre-trained Model Preparation
|
||||
|
||||
#### 4.1.1 Prepare the Dataset
|
||||
|
||||
```shell
|
||||
# Download the example dataset
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_dataset_examples.tar
|
||||
tar -xf ocr_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
#### 4.1.2 Download the Pre-trained Model
|
||||
|
||||
```shell
|
||||
# Download the PP-OCRv5_server_rec pre-trained model
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 Model Training
|
||||
|
||||
PaddleOCR modularizes its code. To train the `PP-OCRv5_server_rec` recognition model, you need to use its [configuration file](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml).
|
||||
|
||||
The training commands are as follows:
|
||||
|
||||
```bash
|
||||
# Single-GPU training (default training method)
|
||||
python3 tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
|
||||
# Multi-GPU training, specify GPU IDs via the --gpus parameter
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.3 Model Evaluation
|
||||
|
||||
You can evaluate the trained weights, such as `output/xxx/xxx.pdparams`, using the following command:
|
||||
|
||||
```bash
|
||||
# Note: Set the path of pretrained_model to a local path. If you use a model you trained and saved yourself, please modify the path and file name to {path/to/weights}/{model_name}.
|
||||
# Demo test set evaluation
|
||||
python3 tools/eval.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 Model Export
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_rec_infer/"
|
||||
```
|
||||
|
||||
After exporting the model, the static graph model will be stored in `./PP-OCRv5_server_rec_infer/` in the current directory. In this directory, you will see the following files:
|
||||
```
|
||||
./PP-OCRv5_server_rec_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
At this point, the secondary development is complete. This static graph model can be directly integrated into the PaddleOCR API.
|
||||
|
||||
## 5. FAQ
|
||||
770
docs/version3.x/module_usage/text_recognition.md
Normal file
770
docs/version3.x/module_usage/text_recognition.md
Normal file
@@ -0,0 +1,770 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本识别模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文本识别模块是OCR（光学字符识别）系统中的核心部分，负责从图像中的文本区域提取出文本信息。该模块的性能直接影响到整个OCR系统的准确性和效率。文本识别模块通常接收文本检测模块输出的文本区域的边界框（Bounding Boxes）作为输入，然后通过复杂的图像处理和深度学习算法，将图像中的文本转化为可编辑和可搜索的电子文本。文本识别结果的准确性，对于后续的信息提取和数据挖掘等应用至关重要。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>86.38</td>
|
||||
<td>8.46 / 2.36</td>
|
||||
<td>31.21 / 31.21</td>
|
||||
<td>81</td>
|
||||
<td rowspan="2">PP-OCRv5_rec 是新一代文本识别模型。该模型致力于以单一模型高效、精准地支持简体中文、繁体中文、英文、日文四种主要语言，以及手写、竖版、拼音、生僻字等复杂文本场景的识别。在保持识别效果的同时，兼顾推理速度和模型鲁棒性，为各种场景下的文档理解提供高效、精准的技术支撑。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>81.29</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>16</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec_doc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv4_server_rec_doc_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_doc_pretrained.pdparams">训练模型</a></td>
|
||||
<td>86.58</td>
|
||||
<td>8.69 / 2.78</td>
|
||||
<td>37.93 / 37.93</td>
|
||||
<td>182</td>
|
||||
<td>PP-OCRv4_server_rec_doc是在PP-OCRv4_server_rec的基础上，在更多中文文档数据和PP-OCR训练数据的混合数据训练而成，增加了部分繁体字、日文、特殊字符的识别能力，可支持识别的字符为1.5万+，除文档相关的文字识别能力提升外，也同时提升了通用文字的识别能力</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>78.74</td>
|
||||
<td>5.26 / 1.12</td>
|
||||
<td>17.48 / 3.61</td>
|
||||
<td>10.5</td>
|
||||
<td>PP-OCRv4的轻量级识别模型，推理效率高，可以部署在包含端侧设备的多种硬件设备中</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>85.19</td>
|
||||
<td>8.75 / 2.49</td>
|
||||
<td>36.93 / 36.93</td>
|
||||
<td>173</td>
|
||||
<td>PP-OCRv4的服务器端模型，推理精度高，可以部署在多种不同的服务器上</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv4_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>70.39</td>
|
||||
<td>4.81 / 1.23</td>
|
||||
<td>17.20 / 4.18</td>
|
||||
<td>7.5</td>
|
||||
<td>基于PP-OCRv4识别模型训练得到的超轻量英文识别模型，支持英文、数字识别</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
> ❗ 以上列出的是文本识别模块重点支持的<b>4个核心模型</b>，该模块总共支持<b>20个全量模型</b>，包含多个多语言文本识别模型，完整的模型列表如下：
|
||||
|
||||
<details><summary> 👉模型列表详情</summary>
|
||||
|
||||
* <b>PP-OCRv5 多场景模型</b>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>中文识别 Avg Accuracy(%)</th>
|
||||
<th>英文识别 Avg Accuracy(%)</th>
|
||||
<th>繁体中文识别 Avg Accuracy(%)</th>
|
||||
<th>日文识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>86.38</td>
|
||||
<td>64.70</td>
|
||||
<td>93.29</td>
|
||||
<td>60.35</td>
|
||||
<td>8.46 / 2.36</td>
|
||||
<td>31.21 / 31.21</td>
|
||||
<td>81</td>
|
||||
<td rowspan="2">PP-OCRv5_rec 是新一代文本识别模型。该模型致力于以单一模型高效、精准地支持简体中文、繁体中文、英文、日文四种主要语言，以及手写、竖版、拼音、生僻字等复杂文本场景的识别。在保持识别效果的同时，兼顾推理速度和模型鲁棒性，为各种场景下的文档理解提供高效、精准的技术支撑。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>81.29</td>
|
||||
<td>66.00</td>
|
||||
<td>83.55</td>
|
||||
<td>54.65</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>16</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>中文识别模型</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec_doc</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv4_server_rec_doc_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_doc_pretrained.pdparams">训练模型</a></td>
|
||||
<td>86.58</td>
|
||||
<td>8.69 / 2.78</td>
|
||||
<td>37.93 / 37.93</td>
|
||||
<td>182</td>
|
||||
<td>PP-OCRv4_server_rec_doc是在PP-OCRv4_server_rec的基础上，在更多中文文档数据和PP-OCR训练数据的混合数据训练而成，增加了部分繁体字、日文、特殊字符的识别能力，可支持识别的字符为1.5万+，除文档相关的文字识别能力提升外，也同时提升了通用文字的识别能力</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>78.74</td>
|
||||
<td>5.26 / 1.12</td>
|
||||
<td>17.48 / 3.61</td>
|
||||
<td>10.5</td>
|
||||
<td>PP-OCRv4的轻量级识别模型，推理效率高，可以部署在包含端侧设备的多种硬件设备中</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv4_server_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-OCRv4_server_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv4_server_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>85.19</td>
|
||||
<td>8.75 / 2.49</td>
|
||||
<td>36.93 / 36.93</td>
|
||||
<td>173</td>
|
||||
<td>PP-OCRv4的服务器端模型，推理精度高，可以部署在多种不同的服务器上</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>72.96</td>
|
||||
<td>3.89 / 1.16</td>
|
||||
<td>8.72 / 3.56</td>
|
||||
<td>10.3</td>
|
||||
<td>PP-OCRv3的轻量级识别模型，推理效率高，可以部署在包含端侧设备的多种硬件设备中</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_SVTRv2_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_SVTRv2_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_SVTRv2_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>68.81</td>
|
||||
<td>10.38 / 8.31</td>
|
||||
<td>66.52 / 30.83</td>
|
||||
<td>80.5</td>
|
||||
<td rowspan="1">
|
||||
SVTRv2 是一种由复旦大学视觉与学习实验室（FVL）的OpenOCR团队研发的服务端文本识别模型，其在PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务中荣获一等奖，A榜端到端识别精度相比PP-OCRv4提升6%。
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ch_RepSVTR_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/ch_RepSVTR_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ch_RepSVTR_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>65.07</td>
|
||||
<td>6.29 / 1.57</td>
|
||||
<td>20.64 / 5.40</td>
|
||||
<td>48.8</td>
|
||||
<td rowspan="1"> RepSVTR 文本识别模型是一种基于SVTRv2 的移动端文本识别模型，其在PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务中荣获一等奖，B榜端到端识别精度相比PP-OCRv4提升2.5%，推理速度持平。</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* <b>英文识别模型</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv4_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv4_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv4_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td> 70.39</td>
|
||||
<td>4.81 / 1.23</td>
|
||||
<td>17.20 / 4.18</td>
|
||||
<td>7.5</td>
|
||||
<td>基于PP-OCRv4识别模型训练得到的超轻量英文识别模型，支持英文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>en_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
en_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/en_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>70.69</td>
|
||||
<td>3.56 / 0.78</td>
|
||||
<td>8.44 / 5.78</td>
|
||||
<td>17.3</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量英文识别模型，支持英文、数字识别</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
* <b>多语言识别模型</b>
|
||||
<table>
|
||||
<tr>
|
||||
<th>模型</th><th>模型下载链接</th>
|
||||
<th>识别 Avg Accuracy(%)</th>
|
||||
<th>GPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>CPU推理耗时（ms）<br/>[常规模式 / 高性能模式]</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>korean_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
korean_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>90.45</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>基于PP-OCRv5识别模型训练得到的超轻量韩文识别模型，支持韩文、英文和数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>latin_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
latin_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>84.7</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>基于PP-OCRv5识别模型训练得到的拉丁文识别模型，支持大部分拉丁字母语言、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>eslav_PP-OCRv5_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
eslav_PP-OCRv5_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/eslav_PP-OCRv5_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>85.8</td>
|
||||
<td>5.43 / 1.46</td>
|
||||
<td>21.20 / 5.32</td>
|
||||
<td>14</td>
|
||||
<td>基于PP-OCRv5识别模型训练得到的东斯拉夫语言识别模型， 支持东斯拉夫语言、英文和数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>korean_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
korean_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/korean_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>60.21</td>
|
||||
<td>3.73 / 0.98</td>
|
||||
<td>8.76 / 2.91</td>
|
||||
<td>9.6</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量韩文识别模型，支持韩文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>japan_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
japan_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/japan_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>45.69</td>
|
||||
<td>3.86 / 1.01</td>
|
||||
<td>8.62 / 2.92</td>
|
||||
<td>9.8</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量日文识别模型，支持日文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>chinese_cht_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
chinese_cht_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/chinese_cht_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>82.06</td>
|
||||
<td>3.90 / 1.16</td>
|
||||
<td>9.24 / 3.18</td>
|
||||
<td>10.8</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量繁体中文识别模型，支持繁体中文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>te_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
te_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/te_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>95.88</td>
|
||||
<td>3.59 / 0.81</td>
|
||||
<td>8.28 / 6.21</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量泰卢固文识别模型，支持泰卢固文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ka_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ka_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ka_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>96.96</td>
|
||||
<td>3.49 / 0.89</td>
|
||||
<td>8.63 / 2.77</td>
|
||||
<td>17.4</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量卡纳达文识别模型，支持卡纳达文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>ta_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
ta_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/ta_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>76.83</td>
|
||||
<td>3.49 / 0.86</td>
|
||||
<td>8.35 / 3.41</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量泰米尔文识别模型，支持泰米尔文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>latin_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
latin_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/latin_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>76.93</td>
|
||||
<td>3.53 / 0.78</td>
|
||||
<td>8.50 / 6.83</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量拉丁文识别模型，支持拉丁文、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>arabic_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
arabic_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/arabic_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>73.55</td>
|
||||
<td>3.60 / 0.83</td>
|
||||
<td>8.44 / 4.69</td>
|
||||
<td>17.3</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量阿拉伯字母识别模型，支持阿拉伯字母、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>cyrillic_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
cyrillic_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/cyrillic_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>94.28</td>
|
||||
<td>3.56 / 0.79</td>
|
||||
<td>8.22 / 2.76</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量斯拉夫字母识别模型，支持斯拉夫字母、数字识别</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>devanagari_PP-OCRv3_mobile_rec</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/\
|
||||
devanagari_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/devanagari_PP-OCRv3_mobile_rec_pretrained.pdparams">训练模型</a></td>
|
||||
<td>96.44</td>
|
||||
<td>3.60 / 0.78</td>
|
||||
<td>6.95 / 2.87</td>
|
||||
<td>8.7</td>
|
||||
<td>基于PP-OCRv3识别模型训练得到的超轻量梵文字母识别模型，支持梵文字母、数字识别</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>
|
||||
<ul>
|
||||
<li>
|
||||
中文识别模型： PaddleOCR 自建的中文数据集，覆盖街景、网图、文档、手写多个场景，其中文本识别包含 1.1w 张图片。
|
||||
</li>
|
||||
<li>
|
||||
ch_SVTRv2_rec：<a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务</a>A榜评估集。
|
||||
</li>
|
||||
<li>
|
||||
ch_RepSVTR_rec：<a href="https://aistudio.baidu.com/competition/detail/1131/0/introduction">PaddleOCR算法模型挑战赛 - 赛题一：OCR端到端识别任务</a>B榜评估集。
|
||||
</li>
|
||||
<li>
|
||||
英文识别模型：PaddleOCR 自建的英文数据集。
|
||||
</li>
|
||||
<li>
|
||||
多语言识别模型：PaddleOCR 自建的多语种数据集。
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
</details>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr text_recognition -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将文本识别的模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextRecognition
|
||||
model = TextRecognition(model_name="PP-OCRv5_server_rec")
|
||||
output = model.predict(input="general_ocr_rec_001.png", batch_size=1)
|
||||
for res in output:
|
||||
res.print()
|
||||
res.save_to_img(save_path="./output/")
|
||||
res.save_to_json(save_path="./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
```bash
|
||||
{'res': {'input_path': 'general_ocr_rec_001.png', 'page_index': None, 'rec_text': '绿洲仕格维花园公寓', 'rec_score': 0.9823867082595825}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入待预测文本行图像的路径
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `rec_text`：表示文本行图像的预测文本
|
||||
- `rec_score`：表示文本行图像的预测置信度
|
||||
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/text_recog/general_ocr_rec_001.png"/>
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TextRecognition`实例化文本识别模型（此处以`PP-OCRv5_server_rec`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-OCRv5_server_rec</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>input_shape</code></td>
|
||||
<td>模型输入图像尺寸，格式为 <code>(C, H, W)</code>。</td>
|
||||
<td><code>tuple|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文本识别模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_ocr_rec_001.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>list</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
如果以上模型在您的场景上效果仍然不理想，您可以尝试以下步骤进行二次开发，此处以训练 `PP-OCRv5_server_rec` 举例，其他模型替换对应配置文件即可。首先，您需要准备文本识别的数据集，可以参考[文本识别 Demo 数据](https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_dataset_examples.tar)的格式准备，准备好后，即可按照以下步骤进行模型训练和导出，导出后，可以将模型快速集成到上述 API 中。此处以文本识别 Demo 数据示例。在训练模型之前，请确保已经按照[安装文档](../installation.md)安装了 PaddleOCR 所需要的依赖。
|
||||
|
||||
|
||||
## 4.1 数据集、预训练模型准备
|
||||
|
||||
### 4.1.1 准备数据集
|
||||
|
||||
```shell
|
||||
# 下载示例数据集
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/data/ocr_rec_dataset_examples.tar
|
||||
tar -xf ocr_rec_dataset_examples.tar
|
||||
```
|
||||
|
||||
### 4.1.2 下载预训练模型
|
||||
|
||||
```shell
|
||||
# 下载 PP-OCRv5_server_rec 预训练模型
|
||||
wget https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-OCRv5_server_rec_pretrained.pdparams
|
||||
```
|
||||
|
||||
### 4.2 模型训练
|
||||
|
||||
PaddleOCR 对代码进行了模块化，训练 `PP-OCRv5_server_rec` 识别模型时需要使用 `PP-OCRv5_server_rec` 的[配置文件](https://github.com/PaddlePaddle/PaddleOCR/blob/main/configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml)。
|
||||
|
||||
|
||||
训练命令如下：
|
||||
|
||||
```bash
|
||||
#单卡训练 (默认训练方式)
|
||||
python3 tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
|
||||
#多卡训练，通过--gpus参数指定卡号
|
||||
python3 -m paddle.distributed.launch --gpus '0,1,2,3' tools/train.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml \
|
||||
-o Global.pretrained_model=./PP-OCRv5_server_rec_pretrained.pdparams
|
||||
```
|
||||
|
||||
|
||||
### 4.3 模型评估
|
||||
|
||||
您可以评估已经训练好的权重，如，`output/xxx/xxx.pdparams`，使用如下命令进行评估：
|
||||
|
||||
```bash
|
||||
#注意将pretrained_model的路径设置为本地路径。若使用自行训练保存的模型，请注意修改路径和文件名为{path/to/weights}/{model_name}。
|
||||
#demo 测试集评估
|
||||
python3 tools/eval.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams
|
||||
```
|
||||
|
||||
### 4.4 模型导出
|
||||
|
||||
```bash
|
||||
python3 tools/export_model.py -c configs/rec/PP-OCRv5/PP-OCRv5_server_rec.yml -o \
|
||||
Global.pretrained_model=output/xxx/xxx.pdparams \
|
||||
Global.save_inference_dir="./PP-OCRv5_server_rec_infer/"
|
||||
```
|
||||
|
||||
导出模型后，静态图模型会存放于当前目录的`./PP-OCRv5_server_rec_infer/`中，在该目录下，您将看到如下文件：
|
||||
```
|
||||
./PP-OCRv5_server_rec_infer/
|
||||
├── inference.json
|
||||
├── inference.pdiparams
|
||||
├── inference.yml
|
||||
```
|
||||
至此，二次开发完成，该静态图模型可以直接集成到 PaddleOCR 的 API 中。
|
||||
|
||||
## 五、FAQ
|
||||
@@ -0,0 +1,372 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# Text Line Orientation Classification Module Tutorial
|
||||
|
||||
## 1. Overview
|
||||
The text line orientation classification module primarily distinguishes the orientation of text lines and corrects them using post-processing. In processes such as document scanning and license/certificate photography, to capture clearer images, the capture device may be rotated, resulting in text lines in various orientations. Standard OCR pipelines cannot handle such data well. By utilizing image classification technology, the orientation of text lines can be predetermined and adjusted, thereby enhancing the accuracy of OCR processing.
|
||||
|
||||
## 2. Supported Model List
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Model</th><th>Model Download Link</th>
|
||||
<th>Top-1 Accuracy (%)</th>
|
||||
<th>GPU Inference Time (ms)<br/>[Normal Mode / High-Performance Mode]</th>
|
||||
<th>CPU Inference Time (ms)</th>
|
||||
<th>Model Storage Size (MB)</th>
|
||||
<th>Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x0_25_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x0_25_textline_ori_pretrained.pdparams">Training Model</a></td>
|
||||
<td>98.85</td>
|
||||
<td>2.16 / 0.41</td>
|
||||
<td>2.37 / 0.73</td>
|
||||
<td>0.96</td>
|
||||
<td>Text line classification model based on PP-LCNet_x0_25, with two classes: 0 degrees and 180 degrees</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_textline_ori_infer.tar">Inference Model</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_textline_ori_pretrained.pdparams">Training Model</a></td>
|
||||
<td>99.42</td>
|
||||
<td>- / -</td>
|
||||
<td>2.98 / 2.98</td>
|
||||
<td>6.5</td>
|
||||
<td>Text line classification model based on PP-LCNet_x1_0, with two classes: 0 degrees and 180 degrees</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ **Note**: The text line orientation classification model was upgraded on May 26, 2025, and `PP-LCNet_x1_0_textline_ori` has been added. If you need to use the pre-upgrade model weights, please click the <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.bak.tar">download link</a>.
|
||||
|
||||
<strong>Test Environment Description:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>Performance Test Environment</b>
|
||||
<ul>
|
||||
<li><strong>Test Dataset：</strong> PaddleX Self-built Dataset, Covering Multiple Scenarios Such as Documents and Certificates, Containing 1000 Images.</li>
|
||||
<li><strong>Hardware Configuration:</strong>
|
||||
<ul>
|
||||
<li>GPU: NVIDIA Tesla T4</li>
|
||||
<li>CPU: Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>Software Environment:</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>Inference Mode Description</b></li>
|
||||
</ul>
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Mode</th>
|
||||
<th>GPU Configuration </th>
|
||||
<th>CPU Configuration </th>
|
||||
<th>Acceleration Technology Combination</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>Normal Mode</td>
|
||||
<td>FP32 Precision / No TRT Acceleration</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>High-Performance Mode</td>
|
||||
<td>Optimal combination of pre-selected precision types and acceleration strategies</td>
|
||||
<td>FP32 Precision / 8 Threads</td>
|
||||
<td>Pre-selected optimal backend (Paddle/OpenVINO/TRT, etc.)</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 3. Quick Integration
|
||||
|
||||
> ❗ Before starting, please install the wheel package of PaddleOCR. For detailed instructions, refer to the [Installation Guide](../installation.en.md).
|
||||
|
||||
You can quickly experience the functionality with a single command:
|
||||
|
||||
```bash
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg
|
||||
```
|
||||
|
||||
<b>Note: </b>The official models would be download from HuggingFace by default. If can't access to HuggingFace, please set the environment variable `PADDLE_PDX_MODEL_SOURCE="BOS"` to change the model source to BOS. In the future, more model sources will be supported.
|
||||
|
||||
You can also integrate the text line orientation classification model into your project. Run the following code after downloading the [example image](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg) to your local machine.
|
||||
|
||||
```bash
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(model_name="PP-LCNet_x0_25_textline_ori")
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
After running, the result obtained is:
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'textline_rot180_demo.jpg', 'page_index': None, 'class_ids': array([1], dtype=int32), 'scores': array([0.99864], dtype=float32), 'label_names': ['180_degree']}}
|
||||
```
|
||||
|
||||
The meanings of the running results parameters are as follows:
|
||||
|
||||
- `input_path`：Indicates the path of the input image.
|
||||
- `page_index`：If the input is a PDF file, it indicates the current page number of the PDF; otherwise, it is `None`.
|
||||
- `class_ids`：Indicates the class ID of the prediction result.
|
||||
- `scores`：Indicates the confidence score of the prediction result.
|
||||
- `label_names`：Indicates the class name of the prediction result.
|
||||
The visualization image is as follows:
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/textline_ori_classification/textline_rot180_demo_res.jpg">
|
||||
|
||||
The explanations for the methods, parameters, etc., are as follows:
|
||||
|
||||
* `TextLineOrientationClassification` instantiates a textline classification model (here, `PP-LCNet_x0_25_textline_ori` is used as an example), and the specific explanations are as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>Model name. If set to <code>None</code>, <code>PP-LCNet_x0_25_textline_ori</code> will be used.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>Model storage path.</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>Device for inference.<br/>
|
||||
<b>For example:</b> <code>"cpu"</code>, <code>"gpu"</code>, <code>"npu"</code>, <code>"gpu:0"</code>, <code>"gpu:0,1"</code>.<br/>
|
||||
If multiple devices are specified, parallel inference will be performed.<br/>
|
||||
By default, GPU 0 is used if available; otherwise, CPU is used.
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>Whether to enable high-performance inference.</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>Whether to use the Paddle Inference TensorRT subgraph engine. If the model does not support acceleration through TensorRT, setting this flag will not enable acceleration.<br/>
|
||||
For Paddle with CUDA version 11.8, the compatible TensorRT version is 8.x (x>=6), and it is recommended to install TensorRT 8.6.1.6.<br/>
|
||||
For Paddle with CUDA version 12.6, the compatible TensorRT version is 10.x (x>=5), and it is recommended to install TensorRT 10.5.0.18.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>Computation precision when using the TensorRT subgraph engine in Paddle Inference.<br/><b>Options:</b> <code>"fp32"</code>, <code>"fp16"</code>.</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
Whether to enable MKL-DNN acceleration for inference. If MKL-DNN is unavailable or the model does not support it, acceleration will not be used even if this flag is set.
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN cache capacity.
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>Number of threads to use for inference on CPUs.</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* Use the `predict()` method of the text line direction classification model to perform inference. This method returns a list of results. In addition, this module also provides the `predict_iter()` method. Both methods accept the same parameters and return the same result format. The difference is that `predict_iter()` returns a `generator`, which processes and retrieves prediction results step by step. It is suitable for handling large datasets or memory-efficient scenarios. You can choose either method based on your actual needs. The `predict()` method accepts the parameters `input` and `batch_size`, which are described in detail below:
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Description</th>
|
||||
<th>Type</th>
|
||||
<th>Default</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Input data to be predicted. Required. Supports multiple input types:<ul>
|
||||
<li><b>Python Var</b>: e.g., <code>numpy.ndarray</code> representing image data</li>
|
||||
<li><b>str</b>:
|
||||
- Local image or PDF file path: <code>/root/data/img.jpg</code>;
|
||||
- <b>URL</b> of image or PDF file: e.g., <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">example</a>;
|
||||
- <b>Local directory</b>: directory containing images for prediction, e.g., <code>/root/data/</code> (Note: directories containing PDF files are not supported; PDFs must be specified by exact file path)</li>
|
||||
<li><b>list</b>: Elements must be of the above types, e.g., <code>[numpy.ndarray, numpy.ndarray]</code>, <code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>, <code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size, positive integer.</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Call the `predict()` method of the text line orientation classification model for inference. This method will return a list of results. In addition, this module also provides a `predict_iter()` method. Both methods accept the same parameters and return the same results, but `predict_iter()` returns a `generator`, which is more suitable for processing large datasets or when you want to save memory. You can choose either method according to your needs. The parameters of the `predict()` method are `input` and `batch_size`, as described below:
|
||||
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Options</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>Data to be predicted, supporting multiple input types</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td>
|
||||
<ul>
|
||||
<li><b>Python variable</b>, such as image data represented by <code>numpy.ndarray</code></li>
|
||||
<li><b>File path</b>, such as the local path of an image file: <code>/root/data/img.jpg</code></li>
|
||||
<li><b>URL link</b>, such as the network URL of an image file: <a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg">Example</a></li>
|
||||
<li><b>Local directory</b>, the directory should contain data files to be predicted, such as the local path: <code>/root/data/</code></li>
|
||||
<li><b>list</b>, the elements of the list should be of the above-mentioned data types, such as <code>[numpy.ndarray, numpy.ndarray]</code>, <code>[\"/root/data/img1.jpg\", \"/root/data/img2.jpg\"]</code>, <code>[\"/root/data1\", \"/root/data2\"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>Batch size</td>
|
||||
<td><code>int</code></td>
|
||||
<td>Any integer</td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* The prediction results are processed, and the prediction result for each sample is of type `dict`. It supports operations such as printing, saving as an image, and saving as a `json` file:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Method</th>
|
||||
<th>Method Description</th>
|
||||
<th>Parameter</th>
|
||||
<th>Parameter Type</th>
|
||||
<th>Parameter Description</th>
|
||||
<th>Default Value</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="3"><code>print()</code></td>
|
||||
<td rowspan="3">Print the results to the terminal</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Whether to format the output content using <code>JSON</code> indentation</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. If set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="3"><code>save_to_json()</code></td>
|
||||
<td rowspan="3">Save the results as a JSON file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. If it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>Specify the indentation level to beautify the output <code>JSON</code> data, making it more readable, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>Control whether to escape non-<code>ASCII</code> characters to <code>Unicode</code>. If set to <code>True</code>, all non-<code>ASCII</code> characters will be escaped; <code>False</code> retains the original characters, only effective when <code>format_json</code> is <code>True</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>Save the results as an image file</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>The path to save the file. If it is a directory, the saved file name will be consistent with the input file name</td>
|
||||
<td>None</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* Additionally, it supports obtaining the visualization image with results and the prediction results through attributes, as follows:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>Attribute</th>
|
||||
<th>Attribute Description</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan="1"><code>json</code></td>
|
||||
<td rowspan="1">Get the prediction result in <code>json</code> format</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan="1"><code>img</code></td>
|
||||
<td rowspan="1">Get the visualization image in <code>dict</code> format</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## 4. Custom Development
|
||||
|
||||
Since PaddleOCR does not natively support training for text line orientation classification, refer to [PaddleX's Custom Development Guide](https://paddlepaddle.github.io/PaddleX/latest/en/module_usage/tutorials/ocr_modules/textline_orientation_classification.html#iv-custom-development) for training. Trained models can seamlessly integrate into PaddleOCR's API for inference.
|
||||
@@ -0,0 +1,339 @@
|
||||
---
|
||||
comments: true
|
||||
---
|
||||
|
||||
# 文本行方向分类模块使用教程
|
||||
|
||||
## 一、概述
|
||||
|
||||
文本行方向分类模块主要是将文本行的方向区分出来，并使用后处理将其矫正。在诸如文档扫描、证照拍摄等过程中，有时为了拍摄更清晰，会将拍摄设备进行旋转，导致得到的文本行也是不同方向的。此时，标准的OCR流程无法很好地应对这些数据。利用图像分类技术，可以预先判断文本行方向，并将其进行方向调整，从而提高OCR处理的准确性。
|
||||
|
||||
## 二、支持模型列表
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模型</th>
|
||||
<th>模型下载链接</th>
|
||||
<th>Top-1 Acc（%）</th>
|
||||
<th>GPU推理耗时（ms）</th>
|
||||
<th>CPU推理耗时 (ms)</th>
|
||||
<th>模型存储大小（MB）</th>
|
||||
<th>介绍</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>PP-LCNet_x0_25_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x0_25_textline_ori_pretrained.pdparams">训练模型</a></td>
|
||||
<td>98.85</td>
|
||||
<td>2.16 / 0.41</td>
|
||||
<td>2.37 / 0.73</td>
|
||||
<td>0.96</td>
|
||||
<td>基于PP-LCNet_x0_25的文本行分类模型，含有两个类别，即0度，180度</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>PP-LCNet_x1_0_textline_ori</td>
|
||||
<td><a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x1_0_textline_ori_infer.tar">推理模型</a>/<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_pretrained_model/PP-LCNet_x1_0_textline_ori_pretrained.pdparams">训练模型</a></td>
|
||||
<td>99.42</td>
|
||||
<td>- / -</td>
|
||||
<td>2.98 / 2.98</td>
|
||||
<td>6.5</td>
|
||||
<td>基于PP-LCNet_x1_0的文本行分类模型，含有两个类别，即0度，180度</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
> ❗ <b>注</b>：文本行方向分类模型于 2025.5.26 升级，并增加 `PP-LCNet_x1_0_textline_ori`，如需使用升级前的模型权重，请点击<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/official_inference_model/paddle3.0.0/PP-LCNet_x0_25_textline_ori_infer.bak.tar">下载链接</a>。
|
||||
|
||||
<strong>测试环境说明:</strong>
|
||||
|
||||
<ul>
|
||||
<li><b>性能测试环境</b>
|
||||
<ul>
|
||||
<li><strong>测试数据集：</strong>PaddleOCR 自建的数据集，覆盖证件和文档等多个场景，包含 1000 张图片。</li>
|
||||
<li><strong>硬件配置：</strong>
|
||||
<ul>
|
||||
<li>GPU：NVIDIA Tesla T4</li>
|
||||
<li>CPU：Intel Xeon Gold 6271C @ 2.60GHz</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><strong>软件环境：</strong>
|
||||
<ul>
|
||||
<li>Ubuntu 20.04 / CUDA 11.8 / cuDNN 8.9 / TensorRT 8.6.1.6</li>
|
||||
<li>paddlepaddle 3.0.0 / paddleocr 3.0.3</li>
|
||||
</ul>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li><b>推理模式说明</b></li>
|
||||
</ul>
|
||||
|
||||
|
||||
<table border="1">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模式</th>
|
||||
<th>GPU配置</th>
|
||||
<th>CPU配置</th>
|
||||
<th>加速技术组合</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>常规模式</td>
|
||||
<td>FP32精度 / 无TRT加速</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>PaddleInference</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>高性能模式</td>
|
||||
<td>选择先验精度类型和加速策略的最优组合</td>
|
||||
<td>FP32精度 / 8线程</td>
|
||||
<td>选择先验最优后端（Paddle/OpenVINO/TRT等）</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
## 三、快速开始
|
||||
|
||||
> ❗ 在快速开始前，请先安装 PaddleOCR 的 wheel 包，详细请参考 [安装教程](../installation.md)。
|
||||
|
||||
使用一行命令即可快速体验：
|
||||
|
||||
```bash
|
||||
paddleocr textline_orientation_classification -i https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg
|
||||
```
|
||||
|
||||
<b>注：</b>PaddleOCR 官方模型默认从 HuggingFace 获取，如运行环境访问 HuggingFace 不便，可通过环境变量修改模型源为 BOS：`PADDLE_PDX_MODEL_SOURCE="BOS"`，未来将支持更多主流模型源；
|
||||
|
||||
您也可以将文本行方向分类模块中的模型推理集成到您的项目中。运行以下代码前，请您下载[示例图片](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/textline_rot180_demo.jpg)到本地。
|
||||
|
||||
```python
|
||||
from paddleocr import TextLineOrientationClassification
|
||||
model = TextLineOrientationClassification(model_name="PP-LCNet_x0_25_textline_ori")
|
||||
output = model.predict("textline_rot180_demo.jpg", batch_size=1)
|
||||
for res in output:
|
||||
res.print(json_format=False)
|
||||
res.save_to_img("./output/demo.png")
|
||||
res.save_to_json("./output/res.json")
|
||||
```
|
||||
|
||||
运行后，得到的结果为：
|
||||
|
||||
```bash
|
||||
{'res': {'input_path': 'textline_rot180_demo.jpg', 'page_index': None, 'class_ids': array([1], dtype=int32), 'scores': array([0.99864], dtype=float32), 'label_names': ['180_degree']}}
|
||||
```
|
||||
|
||||
运行结果参数含义如下：
|
||||
- `input_path`：表示输入图片的路径。
|
||||
- `page_index`：如果输入是PDF文件，则表示当前是PDF的第几页，否则为 `None`
|
||||
- `class_ids`：表示预测结果的类别 id，含有两个类别，即0度和180度。
|
||||
- `scores`：表示预测结果的置信度。
|
||||
- `label_names`：表示预测结果的类别名。
|
||||
|
||||
可视化图片如下：
|
||||
|
||||
<img src="https://raw.githubusercontent.com/cuicheng01/PaddleX_doc_images/refs/heads/main/images/modules/textline_ori_classification/textline_rot180_demo_res.jpg">
|
||||
|
||||
相关方法、参数等说明如下：
|
||||
|
||||
* `TextLineOrientationClassification`实例化文本行方向分类模型（此处以`PP-LCNet_x0_25_textline_ori`为例），具体说明如下：
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td><code>model_name</code></td>
|
||||
<td>模型名称。如果设置为<code>None</code>，则使用<code>PP-LCNet_x0_25_textline_ori</code>。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>model_dir</code></td>
|
||||
<td>模型存储路径。</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>device</code></td>
|
||||
<td>用于推理的设备。<br/>
|
||||
<b>例如：</b><code>"cpu"</code>、<code>"gpu"</code>、<code>"npu"</code>、<code>"gpu:0"</code>、<code>"gpu:0,1"</code>。<br/>
|
||||
如指定多个设备，将进行并行推理。<br/>
|
||||
默认情况下，优先使用 GPU 0；若不可用则使用 CPU。
|
||||
</td>
|
||||
<td><code>str|None</code></td>
|
||||
<td><code>None</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_hpi</code></td>
|
||||
<td>是否启用高性能推理。</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>use_tensorrt</code></td>
|
||||
<td>是否启用 Paddle Inference 的 TensorRT 子图引擎。如果模型不支持通过 TensorRT 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
对于 CUDA 11.8 版本的飞桨，兼容的 TensorRT 版本为 8.x（x>=6），建议安装 TensorRT 8.6.1.6。<br/>
|
||||
对于 CUDA 12.6 版本的飞桨，兼容的 TensorRT 版本为 10.x（x>=5），建议安装 TensorRT 10.5.0.18。
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>precision</code></td>
|
||||
<td>当使用 Paddle Inference 的 TensorRT 子图引擎时设置的计算精度。<br/><b>可选项：</b><code>"fp32"</code>、<code>"fp16"</code>。</td>
|
||||
<td><code>str</code></td>
|
||||
<td><code>"fp32"</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>enable_mkldnn</code></td>
|
||||
<td>
|
||||
是否启用 MKL-DNN 加速推理。如果 MKL-DNN 不可用或模型不支持通过 MKL-DNN 加速，即使设置了此标志，也不会使用加速。<br/>
|
||||
</td>
|
||||
<td><code>bool</code></td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>mkldnn_cache_capacity</code></td>
|
||||
<td>
|
||||
MKL-DNN 缓存容量。
|
||||
</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>cpu_threads</code></td>
|
||||
<td>在 CPU 上推理时使用的线程数量。</td>
|
||||
<td><code>int</code></td>
|
||||
<td><code>10</code></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
* 调用文本行方向分类模型的 `predict()` 方法进行推理预测，该方法会返回一个结果列表。另外，本模块还提供了 `predict_iter()` 方法。两者在参数接受和结果返回方面是完全一致的，区别在于 `predict_iter()` 返回的是一个 `generator`，能够逐步处理和获取预测结果，适合处理大型数据集或希望节省内存的场景。可以根据实际需求选择使用这两种方法中的任意一种。`predict()` 方法参数有 `input` 和 `batch_size`，具体说明如下：
|
||||
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>参数</th>
|
||||
<th>参数说明</th>
|
||||
<th>参数类型</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td><code>input</code></td>
|
||||
<td>待预测数据，支持多种输入类型，必填。
|
||||
<ul>
|
||||
<li><b>Python Var</b>：如 <code>numpy.ndarray</code> 表示的图像数据</li>
|
||||
<li><b>str</b>：如图像文件或者PDF文件的本地路径：<code>/root/data/img.jpg</code>；<b>如URL链接</b>，如图像文件或PDF文件的网络URL：<a href="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_doc_preprocessor_002.png">示例</a>；<b>如本地目录</b>，该目录下需包含待预测图像，如本地路径：<code>/root/data/</code>(当前不支持目录中包含PDF文件的预测，PDF文件需要指定到具体文件路径)</li>
|
||||
<li><b>List</b>：列表元素需为上述类型数据，如<code>[numpy.ndarray, numpy.ndarray]</code>，<code>["/root/data/img1.jpg", "/root/data/img2.jpg"]</code>，<code>["/root/data1", "/root/data2"]</code></li>
|
||||
</ul>
|
||||
</td>
|
||||
<td><code>Python Var|str|list</code></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>batch_size</code></td>
|
||||
<td>批大小，可设置为任意正整数。</td>
|
||||
<td><code>int</code></td>
|
||||
<td>1</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 对预测结果进行处理，每个样本的预测结果均为对应的Result对象，且支持打印、保存为图片、保存为`json`文件的操作:
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>方法</th>
|
||||
<th>方法说明</th>
|
||||
<th>参数</th>
|
||||
<th>参数类型</th>
|
||||
<th>参数说明</th>
|
||||
<th>默认值</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>print()</code></td>
|
||||
<td rowspan = "3">打印结果到终端</td>
|
||||
<td><code>format_json</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>是否对输出内容进行使用 <code>JSON</code> 缩进格式化</td>
|
||||
<td><code>True</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "3"><code>save_to_json()</code></td>
|
||||
<td rowspan = "3">将结果保存为json格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>indent</code></td>
|
||||
<td><code>int</code></td>
|
||||
<td>指定缩进级别，以美化输出的 <code>JSON</code> 数据，使其更具可读性，仅当 <code>format_json</code> 为 <code>True</code> 时有效</td>
|
||||
<td>4</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>ensure_ascii</code></td>
|
||||
<td><code>bool</code></td>
|
||||
<td>控制是否将非 <code>ASCII</code> 字符转义为 <code>Unicode</code>。设置为 <code>True</code> 时，所有非 <code>ASCII</code> 字符将被转义；<code>False</code> 则保留原始字符，仅当<code>format_json</code>为<code>True</code>时有效</td>
|
||||
<td><code>False</code></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td><code>save_to_img()</code></td>
|
||||
<td>将结果保存为图像格式的文件</td>
|
||||
<td><code>save_path</code></td>
|
||||
<td><code>str</code></td>
|
||||
<td>保存的文件路径，当为目录时，保存文件命名与输入文件类型命名一致</td>
|
||||
<td>无</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
* 此外，也支持通过属性获取带结果的可视化图像和预测结果，具体如下：
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>属性</th>
|
||||
<th>属性说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>json</code></td>
|
||||
<td rowspan = "1">获取预测的<code>json</code>格式的结果</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td rowspan = "1"><code>img</code></td>
|
||||
<td rowspan = "1">获取格式为<code>dict</code>的可视化图像</td>
|
||||
</tr>
|
||||
|
||||
</table>
|
||||
|
||||
|
||||
## 四、二次开发
|
||||
|
||||
由于 PaddleOCR 并不直接提供文本行方向分类的训练，因此，如果需要训练文档图像方向分类模型，可以参考 [PaddleX 文本行方向分类二次开发](https://paddlepaddle.github.io/PaddleX/latest/module_usage/tutorials/ocr_modules/textline_orientation_classification.html#_5)部分进行训练。训练后的模型可以无缝集成到 PaddleOCR 的 API 中进行推理。
|
||||
Reference in New Issue
Block a user