lamonov.pavel/PaddleOCR
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

first commit
Some checks are pending
Build/Publish Develop Docs / deploy (push) Waiting to run
Details

Browse Source
This commit is contained in:
Ламонов Павел
2025-07-02 08:57:16 +03:00
commit 56532cc9a9
1901 changed files with 457695 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

213
ppstructure/recovery/README.md Normal file
View File

@@ -0,0 +1,213 @@
English | [简体中文](README_ch.md)
# Layout Recovery
- [1. Introduction](#1)
- [2. Install](#2)
- [2.1 Install PaddlePaddle](#2.1)
- [2.2 Install PaddleOCR](#2.2)
- [3. Quick Start using standard PDF parse](#3)
- [4. Quick Start using image format PDF parse ](#4)
- [4.1 Download models](#4.1)
- [4.2 Layout recovery](#4.2)
- [5. More](#5)
<a name="1"></a>
## 1. Introduction
The layout recovery module is used to restore the image or pdf to an
editable Word file consistent with the original image layout.
Two layout recovery methods are provided, you can choose by PDF format:
- **Standard PDF parse(the input is standard PDF)**: Python based PDF to word library [pdf2docx] (https://github.com/dothinking/pdf2docx) is optimized, the method extracts data from PDF with PyMuPDF, then parse layout with rule, finally, generate docx with python-docx.
- **Image format PDF parse(the input can be standard PDF or image format PDF)**: Layout recovery combines [layout analysis](../layout/README.md)、[table recognition](../table/README.md) to better recover images, tables, titles, etc. supports input files in PDF and document image formats in Chinese and English.
The input formats and application scenarios of the two methods are as follows:
| method | input formats | application scenarios/problem |
| :-----: | :----------: | :----------------------------------------------------------: |
| Standard PDF parse | pdf | Advantages: Better recovery for non-paper documents, each page remains on the same page after restoration<br>Disadvantages: English characters in some Chinese documents are garbled, some contents are still beyond the current page, the whole page content is restored to the table format, and the recovery effect of some pictures is not good |
| Image format PDF parse( | pdf、picture | Advantages: More suitable for paper document content recovery, OCR recognition effect is more good<br>Disadvantages: Currently, the recovery is based on rules, the effect of content typesetting (spacing, fonts, etc.) need to be further improved, and the effect of layout recovery depends on layout analysis |
The following figure shows the effect of restoring the layout of documents by using PDF parse:
<div align="center">
<img src="https://user-images.githubusercontent.com/19808900/195319853-045123c9-f542-4596-b4e4-6081708dfc56.png" width = "700" />
</div>
The following figures show the effect of restoring the layout of English and Chinese documents by using OCR technique:
<div align="center">
<img src="../docs/recovery/recovery.jpg" width = "700" />
</div>
<div align="center">
<img src="../docs/recovery/recovery_ch.jpg" width = "800" />
</div>
<a name="2"></a>
## 2. Install
<a name="2.1"></a>
### 2.1 Install PaddlePaddle
```bash
python3 -m pip install --upgrade pip
# If you have cuda9 or cuda10 installed on your machine, please run the following command to install
python3 -m pip install "paddlepaddle-gpu" -i https://mirror.baidu.com/pypi/simple
# CPU installation
python3 -m pip install "paddlepaddle" -i https://mirror.baidu.com/pypi/simple
````
For more requirements, please refer to the instructions in [Installation Documentation](https://www.paddlepaddle.org.cn/en/install/quick?docurl=/documentation/docs/en/install/pip/macos-pip_en.html).
<a name="2.2"></a>
### 2.2 Install PaddleOCR
- **(1) Download source code**
```bash
[Recommended] git clone https://github.com/PaddlePaddle/PaddleOCR
# If the pull cannot be successful due to network problems, you can also choose to use the hosting on the code cloud:
git clone https://gitee.com/paddlepaddle/PaddleOCR
# Note: Code cloud hosting code may not be able to synchronize the update of this github project in real time, there is a delay of 3 to 5 days, please use the recommended method first.
````
- **(2) Install recovery `requirements`**
The layout restoration is exported as docx files, so python-docx API need to be installed, and PyMuPDF api([requires Python >= 3.7](https://pypi.org/project/PyMuPDF/)) need to be installed to process the input files in pdf format.
Install all the libraries by running the following command:
```bash
python3 -m pip install -r ppstructure/recovery/requirements.txt
````
And if using pdf parse method, we need to install pdf2docx api.
```bash
wget https://paddleocr.bj.bcebos.com/whl/pdf2docx-0.0.0-py3-none-any.whl
pip3 install pdf2docx-0.0.0-py3-none-any.whl
```
<a name="3"></a>
## 3. Quick Start using standard PDF parse
`use_pdf2docx_api` use PDF parse for layout recovery, The whl package is also provided for quick use, follow the above code, for more information please refer to [quickstart](../docs/quickstart_en.md) for details.
```bash
# install paddleocr
pip3 install "paddleocr>=2.6"
paddleocr --image_dir=ppstructure/docs/recovery/UnrealText.pdf --type=structure --recovery=true --use_pdf2docx_api=true
```
Command line:
```bash
python3 predict_system.py \
--image_dir=ppstructure/docs/recovery/UnrealText.pdf \
--recovery=True \
--use_pdf2docx_api=True \
--output=../output/
```
<a name="4"></a>
## 4. Quick Start using image format PDF parse
Through layout analysis, we divided the image/PDF documents into regions, located the key regions, such as text, table, picture, etc., and recorded the location, category, and regional pixel value information of each region. Different regions are processed separately, where:
- OCR detection and recognition is performed in the text area, and the coordinates of the OCR detection box and the text content information are added on the basis of the previous information
- The table area identifies tables and records html and text information of tables
- Save the image directly
We can restore the test picture through the layout information, OCR detection and recognition structure, table information, and saved pictures.
The whl package is also provided for quick use, follow the above code, for more information please refer to [quickstart](../docs/quickstart_en.md) for details.
```bash
paddleocr --image_dir=ppstructure/docs/table/1.png --type=structure --recovery=true --lang='en'
```
<a name="4.1"></a>
### 4.1 Download models
If input is English document, download English models:
```bash
cd PaddleOCR/ppstructure
# download model
mkdir inference && cd inference
# Download the detection model of the ultra-lightweight English PP-OCRv3 model and unzip it
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_det_infer.tar && tar xf en_PP-OCRv3_det_infer.tar
# Download the recognition model of the ultra-lightweight English PP-OCRv3 model and unzip it
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_rec_infer.tar && tar xf en_PP-OCRv3_rec_infer.tar
# Download the ultra-lightweight English table inch model and unzip it
wget https://paddleocr.bj.bcebos.com/ppstructure/models/slanet/paddle3.0b2/en_ppstructure_mobile_v2.0_SLANet_infer.tar
tar xf en_ppstructure_mobile_v2.0_SLANet_infer.tar
# Download the layout model of publaynet dataset and unzip it
wget https://paddleocr.bj.bcebos.com/ppstructure/models/layout/picodet_lcnet_x1_0_fgd_layout_infer.tar
tar xf picodet_lcnet_x1_0_fgd_layout_infer.tar
cd ..
```
If input is Chinese document，download Chinese models:
[Chinese and English ultra-lightweight PP-OCRv3 model](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/README.md#pp-ocr-series-model-listupdate-on-september-8th)、[table recognition model](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/ppstructure/docs/models_list.md#22-表格识别模型)、[layout analysis model](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/ppstructure/docs/models_list.md#1-版面分析模型)
<a name="4.2"></a>
### 4.2 Layout recovery
```bash
python3 predict_system.py \
--image_dir=./docs/table/1.png \
--det_model_dir=inference/en_PP-OCRv3_det_infer \
--rec_model_dir=inference/en_PP-OCRv3_rec_infer \
--rec_char_dict_path=../ppocr/utils/en_dict.txt \
--table_model_dir=inference/en_ppstructure_mobile_v2.0_SLANet_infer \
--table_char_dict_path=../ppocr/utils/dict/table_structure_dict.txt \
--layout_model_dir=inference/picodet_lcnet_x1_0_fgd_layout_infer \
--layout_dict_path=../ppocr/utils/dict/layout_dict/layout_publaynet_dict.txt \
--vis_font_path=../doc/fonts/simfang.ttf \
--recovery=True \
--output=../output/
```
After running, the docx of each picture will be saved in the directory specified by the output field
Field：
- image_dir：test file， can be picture, picture directory, pdf file, pdf file directory
- det_model_dir：OCR detection model path
- rec_model_dir：OCR recognition model path
- rec_char_dict_path：OCR recognition dict path. If the Chinese model is used, change to "../ppocr/utils/ppocr_keys_v1.txt". And if you trained the model on your own dataset, change to the trained dictionary
- table_model_dir：table recognition model path
- table_char_dict_path：table recognition dict path. If the Chinese model is used, no need to change
- layout_model_dir：layout analysis model path
- layout_dict_path：layout analysis dict path. If the Chinese model is used, change to "../ppocr/utils/dict/layout_dict/layout_cdla_dict.txt"
- recovery：whether to enable layout of recovery, default False
- output：save the recovery result path
<a name="5"></a>
## 5. More
For training, evaluation and inference tutorial for text detection models, please refer to [text detection doc](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_en/detection_en.md).
For training, evaluation and inference tutorial for text recognition models, please refer to [text recognition doc](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_en/recognition_en.md).
For training, evaluation and inference tutorial for layout analysis models, please refer to [layout analysis doc](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/ppstructure/layout/README.md)
For training, evaluation and inference tutorial for table recognition models, please refer to [table recognition doc](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/ppstructure/table/README.md)

222
ppstructure/recovery/README_ch.md Normal file
View File

@@ -0,0 +1,222 @@
[English](README.md) | 简体中文
# 版面恢复
- [1. 简介](#1)
- [2. 安装](#2)
- [2.1 安装PaddlePaddle](#2.1)
- [2.2 安装PaddleOCR](#2.2)
- [3.使用标准PDF解析进行版面恢复](#3)
- [4. 使用图片格式PDF解析进行版面恢复](#4)
- [4.1 下载模型](#4.1)
- [4.2 版面恢复](#4.2)
- [5. 更多](#5)
<a name="1"></a>
## 1. 简介
版面恢复就是将输入的图片、pdf内容仍然像原文档那样排列着，段落不变、顺序不变的输出到word文档中等。
提供了2种版面恢复方法，可根据输入PDF的格式进行选择：
- **标准PDF解析(输入须为标准PDF)**：基于Python的pdf转word库[pdf2docx](https://github.com/dothinking/pdf2docx)进行优化，该方法通过PyMuPDF获取页面元素，然后利用规则解析章节、段落、表格等布局及样式，最后通过python-docx将解析的内容元素重建到word文档中。
- **图片格式PDF解析(输入可为标准PDF或图片格式PDF)**：结合[版面分析](../layout/README_ch.md)、[表格识别](../table/README_ch.md)技术，从而更好地恢复图片、表格、标题等内容，支持中、英文pdf文档、文档图片格式的输入文件。
2种方法输入格式、适用场景如下：
| 方法 | 支持输入文件 | 适用场景/存在问题 |
| :-------------: | :----------: | :----------------------------------------------------------: |
| 标准PDF解析 | pdf | 优点：非论文文档恢复效果更优、每一页内容恢复后仍在同一页<br>缺点：有些中文文档中的英文乱码、仍存在内容超出当前页面的情况、整页内容恢复为表格格式、部分图片恢复效果不佳 |
| 图片格式PDF解析 | pdf、图片 | 优点：更适合论文文档正文内容的恢复、中英文文档OCR识别效果好<br>缺点：目前内容恢复基于规则，内容排版效果(间距、字体等)待进一步提升、版面恢复效果依赖于版面分析效果 |
下图展示了通过PDF解析版面恢复效果：
<div align="center">
<img src="https://user-images.githubusercontent.com/19808900/195319840-68fc60ec-ea66-4095-b734-0ec115860341.png" width = "700" />
</div>
下图分别展示了通过OCR技术，英文文档和中文文档版面恢复的效果：
<div align="center">
<img src="../docs/recovery/recovery.jpg" width = "700" />
</div>
<div align="center">
<img src="../docs/recovery/recovery_ch.jpg" width = "800" />
</div>
<a name="2"></a>
## 2. 安装
<a name="2.1"></a>
### 2.1 安装PaddlePaddle
```bash
python3 -m pip install --upgrade pip
# 您的机器安装的是CUDA9或CUDA10，请运行以下命令安装
python3 -m pip install "paddlepaddle-gpu" -i https://mirror.baidu.com/pypi/simple
# 您的机器是CPU，请运行以下命令安装
python3 -m pip install "paddlepaddle" -i https://mirror.baidu.com/pypi/simple
```
更多需求，请参照[安装文档](https://www.paddlepaddle.org.cn/install/quick)中的说明进行操作。
<a name="2.2"></a>
### 2.2 安装PaddleOCR
- **（1）下载版面恢复源码**
```bash
【推荐】git clone https://github.com/PaddlePaddle/PaddleOCR
# 如果因为网络问题无法pull成功，也可选择使用码云上的托管：
git clone https://gitee.com/paddlepaddle/PaddleOCR
# 注：码云托管代码可能无法实时同步本github项目更新，存在3~5天延时，请优先使用推荐方式。
```
- **（2）安装recovery的`requirements`**
版面恢复导出为docx文件，所以需要安装Python处理word文档的python-docx API，同时处理pdf格式的输入文件，需要安装PyMuPDF API([要求Python >= 3.7](https://pypi.org/project/PyMuPDF/))。
通过如下命令安装全部库：
```bash
python3 -m pip install -r ppstructure/recovery/requirements.txt
```
使用pdf2docx库解析的方式恢复文档需要安装优化的pdf2docx。
```bash
wget https://paddleocr.bj.bcebos.com/whl/pdf2docx-0.0.0-py3-none-any.whl
pip3 install pdf2docx-0.0.0-py3-none-any.whl
```
<a name="3"></a>
## 3.使用标准PDF解析进行版面恢复
`use_pdf2docx_api`表示使用PDF解析的方式进行版面恢复，通过whl包的形式方便快速使用，代码如下，更多信息详见 [quickstart](../docs/quickstart.md)。
```bash
# 安装 paddleocr，推荐使用2.6版本
pip3 install "paddleocr>=2.6"
paddleocr --image_dir=ppstructure/docs/recovery/UnrealText.pdf --type=structure --recovery=true --use_pdf2docx_api=true
```
通过命令行的方式：
```bash
python3 predict_system.py \
--image_dir=ppstructure/docs/recovery/UnrealText.pdf \
--recovery=True \
--use_pdf2docx_api=True \
--output=../output/
```
<a name="4"></a>
## 4.使用图片格式PDF解析进行版面恢复
我们通过版面分析对图片/pdf形式的文档进行区域划分，定位其中的关键区域，如文字、表格、图片等，记录每个区域的位置、类别、区域像素值信息。对不同的区域分别处理，其中：
- 文字区域直接进行OCR检测和识别，在之前信息基础上增加OCR检测框坐标和文本内容信息
- 表格区域进行表格识别，记录表格html和文字信息
- 图片直接保存
我们通过版面信息、OCR检测和识别结构、表格信息、保存的图片，对测试图片进行恢复即可。
提供如下代码实现版面恢复，也提供了whl包的形式方便快速使用，代码如下，更多信息详见 [quickstart](../docs/quickstart.md)。
```bash
# 安装 paddleocr，推荐使用2.6版本
pip3 install "paddleocr>=2.6"
# 中文测试图
paddleocr --image_dir=ppstructure/docs/table/1.png --type=structure --recovery=true
# 英文测试图
paddleocr --image_dir=ppstructure/docs/table/1.png --type=structure --recovery=true --lang='en'
# pdf测试文件
paddleocr --image_dir=ppstructure/docs/recovery/UnrealText.pdf --type=structure --recovery=true --lang='en'
```
<a name="4.1"></a>
### 4.1 下载模型
如果输入为英文文档类型，下载OCR检测和识别、版面分析、表格识别的英文模型
```bash
cd PaddleOCR/ppstructure
# 下载模型
mkdir inference && cd inference
# 下载英文超轻量PP-OCRv3检测模型并解压
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_det_infer.tar && tar xf en_PP-OCRv3_det_infer.tar
# 下载英文超轻量PP-OCRv3识别模型并解压
wget https://paddleocr.bj.bcebos.com/PP-OCRv3/english/en_PP-OCRv3_rec_infer.tar && tar xf en_PP-OCRv3_rec_infer.tar
# 下载英文表格识别模型并解压
wget https://paddleocr.bj.bcebos.com/ppstructure/models/slanet/paddle3.0b2/en_ppstructure_mobile_v2.0_SLANet_infer.tar
tar xf en_ppstructure_mobile_v2.0_SLANet_infer.tar
# 下载英文版面分析模型
wget https://paddleocr.bj.bcebos.com/ppstructure/models/layout/picodet_lcnet_x1_0_fgd_layout_infer.tar
tar xf picodet_lcnet_x1_0_fgd_layout_infer.tar
cd ..
```
如果输入为中文文档类型，在下述链接中下载中文模型即可：
[PP-OCRv3中英文超轻量文本检测和识别模型](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/README_ch.md#pp-ocr%E7%B3%BB%E5%88%97%E6%A8%A1%E5%9E%8B%E5%88%97%E8%A1%A8%E6%9B%B4%E6%96%B0%E4%B8%AD)、[表格识别模型](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/ppstructure/docs/models_list.md#22-表格识别模型)、[版面分析模型](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/ppstructure/docs/models_list.md#1-版面分析模型)
<a name="4.2"></a>
### 4.2 版面恢复
使用下载的模型恢复给定文档的版面，以英文模型为例，执行如下命令：
```bash
python3 predict_system.py \
--image_dir=./docs/table/1.png \
--det_model_dir=inference/en_PP-OCRv3_det_infer \
--rec_model_dir=inference/en_PP-OCRv3_rec_infer \
--rec_char_dict_path=../ppocr/utils/en_dict.txt \
--table_model_dir=inference/en_ppstructure_mobile_v2.0_SLANet_infer \
--table_char_dict_path=../ppocr/utils/dict/table_structure_dict.txt \
--layout_model_dir=inference/picodet_lcnet_x1_0_fgd_layout_infer \
--layout_dict_path=../ppocr/utils/dict/layout_dict/layout_publaynet_dict.txt \
--vis_font_path=../doc/fonts/simfang.ttf \
--recovery=True \
--output=../output/
```
运行完成后，恢复版面的docx文档会保存到`output`字段指定的目录下
字段含义：
- image_dir：测试文件，可以是图片、图片目录、pdf文件、pdf文件目录
- det_model_dir：OCR检测模型路径
- rec_model_dir：OCR识别模型路径
- rec_char_dict_path：OCR识别字典，如果更换为中文模型，需要更改为"../ppocr/utils/ppocr_keys_v1.txt"，如果您在自己的数据集上训练的模型，则更改为训练的字典的文件
- table_model_dir：表格识别模型路径
- table_char_dict_path：表格识别字典，如果更换为中文模型，不需要更换字典
- layout_model_dir：版面分析模型路径
- layout_dict_path：版面分析字典，如果更换为中文模型，需要更改为"../ppocr/utils/dict/layout_dict/layout_cdla_dict.txt"
- recovery：是否进行版面恢复，默认False
- output：版面恢复结果保存路径
<a name="5"></a>
## 5. 更多
关于OCR检测模型的训练评估与推理，请参考：[文本检测教程](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/detection.md)
关于OCR识别模型的训练评估与推理，请参考：[文本识别教程](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/doc/doc_ch/recognition.md)
关于版面分析模型的训练评估与推理，请参考：[版面分析教程](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/ppstructure/layout/README_ch.md)
关于表格识别模型的训练评估与推理，请参考：[表格识别教程](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/ppstructure/table/README_ch.md)

13
ppstructure/recovery/__init__.py Normal file
View File

@@ -0,0 +1,13 @@
# copyright (c) 2020 PaddlePaddle Authors. All Rights Reserve.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

155
ppstructure/recovery/recovery_to_doc.py Normal file
View File

@@ -0,0 +1,155 @@
# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
from copy import deepcopy
from docx import Document
from docx import shared
from docx.enum.text import WD_ALIGN_PARAGRAPH
from docx.enum.section import WD_SECTION
from docx.oxml.ns import qn
from docx.enum.table import WD_TABLE_ALIGNMENT
from ppstructure.recovery.table_process import HtmlToDocx
from ppocr.utils.logging import get_logger
logger = get_logger()
def convert_info_docx(img, res, save_folder, img_name):
doc = Document()
doc.styles["Normal"].font.name = "Times New Roman"
doc.styles["Normal"]._element.rPr.rFonts.set(qn("w:eastAsia"), "宋体")
doc.styles["Normal"].font.size = shared.Pt(6.5)
flag = 1
for i, region in enumerate(res):
if not region["res"] and region["type"].lower() != "figure":
continue
img_idx = region["img_idx"]
if flag == 2 and region["layout"] == "single":
section = doc.add_section(WD_SECTION.CONTINUOUS)
section._sectPr.xpath("./w:cols")[0].set(qn("w:num"), "1")
flag = 1
elif flag == 1 and region["layout"] == "double":
section = doc.add_section(WD_SECTION.CONTINUOUS)
section._sectPr.xpath("./w:cols")[0].set(qn("w:num"), "2")
flag = 2
if region["type"].lower() == "figure":
excel_save_folder = os.path.join(save_folder, img_name)
img_path = os.path.join(
excel_save_folder, "{}_{}.jpg".format(region["bbox"], img_idx)
)
paragraph_pic = doc.add_paragraph()
paragraph_pic.alignment = WD_ALIGN_PARAGRAPH.CENTER
run = paragraph_pic.add_run("")
if flag == 1:
run.add_picture(img_path, width=shared.Inches(5))
elif flag == 2:
run.add_picture(img_path, width=shared.Inches(2))
elif region["type"].lower() == "title":
doc.add_heading(region["res"][0]["text"])
elif region["type"].lower() == "table":
parser = HtmlToDocx()
parser.table_style = "TableGrid"
parser.handle_table(region["res"]["html"], doc)
elif region["type"] == "equation" and "latex" in region["res"]:
pass
else:
paragraph = doc.add_paragraph()
paragraph_format = paragraph.paragraph_format
for i, line in enumerate(region["res"]):
if i == 0:
paragraph_format.first_line_indent = shared.Inches(0.25)
text_run = paragraph.add_run(line["text"] + " ")
text_run.font.size = shared.Pt(10)
# save to docx
docx_path = os.path.join(save_folder, "{}_ocr.docx".format(img_name))
doc.save(docx_path)
logger.info("docx save to {}".format(docx_path))
def sorted_layout_boxes(res, w):
"""
Sort text boxes in order from top to bottom, left to right
args:
res(list):ppstructure results
return:
sorted results(list)
"""
num_boxes = len(res)
if num_boxes == 1:
res[0]["layout"] = "single"
return res
sorted_boxes = sorted(res, key=lambda x: (x["bbox"][1], x["bbox"][0]))
_boxes = list(sorted_boxes)
new_res = []
res_left = []
res_right = []
i = 0
while True:
if i >= num_boxes:
break
if i == num_boxes - 1:
if (
_boxes[i]["bbox"][1] > _boxes[i - 1]["bbox"][3]
and _boxes[i]["bbox"][0] < w / 2
and _boxes[i]["bbox"][2] > w / 2
):
new_res += res_left
new_res += res_right
_boxes[i]["layout"] = "single"
new_res.append(_boxes[i])
else:
if _boxes[i]["bbox"][2] > w / 2:
_boxes[i]["layout"] = "double"
res_right.append(_boxes[i])
new_res += res_left
new_res += res_right
elif _boxes[i]["bbox"][0] < w / 2:
_boxes[i]["layout"] = "double"
res_left.append(_boxes[i])
new_res += res_left
new_res += res_right
res_left = []
res_right = []
break
elif _boxes[i]["bbox"][0] < w / 4 and _boxes[i]["bbox"][2] < 3 * w / 4:
_boxes[i]["layout"] = "double"
res_left.append(_boxes[i])
i += 1
elif _boxes[i]["bbox"][0] > w / 4 and _boxes[i]["bbox"][2] > w / 2:
_boxes[i]["layout"] = "double"
res_right.append(_boxes[i])
i += 1
else:
new_res += res_left
new_res += res_right
_boxes[i]["layout"] = "single"
new_res.append(_boxes[i])
res_left = []
res_right = []
i += 1
if res_left:
new_res += res_left
if res_right:
new_res += res_right
return new_res

187
ppstructure/recovery/recovery_to_markdown.py Normal file
View File

@@ -0,0 +1,187 @@
# Copyright (c) 2024 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import re
from ppocr.utils.logging import get_logger
logger = get_logger()
def check_merge_method(in_region):
"""Select the function to merge paragraph.
Determine the paragraph merging method based on the positional
relationship between the text bbox and the first line of text in the text bbox.
Args:
in_region: Elements with text type in the layout result.
Returns:
Merge the functions of paragraph, convert_text_space_head or convert_text_space_tail.
"""
text_bbox = in_region["bbox"]
text_x1 = text_bbox[0]
frist_line_box = in_region["res"][0]["text_region"]
point_1 = frist_line_box[0]
point_2 = frist_line_box[2]
frist_line_x1 = point_1[0]
frist_line_height = abs(point_2[1] - point_1[1])
x1_distance = frist_line_x1 - text_x1
return (
convert_text_space_head
if x1_distance > frist_line_height
else convert_text_space_tail
)
def convert_text_space_head(in_region):
"""The function to merge paragraph.
The sign of dividing paragraph is that there are two spaces at the beginning.
Args:
in_region: Elements with text type in the layout result.
Returns:
The text content of the current text box.
"""
text = ""
pre_x = None
frist_line = True
for i, res in enumerate(in_region["res"]):
point1 = res["text_region"][0]
point2 = res["text_region"][2]
h = point2[1] - point1[1]
if i == 0:
text += res["text"]
pre_x = point1[0]
continue
x1 = point1[0]
if frist_line:
if abs(pre_x - x1) < h:
text += "\n\n"
text += res["text"]
frist_line = True
else:
text += res["text"]
frist_line = False
else:
same_paragh = abs(pre_x - x1) < h
if same_paragh:
text += res["text"]
frist_line = False
else:
text += "\n\n"
text += res["text"]
frist_line = True
pre_x = x1
return text
def convert_text_space_tail(in_region):
"""The function to merge paragraph.
The symbol for dividing paragraph is a space at the end.
Args:
in_region: Elements with text type in the layout result.
Returns:
The text content of the current text box.
"""
text = ""
frist_line = True
text_bbox = in_region["bbox"]
width = text_bbox[2] - text_bbox[0]
for i, res in enumerate(in_region["res"]):
point1 = res["text_region"][0]
point2 = res["text_region"][2]
row_width = point2[0] - point1[0]
row_height = point2[1] - point1[1]
full_row_threshold = width - row_height
is_full = row_width >= full_row_threshold
if frist_line:
text += "\n\n"
text += res["text"]
else:
text += res["text"]
frist_line = not is_full
return text
def convert_info_markdown(res, save_folder, img_name):
"""Save the recognition result as a markdown file.
Args:
res: Recognition result
save_folder: Folder to save the markdown file
img_name: PDF file or image file name
Returns:
None
"""
def replace_special_char(content):
special_chars = ["*", "`", "~", "$"]
for char in special_chars:
content = content.replace(char, "\\" + char)
return content
markdown_string = []
for i, region in enumerate(res):
if not region["res"] and region["type"].lower() != "figure":
continue
img_idx = region["img_idx"]
if region["type"].lower() == "figure":
img_file_name = "{}_{}.jpg".format(region["bbox"], img_idx)
markdown_string.append(
f"""<div align="center">\n\t<img src="{img_name+"/"+img_file_name}">\n</div>"""
)
elif region["type"].lower() == "title":
markdown_string.append(
f"""# {region['res'][0]['text']}"""
+ "".join(
[" " + one_region["text"] for one_region in region["res"][1:]]
)
)
elif region["type"].lower() == "table":
markdown_string.append(region["res"]["html"])
elif region["type"].lower() == "header" or region["type"].lower() == "footer":
pass
elif region["type"].lower() == "equation" and "latex" in region["res"]:
markdown_string.append(f"""$${region["res"]["latex"]}$$""")
elif region["type"].lower() == "text":
merge_func = check_merge_method(region)
# logger.warning(f"use merge method:{merge_func.__name__}")
markdown_string.append(replace_special_char(merge_func(region)))
else:
string = ""
for line in region["res"]:
string += line["text"] + " "
markdown_string.append(string)
md_path = os.path.join(save_folder, "{}_ocr.md".format(img_name))
markdown_string = "\n\n".join(markdown_string)
markdown_string = re.sub(r"\n{3,}", "\n\n", markdown_string)
with open(md_path, "w", encoding="utf-8") as f:
f.write(markdown_string)
logger.info("markdown save to {}".format(md_path))

4
ppstructure/recovery/requirements.txt Normal file
View File

@@ -0,0 +1,4 @@
python-docx
beautifulsoup4
fonttools>=4.43.0
fire>=0.3.0

325
ppstructure/recovery/table_process.py Normal file
View File

@@ -0,0 +1,325 @@
# Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""
This code is refer from: https://github.com/weizwx/html2docx/blob/master/htmldocx/h2d.py
"""
import re
import docx
from docx import Document
from bs4 import BeautifulSoup
from html.parser import HTMLParser
def get_table_rows(table_soup):
table_row_selectors = [
"table > tr",
"table > thead > tr",
"table > tbody > tr",
"table > tfoot > tr",
]
# If there's a header, body, footer or direct child tr tags, add row dimensions from there
return table_soup.select(", ".join(table_row_selectors), recursive=False)
def get_table_columns(row):
# Get all columns for the specified row tag.
return row.find_all(["th", "td"], recursive=False) if row else []
def get_table_dimensions(table_soup):
# Get rows for the table
rows = get_table_rows(table_soup)
# Table is either empty or has non-direct children between table and tr tags
# Thus the row dimensions and column dimensions are assumed to be 0
cols = get_table_columns(rows[0]) if rows else []
# Add colspan calculation column number
col_count = 0
for col in cols:
colspan = col.attrs.get("colspan", 1)
col_count += int(colspan)
return rows, col_count
def get_cell_html(soup):
# Returns string of td element with opening and closing <td> tags removed
# Cannot use find_all as it only finds element tags and does not find text which
# is not inside an element
return " ".join([str(i) for i in soup.contents])
def delete_paragraph(paragraph):
# https://github.com/python-openxml/python-docx/issues/33#issuecomment-77661907
p = paragraph._element
p.getparent().remove(p)
p._p = p._element = None
def remove_whitespace(string, leading=False, trailing=False):
"""Remove white space from a string.
Args:
string(str): The string to remove white space from.
leading(bool, optional): Remove leading new lines when True.
trailing(bool, optional): Remove trailing new lines when False.
Returns:
str: The input string with new line characters removed and white space squashed.
Examples:
Single or multiple new line characters are replaced with space.
>>> remove_whitespace("abc\\ndef")
'abc def'
>>> remove_whitespace("abc\\n\\n\\ndef")
'abc def'
New line characters surrounded by white space are replaced with a single space.
>>> remove_whitespace("abc \\n \\n \\n def")
'abc def'
>>> remove_whitespace("abc \\n \\n \\n def")
'abc def'
Leading and trailing new lines are replaced with a single space.
>>> remove_whitespace("\\nabc")
' abc'
>>> remove_whitespace(" \\n abc")
' abc'
>>> remove_whitespace("abc\\n")
'abc '
>>> remove_whitespace("abc \\n ")
'abc '
Use ``leading=True`` to remove leading new line characters, including any surrounding
white space:
>>> remove_whitespace("\\nabc", leading=True)
'abc'
>>> remove_whitespace(" \\n abc", leading=True)
'abc'
Use ``trailing=True`` to remove trailing new line characters, including any surrounding
white space:
>>> remove_whitespace("abc \\n ", trailing=True)
'abc'
"""
# Remove any leading new line characters along with any surrounding white space
if leading:
string = re.sub(r"^\s*\n+\s*", "", string)
# Remove any trailing new line characters along with any surrounding white space
if trailing:
string = re.sub(r"\s*\n+\s*$", "", string)
# Replace new line characters and absorb any surrounding space.
string = re.sub(r"\s*\n\s*", " ", string)
# TODO need some way to get rid of extra spaces in e.g. text <span> </span> text
return re.sub(r"\s+", " ", string)
font_styles = {
"b": "bold",
"strong": "bold",
"em": "italic",
"i": "italic",
"u": "underline",
"s": "strike",
"sup": "superscript",
"sub": "subscript",
"th": "bold",
}
font_names = {
"code": "Courier",
"pre": "Courier",
}
class HtmlToDocx(HTMLParser):
def __init__(self):
super().__init__()
self.options = {
"fix-html": True,
"images": True,
"tables": True,
"styles": True,
}
self.table_row_selectors = [
"table > tr",
"table > thead > tr",
"table > tbody > tr",
"table > tfoot > tr",
]
self.table_style = None
self.paragraph_style = None
def set_initial_attrs(self, document=None):
self.tags = {
"span": [],
"list": [],
}
if document:
self.doc = document
else:
self.doc = Document()
self.bs = self.options["fix-html"] # whether or not to clean with BeautifulSoup
self.document = self.doc
self.include_tables = True # TODO add this option back in?
self.include_images = self.options["images"]
self.include_styles = self.options["styles"]
self.paragraph = None
self.skip = False
self.skip_tag = None
self.instances_to_skip = 0
def copy_settings_from(self, other):
"""Copy settings from another instance of HtmlToDocx"""
self.table_style = other.table_style
self.paragraph_style = other.paragraph_style
def ignore_nested_tables(self, tables_soup):
"""
Returns array containing only the highest level tables
Operates on the assumption that bs4 returns child elements immediately after
the parent element in `find_all`. If this changes in the future, this method will need to be updated
:return:
"""
new_tables = []
nest = 0
for table in tables_soup:
if nest:
nest -= 1
continue
new_tables.append(table)
nest = len(table.find_all("table"))
return new_tables
def get_tables(self):
if not hasattr(self, "soup"):
self.include_tables = False
return
# find other way to do it, or require this dependency?
self.tables = self.ignore_nested_tables(self.soup.find_all("table"))
self.table_no = 0
def run_process(self, html):
if self.bs and BeautifulSoup:
self.soup = BeautifulSoup(html, "html.parser")
html = str(self.soup)
if self.include_tables:
self.get_tables()
self.feed(html)
def add_html_to_cell(self, html, cell):
if not isinstance(cell, docx.table._Cell):
raise ValueError("Second argument needs to be a %s" % docx.table._Cell)
unwanted_paragraph = cell.paragraphs[0]
if unwanted_paragraph.text == "":
delete_paragraph(unwanted_paragraph)
self.set_initial_attrs(cell)
self.run_process(html)
# cells must end with a paragraph or will get message about corrupt file
# https://stackoverflow.com/a/29287121
if not self.doc.paragraphs:
self.doc.add_paragraph("")
def apply_paragraph_style(self, style=None):
try:
if style:
self.paragraph.style = style
elif self.paragraph_style:
self.paragraph.style = self.paragraph_style
except KeyError as e:
raise ValueError(f"Unable to apply style {self.paragraph_style}.") from e
def handle_table(self, html, doc):
"""
To handle nested tables, we will parse tables manually as follows:
Get table soup
Create docx table
Iterate over soup and fill docx table with new instances of this parser
Tell HTMLParser to ignore any tags until the corresponding closing table tag
"""
table_soup = BeautifulSoup(html, "html.parser")
rows, cols_len = get_table_dimensions(table_soup)
table = doc.add_table(len(rows), cols_len)
table.style = doc.styles["Table Grid"]
num_rows = len(table.rows)
num_cols = len(table.columns)
cell_row = 0
for index, row in enumerate(rows):
cols = get_table_columns(row)
cell_col = 0
for col in cols:
colspan = int(col.attrs.get("colspan", 1))
rowspan = int(col.attrs.get("rowspan", 1))
cell_html = get_cell_html(col)
if col.name == "th":
cell_html = "<b>%s</b>" % cell_html
if cell_row >= num_rows or cell_col >= num_cols:
continue
docx_cell = table.cell(cell_row, cell_col)
while docx_cell.text != "": # Skip the merged cell
cell_col += 1
docx_cell = table.cell(cell_row, cell_col)
cell_to_merge = table.cell(
cell_row + rowspan - 1, cell_col + colspan - 1
)
if docx_cell != cell_to_merge:
docx_cell.merge(cell_to_merge)
child_parser = HtmlToDocx()
child_parser.copy_settings_from(self)
child_parser.add_html_to_cell(cell_html or " ", docx_cell)
cell_col += colspan
cell_row += 1
def handle_data(self, data):
if self.skip:
return
# Only remove white space if we're not in a pre block.
if "pre" not in self.tags:
# remove leading and trailing whitespace in all instances
data = remove_whitespace(data, True, True)
if not self.paragraph:
self.paragraph = self.doc.add_paragraph()
self.apply_paragraph_style()
# There can only be one nested link in a valid html document
# You cannot have interactive content in an A tag, this includes links
# https://html.spec.whatwg.org/#interactive-content
link = self.tags.get("a")
if link:
self.handle_link(link["href"], data)
else:
# If there's a link, dont put the data directly in the run
self.run = self.paragraph.add_run(data)
spans = self.tags["span"]
for span in spans:
if "style" in span:
style = self.parse_dict_string(span["style"])
self.add_styles_to_run(style)
# add font style and name
for tag in self.tags:
if tag in font_styles:
font_style = font_styles[tag]
setattr(self.run.font, font_style, True)
if tag in font_names:
font_name = font_names[tag]
self.run.font.name = font_name