Benchmarking of Table Structure Recognition Algorithms - Revision history

Dimos at 18:01, 27 January 2011

2011-01-27T18:01:21Z

Dimos: /* References */

2010-07-07T09:43:28Z

‎References

Dimos: /* References */

2010-07-06T17:11:26Z

‎References

Dimos: /* Related Ground Truth Data */

2010-07-06T16:35:51Z

‎Related Ground Truth Data

Dimos: /* Description */

2010-07-06T16:35:25Z

‎Description

Dimos: /* Description */

2010-07-06T16:32:07Z

‎Description

Dimos: /* Description */

2010-07-06T16:31:05Z

‎Description

Dimos: Created page with 'Datasets -> Current Page {| style="width: 100%" |- | align="right" | {| |- | '''Created: '''2010-04-30 |- | {{Last updated}} |} |} =Description= The aim of this task is…'

2010-07-06T16:29:23Z

Created page with 'Datasets -> Current Page {| style="width: 100%" |- | align="right" | {| |- | '''Created: '''2010-04-30 |- | {{Last updated}} |} |} =Description= The aim of this task is…'

New page

[[Datasets]] -> Current Page

{| style="width: 100%"
|-
| align="right" |

{|
|-
| '''Created: '''2010-04-30
|-
| {{Last updated}}
|}

|}

=Description=
The aim of this task is to benchmark different table spotting and table structure recognition algorithms. In the past, several algorithms have been developed for table recognition tasks and evaluated on different datasets [1-4]. Additionally different benchmarking measures have been used in the past by different researchers to evaluate the efficiency of their algorithms.

This task comprises 427 images from the publicly available UNLV dataset. We provide the table structure groundtruth and word bounding box results for these document images. These images from the UNLV dataset along with the ground truth XML files and word bounding box OCR files can be used to generate pixel accurate ground truth image files as described in [5]. Appropriate tools to generate these ground truth image files are provided below.

The competing algorithms need to produce the output in the XML format similar to the [[Table Ground Truth for the UW3 and UNLV datasets|sample ground truth]]. Five example output from our t-recs algorithm are included in the set of samples provided for reference. The output from these algorithms are then used to produce result image files which are then compared by the evaluation framework to produce following benchmarking measures:

1. Correct Detections
2. Partial Detections
3. Over Segmentation
4. Under Segmentation
5. Missed
6. False Positives

at the following levels of abstraction:
1. Table
2. Rows
3. Columns
4. Cells
5. Row spanning cells
6. Column spanning cells
7. Row/column spanning cells.

The evaluation software provided below (segment_evaluation.sh, part of the T-truth package) expects ground truth images in the "evaluation/gt" folder and result images in the "evaluation/result" folder. The evaluation algorithm produces a file with the extension ".out" in the evaluation directory for each level of abstraction (table, row, column etc) with semicolon separated benchmarking measures in the following order:

gt_img; result_img; GroundTruthComponents; SegmentationComponents; Number of OverSegmentations; Number of UnderSegmentations; Number of FalseAlarms ; OverSegmentedComponents; UnderSegmentedComponents; False Alarms; Missed; Correct; Partial Matches

These benchmarking measures are described in [5].

=Related Dataset=
* [http://www.isri.unlv.edu/ISRI/OCRtk UNLV Dataset]

=Related Ground Truth Data=
* [[Table Ground Truth for the UW3 and UNLV datasets]]

=Related Software
* [http://www.iapr-tc11.org/dataset/TableGT_UW3_UNLV/t-truth.tar.gz T-truth Software and Samples] (4.5 Mb)

=References=
# T. Kieninger and A. Dengel; Applying the T-RECS table recognition system to the business letter domain.
# J. Hu, R. Kashi, D. Lopresti, and G. Wilfong.Medium-independent table detection.
# S. Mandal, S. Chowdhury, A. Das, and B. Chanda; A simple and e
ective table detection system from document images
# B. Gatos, D. Danatsas, I. Pratikakis, and S. J. Perantonis; Automatic table detection in document images.
# Asif Shahab, Faisal Shafait, Thomas Kieninger and Andreas Dengel, "An Open Approach towards the benchmarking of table structure recognition systems", Proceedings of DAS’10, pp. 113-120, June 9-11, 2010, Boston, MA, USA

----
This page is editable only by [[IAPR-TC11:Reading_Systems#TC11_Officers|TC11 Officers ]].

← Older revision		Revision as of 18:01, 27 January 2011
Line 1:		Line 1:
−	[[Datasets]] -> Current Page	+	[[Datasets]] -> [[Datasets List]] -> Current Page

	{\| style="width: 100%"		{\| style="width: 100%"

@@ Line 67: / Line 67: @@
 =References=
-# T. Kieninger and A. Dengel; Applying the T-RECS table recognition system to the business letter domain.
+# T. Kieninger and A. Dengel, "Applying the T-RECS table recognition system to the business letter domain", Proceedings of the 6th IAPR International Conference on Document Analysis and Recognition, IEEE CS, pp. 518-522, 2001
-# J. Hu, R. Kashi, D. Lopresti, and G. Wilfong.Medium-independent table detection.
+# J. Hu, R. Kashi, D. Lopresti, and G. Wilfong, "Medium-independent table detection", Proceedings of the 8th SPIE Conference on Document Recognition and Retrieval, pp. 291-302, 2000
-# S. Mandal, S. Chowdhury, A. Das, and B. Chanda; A simple and effective table detection system from document images
+# S. Mandal, S. Chowdhury, A. Das, and B. Chanda, "A simple and effective table detection system from document images", International Journal on Document Analysis and Recognition, Vol. 8, Num. 2-3, pp. 172-182, 2006
-# B. Gatos, D. Danatsas, I. Pratikakis, and S. J. Perantonis; Automatic table detection in document images.
+# B. Gatos, D. Danatsas, I. Pratikakis, and S. J. Perantonis, "Automatic table detection in document images", Proceedings of the International Conference on Advances in Pattern Recognition, pp. 612-621, 2005
 # Asif Shahab, Faisal Shafait, Thomas Kieninger and Andreas Dengel, "An Open Approach towards the benchmarking of table structure recognition systems", Proceedings of DAS’10, pp. 113-120, June 9-11, 2010, Boston, MA, USA

@@ Line 63: / Line 63: @@
 * [[Table Ground Truth for the UW3 and UNLV datasets]]
-=Related Software
+=Related Software=
 * [http://www.iapr-tc11.org/dataset/TableGT_UW3_UNLV/t-truth.tar.gz T-truth Software and Samples] (4.5 Mb)

@@ Line 41: / Line 41: @@
 The evaluation software provided below (segment_evaluation.sh, part of the T-truth package) expects ground truth images in the "evaluation/gt" folder and result images in the "evaluation/result" folder. The evaluation algorithm produces a file with the extension ".out" in the evaluation directory for each level of abstraction (table, row, column etc) with semicolon separated benchmarking measures in the following order:
-gt_img; result_img; GroundTruthComponents; SegmentationComponents; Number of OverSegmentations; Number of UnderSegmentations; Number of FalseAlarms ; OverSegmentedComponents; UnderSegmentedComponents; False Alarms; Missed; Correct; Partial Matches
+* gt_img
 These benchmarking measures are described in [5].

@@ Line 15: / Line 15: @@
 =Description=
-[[Image:Table_UNLV_UW3_Results.png|300px|thumb|right|Example results obtained.]]
+[[Image:Table_UNLV_UW3_Results.png|400px|thumb|right|Example results obtained.]]
 The aim of this task is to benchmark different table spotting and table structure recognition algorithms. In the past, several algorithms have been developed for table recognition tasks and evaluated on different datasets [1-4]. Additionally different benchmarking measures have been used in the past by different researchers to evaluate the efficiency of their algorithms.
@@ Line 23: / Line 23: @@
 The competing algorithms need to produce the output in the XML format similar to the [[Table Ground Truth for the UW3 and UNLV datasets|sample ground truth]]. Five example output from our t-recs algorithm are included in the set of samples provided for reference. The output from these algorithms are then used to produce result image files which are then compared by the evaluation framework to produce following benchmarking measures:
-.	Correct Detections
+# Correct Detections
-.	Partial Detections
+# Partial Detections
-.	Over Segmentation
+# Over Segmentation
-.	Under Segmentation
+# Under Segmentation
-.	Missed
+# Missed
-.	False Positives
+# False Positives
 at the following levels of abstraction:
-.	Table
+# Table
-.	Rows
+# Rows
-.	Columns
+# Columns
-.	Cells
+# Cells
-.	Row spanning cells
+# Row spanning cells
-.	Column spanning cells
+# Column spanning cells
-.	Row/column spanning cells.
+# Row/column spanning cells.
 The evaluation software provided below (segment_evaluation.sh, part of the T-truth package) expects ground truth images in the "evaluation/gt" folder and result images in the "evaluation/result" folder. The evaluation algorithm produces a file with the extension ".out" in the evaluation directory for each level of abstraction (table, row, column etc) with semicolon separated benchmarking measures in the following order: