CROHME-II: ICFHR2012 Competition on Recognition of Online Handwritten Mathematical Expressions
CROHME-II is the continuation of its earlier version CROHME2011 which was held during ICDAR 2011. Please visit CROHME web-page for more information.
We hereby present the ICFHR2012 CROHME package.
All data and tools provided here are freely available only for research purpose without any commercial use.
Data
This package provides training and test data from the two previous competitions CROHME 2011 and 2012. Furthermore, thanks to the participants' authorization, we are allowed to distribute the results files from the majority of the submitted systems. Here is the description of all data directories:
- CROHME2011_data : all data from the CROHME 2011 competition:
- CROHME_test : inkml test files without ground truth
- CROHME_testGT : inkml test files with ground truth
- CROHME_train : inkml train files with ground truth
- gram : xml grammars and symbol lists for parts I and II
- CROHME2012_data : all data from the CROHME 2012 competition:
- testData : inkml test files without ground truth
- testDataGT : inkml test files with ground truth
- trainData : inkml train files with ground truth
- gram : xml grammars and symbol lists for parts I, II and III
- lists : lists of inkml files and latex expressions for parts I, II and III
- ParticipantsResults2012 : results files from the participants of the CROHME 2012 competition (note that the result files from VisionObjects are restricted to the participants):
- ResultsTest : inkml results for the test part for each participant (parts I, II, and III)
- ResultsTrain : inkml results for the train part for each participant (only part III)
Tools
CROHME Organizers provide some tools for math expression selection, running the test phase, evaluation and visualisation :
- CROHME2011_data/tokenAndParse.pl and CROHME2012_data/tokenAndParse.pl : perl script to filter latex expressions with regards to xml grammars defining part I, II and III
- evaluationTools/startTestPhase.pl : perl script to run a recognizer over a list of inkml files
- evaluationTools/evalInkml_v1.7.pl : perl script to compare ground-truthed inkml files with inkml results
CROHME Results
CROHME results are summed up in the two papers available in the directory CROHME_papers.
After CROHME 2012 competition two errors have been detected in the test data after the final version of the paper. In the ICFHR paper 488 expressions are used in the test dataset, thus here are the updated results on the 486 valid inkml files:
| STR_REC | SEG | REC/SEG | EXP | EXP1 | EXP2 | EXP3 | STRUCT | |
|---|---|---|---|---|---|---|---|---|
| FAlvaro | 79.91 | 91.94 | 86.35 | 22.84 | 34.77 | 41.15 | 42.39 | 42.39 |
| FS | 55.81 | 71.21 | 85.08 | 3.70 | 6.38 | 9.05 | 10.49 | 9.88 |
| IVC | 79.02 | 87.79 | 91.45 | 25.72 | 36.21 | 39.30 | 40.12 | 37.04 |
| RIT | 72.17 | 87.57 | 87.67 | 9.47 | 18.52 | 23.25 | 24.90 | 23.87 |
| Sabanci | 45.45 | 59.16 | 84.40 | 4.94 | 10.70 | 14.20 | 15.02 | 14.81 |
| UWaterloo | 86.47 | 95.55 | 91.27 | 40.33 | 54.32 | 59.26 | 62.14 | 61.52 |
| VO | 95.79 | 98.84 | 96.91 | 62.96 | 79.42 | 82.30 | 82.51 | 80.86 |
Organizers
Contact Information:
- Dr. Harold Mouchère - harold.mouchere@univ-nantes.fr
- Mr. Dae Hwan Kim - ofreunde.edward@gmail.com
- Dr. Utpal Garain - utpal@isical.ac.in
License and Copyright
These CROHME 2012 (Competition on Recognition of Handwritten Mathematical Expressions 2012 dataset) is Copyright © 11/05/2011, Université de Nantes / CNRS These CROHME 2012 is free software and data; you can redistribute it and/or modify it under the terms of the Creative Commons CC BY-NC-SA 3.0 (Attribution-NonCommercial-ShareAlike 3.0 Unported)
These data and software are distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the Creative Commons License CC BY-NC-SA 3.0 for more details. You should have received a copy of the Creative Commons License along with this program; if not, you can also find the Creative Commons licence on the Creative Commons web site.
In addition, we kindly ask you to acknowledge these CROHME 2012 and its authors in any program or publication in which you use these CROHME 2012. (You are not required to do so; it is up to your common sense to decide whether you want to comply with this request or not.) For general publications, we suggest referencing: ICFHR 2012 - Competition on Recognition of Online Mathematical Expressions (CROHME2012), Mouchère H., Viard-Gaudin C., Garain U., Kim D. H., Kim J. H. Proceedings of the International Conference on Frontiers in Handwriting Recognition, ICFHR 2012, Italy (2012)
Non-free versions of this dataset are available under terms different from those of the Creative Commons. For these alternative terms you must purchase a license from one of the authors' laboratory. Users interested in such a license should contact them for more information.