ANNIE pipeline
English Tokenizer words numbers
Composite Text Density
Hybrid Text Density, Composite Text Density and Text Density are measures proposed in context of Noise Removal, Information Extraction. With the textual information, we propose two measures for the evaluation of the textual importance of tags in web pages: Text Density and Composite Text Density. Once an HTML document is parsed and represented by a…
JAPE rules
JAPE je jezik za pattern matching, NE Transducer – ga koristi
LinkCharNumber
number of all hyperlink characters in its subtree
TRE
TRE is a lightweight, robust, efficient, portable, and POSIX compliant regexp matching library. Key features include the agrep command line tool for approximate regexp matching in the style of grep, an approximate matching library API, portability, wide character and multibyte character support, binary pattern and data support, complete thread safety, consistently efficient matching, low memory…
Visual Importance
Visual Importance