7 #ifndef LOWERCASETOKENIZER_H 8 #define LOWERCASETOKENIZER_H 38 virtual wchar_t normalize(
wchar_t c);
LowerCaseTokenizer performs the function of LetterTokenizer and LowerCaseFilter together. It divides text at non-letters and converts them to lower case. While it is functionally equivalent to the combination of LetterTokenizer and LowerCaseFilter, there is a performance advantage to doing the two tasks at once, hence this (redundant) implementation.
Definition: LowerCaseTokenizer.h:21
boost::shared_ptr< Reader > ReaderPtr
Definition: LuceneTypes.h:547
boost::shared_ptr< AttributeFactory > AttributeFactoryPtr
Definition: LuceneTypes.h:519
boost::shared_ptr< AttributeSource > AttributeSourcePtr
Definition: LuceneTypes.h:520
A LetterTokenizer is a tokenizer that divides text at non-letters. That's to say, it defines tokens a...
Definition: LetterTokenizer.h:19
#define LUCENE_CLASS(Name)
Definition: LuceneObject.h:24
Definition: AbstractAllTermDocs.h:12