Tokenizer base configuration

You can choose among two tokenizer base configurations:

• Ngram with position: logNgramTokenizerPositionFactory
• Ngram without position: logNgramTokenizerFactory

The following example creates an Ngram tokenizer without positional relevance:

=> CREATE TRANSFORM FUNCTION v_txtindex.myNgramTokenizer 
   AS LANGUAGE 'C++' 
   NAME 'logNgramTokenizerFactory' 
   LIBRARY v_txtindex.logSearchLib NOT FENCED;