Tokenizer base configuration

You can choose among several different tokenizer base configurations:.

You can choose among two tokenizer base configurations:

  • Ngram with position: logNgramTokenizerPositionFactory
  • Ngram without position: logNgramTokenizerFactory

The following example creates an Ngram tokenizer without positional relevance:

=> CREATE TRANSFORM FUNCTION v_txtindex.myNgramTokenizer 
   AS LANGUAGE 'C++' 
   NAME 'logNgramTokenizerFactory' 
   LIBRARY v_txtindex.logSearchLib NOT FENCED;