Vertica documentation is moving! Use our new documentation site at https://docs.rocketsoftware.com/bundle/vertica_12.0/page/index.html

Tokenizer base configuration

You can choose among several different tokenizer base configurations:.

You can choose among several different tokenizer base configurations:

Type Position Without Position
Ngram logNgramTokenizerPositionFactory logNgramTokenizerFactory
Words logWordITokenizerPositionFactory logWordITokenizerFactory

Create a logWord tokenizer without positional relevance:

=> CREATE TRANSFORM FUNCTION v_txtindex.fooTokenizer AS LANGUAGE 'C++' NAME 'logWordITokenizerFactory' LIBRARY v_txtindex.logSearchLib NOT FENCED;