NCTokenParser

A tokenizer that splits given text into the list of NCToken objects. This is one of the user-defined components of the processing pipeline. See NCPipeline for documentation on the token parser place in the overall processing pipeline.

See also:
Source:
NCTokenParser.scala
class Object
trait Matchable
class Any

Value members

Abstract methods

Splits given text into list of tokens.

Splits given text into list of tokens.

Value parameters:
text

Input text to split.

Returns:

List of split token. Can be empty but should never be null.

Source:
NCTokenParser.scala