NCEnStopWordsTokenEnricher

Stopword token enricher for English (EN) language. Stopwords are the words which are filtered out (i.e. stopped) before processing of natural language text because they are insignificant.

This enricher adds stopword boolean metadata property to the token instance if the word it represents is an English stopword. The value true of this metadata property indicates that this word is detected as a stopword, false value indicates otherwise. This implementation works off the algorithm that uses an internal list of English stopwords as well as a procedural logic to determine the stopword status of the token. This algorithm should work fine for most of the general uses cases. User can also add additional stopwords or exceptions for the existing ones using corresponding parameters in NCEnStopWordsTokenEnricher constructor.

More information about stopwords can be found at https://en.wikipedia.org/wiki/Stop_word.

NOTE: this implementation requires lemma and pos string metadata properties that contain token's lemma and part of speech accordingly. You can configure NCOpenNLPTokenEnricher with the model for English language that would provide these metadata properties before this enricher in your pipeline.