Data Model

Model Overview

Data model is a central concept in NLPCraft defining interface to your data sources like a database or a SaaS application. NLPCraft employs model-as-a-code approach where entire data model is an implementation of NCModel interface which can be developed using any JVM programming language like Java, Scala, Kotlin, or Groovy.

A data model defines:

• Set of model elements (a.k.a. named entities) to be detected in the user input.
• Zero or more intent callbacks.
• Common model configuration and various life-cycle callbacks.

Note that model-as-a-code approach natively supports any software life cycle tools and frameworks like various build tools, CI/SCM tools, IDEs, etc. You don't have to use additional web-based tools to manage some aspects of your data models - your entire model and all of its components are part of your project source code.

Model Dataflow

User request starts with the user application (like a chatbot or NLI-based system) making a REST call using NLPCraft REST API. That REST call carries among other things the input text and data model ID, and it arrives first to the REST server.

Upon receiving the user request, the REST server performs NLP pre-processing converting the input text into a sequence of tokens and enriching them with additional information.

Once finished, the encrypted sequence of tokens is sent further down to the probe where the requested data model is deployed.

Upon receiving that sequence of tokens, the data probe further enriches it based on the user data model and matches it against declared intents. When a matching intent is found its callback method is called and its result travels back from the data probe to the REST server and eventually to the user that made the REST call.

Read more about details of user request workflow and intent matching in Intent Matching section.

Model Lifecycle

Data model is an implementation of NCModel interface. NCModel interface has defaults for most of its methods. These are the only methods that need to be implemented by its sub-class:

• getId()
• getName()
• getVersion()

You can either implement NCModel interface directly or use one of the adapters (recommended in most cases):

• NCModelAdapter - when entire model definition is in sub-class source code.
• NCModelFileAdapter - when using external JSON/YAML declaration for model definition.

Note that you can also use 3rd party IoC frameworks like Spring to construct your data models. See NCModelFactory for more information.

Deployment

Data models get deployed to and hosted by the data probes - a lightweight container whose job is to host data models and securely transfer requests between REST server and the data models. When a data probe starts it reads its configuration to see which models to deploy.

Note that data probes don't support hot-redeployment. To redeploy the data model you need to restart the data probe. Note also that data probe can be started in embedded mode, i.e. it can be started from within an existing JVM process like user application.

Callbacks

There are two callbacks on NCModel interface (by way of extending NCLifecycle interface) that you can optionally override to affect the the default lifecycle behavior:

• onInit() - called right after the model was loaded and deployed.
• onDiscard() - called to discard the data model when and only when data probe is orderly shutting down.

Note that there are also several callbacks that you can override to affect model behavior to perform logging, debugging, statistic or usage collection, explicit update or initialization of conversation context, security audit or validation:

• onParsedVariant(...)
• onContext(...)
• onMatchedIntent(...)
• onResult(...)
• onError(...)
• onRejection(...)

Model Configuration

Apart from mandatory model ID, name and version there is a number of static model configurations that you can set. All of these properties have sensible defaults that you can override, when required, in either sub-classes or via external JSON/YAML declaration:

• getAdditionalStopWords
• getEnabledBuiltInTokens
• getExamples
• getExcludedStopWords
• getJiggleFactor
• getMaxFreeWords
• getMaxSuspiciousWords
• getMaxTokens
• getMaxTotalSynonyms
• getMaxUnknownWords
• getMaxWords
• getMetadata
• getMinNonStopwords
• getMinTokens
• getMinWords
• getSuspiciousWords
• isDupSynonymsAllowed
• isNonEnglishAllowed
• isNoNounsAllowed
• isNotLatinCharsetAllowed
• isNoUserTokensAllowed
• isPermutateSynonyms
• isSwearWordsAllowed

External JSON/YAML Declaration

You can move out all the static model configuration into an external JSON or YAML file. To load that configuration you need to use NCModelFileAdapter adapter when creating your data model. Here are JSON and YAML templates and you can find more details in NCModel Javadoc and in examples.

{
     "id": "user.defined.id",
     "name": "User Defined Name",
     "version": "1.0",
     "description": "Short model description.",
     "enabledBuiltInTokens": ["google:person", "google:location"]
     "macros": [],
     "metadata": {},
     "elements": [
         {
             "id": "x:id",
             "description": "",
             "groups": [],
             "parentId": "",
             "synonyms": [],
             "metadata": {},
             "values": []
         }
     ],
     ...
     "intents": []
}
                

id: "user.defined.id"
name: "User Defined Name"
version: "1.0"
description: "Short model description."
macros:
enabledBuiltInTokens:
elements:
  - id: "x:id"
    description: ""
    synonyms:
    groups:
    values:
    parentId:
    metadata:
...
intents:
                

Model Elements

Data model element defines a semantic entity that will be detected in the user input. A model element typically is one or more individual words that have a consistent semantic meaning and typically denote a real-world object, such as persons, locations, number, date and time, organizations, products, etc. Such object can be abstract or have a physical existence.

Model element is an implementation of NCElement interface. NCModel provides its elements via getElements() method. Typically, you create model elements by either:

• Implementing NCElement interface directly, or
• sing JSON or YAML static model configuration (the preferred way in most cases).

Note that when you use external static model configuration with JSON or YAML you can still modify it after it was loaded using NCModelFileAdapter adapter. It is particular convenient when synonyms or values are loaded separately from, or in addition to, the model elements themselves, i.e. from a database or another file.

Although model element and named entity describe a similar concept, the NLPCraft model elements provide a much more powerful instrument. Unlike named entities support in other projects NLPCraft model elements have number of unique capabilities:

• New model elements can be added declaratively via token DSL, regex and macro expansion.
• New model elements can be also added programmatically for ultimate flexibility.
• Model elements can have many-to-many group memberships.
• Model elements can form a hierarchical structure.
• Model elements are composable, i.e. a model element can use other model elements in its definition.
• Model elements can be declared with user defined metadata.
• Model elements provide normalized values and can define their own "proper nouns".
• Model elements can compose named entities from many 3rd party libraries.
• All properties of model elements (id, groups, parent & ancestors, values, and metadata) can be used in token and intent DSLs.

User vs. Built-In Elements

Additionally to the model elements that are defined by the user in the data model (i.e. user model elements) NLPCraft provides its own named entities as well as the integration with number of 3rd party projects. You can think of these built-in elements as if they were implicitly defined in your model - you can use them in exactly the same way as if you defined them yourself. You can find more information on how to configure external token providers in Integrations section.

Note that you can't directly change group membership, parent-child relationship or metadata of the built-in elements. You can, however, "wrap" built-in entity into your own one using ^^id == 'external.id'^^ token DSL expression where you can define all necessary additional configuration properties (more on that below).

NLPCraft uses fully deterministic named entity recognition and is not based on statistical approaches that would require pre-existing marked up data sets and extensive training. For each model element you can either provide a set of synonyms to match on or specify a piece of code that would be responsible for detecting that named entity (discussed below). A synonym can have one or more individual words. Note that element's ID is its implicit synonym so that even if no additional synonyms are defined at least one synonym always exists. Note also that synonym matching is performed on normalized and stemmatized forms of both a synonym and user input.

Here's an example of a simple model element definition in JSON:

            ...
            "elements": [
                {
                    "id": "transport.vehicle",
                    "description": "Transportation vehicle",
                    "synonyms": [
                        "car",
                        "truck",
                        "light duty truck"
                        "heavy duty truck"
                        "sedan",
                        "coupe"
                    ]
                }
            ]
            ...
        

During synonym matching NLPCraft uses jiggle factor to rearrange (or "jiggle") the individual words in the user input in attempt to match a given synonym. Jiggle factor is a measure of how much sparsity is allowed when user input words are reordered in attempt to match the multi-word synonyms. Zero means no reordering is allowed. One means that a word can move only one position left or right, and so on. Empirically the value of 2 proved to be a good default value in most cases. Note that larger values mean that synonym words can be almost in any random place in the user input which makes synonym matching less meaningful.

While adding multi-word synonyms looks somewhat trivial - in real models, the naive approach can lead to thousands and even tens of thousands of possible synonyms due to words, grammar, and linguistic permutations - which quickly becomes untenable if performed manually.

NLPCraft provides an effective tool for a compact synonyms representation. Instead of listing all possible multi-word synonyms one by one you can use combination of following expressions:

• Macros
• Regular expressions
• Option Groups
• Token DSL

Each whitespace separated string in the synonym can be either a regular word (like in the above transportation example where it will be matched on using its normalized and stemmatized form) or one of the above expression.

Note that this universal synonyms definition is used in the following NCElement methods:

• getSynonyms() - gets synonyms to match on.
• getValues() - get values to match on (see below).

Listing all possible multi-word synonyms for a given element can be a time-consuming task. Macros together with option groups allow for significant simplification of this process. Macros let you give a name to an often used set of words or option groups and reuse it without repeating those words or option groups again and again. A model provides a list of macros via getMacros() method on NCModelView interface. Each macro has a name in a form of <X> where X is just any string, and a string value. Note that macros can be nested (but not recursive), i.e. macro value can include references to other macros. When macro name X is encountered in the synonym it gets recursively replaced with its value.

Here's a code snippet of macro definitions using JSON definition:

            "macros": [
                {
                    "name": "<A>",
                    "macro": "aaa"
                },
                {
                    "name": "<B>",
                    "macro": "<A> bbb"
                },
                {
                    "name": "<C>",
                    "macro": "<A> bbb {z|w}"
                }
             ]
        

Option groups are similar to wildcard patterns that operates on a single word base. One line of option group expands into one or more individual synonyms. Option groups is the key mechanism for shortened synonyms notation. The following examples demonstrate how to use option groups.

Consider the following macros defined below (note that macros <B> and <C> are nested):

Then the following option group expansions will occur in these examples:

Specifically:

• {A|B} denotes either A or B.
• {A|B|*} denotes either A or B or nothing.
• Excessive curly brackets are ignored, when safe to do so.
• Macros cannot be recursive but can be nested.
• Option groups can be nested.
• '\' (backslash) can be used to escape '{', '}', '|' and '*' special symbols used by the option groups.
• Excessive whitespaces are trimmed when expanding option groups.

We can rewrite our transportation model element in a bit more efficient way using macros and option groups. Even though the actual length of definition hasn't changed much it now auto-generates many dozens of synonyms we would have to write out manually otherwise:

            ...
            "macros": [
                {
                    "name": "<TRUCK_TYPE>",
                    "macro": "{ {light|super|heavy|medium} duty|half ton|1/2 ton|3/4 ton|one ton}"
                }
             ]
            "elements": [
                {
                    "id": "transport.vehicle",
                    "description": "Transportation vehicle",
                    "synonyms": [
                        "car",
                        "{<TRUCK_TYPE>|*} {pickup|*} truck"
                        "sedan",
                        "coupe"
                    ]
                }
            ]
            ...
        

Any individual synonym word that starts and ends with // (two forward slashes) is considered to be Java regular expression as defined in java.util.regex.Pattern. Note that regular expression can only span a single word, i.e. only individual words from the user input will be matched against given regular expression and no whitespaces are allowed within regular expression. Note also that option group special symbols {, }, | and * have to be escaped in the regular expression using \ (backslash).

For example, the following synonym:

        "synonyms": [
            "{foo|//[bar].+//}}"
        ]
        

will match word foo or any other strings that start with bar as long as this string doesn't contain whitespaces.

Model element can have an optional set of special synonyms called values or proper nouns for this element. Unlike basic synonyms, each value is a pair of a name and a set of standard synonyms by which that value, and ultimately its element, can be recognized in the user input. Note that the value name itself acts as an implicit synonym even when no additional synonyms added for that value.

When a model element is recognized it is made available to the model's matching logic as an instance of the NCToken interface. This interface has a method getValue() which returns the name of the value, if any, by which that model element was recognized. That value name can be further used in intent matching.

To understand the importance of the values consider the following changes to our transportation example model:

            ...
            "macros": [
                {
                    "name": "<TRUCK_TYPE>",
                    "macro": "{light duty|heavy duty|half ton|1/2 ton|3/4 ton|one ton|super duty}"
                }
             ]
            "elements": [
                {
                    "id": "transport.vehicle",
                    "description": "Transportation vehicle",
                    "synonyms": [
                        "car",
                        "{<TRUCK_TYPE>|*} {pickup|*} truck"
                        "sedan",
                        "coupe"
                    ],
                    "values": [
                        {
                            "value": "mercedes",
                            "synonyms": ["mercedes-ben{z|s}", "mb", "ben{z|s}"]
                        },
                        {
                            "value": "bmw",
                            "synonyms": ["{bimmer|bimer|beemer}", "bayerische motoren werke"]
                        }
                        {
                            "value": "chevrolet",
                            "synonyms": ["chevy"]
                        }
                    ]
                }
            ]
            ...
        

With that setup transport.vehicle element will be recognized by any of the following input string:

• car
• benz (with value mercedes)
• 3/4 ton pickup truck
• light duty truck
• chevy (with value chevrolet)
• bimmer (with value bmw)
• transport.vehicle

Note that element value can be used in token and intent DSLs.

Each model element belongs to one or more groups. Model element provides its groups via getGroups() method. By default, if element group is not specified, the element ID will act as its default group ID.

Group membership is a quick and easy way to organise similar model elements together and use this categorization in token and intent DSL.

Note that the proper grouping of the elements is also necessary for the correct operation of Short-Term-Memory (STM) in the conversational context when using intent-based matching. See NCConversation for mode details.

Consider a NCToken that represents a previously found model element that is stored in the conversation. Such token will be overridden in the conversation by the more recent token from the same group - a critical rule of maintaining the proper conversational context.

Note that token's groups can be used in token and intent DSLs.

Each model element can form an optional hierarchical relationship with other element by specifying its parent element ID via getParentID() method. The main idea here is that sometimes model elements can act not only individually but their place in the hierarchy can be important too for token and intent DSL.

For example, we could have designed our transportation example model in a different way by using multiple model elements linked with this hierarchy:

+-- vehicle
|     +--truck
|     |    |-- light.duty.truck
|     |    |-- heavy.duty.truck
|     |    +-- medium.duty.truck
|     +--car
|     |   |-- coupe
|     |   |-- sedan
|     |   |-- hatchback
|     |   +-- wagon
        

Then in our intent DSL, for example, we could look for any token with root parent ID vehicle or immediate parent ID truck or car without a need to match on all current and future individual sub-IDs:

            "intent=vehicle.intent term={ancestors @@ 'vehicle'}"
            "intent=truck.intent term={parent == 'truck'}"
            "intent=car.intent term={parent == 'car'}"
        

Token DSL

Any individual synonym word that that starts and ends with ^^ is a token DSL expression. A token DSL expression inside of ^^ ... ^^ markers allows you to define a predicate on already parsed and detected token. It is very important to note that unlike all other synonyms the token DSL predicate operates on a already detected token, not on an individual unparsed word.

Token DSL allows you to compose named entities, i.e. use one name entity when defining another one. For example, we could define a model element for the race car using our previous transportation example (note how synonym on line 18 references the element defined on line 4):

            ...
            "elements": [
                {
                    "id": "transport.vehicle",
                    "description": "Transportation vehicle",
                    "synonyms": [
                        "car",
                        "truck",
                        "{light|heavy|super|medium} duty {pickup|*} truck"
                        "sedan",
                        "coupe"
                    ]
                },
                {
                    "id": "race.vehicle",
                    "description": "Race vehicle",
                    "synonyms": [
                        "{race|speed|track} ^^id == 'transport.vehicle'^^"
                    ]
                }

            ]
            ...
        

Another often used use case is to wrap 3rd party named entities to add group membership, metadata or hierarchical relationship to the externally detected named entity. For example, you can wrap google:location token and add group membership for my_group group:

            ...
            "elements": [
                {
                    "id": "google.loc.wrap",
                    "description": "Wrapper for google location",
                    "groups": ["my_group"],
                    "synonyms": [
                        "^^id == 'google:location'^^"
                    ]
                }

            ]
            ...
        

Token DSL is a simple expression language for defining a single predicate over a token - a detected model element. Remember that unlike token DSL all other types of synonyms work with simple words (vs. tokens). Here's a full ANTLR4 grammar for token DSL. Note that this is exactly the same syntax as used by intent DSL for token predicates in intents - except for aliases which we will explain below.

Here's an example of token DSL defining a synonym for the population of any city in France:

            "synonyms": [
                "population {of|for} ^^[city](id == 'nlpcraft:city' && lowercase(~city:country) == 'france')^^"
            ]
        

Few notes on token DSL syntax:

• This synonym defines a composed named entity, i.e. named entity that consists of other named entities. In our example, we utilize token nlpcraft:city along with other basic synonym.
• Token DSL expression always results in one and only one token when matched, however, the synonym can have multiple token DSL expressions.
• Token DSL expression can have optional alias ([city]) that can be used in other token DSL expressions when referencing the token matched by that expression.
• You can get all participant nested tokens, if required, using NCToken#getPartTokens() method call chain. You can also reference participant tokens in the token DSL expression itself by using dot-notation (see below) with either token IDs or aliases.
• All string values should be places in single quotes, as in 'some string'. For numeric literals you can use underscores to help readability, i.e. ~list:size >= 1_000_000
• You can use null, true and false literals as a values.
• Individual token expressions can be combined with &&, || and ! logical combinators and ( ) brackets that obey standard precedence rules.

The individual token DSL expression can be one of the following forms:

            {qual}param op value
            func({qual}param) op value
        

The {qual}param is the left side parameter and it can have optional qualifier (qual). Qualifier allows to reference participant tokens either by their ID or their DSL expression's alias using dot-notation. For example:

The param itself can be one of the following literals:

The optional func function can alter the value of the left-side parameter. Only one function call is allowed, i.e. function calls cannot be nested. The primary use case for functions is dealing with 3rd party metadata where you don't have a direct control on the values supplied from 3rd party named entity providers. The following functions are supported:

The op (operation) can be one of the following:

Individual token expressions can be combined with &&, || and ! logical combinators and ( ) brackets that obey standard precedence rules as well as short-cut processing of logical && and || combinators. For example:

^^[alias](my:list[0] >= 1_000_000 && alias1.groups @@ 'clients')^^
^^(id == 'myid' && ~score > 10) || (alias1.groups @@ 'clients' && ~score <= 10)^^

In cases when declarative synonyms (macros, option groups, regexp and token DSL) are not expressive enough you create your model element recognizer programmatically:

• Model provides its custom parsers via getParsers() method.
• Custom parser is defined by the following classes: NCCustomElement, NCCustomParser and NCCustomWord.

Model Logic

When a user sends its request via REST API it is received by the REST server. Upon receipt, the REST server does the basic NLP processing and enriching. Once finished, the REST server sends the enriched request down to a specific data probe selected based on the requested data model.

The model logic is defined in intents, specifically in the intent callbacks that get called when their intent is chosen as a winning match against the user request. Below we will quickly discuss the key APIs that are essential for developing intent callbacks. Note that this does now replace a more detailed Javadoc documentation that you are encouraged to read through as well:

• Interface NCModelView
• Interface NCIntentMatch
• Interface NCContext
• Interface NCRequest
• Interface NCToken
• Class NCResult

Interface NCModelView

This interface provides read-only view on data model. Model view defines a declarative, or configurable, part of the model. All properties in this interface can be defined or overridden in JSON/YAML external presentation when used with NCModelFileAdapter adapter.

Interface NCIntentMatch

This interface defines a context of a particular intent match. It can be passed into the callback of the matched intent and provides the following:

• ID of the matched intent.
• Specific parsing variant that was matched against this intent.
• Access to the original query context (NCContext).
• Various access APIs for intent tokens.

Interface NCContext

This interface provides all available data about the parsed user input and all its supplemental information. It's accessible from NCIntentMatch interface and provide large amount of information to the intent callback logic:

• Server request ID. Server request is defined as a processing of one user input sentence.
• Reference to NCConversation for controlling STM of conversation manager and dialog flow.
• Reference to NCModelView instance that the intent callback method belongs to giving access to entire static model configuration.
• Reference to NCRequest that provides detailed information about the user input.
• List of parsing variants provided by getVariants() method. When the user sentence gets parsed into individual tokens (i.e. detected model elements) there is generally more than one way to do it. This ambiguity is perfectly fine because only the data model has all the necessary information to select one parsing variant that fits that model the best. Without the data model there isn't enough context to determine which variant is the best fitting. Method getVariants() returns list of all parsing variants for a given user input.

Interface NCRequest

NCRequest interface is one of the several important entities in Data Model API that you as a model developer will be working with. You should review its Javadoc but here is an outline of the information it provides:

• Information about the user that issued the request.
• User agent and remote address, if any available, of the user's application that made the initial REST call.
• Original request text, timestamp of its receipt, and server request ID.

Interface NCToken

NCToken object is another key abstraction in Data Model API. A token is a detected model element and is a part of a fully parsed user input. Sequence of tokens represents parsed user input. A single token corresponds to a one or more words, sequential or not, in the user sentence.

Most of the token's information is stored in map-based metadata accessible via getMetadata() method. Depending on the token ID each token will have different set of metadata properties. Some common NLP properties are always present for tokens of all types.

Class NCResult

This class defines data model result returned from model's intent callbacks. Result consists of the text body and the type. The type is similar in notion to MIME types. Intent callbacks must use this class to provide their results.

Built-In Tokens

NLPCraft provides a number of built-in model elements (i.e. tokens) including the integration with several popular 3rd party NER frameworks. Table below provides information about these built-in tokens. Section about token metadata provides further information about metadata that each type of token carries.

Built-in tokens have to be explicitly enabled on both the REST server and in the model. See nlpcraft.server.tokenProviders configuration property and NCModelView#getEnabledBuiltInTokens() method for more details. By default, only NLPCraft tokens are enabled (token ID starting with nlpcraft).

Token Metadata

Each token has different set of metadata. Sections below describe metadata for each built-in token supported by NLPCraft:

• Token ID nlpcraft:nlp
• Token ID nlpcraft:date
• Token ID nlpcraft:num
• Token ID nlpcraft:city
• Token ID nlpcraft:continent
• Token ID nlpcraft:subcontinent
• Token ID nlpcraft:region
• Token ID nlpcraft:country
• Token ID nlpcraft:metro
• Token ID nlpcraft:coordinate
• Token ID nlpcraft:sort
• Token ID nlpcraft:limit
• Token ID nlpcraft:relation
• Token ID stanford:xxx
• Token ID spacy:xxx
• Token ID google:xxx
• Token ID opennlp:xxx

This token's metadata provides common basic NLP properties that are part of any token. All tokens without exception have these metadata properties. This metadata represents a common set of NLP properties for a given token. All these metadata properties are mandatory.

This token denotes a date range including single days. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory.

This token denotes a single numerical value or a numeric condition. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

Following table provides possible values for nlpcraft:num:unit and nlpcraft:num:unittype properties:

This token denotes a city. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

Following tables provides possible values for nlpcraft:city:countrymeta map. The data is obtained from The United Nations Statistics Division datasets:

Following tables provides possible values for nlpcraft:city:citymeta map. The data is obtained from The United Nations Statistics Division datasets:

This token denotes a continent. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

This token denotes a subcontinent. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

This token denotes a metro area. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

This token denotes a geographical region. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

Following tables provides possible values for nlpcraft:region:countrymeta map. The data is obtained from The United Nations Statistics Division datasets:

This token denotes a country. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

Following tables provides possible values for nlpcraft:country:countrymeta map. The data is obtained from The United Nations Statistics Division datasets:

This token denotes a latitude and longitude coordinate. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

This token denotes a sorting or ordering function. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

This token denotes a numeric limit value (like in "top 10" or "bottom five"). Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

This token denotes a numeric limit value (like in "top 10" or "bottom five"). Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

These tokens denote xxx that is a lower case name of the named entity in Google APIs, i.e. google:person, google:location, etc. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

These tokens denote xxx that is a lower case name of the named entity in Stanford CoreNLP, i.e. stanford:person, stanford:location, etc. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

These tokens denote xxx that is a lower case name of the named entity in spaCy, i.e. spacy:person, spacy:location, etc. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.

These tokens denote xxx that is a lower case name of the named entity in Apache OpenNLP, i.e. opennlp:person, opennlp:money, etc. Additionally to nlpcraft:nlp:xxx properties this type of token will have the following metadata properties all of which are mandatory unless otherwise noted.