Apache NLPCraft - Natural Language Interface

Overview

As mentioned in overview section the REST server and data probe are the two runtime components that you need to run when using NLPCraft. Data probes are used to deploy and host data model, while REST server (or a cluster of servers) is used to accept client REST call and route them to the data models via data probes.

REST Server vs. Data Probe

It's important to remember why REST server is a separate component from a data probe. While a typical deployment would have only one REST server (or a cluster of REST servers behind a single load balancer), there are maybe multiple data probes hosting different data models deployed in different physical locations, managed through different life cycles and requiring different security and network configurations.

Moreover, REST server is a heavy and resource consuming component that is built around Apache Ignite distributing in-memory computing capabilities - while the data probe is a lightweight data model container. During the development and testing of data models, the developers need to frequently redeploy data models by restarting the data probe. If the REST server and the data probe would be one component - this process would be very inefficient.

Fro the rest of this section we assume that NLPCraft was downloaded and installed via ZIP archive. However, all instructions below are fully applicable to any type of installation.

Once NLPCraft is downloaded as a ZIP archive and unzipped, the current directory will look like this:

├── LICENSE
├── bin
├── sql
├── build
│   └── apache-nlpcraft-0.7.0
│       ├── nlpcraft.conf
│       ├── ignite.xml
│       ├── log4j2.xml
│       └── apache-nlpcraft-0.7.0-all-deps.jar
├── javadoc
├── openapi
└── src

Regardless of how NLPCraft was installed it comes with a single executable JAR file that includes all necessary dependencies: build/apache-nlpcraft-0.7.0/apache-nlpcraft-0.7.0-all-deps.jar. This JAR file includes binaries for data probe, data model APIs and the REST server.

REST Server

As mentioned above REST server (or a cluster of servers) is used to accept client REST calls and route them to the data model via data probes. Note that both data probe and the REST server start the same way.

REST server can be stared in a standard way from either the command line or IDE such as Eclipse or IntelliJ IDEA:

                    $ cd build
                    $ java -Xms1024m -jar apache-nlpcraft-0.7.0-all-deps.jar -server

                    $ cd build
                    $ java -Xms1024m -cp apache-nlpcraft-0.7.0-all-deps.jar org.apache.nlpcraft.NCStart -server

Configure run configuration with the main class org.apache.nlpcraft.NCStart. Note that org.apache.nlpcraft.NCStart class starts both the REST server and the data probe and is the class that is configured as Main-Class in apache-nlpcraft-0.7.0-all-deps.jar JAR file manifest.

                    $ docker run -m 8G -p 8081:8081 -p 8201:8201 -p 8202:8202 nlpcraftserver/server:0.7.0

By default, the Docker image runs with a default configuration. See configuration section on how to provide custom configuration via environment variables for the REST server running inside of Docker container.

Parameters:

-server: Mandatory parameter to indicate that you are starting the REST server.
-config=path: Optional parameter to provide configuration file path. Server will automatically look for nlpcraft.conf configuration file in the same directory as apache-nlpcraft-0.7.0-all-deps.jar file. If the configuration file has different name or in different location use -config=path parameter where path is an absolute path to the configuration file. Note that the server and the data probe can use the same file for their configuration (just like the default nlpcraft.conf contains configuration for both the server and the data probe).
-igniteConfig=path: Optional parameter to provide Apache Ignite configuration file path. Note that Apache Ignite is used as a cluster computing plane and a default distributed storage. Server will automatically look for ignite.xml configuration file in the same directory as apache-nlpcraft-0.7.0-all-deps.jar file. If the configuration file has different name or in different location use -igniteConfig=path parameter where path is an absolute path to the Ignite configuration file.

Java Memory

Make sure to allocate enough memory for server JVM using -Xms JVM option, i.e. -Xms1024m. Many 3rd party NLP engines like Stanford CoreNLP are very memory intensive and may require several GBs of JVM heap allocated depending on the models used. Note that when server JVM has insufficient heap memory the Apache Ignite may throw the following warning logs:

                Jul-22 13:27:56 [INFO ] ...
                Jul-22 13:28:08 [WARN ] Possible too long JVM pause: 11364 milliseconds.
                Jul-22 13:28:11 [INFO ] ...

The abnormally long GC pauses (over 5s) can be caused by the excessive memory swapping performed by OS due to insufficient JVM heap memory.

When the REST server successfully started you should see the log output similar to this:

                _   ____      ______           ______
               / | / / /___  / ____/________ _/ __/ /_
              /  |/ / / __ \/ /   / ___/ __ `/ /_/ __/
             / /|  / / /_/ / /___/ /  / /_/ / __/ /_
            /_/ |_/_/ .___/\____/_/   \__,_/_/  \__/
                   /_/

            Server
            Version: 0.7.0
            Copyright (C) 2020 Apache Software Foundation.

            ...

            +-------------------------+
            | Server started [19.35s] |
            +-------------------------+


            Mar-11 23:21:04 [INFO ] REST server is listening on 'localhost:8081'.

Data Probe

Just like the REST server the data probe can be started in a standard way from either the command line or IDE such as Eclipse or IntelliJ IDEA:

                    $ cd build
                    $ java -Xms1024m -jar apache-nlpcraft-0.7.0-all-deps.jar -probe

NOTE: when using executable JAR to start the data probe you cannot add your own model classes to the classpath. You should either package your classes into JAR file and configure probe accordingly - or use -cp option in command line.

                    $ cd build
                    $ java -Xms1024m -cp apache-nlpcraft-0.7.0-all-deps.jar:/my/project/classes org.apache.nlpcraft.NCStart -probe -config=/my/project/probe.conf

Directory /my/project/classes should contain all compiled classes for your models. Make sure to replace /my/project/classes and /my/project/probe.conf with the actual paths.

Configure run configuration with the main class org.apache.nlpcraft.NCStart.

Class org.apache.nlpcraft.NCStart

Note that org.apache.nlpcraft.NCStart class starts both the REST server and the data probe and is the class that is configured as Main-Class in apache-nlpcraft-0.7.0-all-deps.jar JAR file manifest.

Parameters:

-probe: Mandatory parameter to indicate that you are starting a data probe.
-config=path: Optional parameter to provide probe configuration file path. Data probe will automatically look for nlpcraft.conf configuration file in the same directory as apache-nlpcraft-0.7.0-all-deps.jar file. If the configuration file has different name or in different location use -config=path parameter where path is an absolute path to the data probe configuration file. Note that the server and the data probe can use the same file for their configuration (just like the default nlpcraft.conf contains configuration for both the server and the data probe).

Adding Your Classes

Note that when you are using a command line to start the probe you can also add your own classes that implement your models. To do that you need to use -cp option instead of -jar and construct your JVM classpath to include both the apache-nlpcraft-0.7.0-all-deps.jar as well as directory where your compiled Java code is located:

                    $ cd build
                    $ java -cp apache-nlpcraft-0.7.0-all-deps.jar:/my/project/classes org.apache.nlpcraft.NCStart -probe -config /my/project/probe.conf

Make sure to replace /my/project/classes with your own directory where your compiled model classes are located. Note that you need to specify class org.apache.nlpcraft.NCStart explicitly in this case.

When the data probe started you should see the log output similar to this:

            _   ____      ______           ______
           / | / / /___  / ____/________ _/ __/ /_
          /  |/ / / __ \/ /   / ___/ __ `/ /_/ __/
         / /|  / / /_/ / /___/ /  / /_/ / __/ /_
        /_/ |_/_/ .___/\____/_/   \__,_/_/  \__/
               /_/

        Data Probe
        Version: 0.7.0
        Copyright (C) 2020 Apache Software Foundation.

        Mar-11 23:25:52 [INFO ] Probe Configuration:
        +--------------------------------------------------------------------+
        | Probe ID        | all.examples                                     |
        | Probe Token     | 3141592653589793                                 |
        | API Version     | 0.7.0, 2019-03-01                                |
        | Down-Link       | localhost:8202                                   |
        | Up-Link         | localhost:8201                                   |
        +-----------------+--------------------------------------------------+
        | Models          | org.apache.nlpcraft.examples.alarm.AlarmModel    |
        |                 | org.apache.nlpcraft.examples.echo.EchoModel      |
        |                 | org.apache.nlpcraft.examples.time.TimeModel      |
        |                 | org.apache.nlpcraft.examples.weather.WeatherModel|
        +-----------------+--------------------------------------------------+
        | Lifecycle       |                                                  |
        | JARs Folder     |                                                  |
        +--------------------------------------------------------------------+

        ...

        Mar-11 23:25:56 [INFO ] Models deployed: 5

        +================================================================================+
        |        Model ID        |           Name           | Ver. | Elements | Synonyms |
        +================================================================================+
        | nlpcraft.alarm.ex      | Alarm Example Model      | 1.0  | 1        | 419      |
        | nlpcraft.weather.ex    | Weather Example Model    | 1.0  | 3        | 9045     |
        | nlpcraft.time.ex       | Time Example Model       | 1.0  | 1        | 432      |
        | nlpcraft.echo.ex       | Echo Example Model       | 1.0  | 0        | 0        |
        +--------------------------------------------------------------------------------+

        ...

        +--------------------------+
        | Probe started [5.12 sec] |
        +--------------------------+

        ...

        Mar-11 23:25:58 [INFO ] Server connection established.

Configuration

Both REST server and the data probe use Typesafe Config for their configuration:

Both the server and the data probe come with default configuration available in build/nlpcraft.conf file.
Custom configuration or default overrides can be placed into a file or provided via environment variables.
Configuration files use HOCON file format.
Server and probe configuration can be placed in the same file (as it is shipped by default in build/nlpcraft.conf file) or kept in separate files. When kept in separate files each file would have either nlpcraft.server or nlpcraft.probe sub-section.

By default, when REST server or data probe start they look for nlpcraft.conf configuration file in the same directory as apache-nlpcraft-0.7.0-all-deps.jar file and the on their classpath. You can change this behavior with -config=path parameter.

Default configuration is available in build/nlpcraft.conf file and it is extensively documented. It has subsections for the server and probe configuration. When server and the probe use different files these whole sections should be placed into an individual files:

Server configuration file (e.g. server_nlpcraft.conf):

nlpcraft {
    server {
        ...
    }
}

Probe configuration file (e.g. probe_nlpcraft.conf):

nlpcraft {
    probe {
        ...
    }
}

Custom Configuration

While you can change configuration file or files for your own needs (and use -config=... parameter described above to provide path to that file) it is often more convenient to use the default configuration file and just change one or two properties in it. You can accomplish this by using standard HOCON overriding via environment variables:

Set probe or server JVM system property -Dconfig.override_with_env_vars=true which will instruct configuration framework to look for external overrides.
For each configuration property x.y.z set the overriding environment variable CONFIG_FORCE_x_y_z=some_value
See more details on HOCON documentation.

Consider the following snippet of NLPCraft configuration:

nlpcraft {
    probe {
        models = "com.nlp.MyModel"
    }

    server {
        lifecycle = "org.apache.nlpcraft.server.lifecycle.opencensus.NCJaegerExporter"

        rest {
            host = "0.0.0.0"
            port = 8081
            apiImpl = "org.apache.nlpcraft.server.rest.NCBasicRestApi"
        }
    }
}

You can override these properties with the following environment variables:

CONFIG_FORCE_nlpcraft_server_rest_host=1.2.3.4
CONFIG_FORCE_nlpcraft_server_lifecycle="org.nlp.Lifecycle1, org.nlp.Lifecycle1"
CONFIG_FORCE_nlpcraft_probe_models="com.nlp.MyModel, com.nlp.AnotherModel"

Examples

Note that all examples that come with NLPCraft have instructions that use environment variable overriding for running their data probes. They use default nlpcraft.conf file and just override one nlpcraft.probe.models property (see above) to specify what model the data probe needs to deploy.

ANSI Colors

Both NLPCraft server and probe use ANSI coloring via ANSI escape sequences for their log output by default. ANSI coloring provides easer console log comprehension and modern estethics.

However, there are cases when either specific console does not support ANSI escape sequences, or specific color schema isn't suitable or log being redirected to a file or piped to downstream system. In these cases you need to disable ANSI coloring to avoid polluting log with unprocessed ANSI escape codes.

You can disable ANSI coloring in either server, probe or both by supplying the following system property to JVM process: -DNLPCRAFT_ANSI_COLOR_DISABLED=true

Testing

It is a good practice to run units tests during routine builds using Maven (or others CI toolchains). To test data models you need to have a running server and then start one or more data probes with models you want to test. While doing this from IDE can be trivial enough, doing this from Maven can be tricky.

The challenge is that from the Maven build you need to start the server, wait til its fully started and initialized, and only then start issuing REST calls, start data probes or run tests that use embedded probes. When done manually (e.g. from IDE) you can visually observe when the server finished its startup and then manually launch the tests. In Maven, however, you need to use a special plugin to accomplish the same in automated fashion.

Probe Waits For Server

Technically, when a data probe starts up it will initialize, load the models, and will automatically wait for the server to get online if it isn't yet (as well as periodically check for it). Once server is online the data probe will automatically connect to it. However, if the unit tests don't use data probe and just issue REST calls then these tests have to somehow wait for the server to get online.

To overcome this challenge you can use process-exec-maven-plugin Maven plugin.

To get around this problem NLPCraft uses process-exec-maven-plugin Maven plugin in its own build. This plugin allows to start the external process and use configured URL endpoint to check whether or not the external process has fully started. This works perfect with NLPCraft server health check REST call. The plugin can be configured in the following way for your own project (taken directly from NLPCraft pom.xml):

<plugin>
    <groupId>com.bazaarvoice.maven.plugins</groupId>
    <artifactId>process-exec-maven-plugin</artifactId>
    <version>0.9</version>
    <executions>
        <execution>
            <id>pre-integration-test</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>start</goal>
            </goals>
            <configuration>
                <name>server</name>
                <healthcheckUrl>http://localhost:8081/api/v1/health</healthcheckUrl>
                <waitAfterLaunch>180</waitAfterLaunch>
                <processLogFile>${project.build.directory}/server.log</processLogFile>
                <arguments>
                    <argument>java</argument>
                    <argument>-Xmx4G</argument>
                    <argument>-Xms4G</argument>
                    <argument>--add-exports=java.base/jdk.internal.misc=ALL-UNNAMED</argument>
                    <argument>--add-exports=java.base/sun.nio.ch=ALL-UNNAMED</argument>
                    <argument>--add-exports=java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED</argument>
                    <argument>--add-exports=jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED</argument>
                    <argument>--add-exports=java.base/sun.reflect.generics.reflectiveObjects=ALL-UNNAMED</argument>
                    <argument>--add-opens=jdk.management/com.sun.management.internal=ALL-UNNAMED</argument>
                    <argument>--illegal-access=permit</argument>
                    <argument>-DNLPCRAFT_ANSI_COLOR_DISABLED=true</argument>
                    <argument>-Djdk.tls.client.protocols=TLSv1.2</argument>
                    <argument>-jar</argument>
                    <argument>${project.build.directory}/${nlpcraft.all.deps.jar}</argument>
                    <argument>-server</argument>
                </arguments>
            </configuration>
        </execution>
        <execution>
            <id>stop-all</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>stop-all</goal>
            </goals>
        </execution>
    </executions>
</plugin>

NOTES:

On line 14 we specify the URL endpoint to check whether or not our server is online. We use /health localhost REST call for that.
On line 16 we redirect the output from server to a dedicated file to avoid interleaving log from server and log from data probe in the same console (where we are running the Maven build from). Such interleaving will make the combined log unreadable and can cause output problem for the console due to mixed up ANSI escape sequences from server and data probe.
Since the server log is piped out to a separate file we disable ANSI coloring for the server on the line 28.

Avoid Interleaving Logs

When running both server and the data probe(s) from the Maven build it is important to avoid interleaving logs from the server and the probe. Such interleaving will make the combined log in Maven unreadable and can cause console malfunction due to mixed up ANSI escape codes. It is idiomatic in such cases to:

Disable ANSI coloring for the server via -DNLPCRAFT_ANSI_COLOR_DISABLED=true
Pipe out the server log into a dedicated file using Maven plugins like process-exec-maven-plugin.

Server & Probe