Using entities to extract data

A core feature of the AI engine is the ability to extract data from user input using entities

Entities

Entities contain the variables we want to extract from the text a user sents. For example, when a user sends I want to fly from Amsterdam to San Francisco we want to detect that Amsterdam and San Francisco are a place of departure and arrival.

Within the intent view you can mark entities by selecting them and assigning an entity type. To train the entity classifier annotate lots of examples. When you annotate lots of cities for example, the classifier will learn to recognize cities.

This means it will also pick up on cities you didn’t explicitly marked. Of course you are not limited to cities: you could create a food entity, an animal entity or a movie entity, or something else entirely, you name it!

TIP: Mark and name entities consistently

Make sure you always mark every example. Do not skip any, as this harms the classification process.

Validation and formatting

We allow the usage of various entity types that allow validation and data transformation. For example the system will convert any entity containing tomorrow to a UTC datetime format if it’s a date entity type.

System entity types

List entity type

List entity types work in a similar way as the Text entity type. There is one important difference. Lists provide a way to give boundaries to what the AI will match.

Let’s say we take the above example of a departure and destination city. Perhaps you only want a select number of cities a person can travel to. You can limit these options by creating a custom entity list type named city. Within this entity type you provide only matched city names that are valid to travel to.

When the AI detects an entity of this custom entity type it will determine if it matches. The AI could find an entity that is not valid in relation to the type of entity, For example, I want to travel to Washington could be matched in the above example, but since Washington is not in our list, it’s immediately dropped

Tip: Matching is fuzzy

No need to add typos as synonyms, matching is case insensitive and works with grammar errors

Exact matching

By default custom entities will do an exact match. When you turn of exact matching, custom entities will work the same way as a regular Text entity type works.

Using matches in code

Extracted entities can be used within Code, webhooks and reply text templates. They always come in the form of an array, even when only one match is found.

// Example payload inside cloud code with a destination match
{
    params: {
        destination: [{
            match: "NYc",
            value: "New York",
            id: "NY"        
        }]
    }
}

Multiple destinations would look like

// Example payload inside cloud code with a 2 destination matches
{
    params: {
        destination: [{
            match: "NYc",
            value: "New York",
            id: "NY"        
        },{
            match: "Amsterdam",
            value: "Amsterdam",
            id: "AMS"
        }]
    }
}