Using entities to extract data
A core feature of the AI engine is the ability to extract data from user input using entities
Entities
Entities contain the variables we want to extract from the text a user sents. For example, when a user sends I want to fly from Amsterdam to San Francisco
we want to detect that Amsterdam
and San Francisco
are a place of departure and arrival.
Within the intent view you can mark entities by selecting them and assigning an entity type. To train the entity classifier annotate lots of examples. When you annotate lots of cities for example, the classifier will learn to recognize cities.
This means it will also pick up on cities you didn’t explicitly marked. Of course you are not limited to cities: you could create a food entity, an animal entity or a movie entity, or something else entirely, you name it!
TIP: Mark and name entities consistently
Make sure you always mark every example. Do not skip any, as this harms the classification process.
Validation and formatting
We allow the usage of various entity types that allow validation and data transformation. For example the system will convert any entity containing tomorrow
to a UTC datetime
format if it’s a date
entity type.
System entity types
Name | Examples |
---|---|
Text | This is the most used entity type. The AI will match anything you train it using your examples. If you’d like to match music artists, train the AI with examples like
Madonna
,
Michael Jackson
,
The Weeknd
etc |
Date | Tomorrow , next week , 1e of July |
Time | 12am , 23:59 , now, tomorrow at 8am |
Number | 2453 . twenty one |
[email protected] | |
URL | examplecorp.com |
Distance | 200 meters , 12 miles , 1.5cm |
Text | Anything from names to cities. |
List entity type
List entity types work in a similar way as the Text
entity type. There is one important difference. Lists provide a way to give boundaries to what the AI will match.
Let’s say we take the above example of a departure and destination city. Perhaps you only want a select number of cities a person can travel to. You can limit these options by creating a custom entity list type named city. Within this entity type you provide only matched city names that are valid to travel to.
When the AI detects an entity of this custom entity type it will determine if it matches. The AI could find an entity that is not valid in relation to the type of entity, For example, I want to travel to Washington
could be matched in the above example, but since Washington
is not in our list, it’s immediately dropped
Tip: Matching is fuzzy
No need to add typos as synonyms, matching is case insensitive and works with grammar errors
Exact matching
By default custom entities will do an exact match. When you turn of exact matching, custom entities will work the same way as a regular Text
entity type works.
Using matches in code
Extracted entities can be used within Code, webhooks and reply text templates. They always come in the form of an array, even when only one match is found.
// Example payload inside cloud code with a destination match
{
params: {
destination: [{
match: "NYc",
value: "New York",
id: "NY"
}]
}
}
Multiple destinations would look like
// Example payload inside cloud code with a 2 destination matches
{
params: {
destination: [{
match: "NYc",
value: "New York",
id: "NY"
},{
match: "Amsterdam",
value: "Amsterdam",
id: "AMS"
}]
}
}