1. Plug it In
Welcome to the SLN-TLHMI-IOT Getting Started guide. This guide walks
you through the process of getting up and running with your
SLN-TLHMI-IOT board. This guide takes you through the steps of
unboxing your kit, running the out-of-box coffee machine demo
application, as well as downloading, modifying and debugging the
firmware source code for your kit.
Although the SLN-TLHMI-IOT supports multiple out of box applications
(which can be selected by pressing and holding SW1, SW2 or SW3 on
the back of the kit), this guide will focus on the Coffee Machine
out of box demo.
The Coffee Machine application demonstrates not only traditional
touch-based controls, but also the face recognition, multi-language
far-field voice recognition and graphics capabilities of the kit.
Before we begin, make sure to check the box your kit came in for any marks or damage and should you find
anything, be sure to report it to
your local NXP representative.
The kit your box arrived in with few different things, such
- A packing list paper
- Fully assembled TLHMI kit
- USB-C cable
1.3 Saving a New Face
Let us get started with a demonstration of this application’s out-of-box features.
To demonstrate the SLN-TLHMI-IOT's ability to create tailored custom experiences for each saved user, we are
using our face to save a custom coffee profile.
Before diving in, let us take a quick tour of the home screen layout.
The home screen is divided into two halves: the camera preview pane on the left, and the coffee selection menu
on the right.
Additionally, there is a Start button in the lower-left corner of the screen and a language selector drop-down
menu in the top-right corner of the screen.
On the camera preview pane, a bounding box is provided to help you
properly align your face so that it may be identified by the kit.
The different colors of the bounding box are used to indicate face
recognition status. By default, the box is colored blue to indicate
that no face has been detected.
The color red is used to indicate that an unrecognized face has been
detected. Align your face with the camera until the bounding box
turns red. This should take < 1s if done properly
Once your face has been detected by the camera, it can later be used
to associate with your preferred coffee selection.
Now that your face has been detected, you are ready to select your
coffee preferences. The coffee configuration options can be seen on
the right half of the home screen as shown in Figure 10. These
options include coffee type, size and strength.
To choose your coffee preferences, follow the steps below:
Select the coffee type by pressing the icon associated with your
Next, click the checkbox selector next to your preferred size.
Finally, select the checkbox selector next to your preferred
The coffee selection has been customized. Before moving on, ensure
that your face has been detected by the camera (indicated by a red
bounding box) before pressing the Start button to begin brewing
After pressing the start button, a brewing animation is shown. The
brewing screen displays a progress bar and a summary of the chosen
selections on the left.
Once the brewing animation is finished, the chosen coffee is shown
on the final screen.
After a brief delay, a voice prompt plays via the onboard
speakers, asking “Save your coffee selection? Confirm or Cancel.”
Say “Confirm” to save this order. Answering “Confirm” associates
the coffee that was brewed with your face. Next time your face is
recognized, the coffee machine automatically asks whether you
would like to reuse your saved order.
After your order is complete and the prompt has been responded to,
the device will automatically reenter the standby screen that was
shown at startup.
1.4 Modifying a Saved Order
Now that we have demonstrated the core features using traditional
touch-based controls, we will walk through the next generation
hands-free voice control capabilities made possible by the
processing power of the i.MX RT117H.
First, wake the board out of Standby mode using the “Hey NXP” wake
With the board active once more, align your face inside the bounding
box until the color of the box turns green.
A green bounding box indicates that you are a recognized user. The
coffee machine plays an audio prompt asking if you would like to
order “another Espresso? Confirm or Cancel?”
Responding with “Confirm” immediately takes you to the brewing
screen again, while “Cancel” can be used to choose a different
Instead of reusing our saved order, respond to the prompt with the
“Cancel” voice command, allowing us to choose something different.
This time around, however, we will use hands-free voice commands
instead of the touch controls we used previously.
Follow the steps below:
Set the coffee type by using the “Cappuccino” voice command.
Successfully recognized voice commands are indicated by a green
LED and brief audio tone.
Now, update the coffee size using the “Medium” voice command.
Set the strength to the maximum using the “Strong” voice command.
Finally, start the brewing animation again using the “Start” voice
Once the animation is completed, and the final screen is reached,
you are prompted again to save your new order (replacing the
Responding to the prompt with “Confirm” overwrites your previously
saved order with your latest selection. Now, the next time your face
is recognized it automatically uses your updated order.
1.5 Deleting a Saved Face
Now that we have demonstrated saving a face and updating a coffee order, let’s finally demonstrate deleting our face from the
local face database.
To do so, reawaken the board from Standby mode using the “Hey NXP” wake word and/or by touching the screen.
After the display turns back on, align your face with the onscreen bounding box. Once your face is recognized, use the “Delete”
voice command to delete your face from the database. Alternatively, you can click the red X icon in the top-left corner of the
screen to remove the recognized face.
After successfully saving a new coffee machine user, modifying and deleting the order, we have completed our introduction of the
TLHMI coffee machine out-of-box demo experience.
Now that you are familiar with the basics, see the
Smart HMI User Guide for a
comprehensive list of all the outof- box features available in SLN-TLHMI-IOT. The guide also includes the additional demo
applications available such as the smart elevator and smart panel applications.