Smart Home Skill Testing - Sample and Guide
Learn how to quickly and comprehensively automate testing for Smart Home skills.
This is a simple project that demonstrates automated testing for a Smart Home skill.
Use this as a template for setting up automated testing for your own Smart Home skill.
It uses Bespoken's Virtual Device API to interact with Smart Home devices via Alexa programmatically. The test is run on a regular interval, optionally reporting results to DataDog for more advanced reporting and notifications.
This example also demonstrates an external API being used - when working with Smart Home devices, it is a best practice to use a "source of truth" to ensure that what Alexa says happened actually did happen. Most Smart Home devices come with their own dedicated API, and we show you how to use one as part of a complete test.
For the purpose of this sample, we just use a mock of an external API. For a real test, an implementer should replace the mock with their real API calls.
There are two test suites - single-device-test.js and multi-device-test.js.
The first one shows testing a single device, the second one shows a slightly more complex test that uses a CSV file to test a number of devices.
The basic sequence is simple:
- Initialize the device to a "clean" state - this uses our external API
- Interact with Alexa - "alexa, turn the bedroom lamp off"
- Check with the device via the external API to confirm the result
- Report the results to DataDog [OPTIONAL]
For our multi-device test, the device names being tested come from a CSV file. For those wishing to test many different devices, this example shows a simple but powerful approach to doing this.
To get setup, you will need the following (and some of this you likely already have):
- Node.js installed
- Project dependencies installed:
Open a command-line window, and run
npm installin the directory where this project is installed - A Bespoken Virtual Device token: Follow these directions to get one - Bespoken Virtual Device setup.
- [OPTIONAL] A DataDog account: DataDog is a nice tool for capturing and visualizing metrics as well as configuring notifications based on these metrics.
The virtual device token as well as the Data Dog API key should go into the .env file. If no DataDog API key is supplied, the step to report results to DataDog will be skipped.
Once configured, you can run the tests:
npm test
These tests are configured to be run on a five-minute interval via Circle CI.
The Circle CI workflow to do this is in this file: ./.circleci/config.yml.
For more detailed information, the Circle CI docs are here.
If you setup your own automated test using Circle CI, be sure to set the environment variables within Circle.
To read more about this project, check out our Deako case study this was based on LINK HERE.
Take a look at this nice graph from DataDog of the uptime for our device being tested:
Interested in automated testing for Home Automation? Have questions for us?
Reach out to [email protected] or chat with us on Gitter.