I recently attended the Virtual Reality (VR) Hackathon 2016 in Seattle. Although I ultimately didn’t work on a very exclusive “VR” hack. I got to meet a lot of interesting people, look at some really cool VR tech, and learned how to hack the Amazon Echo.

The Seattle VR Hackathon 2016 ran the 3rd weekend in April at the University of Washington. It started Friday night and ran until Sunday afternoon.

During the Hackathon Friday night mixer, we formed into groups. The group I joined consisted of three people. After forming the group, we began to decide what we would do for our hack. One of the back-end developers in the group wanted to do something with the Amazon Echo hardware. The other developer and myself wanted to do data visualization in VR. We tossed around several ideas and decided the Amazon Echo, with the use of its AI, named Alexa, would be a great utility to assist users while immersed in VR. Unfortunately, this meant that the data visualization would need to wait. The clouds slowly parted and the plan materialized. We would use Alexa as our personal assistant while in VR.

Because you can become so involved in the VR world with goggles on, you need a connection to the real world in order to do things like order a pizza, call your mom, or text your friends to join you in a game. The idea we had was a utility function that would allow you to send communications to the real world without interrupting game play – to provide you the ability to send your friend a text message and tell him/her to come join you in the game, by simply asking Alexa to do it. Once the plan was set, we parted for the night.

Saturday morning we met up again, and after a primer tutorial by the Amazon team, we confirmed that the Echo would work great for our project. We got started putting the pieces of the puzzle together. In order to accomplish it we would need to use a few different services.

• Amazon Echo for voice interaction. Amazon Echo uses an AI and responds to the name Alexa.

• Amazons AWS Lambda for handling the logic and programmatic responses to our requests. AWS Lambda is a serverless compute service that runs your code in response to events and automatically manages the underlying compute resources for you. You can use AWS Lambda to extend other AWS services with custom logic, or create your own back-end services that operate at AWS scale, performance, and security.

• Twilio for making telephone calls and sending text messages. Twilio allows software developers to programmatically make and receive phone calls and send and receive text messages using its web service APIs.

By creating a set of what is termed as “Alexa skills,” we could trigger programmed events to be passed using speech recognition from Alexa and get scripted interaction with people in the real world. The functions called by “Alexa skills” were handled by the AWS service Lambda. Using Lambda, we could write functions in node.js, python, or java. The functions would make web services calls to Twilio. Using the Twilio API, we were able to write scripts to make the telephone call or send a text message. Once the person received the phone call and text, the Amazon Echo would respond back to the user in a nice voice with a programmatic response like “message sent.”

By the end of Saturday we had a working prototype where a user could say out loud “Alexa, call John!” and immediately the user would receive a text message confirming the call had been sent. Alexa would then respond “Done!” and the person “John” would receive a telephone call, on his cell phone, from a Twilio robot telling him the words we had programmed it to say. With a little knuckle grease, some refreshers to node.js, a few facepalms, the pieces of the puzzle weren’t that difficult to put together.

The last day of the Hackathon came and we presented to the judges and other hackers. As with many of the hacker events, there were a bunch of attendees, which made 24+ teams and some really good hacks. Our hack only loosely fit in with two of the six categories, and we didn’t win in either of those categories. Although we didn’t win any prizes, we had a great time and we learned some valuable skills that I will definitely use on other projects. All that said, it was a lot of fun and I would do it again.

~Xandon Frogget