Alexa Game Design Interaction Design UI Design

Developing for Amazon Alexa

This summer when I spoke at PixelPop Festival, I went to a talk by Mike Lee about developing games for Amazon Alexa. He was working for a game studio that created a sort of “choose your own adventure” interactive radio play. Silvia and I have been writing a series of radio plays, so I was intrigued by the idea and decided to learn how to develop for Alexa.

I started researching, and quickly learned that most people start out with a tutorial to create a trivia game skill. This tutorial takes you through the whole process from getting your dev environment set up to publishing your skill in the Alexa Skill Store. Theoretically the only coding you need to do is to write your own questions in a big ol’ JavaScript object, although I ended up doing a bit more customization.

I needed a topic for my trivia game. Naturally I chose Sherlock Holmes, since I had just finished reading the whole Holmes canon. The tutorial suggested that the most successful trivia skills have over 100 questions, so I got to work writing them. This probably took up the biggest chunk of time of the whole project. It was a challenge to not only come up with 100 questions, but three incorrect answers for each question.

Holmes uses a magnifying glass to investigates the word RACHE written on the wall.
Illustration By D. H. Friston from A Study in Scarlet

The next challenge was pronunciation. Alexa was pronouncing stuff weirdly, or just using the wrong sense of the word, as in “His Last Bow.” Alexa supports Speech Synthesis Markup Language (SSML), which I used to correct most of this.
<phoneme alphabet="ipa" ph="mʌnki">monkey</phoneme>

<w role="amazon:SENSE_1">Bow</w>
But then there were words from other languages, like the famous German word RACHE written in blood on the wall in A Study In Scarlet. Alexa (or maybe SSML in general?) does not support the use of German phonemes within English language text. So I had to approximate it as best I could with English phonemes, which, as it turns out, is not very accurate.

It was actually pretty fun to figure out how words are supposed to be pronounced that I had only read and never heard. There are some things that Doyle just made up, like The Mazarin Stone. My go-to source for pronunciation was the Jeremy Brett Granada series, in which I learned, among other things, that Mazarin is pronounced as in the French. I was surprisingly able to recreate this pronunciation fairly well with English phonemes.

Another issue I had was Alexa overdoing the upward inflection at the end of the sentence to indicate questions. I tried to fix this by replacing the question marks with periods. This fixed the inflection, but created another very curious issue that I tried to debug for hours to no avail. For some reason if there are any periods in the text of the question, instead of reading the four multiple choice answers, Alexa will say the letters “Q” “U” “E” and “S” in a random order. The bug was reported here, but the issue was closed without being fixed. Hopefully it will get some attention from the tutorial developers.

Overcoming the pronunciation issues and a few other minor bugs, I published Sherlock Holmes Trivia in the Alexa Skill Store in November, and so far I’ve had almost 250 unique users. I’d eventually like to add some more features, like different levels of difficulty. It is a rather challenging quiz, but I hope you’ll give it a try!

Game Design Interaction Design UI Design Usability UX Design Virtual Reality

Virtual Reality Puzzle Game


My most recent VR project is a simple Simon-esque game in which the player must select floating orbs in the correct sequence in order to pass through a dungeon. This was the fourth project for Udacity’s VR Developer Nanodegree. I enjoyed this project a lot because there was more emphasis on designing a good user experience. I got to put my UX skills to use to create personas, draw sketches, conduct user tests, and iterate on the design.



One of the first things I did was create three personas of different users who might be interested in this game. I already wrote about this in a previous post, where you can also download the persona template that I designed. You can click the images below to see them bigger.


I made several sketches throughout the process. I haven’t make all of the sketches a reality yet, but I have a clear idea of what I want the final experience to be like. I added several bits of backstory to the game to make the player more invested in completing the puzzle.

Working in Unity

I created the game in four stages, with user testing and iteration after each stage.

  1. Create the environment (3D models and lighting). Do user testing for mood and scale.
  2. Create the UI for starting and resetting the game. Test for scale, placement, legibility, and clarity.
  3. Add camera motion and program UI buttons. Test for speed and comfort to avoid simulation sickness.
  4. Program game mechanics. Test for playability, understanding of rules, and overall experience.

User Testing and Iteration

Below are videos of my first playtest and my most recent playtest, so you can see the progress made so far.


Some of the things discovered and fixed from playtesting included:

  • Some people felt too short. Adjusted the camera position and the overall scale of the environment.
  • The UI was much too close to the camera at first. Repositioned it farther away.
  • The start UI blocked the view of the dungeon, so it was really disorienting when the camera started moving. I fixed this by making the UI partially transparent.
  • The camera motion was too fast. Slowed it down to create smoother transition into dungeon.
  • The positioning of the orbs was too high to be able to comfortably select the top two. Repositioned them to be more in the line of sight.


And these are some things that I haven’t fixed yet:

  • There is a glitch where the back wall of the dungeon changes color when you look at it.
  • Some people do not recognize the “Simon-esque” aspect of the game, so they don’t know what to do when the orbs light up.
  • The negative feedback sound when you select the wrong orb is not clear enough.

Current State

Here is gameplay video of the game in its current state, as of Oct. 1, 2017. For some reason the screen recording software did not capture the ambient environment sounds.


Planned Improvements

  • Incorporate all the story elements to give players more choices (see sketches above).
  • Program a lose condition if they select the wrong orb too many times.
  • Program different levels of difficulty.
  • Create either clear instructions about how to play or hints if the player gets stuck.
  • Animate the orbs to move out of the way upon completing the puzzle.
  • Give each orb a unique pitch and color.
  • Create background scenery.
  • Improve UI design.
  • Improve sound design.


UI Design

UI sketching supplies

I pretty much always start on paper when sketching out new ideas (I even write code on paper sometimes). So I really nerded out when I discovered UI Stencils—they have a pixel ruler!! I love anything that brings the digital into the physical world. I splurged and bought myself a bunch of sketching supplies, with the goal of really working on my wireframing and storyboarding chops this summer.

pens, markers, and stencils for making UI sketches
The supplies I used to make my first sketch.

I’m especially happy with what they call the Everyday Carry Kit. It comes with a set of three dot-grid notebooks, a stencil, and a Zebra mechanical pencil that are all small enough to fit in my pocket (at least in the summer while I’m wearing shorts). This has been great way for me to just start sketching something whenever I have a spare moment. For my first sketch, I just started playing around with the stencil, without having a plan of what I was making. It gradually turned into one step of an account setup interface where you would start adding friends.

A pencil sketch of a UI for adding friends to your network.
The initial sketch in pencil. This just came together as I was trying out different parts of the stencil.

The stencil is awesome, although there are some issues. It is hard to align stuff correctly, because the stencil covers up the other parts of the sketch that you are trying to align to. You can see this in the poor alignment of the user icons and the plusses inside the circles. I started to get better at this by really paying attention to the dot-grid, as you can see in the check boxes.

The other purchase I’m really nerding out about is a set of gray Copic markers. They really help set off different parts of your design hierarchy, and make your paper wireframes look more professional. I LOVE the ultra-subtle ‘N0’ color, which I used for the background. I found the Copics cheaper on Amazon than on the UI Stencils site. I also used some other pens and markers to add outlines and annotations.

A finished wireframe sketch in pen and marker.
The finished sketch using gray Copics to further distinguish design elements.

I’m looking forward to devoting some serious time to developing my pen-to-paper skills and making these look even more professional. One thing I realized I need to work on is my lettering. I have okay handwriting, but I’d like to get it looking a little more uniform.

Supplies used