Joe Fletcher – Touch and Gesture Computing

Play Joe Fletcher – Touch and Gesture Computing
Sign in to queue


Joe Fletcher works in the Surface team, and is focused on identifying design approaches which best utilize touch-based interfaces for both Surface as well as Windows 7. In this episode we will discuss with Joe some of the concepts that he will be expanding upon in his upcoming session at MIX09. If you have specific issues you’d like to see him cover in his session, please leave a comment here letting him know!


Here is the description for the session he will be doing at MIX09:


Touch and Gesture Computing, What You Haven’t Heard

by: Joe Fletcher


Learn about new patterns, behaviors, and design approaches for touch and gesture interfaces from a practitioners point of view.  Learn early lessons from applied knowledge of touch applications, devices, and design methods.



Download this episode

The Discussion

  • User profile image
    Regarding having audible cues when typing on a virtual keyboard, does Windows 7:

    1) Allow you to set whether there is audio feedback whenever you type on the virtual keyboard?
    2) Allow you to have virtual keyboard audio feedback but no other sound from the computer and vice versa?
    3) Allow you to independently set the volume of the virtual keyboard audio feedback?

    Thanks in advance for your response.

    P. Douglas
  • User profile image
    I loved this video and I will definetly attend this session at MIX.

    I hope you guys release some API for Windows Mobile and touch.
    And I'm really excited to see a Surface computer because I've never seen one in real life.

    At the session could you talk more about some of the programming aspects of touch. For example, how do you record a motion? It would be awesome if you could set up an event handler to fire if the user performed a certain finger gesture. Have in Visual Studio a gesture creator to capture a gesture the developer draws and then setup an event handler (this all is wishful thinking).
  • User profile image

    That's actually pretty much how the surface sdk already works. There are gesture events, and you can subscribe to them happening at any level in your app. There isn't yet a great way to define your own gestures, and there aren't a lot of built-in gestures (just the basics) but I expect that will change with the next SDK version.

    Looking forward to your talk at MIX. What I'm really interested in lately is how Microsoft is viewing touch as a whole. It is presenting touch in so many different ways to consumers, and I'm starting to wonder if they will ever pull it all together into a coherent story. Right now its quite fragmented. Already I'm getting lots of questions from people who confuse Surface and Win7, or Tablets and Touchsmart, or Windows Mobile touch screens vs. Windows 7 touch. I think Microsoft needs a Touch Czar. Smiley

  • User profile image


    From the public Windows 7 beta, I know the virtual keyboard does *have sound. I’m not sure if you can customize it besides on/off through a system volume. It doesn’t appear to have any options in the keyboard itself.



    Thanks ^__^ I’m not as plugged into the Windows Mobile side of things, so I’m not sure. As for Surface computers, there will definitely be some at MIX for you to check out. How we record motion isn’t part of my talk right now, but if you connect with me before/after the talk I can explain a little bit of how it works. In short, Microsoft Surface is a vision system and uses cameras inside the unit to see what people are doing. We watch motions, then react to them through the system. You mention an easy way to “capture” a gesture. Due to current technology differences, that would be tough to go cross platform I believe, but I’m not as knowledgeable of the programming aspects. Seraph had good points as well.



    You’re not the only one who is thinking about the cross MS touch story. Microsoft is huge, and computer users will eventually dictate norms. It may take a little while for everything to click though.

    Thanks! Joe Fletcher

Add Your 2 Cents