Joe Fletcher – Touch and Gesture Computing

The Discussion

• 

  Regarding having audible cues when typing on a virtual keyboard, does Windows 7:
  
  1) Allow you to set whether there is audio feedback whenever you type on the virtual keyboard?
  2) Allow you to have virtual keyboard audio feedback but no other sound from the computer and vice versa?
  3) Allow you to independently set the volume of the virtual keyboard audio feedback?
  
  Thanks in advance for your response.
  
  P. Douglas

• 

  I loved this video and I will definetly attend this session at MIX.
  
  I hope you guys release some API for Windows Mobile and touch.
  And I'm really excited to see a Surface computer because I've never seen one in real life.
  
  At the session could you talk more about some of the programming aspects of touch. For example, how do you record a motion? It would be awesome if you could set up an event handler to fire if the user performed a certain finger gesture. Have in Visual Studio a gesture creator to capture a gesture the developer draws and then setup an event handler (this all is wishful thinking).

• 

  That's actually pretty much how the surface sdk already works. There are gesture events, and you can subscribe to them happening at any level in your app. There isn't yet a great way to define your own gestures, and there aren't a lot of built-in gestures (just the basics) but I expect that will change with the next SDK version.
  
  @JoeF
  Looking forward to your talk at MIX. What I'm really interested in lately is how Microsoft is viewing touch as a whole. It is presenting touch in so many different ways to consumers, and I'm starting to wonder if they will ever pull it all together into a coherent story. Right now its quite fragmented. Already I'm getting lots of questions from people who confuse Surface and Win7, or Tablets and Touchsmart, or Windows Mobile touch screens vs. Windows 7 touch. I think Microsoft needs a Touch Czar.

• 

  @pDoug

  From the public Windows 7 beta, I know the virtual keyboard does *have sound. I’m not sure if you can customize it besides on/off through a system volume. It doesn’t appear to have any options in the keyboard itself.

   

  @ChazB

  Thanks ^__^ I’m not as plugged into the Windows Mobile side of things, so I’m not sure. As for Surface computers, there will definitely be some at MIX for you to check out. How we record motion isn’t part of my talk right now, but if you connect with me before/after the talk I can explain a little bit of how it works. In short, Microsoft Surface is a vision system and uses cameras inside the unit to see what people are doing. We watch motions, then react to them through the system. You mention an easy way to “capture” a gesture. Due to current technology differences, that would be tough to go cross platform I believe, but I’m not as knowledgeable of the programming aspects. Seraph had good points as well.

   

  @Seraph

  You’re not the only one who is thinking about the cross MS touch story. Microsoft is huge, and computer users will eventually dictate norms. It may take a little while for everything to click though.
  
  
  Thanks! Joe Fletcher