CES 2010: NUI with Bill Buxton

A few quick comments on your post:

Yes, "painter's shoulders" is an issue.  If we are working on a kiosk or a whiteboard, things may be fine.   In the former, the interaction will likely be brief, and in the latter, we can get close enough that the arms need not be extended out so far as to cause excess fatigue.  On the other hand, when sitting, and reaching over a keyboard, desk, etc. to a vertical monitor, we can generally expect less use and more fatigue, due to extent of reach.  There is a reason that drafting tables were in a sloping desk rather than vertical wall format!

As for human skill, the interesting thing is that the essence of skill acquisition (and hence skill utilization and transfer) is the notion of "chunking".  (See  the following if you want more background:  http://billbuxton.com/chunking.html).  This prompts me to go beyond wondering if/when I should use touch or stylus, and rather - recognizing that I have two hands, and can use them together - consider when it is best to use them together:  for example, touch in my non-dominant hand, and stylus in my dominant one.  This reflects what we do in the everyday physical world, and provides a mechanism to "chunk" together low level tasks into a higher level one, using an existing skill.  We have to stop looking at technologies as being in competition.

As for operating devices while driving, I don't think that we have a disagreement - however, your comment suggests to me that I should make the point more strongly than I did.  You will note that in the interview I did say minimizing cognitive interferance as well as visual and manual.  As well, I qualified things by saying something, "to the extent that it can be done safely."  But you are right, while having the perfect voice only phone (let's pretend that there could be such a thing) sitting on the passenger seat, and speaking to a remote person using it, it is definately NOT the same as having the same conversation with a live person sitting in the same seat under the same conditions.  The real person has cose to the same ability to perceive the world and context in which you are driving as you, so knows why you go silent, or when they need to shut up for a moment.  Hence, the driver's ability to flow in and out of the conversation with minimal distraction/overhead, is much better than it would be with the person on the phone, who as no idea that something was about to happen that really needed your full attention.  The reality is, that there are a large number of things that already distract us from our primary task of driving (tuning the radio, adjusting the heat, sipping on a soda, listening to an interview on the radio, thinking about your spouse ...).  So the real question is, where do we draw the line?  The related question is this:  with an understanding of the issues above, by worthy design, how low can we get the interference of a remote conversation?  Can we get it below the threshold of acceptable risk?  For example, would augmenting the audio with video that gives the remote person a better sence of context help?  I'm not advocating a design, or even suggesting that there is an acceptable solution.  The fact is, unless I ask such questions, I can't answer the question.  But you are absolutely right:  to even begin, we have to be aware of the issue, or we won't begin asking them.  What I do know is that it is certainly unacceptable to text, read email, search for an address, etc. by any combination of voice and touch while driving.  We both know that.  Thanks for the reminder.

Thanks for the comments.

Bill