NUIverse - Reach out and touch...
- Posted: May 15, 2013 at 6:00AM
- 4,990 views
I don't normally blog about projects that don't have source, but this project from Dr. Dave is just cool so I'm making an exception. Plus since it has an extensibility model, I guess that makes it code'able? Kind of? Anyway, it's cool and it's fun...
NUIverse is an application, written to give me a sandbox for exploring “Natural User Interaction” with a large, multi-dimensional dataset. This build for the Samsung SUR40 with Microsoft PixelSense demonstrates multi-touch gestures, multi-user and multi-directional UI, and object-interaction with transparent Byte Tags. This site provides brief instructions for installing, using, and configuring NUIverse.
NUIverse is not intented as an accurate simulation of the solar system and surrounding universe, since it was written in my spare time and the focus was on interaction rather than accuracy. There are many existing applications available for this purpose, such as Microsoft WorldWide Telescope, Celestia, Space Engine etc.
As previously discussed, my manipulation processor now supports WM_TOUCH messages, which means that I can do native multitouch on both Windows 7 and Windows 8. I have therefore updated NUIverse for a Windows-release, as shown below in Figure 1.
There are some key differences to the PixelSense-release, as follows:
- No support for tagged objects, since it does not use the Surface 2.0 runtime, nor require PixelSense hardware (though it will run on the latter outside of the Surface Shell).
- Since horizontal form-factor multitouch hardware is generally less-common than vertical form-factors, I have added a single-orientation configuration setting. This is true by default, since even if mounted horizontally, many touchscreens will not deliver the multitouch performance required for simultaneous multi-user interaction.
- Since the Surface Shell added chrome to close the application and by default the application runs full-screen, either drag in a menu control (see note below) and use the exit menu, or press ESC if a keyboard is present.
One of the current issues when running a full-screen desktop app on Windows 8 is that the operating system captures initial touches used for edge-swipes. If touch is maintained after an initial edge-swipe, further edge-swipes are not captured and therefore will add NUIverse controls to the screen. An alternative is to touch the screen and simultaneously edge-swipe, or to use two fingers when edge-swiping.
Several key configuration settings (in NUIverse.exe.config) are worth mentioning. Note that there is no graphical interface for these settings, and that the configuration file needs to be edited by hand (I would recommend saving a copy first):
- PixelWidth and PixelHeight control the resolution used for both windowed and full-screen mode.
- FullScreen controls whether the application runs full-screen (true) or windowed (false).
- For the configuration settings specified in mm to work correctly, set PixelsPerMm to the appropriate value, taking account physical screen size and either PixelWidth or PixelHeight (square pixels are assumed).
To install NUIverse for Windows, proceed as folows:
Here's a snap of the app running on my Windows 8 notebook;
I said there was extensibility?
I previously mentioned that I had implemented an extensibility model, and thought it useful to discuss an example of adding a simple model to earth orbit, as shown below in Figure 1 (further images of which are in the gallery).
Figure 1. Model added to earth orbit. Colonial Raptor model (based on new TV series) by Coxxon.
The "extra" is defined as a folder containing the following items:
- A model in XNB format. XNA has built-in content importers for .x and .fbx (2009.1) formats.
- An optional pair of textures for both diffuse and emissive textures. These are standard image files.
- An XML file defining the "extra", in this case as shown below in Listing 1.
This configuration file specifies that the model should be added to the planetoid "earth" in the "solar" system, both of which are defined in system.xml configuration file.
In order to scale the model correctly, a scale factor is applied to normalize the model to unit length. This can either be applied in the XML scale attribute, or specified in the XNA content processor scale attribute, in which case the XML attribute can be set to 1. A size attribute then defines the maximum length of the model in km. The Colonial Raptor shown in Figure 1 was defined with a size of 8.6m.
The textures are defined in sub-folders "texture" and "emissive". If an emissive texture is not available, an all-black image (e.g. JPEG file) can be used.
The rotation period defines how long it takes for the model to rotate while orbiting the planetoid. If this is the same as the P orbital element, then the same face of the model is presented to the planetoid throughout the orbit. The remaining standard orbital elements specify that the model is in a circular equatorial orbit at an altitude of 500km (the earth has a radius of 6,371km).