Getting your WP7 Desktop integration out of park with PAARC, the Phone as a Remote Control library
Today's Mobile Monday project is one that I've been watching for while and one that I think can lead to some very cool applications.
You have your Windows app. You have your Windows Phone 7. Wouldn't it be cool if you could control your Windows app from your phone? Turning your phone into a very powerful, configurable and flexible controller? Turning it into a remote control for your Windows app? But how would you write that? How would you connect the phone to the app? Send commands and status back a forth?
That's the thing. You don't have to write it! Peter Kuhn (aka Mister Goodcat) has already written it for you and even open sourced the source too!
Shortly after I got my Windows Phone I had the idea of creating an app that lets me remotely control my desktop computer. The reason for that was that lazy me needed to use a clumsy and error-prone full-size wireless keyboard and mouse when I wanted to surf the web from my couch, using the small media PC connected to my TV set. Wouldn't it be nicer to use the much more handy mobile phone as input device? Unfortunately the RTM version of Windows Phone was missing some networking features for that – using web services or http wasn't what I was looking for. The Mango release changed that, because TCP/UDP socket support was added to the platform.
After I started working on the app I quickly realized that it would be really cool to not just have this feature for my own purposes, but to create a generic library which could be integrated into your own applications, both on the phone and desktop side. That was the moment when I decided to create PAARC ("Phone as a Remote Control") – sorry for the boring name, I couldn't come up with something better and more exciting .
PAARC has the following features:
- Uses TCP and UDP sockets for maximum performance and responsiveness (watch the demo video to see for yourself), but handles all the networking and threading stuff for you behind the scenes.
- Requires minimum technical knowledge from your end-users by e.g. using a server discovery mechanism – no cryptic networking terms to handle, "one-click" connect feature.
- Supports raw (multi-)touch input, the built-in phone gestures like taps, flicks, pinching and dragging, acquisition of all sensor data (accelerometer, compass, gyroscope, combined motion) and text input.
- Comes with a reference WP7 client with support for tombstoning/deactivation and all supported data types, so you don't have to create a separate app to get started (but you can do that, of course, or integrate the client-side library into your already existing phone app).
- Allows full configuration and control from the host application (your .NET desktop app).
- Has built-in remote tracing features to track down problems.
- Comes with a few demo applications you can learn from and is fully documented.
Ever since the first announcement of my "Phone as a Remote Control" project the goal was to work towards a release of the reference app in the Windows Phone Marketplace. Over the last month I've added some improvements and fixes, and I'm glad to announce that last weekend a first version has been accepted and is now publicly available – for free and without advertisement. You should now be able to find the app doing a search for "PAARC" in the Marketplace on your phone, or by visiting this web link. In this post, I'll talk about what reaching this milestone means for you as a developer, and what's in the box for normal users.
Windows Phone PC Controller
One aspect of the now available reference client is of course to function as a showcase for the open source library. To bring it to a meaningful use, one of the included samples available in the project has been extended and evolved into something a bit mature: the Windows Phone PC Controller. This is a .NET desktop application that allows you to transmit phone sensor/touch input to a desktop computer and translate it into operating system level mouse commands which effectively turns your phone into a remote controlling input device for any computer. Again, the source code for this app also is part of the open source project; as a convenience, a pre-compiled installer for it is available to the average user on this stub web page (which also contains some more user-friendly information and FAQs):
The Windows Phone PC Controller project allows you to control any desktop or media PC using your Windows Phone as input device. Connect your phone to the PC using your Wi-Fi connection with a single tap, and all the touch input, gestures and even accelerometer data (if you like) are transmitted and translated into mouse actions. You also can enter text on the phone that is converted to keyboard input on the PC - couch surfing never was easier!
To make this work, you need first need to get two pieces:
- the PC Controller desktop application
- the app that runs on your phone
The Windows Phone PC Controller desktop application can be downloaded from this page. It comes as a convenient installer. Download, run and follow the simple instructions to automatically install it on the computer you want to control.
Windows Phone PC Controller 1.0 (MSI package, ~640 KBytes)
(Windows 7 32/64-bit)
The phone application is named PAARC ("Phone as a Remote Controller") and can be downloaded and installed from the Marketplace, directly on your phone, or using the web link here:
PAARC in the Marketplace
(Requires a Windows Phone 7.5 "Mango" compatible device)
Simply start the desktop application on your PC. No need to change anything, the default configuration settings should be a great start. The application now sits and waits for incoming phone connections. You can even minimize it to the system tray to get it out of your way, and simply leave it running all day long.
Now start the phone application and hit the Auto-Connect button. Your phone should connect to your PC, and you can use the touch input to control your desktop! In case the automatic connection fails, you can enter the IP address of your PC manually. This address is shown in the PC application for your convenience.
Phone as a Remote Control
The paarc library is a project aimed at developers that want to use a Windows Phone as input device for any .NET application running on a remote/desktop computer. It consists of different included components ranging from the core library, a reference implementation on the phone and several samples on the PC side (more on all this below). The library doesn't make any assumptions about the remote .NET application per se, which allows you to connect a phone to any kind of desktop application or game, for example to use it as sensor, as touch input device, game controller, remote keyboard, or whatever else you intend it for.
For a quick live demo of the library in action with some basic explanations, take a look at this 7-minute video:
Componentspaarc consists of the following parts:
- Core library: the core components are made of the following four parts:
- Two client-side Windows Phone assemblies that are able to acquire data from different phone sources, and to connect to a remote server using sockets. The communication assembly is able to receive configuration and control commands from a connected server, send data back to the server, and to use multicast to discover a server on the same network automatically. The data acquisition assembly includes the following possible data sources:
- All built-in sensor APIs of Windows Phone, like the accelerometer, compass, gyroscope and combined motion API (partly restricted by what the device hardware actually supports)
- Raw touch input from the touch panel, including multiple simultaneous touch points
- All built-in high-level touch gestures (tap, double-tap, hold, flick, free/horizontal/vertical drag, pinch)
- A custom drag gesture mode to simplify dragging
- Text input using the built-in SIP keyboard.
- A server-side .NET 4 assembly that listens for incoming client broadcasts or direct connections and is able to both send control and configuration commands to a connected phone as well as receive data that is acquired on the phone.
- A shared assembly for the used data types, interfaces and more.
- A reference Windows Phone client application: this application makes use of the Windows Phone assemblies of the core library. The UI allows connecting to a remote server and provides features to perform user input for all above listed gestures, raw touch input, sensors and text. The intended use is for developers that do not want or require to create a custom phone application, and the ultimate goal is to publish this app to the Marketplace for public use, once it has reached a mature state.
- Sample desktop applications: The project includes a few sample applications to demonstrate the use of the library. In particular, the following applications are included:
- Cube: Demonstrates how to use sensors (the accelerometer), gestures and raw touch input to control content in an application. The rotation of a 3D cube is controlled using the accelerometer, zoom is handled using pinch gestures, and translations are performed by raw touch input. A good starting point for developers of .NET applications to learn using the library quickly.
- ConsoleApplication: a simple command line server that allows sending a small subset of the available commands to the phone, and to display data received from the phone. Handy for quick tests and to learn the basics, can easily extended for your own needs.
- PCController: the most sophisticated sample and the original motivation to start developing this library. This sample converts the input received from the phone into operating system commands to manipulate the mouse cursor. It effectively lets you control your desktop computer using gestures, raw touch, accelerometer and text input, including features like: left, right and double mouse clicks, moving the cursor in relative and absolute positioning mode, drag and drop, two-finger scrolling/mouse wheel emulation. This sample is not meant as a simple demonstration or to easily learn the use of the library, but a real-world example for a more advanced use.
- LogService: A supporting helper application for troubleshooting and diagnosis.
And this wouldn't be a Coding4Fun blog post if we didn't at least take a peek at the code...
Note: If you want to actually play with, look at the Source in Visual Studio you'll need the Windows Phone SDK 7.1(5) and the Portable Library Tools installed. If you get a Project cannot be loaded/Project Type Not Supported error it's likely you're missing one of these (I was missing the Portable Library Tools... doh!)
A couple other notes on playing with the Source in Visual Studio. A) Nuget is your friend. I ran into a number of references that were not on my machine. To fix them I just used "Manage Nuget Packages" and added them as needed (BYW Your Last About/Options Dialog can be found searching for "YLAD" and "YLOD"). B) If you run into a build error of "cannot find nuget.exe" grab the download from here and put the exe in the "PAARC_0.9_src\.nuget" folder (or edit all the Projects and remove the build target... for me just grabbing the exe was easier).
For more details on compiling the code, check out Compile the Code.
Here's a snap of the v0.9 Solution;
As you can see there's a number of samples and such for you to check out play with.
Here's the app and sample running on my notebook.
- If you're having problems, you'll want to check out Troubleshooting problems
- And finally if you're wondering how you can use this in your projects, Use the library in your own projects
- Also if you're running into a network error when using the Emulator, check out http://vimeo.com/31514282 @ 24:06 (This helped me get this sample running...)
If you've every thought about how cool it would be to use your Windows Phone 7 device as an Über-remote control, then this project and library is just waiting for you to go check out...