So why am I highlighting it here? But it looks like a awesome resource that you can code against, build awesome app's using it's data, and it's all open, as in open, open, open...

Digital Public Library of America (dp.la)

The Digital Public Library of America brings together the riches of America’s libraries, archives, and museums, and makes them freely available to the world. It strives to contain the full breadth of human expression, from the written word, to works of art and culture, to records of America’s heritage, to the efforts and data of science. The DPLA aims to expand this crucial realm of openly available materials, and make those riches more easily discovered and more widely usable and used, through its three main elements:

1. A portal that delivers students, teachers, scholars, and the public to incredible resources, wherever they may be in America. Far more than a search engine, the portal provides innovative ways to search and scan through the united collection of millions of items, including by timeline, map, format, and topic.

2. A platform that enables new and transformative uses of our digitized cultural heritage. With an application programming interface (API) and maximally open data, the DPLA can be used by software developers, researchers, and others to create novel environments for learning, tools for discovery, and engaging apps.

3. An advocate for a strong public option in the twenty-first century. For most of American history, the ability to access materials for free through public libraries has been a central part of our culture, producing generations of avid readers and a knowledgeable, engaged citizenry. The DPLA will work, along with like-minded organizations and individuals, to ensure that this critical, open intellectual landscape remains vibrant and broad in the face of increasingly restrictive digital options. The DPLA will seek to multiply openly accessible materials to strengthen the public option that libraries represent in their communities.

Open platform, open data and open API...

The platform (so you can see how the built it all, which is kind of cool)

The data;

The API...

Welcome!

Welcome to the DPLA API Codex! This is where you can find authoritative documentation for the DPLA API and resources you can use to make the most of it. We’re happy to have you. (Wondering why we built an API in the first place?  Check out our philosophy.)

What now?

• If you’ve never played (or worked) with an API before, head on over to API Basics.  We’ll get you started accessing some of our data.
• If you’re already familiar with APIs in general and you just want to know how to get at our data, skip the basics and dive right into requests (Make sure you get an API key first.)
• Once you’ve got some shiny new JSON-LD, how can you tell what it all means?  In responses, we explore the structure of objects and define the fields within those objects that the API will return.
• Ready to start building something against our API, but want to know how we operate? We’ve got you. Check out our policies.
• Uh oh.  You’re lost. We’re lost. Everything’s broken. I’m confused.  Time to check out our troubleshooting and FAQ page!
• Confused by the terminology we’re using? Check out the glossary of terms.
• Did we miss/break/overlook/forget something? Tell us.

The essentials

Need to start right now? Here’s a cheat sheet.

• http://api.dp.la/v2 is the base URL of the DPLA API.
• items and collections are the two resource types you can request.
• In order to pass requests to the API, you’ll need to request an API key first. You also need to append &api_key=$YOUR_KEY (where $YOUR_KEY is your 32-character key) to any requests you make.
• All records are returned as structured, wonderful, JSON-LD objects.   And we have reasons for that.
• We’ve got a boatload of fields you can query, and a whole bunch of ways you can query them.
• Dive head first into a bucket full of kitten metadata: http://api.dp.la/v2/items?q=kittens&api_key=. (Copy and paste the URL into your browser and add your API key on the end.)

...

API Basics

...

A request is a URL sent to the web server over HTTP with the expectation of getting resource items back in the form of human-readable text or data. The URL supplies the web server with everything it needs to create and return a correct response. This is called a RESTful approach to API design and is employed by the DPLA API.

Who should use this?

This API is intended for use by large organizations and lone individuals alike, and our audience’s level of technical sophistication is deliberately quite varied.

How do I use it?

All you need to do to use the DPLA API is:

1. Request an API key from DPLA.
2. Fire up your web browser and enter your request into the URL bar.
3. Read the response.

The DPLA API offers metadata (and meta-metadata) on two types of resources: items and collections. In short, items represent single physical objects indexed by a DPLA data provider, and collections are logical groupings of items. For simplicity’s sake, let’s try a simple search for items—i.e., a full-text search of all of the fields associated with items for a single term.

Let’s say you really want to know more about weasels. Compose your search query by taking the URL of the API and append your request to it like so:

http://api.dp.la/v2/items?q=weasels&api_key=

http://api.dp.la/v2 specifies the API and the correct version; items indicates the type of resource you’re requesting; ?q= begins your request, and the unadorned, unqualified search term weasels will be interpreted as a request for a full-text search.

Remember: You must enter your 32-character API key after the &api_key= parameter in included in every example request, and in general.

Now let’s look at typical results (to simplify things, we’re only reproducing the first result here):

...

Open, open, open... Now it's up to you to build some cool apps (hopefully Windows apps . If you do, let us know...

 For more information, please see the additional topic here:

• Make your site touch-ready

See also related videos here:

• Web Standards: How to use XMLHttpRequest

• [2:10] MSDN Magazine Windows 8 Special
• [2:57] NuGet for Windows Store App Developers (Howard Dierking)
• [3:44] Inside the Kinect for Windows SDK Update with Peter Zatloukal and Bob Heddle (Peter Zatloukal, Bob Heddle), Kinect for Windows releases SDK update and launches in China
• [5:09] Web Platform Docs Debuts with Developer Resources (Eliot Graff)
• [6:13] Anders & Lars: TypeScript, JavaScript, and Dart (Anders Hejlsberg, Lars Bak)
• [7:09] Channel 9 Highlight: DevNuggets [Returns] with Leverage the maps app in your Windows Store (G. Andrew Duthie)
• [8:00] The essential guide to testing your Windows 8 game (Tom Rouse)
• [8:38] HTML5 Game Starter Kit for Windows 8 (Petri Wilhelmsen)
• [9:41] Large Datasets in Windows 8: Part 2 – Syncing Datasets to SQLite, Part 1, (Matt Hudson)
• [10:39] Improving Your App's Performance with PerfView (Brandon Bray)

Picks of the Week!

• Brian's Pick of the Week:[11:30] In-Place: Interacting with Large Displays (Rob Knies)
• Dan's Pick of the Week:[12:26] Freehand 3D Computer Interaction Without Gloves

Microsoft Principal Architect Evangelist Brian Loesgen speaks with Bryan Hall, co-founder of San Diego, CA-based startup Embarke. Bryan explains why they found in using the Azure platform compelling. Bryan shows the interfaces in the Embarke API, and shares best practices they learned along the way.

About Embarke

Embarke is the distribution center for all of your user's social conversations.  Your customers can initiate conversations with their contacts, regardless of which networks they're connected through. For example: One of your users could start a 4-way conversation with someone on Facebook, another in Email, and another on Twitter. None of those people would have to go anywhere special to be a part of that conversation.

Kinect App Site

The Single-Page Kinect App Site template is an easily configured, simple template with all the elements in place, intended as a quick-start for any Kinect developer to promote their app.

View the default site here: Kinect App Site Demo

Idea for this project came when I want to publish some Kinect app of mine. Recently have seen http://wp7appsite.codeplex.com/ and I decided to make something similar but for Kinect enabled applications.

The HTML is fully commented to make setup and customization as quick and easy as possible.

The site uses:

• Twitter Bootstrap
• Tweet!

Project Information URL: http://kinectappsite.codeplex.com

Project Download URL: http://kinectappsite.codeplex.com/releases

Project Source URL: http://kinectappsite.codeplex.com/SourceControl/list/changesets

Contact Information:

• Blog: http://tomek.kownet.info/
• Twitter: @tomkowalczyk

Basicamente, quando se inicia um novo projeto com WebMatrix, do início ao fim, três elementos básicos devem estar presentes: ideia, personalização/desenvolvimento e publicação. Este artigo trata do primeiro elemento, isto é, como o WebMatrix pode com suas características funcionais atender "a ideia" do usuário.

Tipos de projetos disponíveis

Uma ferramenta de desenvolvimento deve atender seus usuários em diversos aspectos. Nesse sentido, em termos de projetos, o WebMatrix disponibiliza basicamente dois modelos: templates básicos temáticos (implementados com sintaxe Razor) e galeria de aplicativos (Web App Gallery). A seguir, apresentamos cada um deles individualmente com suas características associadas.

Templates: são 6 (seis) opções para criação de aplicações baseadas em modelos já prontos. Esta opção torna-se interessante principalmente se o usuário encontra-se em nível junior ou é novo no universo de desenvolvimento para web e está utilizando a ferramenta para aprender como construir aplicações utilizando a sintaxe Razor utilizando banco de dados e helpers. Outro aspecto importante em relação aos templates é que, em algum momento, determinado template pode atender de fato as necessidades de determinado, exigindo do responsável pelo desenvolvimento apenas personalizações. Os templates disponíveis atualmente são:

• Empty Site: projeto vazio. Neste modelo, tudo deve ser criado pelo usuário. De forma geral, esta opção atende a desenvolvedores que pretendem criar soluções esepcíficas, não atendidas pelos modelos oferecidos pela ferramenta.
• Starter Site: projeto que traz um site básico. O template é todo construído com sintaxe Razor, HTML 5, CSS 3, Helpers (autenticação de usuários, etc.).
• Bakery: projeto que traz um modelo de site para padaria. De igual forma, o modelo é implementado em Razor e webstandards.
• Photo Gallery: projeto para galeria de fotos. Todas as características presentes nos modelos anteriores podem ser encontradas aqui.
• Calendar: projeto que implementa um controle de calendário.
• Personal Site: modelo que sugere a estrutura de um site pessoal, trazendo helpers para integração com twitter, facebook, etc.

A Figura 1 apresenta a janela exibida pelo WebMatrix após clicar na opção "Template" da página inicial.

Figura 1. A tela com as opções de templates do WebMatrix

Web App Gallery: como o próprio nome sugere, a Web App Gallery (WAG) é um repositório de aplicativos web terceiros com uma característica fundamental: todos são open-source, isto é, podem ser utilizados livremente sem custo adicional. As aplicações disponíveis na WAG atendem a diferentes demandas: blogs, lojas virtuais, wikis, fóruns, CMS, dentre outros. A grande vantagem desta característica com WebMatrix é o fato de que esta última oferece todos os recursos necessários para executar a personalização, publicação e demais operações com as web apps provenientes da WAG, isto é, tudo o que o usuário precisa em um único lugar.

A Figura 2 apresenta a tela específica de aplicações oferecidas através da Web App Gallery.

Figura 2. Web App Gallery

Dentre as aplicações presentes na WAG, destacam-se as já populares:

• Wordpress: indiscutivelmente a mais popular ferramenta para gerenciamento de conteúdos e mídias no formato de blog. Desenvolvida em linguagem PHP com banco de dados MySQL.
• Joomla!: CMS para gerenciamento de conteúdos web em formato de portal. Desenvolvida em linguagem PHP com banco de dados MySQL.
• nopCommerce: uma das lojas virtuais mais completas e utilizadas no mundo.
• Dentre outras.

Conclusões

Conhecer os tipos de projetos disponibilizados pelo WebMatrix é importante pois passa a ideia de como a ferramenta pretende atender as demandas de seus usuários.

O WebMatrix é uma ferramenta interessante para você se seu objetivo é iniciar de forma correta (entenda-se utilizando conceitos e tecnologias corretas para este cenário) no universo web ou se é desenvolvedor PHP em qualquer nível e procura uma opção interessante em termos de IDE para construir suas aplicações.

Ferramentas como Visual Studio, Visual Studio Express e outras, falam para públicos distintos, sem gerar atrito com WebMatrix, conforme mencionado no primeiro módulo deste treinamento.

The holidays are just about to end, but there's still time to boost your knowledge or learn something totally new.

Durint these 2 weeks we'll be giving the fourth and final topic of this summer issue their place in the sun, namely the Web and the web browser. Click through and you'll find all the best resources, presented based on the amount of time you want to spend on it.

Explore and dive into Web solutions

What is the most used application on a computer nowadays? Most likely it will be the web browser.  The Web and the web browser have quickly become a user's natural habitat, a familiar environment, and with Web 2.0, a place to work, interact and play.

You as an independent software vendor (ISV) can take advantage of this in many ways.  The Web provides you a marketing platform to reach out to millions of potential customers for your application.  In that case, you will want to fully show the beauty of the web on your website to stand out and attract these customers.

On the other hand, you can use the Web as your application platform to build web applications or to extend your desktop application with existing web services (e.g. mapping unctionality, social media integration, etc.) to augment your core application usiness value.

Have  look at our website and discover a handpicked choice of articles, white papers nd videos explaining what Microsoft web technologies can bring to you as an ISV.

Speaker : Nick Trogh

Other interesting Summer Campaigns :

-For IT Professionals : Virtualisation(Part 1), Desktop Deployment,Virtualisation (Part 2), Best of TechDays 2011

-For Architects : Cloud Power, the Windows AzurePlatform, Cloud Patterns and Practices, Windows Azure Platform Interoperability

-For Developers : Windows Phone 7 Mango, Visual Studio 2010 ALM, HTML5, The Windows Azure Platform

Speaker: Ronnie Saurenmann

Presentation (PPTX)

Demos (ZIP)

Speaker: Ken Casada

Presentation (PPTX)

Speaker: Ronnie Saurenmann

Presentation (PPTX)

Demos (ZIP)

Surfing the web is by far the number 1 thing people do on their PCs.  From Facebook, Flickr and Foursquare, to 20min.ch, NZZ Online and Swiss.com, people are spending more and more of their time in a web browser.  And yet, today’s web experience remains flat and contained compared to what’s possible in native apps.  With HTML5 and hardware acceleration, that’s about to change. 

Speaker: Ari Bixhorn

Presentation (PPTX)

Speaker: Sascha P. Corti

Presenation (PPTX)

Opening Keynote: What's new in the Microsoft Web Platform

by Scott Hanselman

“January Web Release” resources

• Scot Hanselman’s blog
  
  
• ASP.NET MVC 3 download
  
• ASP.NET MVC 3: get started
  
• NuGet
  
• WebMatrix: download
  
• WebMatrix: getting started videos
  
• Install Visual Web Developer Express, SQL Express, SQL Management Studio, and ASP.NET MVC 3 with Razor
  
• IIS Express 7.5
  
• Entity Framework Code First
  

Whilst there are now a number of Windows 7 theme packs with collections of the Best of Bing images,  new this month is a Bing desktop gadget which combines the daily image with a search bar – so it’s even easier to search the web, and you don’t miss out on the image of the day.

There’s a great write up of the gadget over at My Microsoft Life, otherwise jump straight to the gadget over at the Windows 7 Desktop Gadget showcase. 

Quark Product Manager Akshay Mahajan talks with Microsoft ISV Architect Evangelist Bruce Kyle about the new product. In the demo, Akshay shows how the product takes the same content and publishes to a white paper, Website, and to a mobile phone. Aksay describes the lessons learned and advises software developers on what they can do to help make their next SharePoint project successful.

About Quark Publishing System

Quark Publishing System (QPS) is the foundation for Quark Dynamic Publishing Solution, which helps you improve productivity, cut costs, and successfully publish to multiple channels by adopting a collaborative and automated publishing process.

Learn More About SharePoint Development

Get started with development on SharePoint 2010 Developer Training Course on Channel 9.

For the latest news for developers, see the USISVDE team blog.

Other ISV Videos

For videos on developing for Windows Phone 7, see:

• Teenager Makes Money Writing Windows Phone 7 Apps with AdCenter
• Talking  with Karmak.com about business applications on Windows Phone 7
• BinaryDads talk about building applications for Windows Phone 7
• Game Design and Development in Silverlight for Windows Phone 7
• Windows Phone 7 Game Begins Life in WPF -- Now a Silverlight,  Ad Supported App
• XNA Game Development on Windows Phone 7 in One Sitting

For videos on Windows Azure Platform, see:

• Azure Email-Enables Lists, Low-Cost Storage for SharePoint
• Crowd-Sourcing Public Sector App for Windows Phone, Azure
• Food Buster Game Achieves Scalability with Windows Azure
• BI Solutions Join On-Premises To Windows Azure Using Star Analytics Command Center
• StorSimple Integrates On-Premises, Cloud Storage with Windows Azure
• NewsGator Moves 3 Million Blog Posts Per Day on Azure

For other videos about independent software vendors (ISVs):

• Quark Extends SharePoint for Dynamic Publishing
• ProModel Adds Simulation, Visualization to Microsoft Project
• Slingbox Maker Sling Media Describes Cross-Platform Commitment to Silverlight
• Thumb-Driven Workflow on Windows 7 Slates from Blue Dot Solutions

For more news, tips, and links to developer training, see the US ISV Community blog.

WebMatrix was recently released and helps make creating websites super easy.  WebMatrix is everything you need to build Web sites using Windows. It includes IIS Developer Express (a development Web server), ASP.NET (a Web framework), and SQL Server Compact (an embedded database). It streamlines Web site development and makes it easy to start Web sites from popular open-source apps. The skills and code you develop with WebMatrix transition seamlessly to Visual Studio and SQL Server.

All you do is create the application, click publish, and a fully functional site is up and working.  How it runs on your local computer is how it will run on the website!

What is even cooler is it connects you to a world of popular and free open-source Web applications, including DotNetNuke, Umbraco, WordPress, Joomla!, and more!

If you want to learn on how to get started, head over to http://www.microsoft.com/web/webmatrix/learn/.

WebMatrix helps developers learn standards-based Web development, explains the text on its new website, using technologies like HTML, CSS and JavaScript. And it even connects you to open source apps like DotNetNuke, Umbraco, WordPress, Joomla!, and more, all available from the built-in gallery. (You’ll probably remember seeing these apps in the Windows Web App Gallery, which was also launched over the past year).

The interface WebMatrix displays when you launch it the first time is a simple wizard-based welcome screen that lets you pick whether you want to create a site from the Web Gallery, a template (several are included) or a “site from folder.”  This may be too simple for advanced developers, but that’s the point. WebMatrix is intended for “web agencies, small development teams within SMBs, hobbyists, and students,” writes Microsoft’s Laurence Moroney on The Officical Microsoft Web Platform Team Blog. Moroney also details how to get started with the tool and do other nifty things, like displaying a Twitter feed, for example.

To download WebMatrix now, you can do so from here. (But note the “beta” label, OK?)

• Starting WebMatrix (13:02)
• Downloading & Running Umbraco CMS (13:40 - 21:05)
• SEO and Broken Links report (21:05 - 23:08)
• How to publish a Web site using WebMatrix (23:08 - 24:20 
• Building basic Web sites using the new Razor syntax in VBHTML/CSHTML files, including editing HTML, adding headers and footers, database access, HTML forms and more (23:08 - 44:28)
• Content Helpers - Content Helpers are wrappers for common tasks and 3rd party services, including database access, adding video to a site, captcha, photo processing, Twitter,  and more (44:28 - 52:40)
• Smarty Route - How routing works without explicit configuration and without file extensions (52:40 - 54:44)

Joomla!’s Sam Moffatt shares, “…it's a big deal to have a major company contributing to the codebase. And yes, the fact that it's Microsoft contributing to a GPL project is big news.” But he says it’s also a natural progression, considering all the work that has been going on for some time now between the two companies.

Already, Joomla! has worked with Microsoft to get their package up on the Web Platform Installer where it now has 32,000 downloads, for example. They’ve also worked with Microsoft on optimizing Joomla! for better IIS compatibility, something that means “opening a whole new territory of possibilities including in many enterprises,” says Moffatt. (Previously, Joomla! was more Apache-focused.)

The Microsoft-contributed code will be made available in Joomla! 1.6, as noted above, but if you want to try it now with 1.5, there’s a workaround for that here.

MSDN Radio is a weekly Developer talk-show that helps answer your questions about the latest Microsoft news, solutions, and technologies. We dive into the challenges of deciphering today’s technology stack. Attendees can dial-in during the show to talk with the hosts on the air. To register for upcoming shows visit www.MSDNRadio.com.

Host and Guest: Mike Benkovich, Senior Developer Evangelist, Microsoft Corporation, and Joe Stagner, Senior Program Manager, Microsoft Corporation

Mike Benkovich delivers technical presentations around the U.S. as a developer tools evangelist on the MSDN team at Microsoft. He has worked in a variety of professional roles, including architect, project manager, developer, and technical writer.

Joe Stagner works at Microsoft with the Developer Tools and Platforms team and as spent almost ten years at Microsoft working with the developer community. In addition to Microsoft’s ASP.NET technology Joe focuses on integrating .NET with third part technology like PHP, Java, etc.

When the SLARToolkit was released, I finally had time to implement real time face detection using the Silverlight 4 webcam API.

This article will describe the simple facial recognition method that searches for a certain sized skin color region in a webcam snapshot. This technique is not as perfect as a professional computer vision library like OpenCV and the Haar-like features they use, but it runs in real time and works for most webcam scenarios.

Demo Application

You need a webcam and at least the Silverlight 4 runtime installed to run the sample. At the moment the release candidate is available for Windows and Mac. The facial region should be illuminated well and the background should be in skin color contrast to get the best results.

Open the sample

How To Use

You can start and stop the webcam with the Button, or you can load an image from disk with the Button. Use the ComboBox to change the demo mode from “Highlight” to “Image.” “Highlight” simply draws a red ellipse around the detected face and “Image” overlays the facial region with an image. The ape's head is the default image (Figure 11), but it's possible to apply a different picture by entering its URI in the TextBox. You can use the Slider controls to alter the skin color thresholds in the YCbCr color space (see Step 2: Filtering the Skin Color). Save the result of the face detection (including the overlaid image) to disk with the Button.

When you click the Button for the first time, you'll need to give permission for the capturing. This application uses the default Silverlight capture devices. You can specify default video and audio devices with the Silverlight Configuration. Just press the right mouse button over the application, click "Silverlight" in the context menu and select the "Webcam / Mic" tab to set them.

How It Works

The idea is to take snapshots from the webcam, filter the skin color out using color thresholds, apply filters to reduce the noise, find the facial region—and then do fun things with this information. In the following sections I'll show you how to do this using a simple skin color segmentation approach. For clarity, I've reduced code listings to only what's relevant here. The complete source code is available at CodePlex and licensed under the Ms-PL.

Figure 1: The six steps

Step 1: Capturing the Webcam

The Silverlight 4 webcam API is easy to use. The CaptureSource class provides the webcam stream and you can use it as the source of a VideoBrush, which in turn fills a rectangle to show the video feed from the webcam. We'll use the default video capture device, but you can also iterate CaptureDeviceConfiguration class to get all the capture devices on the system. The user can specify the default video and audio devices with the Silverlight configuration; he or she only has to press the right mouse button over the Silverlight application, click "Silverlight" in the context menu and select the "Webcam / Mic" tab to set them.

The initialization of the webcam:

C#

// Create capturesource and use the default video capture device 
captureSource = new CaptureSource();
captureSource.VideoCaptureDevice = CaptureDeviceConfiguration.GetDefaultVideoCaptureDevice();
captureSource.CaptureImageCompleted += new EventHandler<CaptureImageCompletedEventArgs>(captureSource_CaptureImageCompleted);

// Start capturing
if (captureSource.State != CaptureState.Started)
{
   // Create video brush and fill the WebcamVideo rectangle with it
   var vidBrush = new VideoBrush();
   vidBrush.Stretch = Stretch.Uniform;
   vidBrush.SetSource(captureSource);
   WebcamVideo.Fill = vidBrush;

   // Ask user for permission and start the capturing
   if (CaptureDeviceConfiguration.RequestDeviceAccess())
   {
      captureSource.Start();
   }
}

Now that we have the webcam up and running, we need to take snapshots from the video stream to feed the face detection. You can use a custom VideoSink implementation for this task, but the CaptureSource also provides the built-in CaptureImageAsync method, which is easier to use. The performance is not much worse than a custom VideoSink.

C#

// Part of the RunUpdate method of the MainPage class
var dispatcherTimer = new DispatcherTimer();
dispatcherTimer.Interval = new TimeSpan(0, 0, 0, 0, 40); // 25 fps
dispatcherTimer.Tick += (s, e) =>
{
   // Process camera snapshot if started
   if (captureSource.State == CaptureState.Started)
   {
      // CaptureImageAsync fires the captureSource_CaptureImageCompleted event
      captureSource.CaptureImageAsync();
   }
};
dispatcherTimer.Start();

// captureSource_CaptureImageCompleted event handler of the MainPage class
private void captureSource_CaptureImageCompleted(object sender, CaptureImageCompletedEventArgs e)
{
   // Process camera snapshot
   Process(e.Result);
}

We use a DispatcherTimer here to call the CaptureImageAsync method. Every time the capturing is done, the CaptureImageAsync method fires a CaptureImageCompleted event, and the EventArgs provides a WriteableBitmap that contains a completed snapshot. The DispatcherTimer is initialized to perform this capturing task every 40 ms.

Figure 2: The result of the first step is a webcam stream

Step 2: Filtering the Skin Color

After grabbing an image from the webcam, you need to filter the skin color; this lets you find the face in a following step. The WriteableBitmap provided through the CaptureImageCompleted event uses the RGB color space to represent the pixels and is actually just a 32 bit integer array that stores the alpha, red, green and blue (ARGB) byte components for all the pixels.

The RGB color space has some major drawbacks when it comes to filtering certain color ranges. For example: we'd like to filter the skin color with a robust method that's not too much affected by the brightness of the image, but doing this in the RGB color space would imply a larger color space volume—so you can't use simple value thresholding.

Fortunately, there are other color spaces available that don't suffer from such problems. The HSV color space, for example, defines the color in its three components as Hue, Saturation and Value (Brightness), where the actual color (Hue) is represented as a circle from 0° to 360° and the brightness is the height of a cylinder.

Since the relevant skin color Hue ranges from 0° - 60° and 300° - 360° (which involves extra calculations) and the RGB to HSV conversion is more computationally expensive than other color space conversions, we use the YCbCr color space for the skin color filtering. YCbCr stores the brightness in the Y component and the chroma (color) information in the Cb component as blue-difference and in Cr component as red-difference. The RGB-YCbCr conversion can be done with simple addition and multiplication operations. The Y component ranges from 0 to 1, Cb and Cr from -0.5 to 0.5.

I tested several photos of different colored persons, and found that the following values cover most skin color ranges (except Martian or Avatar's Navis, perhaps).

Y = [0, 1] Cb = [-0.15, 0.05] Cr = [0.05, 0.20]

The sample application also provides Sliders to change these thresholds dynamically.

Figure 3 illustrates the YCbCr color range that's used for thresholding. The Y is constant at 0.5; the Cb lower threshold is left and the upper threshold is right; the Cr lower threshold is at the top and the upper threshold is at the bottom.

Figure 3: YCbCr color range. Y=0.5 Cb=[-0.15, 0.05] Cr=[0.05, 0.20]

This the source code used to generate the bitmap shown in Figure 3:

C#

// Visualize method of the HistogramVisualizer class
public void Visualize(WriteableBitmap surface)
{
   var w = surface.PixelWidth;
   var h = surface.PixelHeight;
   var pixels = surface.Pixels;
   var min = this.Min;
   var max = this.Max;
   int i;
   float xf, yf, cb, cr;

   // Use constant Y
   float v = min.Y + (max.Y - min.Y) * YFactor;

   // Interpolate between min and max and set pixel
   for (int y = 0; y < h; y++)
   {
      for (int x = 0; x < w; x++)
      {
         i = y * w + x;
         xf = (float)x / w;
         yf = (float)y / h;
         cb = min.Cb + (max.Cb - min.Cb) * xf;
         cr = min.Cr + (max.Cr - min.Cr) * yf;
         pixels[y * w + x] = new YCbCrColor(v, cb, cr).ToArgbColori();
      }
   }
}

Since the WriteableBitmap camera snapshot uses the RGB color space, we have to convert from RGB to YCbCr before we can apply the thresholds:

C#

// FromArgbColori method of the YCbCrColor class
public static YCbCrColor FromArgbColori(int color)
{
   // Extract the RGB components from the int color and convert to the range [0, 1]
   const float f = 1f / 255f;
   var r = (byte)(color >> 16) * f;
   var g = (byte)(color >> 8) * f;
   var b = (byte)(color) * f;

   // Create new YCbCr color from RGB color
   var y = 0.299f * r + 0.587f * g + 0.114f * b;
   var cb = -0.168736f * r + -0.331264f * g + 0.5f * b;
   var cr = 0.5f * r + -0.418688f * g + -0.081312f * b;

   return new YCbCrColor(y, cb, cr);
}

During the thresholding process, each pixel of the WriteableBitmap is converted from RGB to YCbCr and tested against the defined upper and lower threshold:

C#

// Process method of the ColorRangeFilter class
public WriteableBitmap Process(WriteableBitmap snapshot)
{
   var p = snapshot.Pixels;
   var result = new WriteableBitmap(snapshot.PixelWidth, snapshot.PixelHeight);
   var rp = result.Pixels;

   // Threshold every pixel
   for (int i = 0; i < p.Length; i++)
   {
      var ycbcr = YCbCrColor.FromArgbColori(p[i]);
      if (ycbcr.Y >= LowerThreshold.Y && ycbcr.Y <= UpperThreshold.Y
       && ycbcr.Cb >= LowerThreshold.Cb && ycbcr.Cb <= UpperThreshold.Cb
       && ycbcr.Cr >= LowerThreshold.Cr && ycbcr.Cr <= UpperThreshold.Cr)
      {
         rp[i] = 0xFFFFFF;
      }
   }

   return result;
}

After the thresholding is performed and the pixel falls in the skin color range, a white pixel is written to the resulting WriteableBitmap. This results in a binary black & white image that masks skin color and is then used in the next step.

Figure 4: The result of the second step is a skin color filtered binary image

Step 3: Reducing Noise with Erosion

The skin color-filtered image contains tiny pixel areas from the background that are mostly caused by image noise. Image noise can prevent a clear segmentation in later steps, so we have to remove it.

Erosion is a common image filter you can use for this task. The morphological Erosion operator reduces the boundaries of the colored regions so that very small regions are removed and only larger regions remain. It works by iterating over all pixels of the WriteableBitmap and testing to see if the neighboring pixels of the current pixel c are empty (zero) or not. If one of the neighboring pixels is empty, the current pixel c has to be a boundary pixel and thus must be removed (set to black). How many neighboring pixels are tested depends on the concrete implementation. This set of test coordinate points is usually called a kernel in image processing.

Since the previous step produced a black & white image, we can use a simple binary Erosion operator here. It turns out that a 5 x 5 kernel is optimal for this use case. The for-loop that would usually be used to implement a generic Erosion operator is unrolled into 5 * 5 = 25 neighbor pixel tests for better performance.

C#

// Process method of the Erode5x5Filter class
public WriteableBitmap Process(WriteableBitmap input)
{
   var p = input.Pixels;
   var w = input.PixelWidth;
   var h = input.PixelHeight;
   var result = new WriteableBitmap(w, h);
   var rp = result.Pixels;
   var empty = CompareEmptyColor; // = 0
   int c, cm;
   int i = 0;

   // Erode every pixel
   for (int y = 0; y < h; y++)
   {
      for (int x = 0; x < w; x++, i++)
      {
         // Middle pixel
         cm = p[y * w + x];
         if (cm == empty) { continue; }

         // Row 0
         // Left pixel
         if (x - 2 > 0 && y - 2 > 0)
         {
            c = p[(y - 2) * w + (x - 2)];
            if (c == empty) { continue; }
         }
         // Middle left pixel
         if (x - 1 > 0 && y - 2 > 0)
         {
            c = p[(y - 2) * w + (x - 1)];
            if (c == empty) { continue; }
         }
         if (y - 2 > 0)
         {
            c = p[(y - 2) * w + x];
            if (c == empty) { continue; }
         }
         if (x + 1 < w && y - 2 > 0)
         {
            c = p[(y - 2) * w + (x + 1)];
            if (c == empty) { continue; }
         }
         if (x + 2 < w && y - 2 > 0)
         {
            c = p[(y - 2) * w + (x + 2)];
            if (c == empty) { continue; }
         }

         // Row 1
         // Left pixel
         if (x - 2 > 0 && y - 1 > 0)
         {
            c = p[(y - 1) * w + (x - 2)];
            if (c == empty) { continue; }
         }


         // ... 
         // ... Process the rest of the 24 neighboring pixels
         // ...

         
         // If all neighboring pixels are processed 
         // it's clear that the current pixel is not a boundary pixel.
         rp[i] = cm;
      }
   }
   
   return result;
}

Figure 5: The result of the third step is a reduced-noise image

Step 4: Expanding with Dilation

In addition to removing noise, Erosion shrinks the face region. Unfortunately this causes some holes— especially in the area around the eyes—to arise or enlarge, which can lead to faulty color segmentation. That's where the other fundamental morphology operator, Dilation, comes into play

The Dilation enlarges the boundaries and expands an area by iterating over all pixels of the WriteableBitmap. This time, though, it's checked if one of the neighboring pixels of the current pixel c is not empty (white). If only one of the neighboring pixels is not empty, the current pixel c will be set to white.

The best results are achieved when the Dilation with a 5 x 5 kernel is applied three times.

C#

// Process method of the Dilate5x5Filter class which is applied 3 times
public WriteableBitmap Process(WriteableBitmap input)
{
   var p = input.Pixels;
   var w = input.PixelWidth;
   var h = input.PixelHeight;
   var result = new WriteableBitmap(w, h);
   var rp = result.Pixels;
   var r = this.ResultColor;
   var empty = CompareEmptyColor; // = 0
   int c, cm;
   int i = 0;

   // Dilate every pixel
   for (int y = 0; y < h; y++)
   {
      for (int x = 0; x < w; x++, i++)
      {
         // Middle pixel
         cm = p[y * w + x];
         // Is the pixel empty? 
         // If not we set the result and continue with the next pixel
         if (cm != empty) { rp[i] = r; continue; }

         // Row 0
         // Left pixel
         if (x - 2 > 0 && y - 2 > 0)
         {
            c = p[(y - 2) * w + (x - 2)];
            // If only one of the neighboring pixels is not empty,
            // we set the result and continue with the next pixel.
            if (c != empty) { rp[i] = r; continue; }
         }
         // Middle left pixel
         if (x - 1 > 0 && y - 2 > 0)
         {
            c = p[(y - 2) * w + (x - 1)];
            if (c != empty) { rp[i] = r; continue; }
         }
         if (y - 2 > 0)
         {
            c = p[(y - 2) * w + x];
            if (c != empty) { rp[i] = r; continue; }
         }
         if (x + 1 < w && y - 2 > 0)
         {
            c = p[(y - 2) * w + (x + 1)];
            if (c != empty) { rp[i] = r; continue; }
         }
         if (x + 2 < w && y - 2 > 0)
         {
            c = p[(y - 2) * w + (x + 2)];
            if (c != empty) { rp[i] = r; continue; }
         }

         // Row 1
         // Left pixel
         if (x - 2 > 0 && y - 1 > 0)
         {
            c = p[(y - 1) * w + (x - 2)];
            if (c != empty) { rp[i] = r; continue; }
         }


         // ... 
         // ... Process the rest of the 24 neighboring pixels
         // ...         
      }
   }
   
   return result;
}

Figure 6: The result of the fourth step after applying the Dilation three times

Step 5: Finding the face with histogram segmentation

The skin color-filtered, eroded and dilated image is a good starting point for image segmentation, the process of portioning an image into multiple sets of pixels (segments). Image segmentation is typically used to find the location of certain objects.

There are many different techniques available; a fast and simple method is histogram-based segmentation. An image histogram is a statistical representation of all the pixels present in the image. Most image editing tools have a color histogram functionality, which is commonly implemented as a graph that visualizes the amount of certain colors in an image. Figure 7 shows the color histogram of the sample image where the x-axis of the graph represents the brightness of each color component from 0 to 255, and the y-axis represents the number of pixels for each intensity.

Figure 7: Color histogram of the sample image

Since the skin color filtering produced a binary image, the histogram contains only two values for the number of black and white images and doesn‘t help us to find the skin segment. To fix this, we need the location of the white pixels maximum for the y-axis (rows) and x-axis (columns). Counting the white pixels separately for the rows and columns accomplishes this.

Figure 8 shows the row and column histogram for the webcam image where the blue color stands for row and the green for the column distribution of white pixels. The yellow lines highlight the maximum value for each.

Figure 8: Row and column histogram of the webcam image

C#

// FromWriteableBitmap method of the Histogram class
public static Histogram FromWriteabelBitmap(WriteableBitmap input)
{
   var p = input.Pixels;
   var w = input.PixelWidth;
   var h = input.PixelHeight;
   var histX = new int[w];
   var histY = new int[h];
   var empty = CompareEmptyColor; // = 0

   // Create row and column statistics / histogram
   for (int y = 0; y < h; y++)
   {
      for (int x = 0; x < w; x++)
      {
         if (p[y * w + x] != empty)
         {
            histX[x]++;
            histY[y]++;
         }
      }
   }

   return new Histogram(histX, histY);
}


// The constructor of the Histogram class that is used in the FromWriteableBitmap method
public Histogram(int[] histX, int[] histY)
{
   X = histX;
   Y = histY;

   // Find maximum value and index (coordinate) for x
   int ix = 0, iy = 0, mx = 0, my = 01;
   for (int i = 0; i < histX.Length; i++)
   {
      if (histX[i] > mx)
      {
         mx = histX[i];
         ix = i;
      }
   }

   // Find maximum value and index (coordinate) for y
   for (int i = 0; i < histY.Length; i++)
   {
      if (histY[i] > my)
      {
         my = histY[i];
         iy = i;
      }
   }

   // Keep results in member variables
   Max = new Vector(mx, my);
   MaxIndex = new Vector(ix, iy);
}

The code that draws the row and column histogram into the Dilation image shown in Figure 8 uses the WriteableBitmapEx library's DrawLine method:

C#

// Visualize method of the HistogramVisualizer class
public void Visualize(WriteableBitmap surface)
{
   var w = surface.PixelWidth;
   var h = surface.PixelHeight;
   var scale = this.Scale;
   var histogram = this.Histogram;
   var histX = histogram.X;
   var histY = histogram.Y;

   // Histogram X
   for (int x = 0; x < w; x++)
   {
      var hx = histX[x];
      if (hx != 0)
      {
         var norm = (int)(((float)hx / histogram.Max.X) * scale);
         surface.DrawLine(x, h - 1, x, h - norm, Colors.Green);
      }
   }
   // Draw max
   surface.DrawLine(histogram.MaxIndex.X, h - 1, histogram.MaxIndex.X, 0, Colors.Yellow);


   // Histogram Y
   for (int y = 0; y < h; y++)
   {
      var hy = histY[y];
      if (hy != 0)
      {
         var norm = (int)(((float)hy / histogram.Max.Y) * scale);
         surface.DrawLine(w - 1, y, w - norm, y, Colors.Blue);
      }
   }
   // Draw max
   surface.DrawLine(w - 1, histogram.MaxIndex.Y, 0, histogram.MaxIndex.Y, Colors.Yellow);
}

Figure 9: The result of the row and column histogram determination

Now that we know the x- and y-coordinates of the white pixels' start / end and where the maximum peak is, we can use this information to perform the actual histogram-based segmentation. The simple approach we'll use here finds initial seeds where the histogram value is above half of the histogram's maximum. In the next sub step, the algorithm starts at the found seeds and tries to detect the segment's boundaries where the histogram value is below a certain threshold. Usually the face region will be the largest segment in the result list, which is why the segments are sorted by size.

This approach is not perfect, but it's fast enough for real time and works for common webcam scenarios.

C#

// Process method of the HistogramMinMaxSegmentator class
public IEnumerable<Segment> Process(WriteableBitmap input)
{
   var hx = Histogram.X;
   var hy = Histogram.Y;
   var histUpperThreshold = Histogram.Max * 0.5f;

   // Find seeds for the segmentation:
   // All the x and y histogram indices where the value is above 
   // the half maximum and have a min distance 
const int step = 10;
   // x
   var ix = GetIndicesAboveThreshold(Histogram.MaxIndex.X, -step, hx, histUpperThreshold.X);
   ix.AddRange(GetIndicesAboveThreshold(Histogram.MaxIndex.X + step, step, hx, histUpperThreshold.X));
   // y
   var iy = GetIndicesAboveThreshold(Histogram.MaxIndex.Y, -step, hy, histUpperThreshold.Y);
   iy.AddRange(GetIndicesAboveThreshold(Histogram.MaxIndex.Y + step, step, hy, histUpperThreshold.Y));


   // Find the boundaries for the segments defined by the seeds
   var segments = new List<Segment>();
   foreach (var y0 in iy)
   {
      foreach (var x0 in ix)
      {
         var segment = new Segment(0, 0, 0, 0);
         segment.Min.X = GetIndexBelowThreshold(x0, -1, hx, ThresholdLuminance.X);
         segment.Max.X = GetIndexBelowThreshold(x0, 1, hx, ThresholdLuminance.X);
         segment.Min.Y = GetIndexBelowThreshold(y0, -1, hy, ThresholdLuminance.Y);
         segment.Max.Y = GetIndexBelowThreshold(y0, 1, hy, ThresholdLuminance.Y);
         segments.Add(segment);
      }
   }

   // Order by the largest segment
   return segments.OrderByDescending(s => s.DiagonalSq);
}

// GetIndicesAboveThreshold method of the HistogramMinMaxSegmentator class
private List<int> GetIndicesAboveThreshold(int start, int step, int[] hist, int threshold)
{
   var result = new List<int>();
   int hi;
   for (int i = start; i < hist.Length && i > 0; i += step)
   {
      hi = hist[i];
      if (hi > threshold)
      {
         result.Add(i);
      }
   }
   return result;
}

// GetIndexBelowThreshold method of the HistogramMinMaxSegmentator class
private int GetIndexBelowThreshold(int start, int step, int[] hist, int threshold)
{
   int result = start, hi;
   for (int i = start; i < hist.Length && i > 0; i += step)
   {
      hi = hist[i];
      result = i;
      if (hi < threshold)
      {
         break;
      }
   }
   return result;
}

Step 6: Overlaying the detected face

After we get information about the position of the face, we can finally do something useful and fun with this data.

Highlight facial region

The segment information calculated in the previous step contains the x- / y-coordinates of the segment's center and the width / height. It's easy to highlight the facial region with this data (Figure 10): We use an empty image that's laid over the webcam output and draw a red ellipse into the image's bitmap using the WriteableBitmapEx library's DrawEllipse method.

Figure 10: Highlighted facial region

C#

// Overlay method of the MainPage class
private void Overlay(IEnumerable<Segment> foundSegments, int w, int h)
{
   // Highlight the found segments with a red ellipse
   var result = new WriteableBitmap(w, h);
   foreach (var s in foundSegments)
   {
      // Uses the segment's center and half width, height
      var c = s.Center;
      result.DrawEllipseCentered(c.X, c.Y, s.Width >> 1, s.Height >> 1, Colors.Red);
   }
   ImgResult.Source = result;
}

Overlay facial region

Highlighting the face is nice (although not much fun), but we can also use the segment information to put an image over the face and move / scale it to the appropriate position / size. In the first version of the application I've used a photo of Chuck Norris, but I changed it (due to legal issues). Fortunately I found a funny snapshot of an orangutan that Ethan Hein took and released under the Creative Commons license. The ape's head is the default overlay image (Figure 11), but you could apply a different picture by entering its URI in a TextBox.

C#

// TransformOverlaidImage method of the MainPage class
private void TransformOverlaidImage(IEnumerable<Segment> foundSegments, int w, int h)
{
   // Set width and height of image by using the first segment's information
   var s = foundSegments.First();
   var iw = s.Width;
   var ih = s.Height;
   ImgOverlay.Width = iw;
   ImgOverlay.Height = ih;

   // Create the transform to keep the image's size and position in sync with the facial region
   var transform = new TransformGroup();
   
   // Scale image and move it to the segment's position
   transform.Children.Add(new ScaleTransform { 
      ScaleX = 1.5, 
      ScaleY = 1.5, 
      CenterX = iw >> 1, 
      CenterY = ih >> 1 });
   transform.Children.Add(new TranslateTransform { X = s.Min.X, Y = s.Min.Y + 10 });

   // Calcualte scale from Bitmap to actual size and create the transformation
   var sx = (GrdContent.ActualWidth / w);
   var sy = (GrdContent.ActualHeight / h);
   transform.Children.Add(new ScaleTransform { ScaleX = sx, ScaleY = sy });
   
   // Apply transformation
   ImgOverlay.RenderTransform = transform;
}

Voilà the iApe:

Figure 11: Result of the sixth step after overlaying the orangutan head

The Extras

Beside the algorithms we discussed above, the application also has some extra functionality. You don't need these for detection itself, but they're worth explanation.

Take result snapshot

You can take a snapshot of the final result (including the overlay) with the Button and send it to your friends, or use it as a profile picture or for your next passport. The code behind this functionality uses the WriteableBitmapEx library's WriteTga method to save the result into a TGA image file.

C#

// BtnSnapshot_Click method of the MainPage class
private void BtnSnapshot_Click(object sender, RoutedEventArgs e)
{
   // Render
   var wb = new WriteableBitmap(GrdContent, null);

   // Init SaveFileDialog
   var saveFileDlg = new SaveFileDialog
   {
      DefaultExt = ".tga",
      Filter = "TGA Image (*tga)|*.tga",
   };

   // SaveFileDialog.ShowDialog() can only be called from user-initiated code 
   // like an event handler, otherwise a SecurityException is thrown.
   if (saveFileDlg.ShowDialog().Value)
   {
      using (var dstStream = saveFileDlg.OpenFile())
      {
         wb.WriteTga(dstStream);
      }
   }
}

Load detection image

If you don't have a webcam or just want to try a photo with the skin color face detection, you can use the Button to load an image from disk. The code behind the Button opens a file dialog and fills a WriteableBitmap with the image's data. If it's larger than the size of the element that will show it, the picture will be resized with WriteableBitmapEx library's Resize method.

C#

// BtnOpenPic_Click method of the MainPage class
private void BtnOpenPic_Click(object sender, RoutedEventArgs e)
{
   // OpenFileDialog.ShowDialog() can only be called from user-initiated code 
   // like an event handler, otherwise a SecurityException is thrown.
   var openFileDialog = new OpenFileDialog();
   if (openFileDialog.ShowDialog().Value)
   {
      // Open the stream and load the image
      using (var stream = openFileDialog.File.OpenRead())
      {
         // Fill WriteableBitmap from stream
         var bmpImg = new BitmapImage();
         bmpImg.SetSource(stream);
         loadedPicture = new WriteableBitmap(bmpImg);
         
         // Resize the image if it's too large
         var w = (int)GrdContent.Width;
         var h = (int)GrdContent.Height;
         if (loadedPicture.PixelWidth > w || loadedPicture.PixelHeight > h)
         {
            loadedPicture = loadedPicture.Resize(w, h, WriteableBitmapExtensions.Interpolation.Bilinear);
         }
      }
   }
}

Conclusion

This article demonstrated how to implement a simple facial recognition system with Silverlight's new webcam feature and basic histogram-based skin color segmentation. This technique is not perfect, but it works in most webcam scenarios and in real time. There's definitely room for improvement in the segmentation method and in performance.

If you like this article, I'll write a second part that covers a better segmentation approach, overall performance optimization and an extended demo.

And if you want to try this out and learn more, the links to the live demo app and source code are at the top of the article!

About The Author

René Schulte is a .Net and Silverlight software developer from Dresden, Germany. He has a passion for computer graphics, physical simulations, AI and algorithms and loves C#, Shaders, Augmented Reality and the WriteableBitmap. He started the SLARToolkit, the WriteableBitmapEx and the Matrix3DEx Silverlight open source projects and has a Silverlight website powered by real time soft body physics. Contact information can be found on his Silverlight website or blog or via Twitter.