Wir, Microsoft, sind in den vergangenen Monaten von einer steigenden Anzahl von Kunden in diesem Bereich zu Rate gezogen worden. Diese Kunden aus der fertigenden Industrie kommen zu uns mit einem oft langjährigen Erfahrungsschatz in der Automation und Sammlung von Telemetriedaten zu ihren Produkten, sehen sich allerdings vor beträchtlichen Problemen, mehrere zehntausend oder auch Millionen von Geräten gleichzeitig an einem föderierten System zu betreiben, bidirektionale Kommunikation herzustellen, und die Informationsflut zu beherrschen..

Diese Episode hier auf "Subscribe!" ist die deutschsprachige Version der ersten Episode zu diesem Themenbereich, und dreht sich um IoT/M2M im Allgemeinen, die Kommunikationspatterns, und um die Verbindungsoptionen und -problem wobei ich spezifisch auch auf VPN und die damit verbundenen, aber leider nicht offensichtlichen Probleme eingehe. 

Mehr zum Thema [Englisch]:

• June 2012 MSDN Magazine "Using Service Bus for Things" (must read)
• July 2012 MSDN Magazine "A Smart Thermostat on the Service Bus"
• Internet of Things: Is VPN a False Friend?
• AMQP Support in Windows Azure Service Bus 

This is the first in a series of episodes I'll do on the IoT/M2M subject area here on "Subscribe!" and in this first one, I'm introducing the problem space, talk about patterns, and am talking about the connectivity options and challenges around devices, specifically also discussing VPN.

More on the subject:

• June 2012 MSDN Magazine "Using Service Bus for Things" (must read)
• July 2012 MSDN Magazine "A Smart Thermostat on the Service Bus"
• Internet of Things: Is VPN a False Friend?

More resources:

• Windows Azure Service Bus
• Latest Service Bus .NET client library on NuGet
• AMQP 1.0 support in Windows Azure Service Bus
• How to use AMQP 1.0 with the Service Bus .NET API
• Service Bus AMQP 1.0 Developer's Guide
• Apache Qpid AMQP 1.0 JMS library
• Apache Qpid Proton (Linux only; Windows coming soon)

You can get the bits either inlined into the new Windows Azure SDK via the Web Platform Installer or, singly, via NuGet using the keyword "WindowsAzure.ServuiceBus" or http://nuget.org/packages/WindowsAzure.ServiceBus/

We talk about authentication and authorization options and token flow, Santosh explains the new shared-access key model in Service Bus, and how Authorization rules play a role there.

The motioned SbAzTool sample is here and you can find more about the ACS integration in the docs. The shared access key documentation is forthcoming.

• Geo-replication with Service Bus Brokered Messages
• Geo-replication with Service Bus Relayed Messages
• Service Bus Durable Message Sender

The mandatory participation in the secure development lifecycle is all part of Microsoft's ongoing, decade-old trustworthy computing initiative, which has resulted in Microsoft's software and services to embody security best practices. More information about out secure development lifecycle can be found at http://microsoft.com/sdl

Barry and I sat down to discuss the SDL, but also touch on 0-day vulnerabilities and their handling, and like anyone involved in security he has some interesting war stories to share and also some insights from recent security conferences that seem worth paying more attention to and if it's only to learn about the creativity with which people get through closed doors.

In dieser Subscribe! Episode, die ganz ausnahmsweise mal auf Deutsch ist, sprechen Kollege Ruppert Koch und ich übers Arbeiten in Redmond und das Bewerbungs- und Auswahlverfahren für Kandidaten wie es in unserem unmittelbaren Umfeld praktiziert wird. Wir erläutern auch, welche Starthilfen Microsoft den neuen Kollegen an die Hand gibt.

Wenn es Euch interessiert, vielleicht für einige Jahre oder länger im Microsoft Hauptquartier in den USA an neuen Technologien mitzuarbeiten, schaut Euch das Video an und geht dann auf http://careers.microsoft.com um Euch nach passenden Stellen für Eure Qualifikationen und Interessen umzusehen, und dann einfach mal eine Bewerbung vorzubereiten und abzusenden.

Bewerbungen werden vertraulich behandelt, also gibt's nichts zu verlieren

That doesn't let you off the hook from watching you app, but saves you from a lot of depth troubleshooting that you'd otherwise have to deal with if, for instance, you were running middleware like a messaging system yourself.

For this episode I sat down with my colleague Mohamed F. Ahmed, who organizes, amongst other important things, our servicing strategy. 

We talk about how we proactively monitor our own systems and the other platform features we depend on, and how we actively observe logs to catch reliability issues and address privacy concerns as we do that.

Mohamed and I also discuss the layered structure of our world-wide 24h/365d servicing team, with a 1st level live-site operations crew with a constantly refined operations handbook for known behaviors, backed up by on-call product team crews who investigate, if needed at 3 in the morning, and either fix issues, guide customers to solutions, or provide new procedures for the operations handbook.

We talk about the various issue classifications, including the ones that will or would lead us to giving up on an entire cluster or even datacenter facility and fail over to a different one, and how we're structurally set up to learn from past mistakes and to improve our processes, which includes weekly reviews with executive leadership.

If you run a service yourself, in Windows Azure or elsewhere, you may want to make some time to watch this. 

As the app stores grow, there's increasing competitive pressure on organizations of all sizes to increase the direct consumer engagement through apps on mobile devices and tablets, and doing so is often quite a bit of a scalability leap from hundreds or thousands of concurrent internal clients to millions of direct consumer clients.

Mobile Services is there to help and can, also in conjunction with Service Bus and other services form Microsoft and partners, act as a new kind of gateway to enterprise data and compute assets.  

You can find the slide deck on SkyDrive.

The motivation for doing this talk is fairly straightforward. We see the "ESB" term and the architectural notion behind the ESB showing up in many customer engagements and also in RFPs. And we see them showing up in contexts where the goal is scale and availability. The reality is that you can't have both a central control model that the ESB theory promises and yet run a scalable and robust federation of services. The goal of this talk is to highlight the paradox and to present an alternative that's proven in the architecture of large distributed systems.

This talk is obviously another controversial one, so I'm looking forward to comments here or on Twitter under @clemensv

 [Sorry for the somewhat dim light and the ambient A/C noise in the background. That's a daylight conference room in Redmond under Northwest Spring skies and our humming building 24]

Since 1h 52m may be a tad bit long to watch in one go, here are some time markers that you can jump to for hearing about particular feature areas you may be interested in:

• 0h 00m 00s Service Bus Intro
• 0:05:55 - Queues and Topics - Basics
• 0:15:15 - Programming Model Options and Protocols
• 0:23:05 - Message Model, Dimensions, and Mappings
• 0:33:37 - Message Delivery Options and Receive Operation Styles
• 0:39:20 - Service Pricing Model
• 0:42:52 - Brokered Messaging Patterns
• 0:51:18 - Select Composite Patterns
• 0:55:20 - Correlation Patterns
• 1:03:05 - Messaging Features - Send/Receive
• 1:11:24 - Topic Filters and Actions
• 1:18:11 - Sessions
• 1:28:49 - Transaction Support
• 1:34:25 - Time-To-Live  and Scheduling
• 1:37:15 - Deadlettering
• 1:38:50 - Duplicate Detection
• 1:40:41 - Prefetching
• 1:42:12 - Auto Forwarding
• 1:46:00 - General Guidance - Resiliency and Async Operations
• 1:51:38 - Closing

Feedback and questions are welcome as always.

The slide deck can be viewed/downloaded from here; feel free to use the slide-deck for your own Service Bus presentations at conferences, or user-group events or within your team.

I don't think it is. In fact, it's the exact opposite of loose coupling and negates a lot of the advantages of opting for a messaging middleware system and mostly all advantages of opting into open standard protocols like HTTP and AMQP.

That said, it's difficult to blame the folks in the .NET developer community who've arrived at that practice, ultimately having been led down that path by -- as I am meanwhile convinced -- the late 1990s industry choice of picking XML Schema (XSD) to describe the content of messages. XML Schema, with all of its complexity, is a reasonably good way to describe the syntax of XML-based markup languages of which there are many and where aspects like element order and attribute substitution many other complexities of XSDs might make some sense. XSD's type system, which allows for restrictions that fairly directly map to inheritance on OO languages led developers to think of messages as objects - and serialization frameworks that allow for mapping the XSD type model into class hierarchy helped with that impression.

Messages are not objects. Messages don't care about the version control history. Messages don't care about whether a subset of the data they carry also similarly appears in some other message.

Messages are flat. Maps of keys to values. The values are simple-typed or, again, maps or arrays or values or maps. JSON embodies that model. If you ignore XSD and allow for the same simple constraints as JSON, XML also embodies that model.

If you take a look, you'll start noticing that we and a lot of others in the industry have moved to document messages in simpler ways than schemas.

• Service Bus Queue: http://msdn.microsoft.com/en-us/library/windowsazure/hh780773.aspx
• Twitter Tweet: https://dev.twitter.com/docs/platform-objects/tweets 
• Facebook Post https://developers.facebook.com/docs/reference/api/post/ 

(BTW, if we could easily do it without breaking backwards compatibility we'd even drop the namespace in the SB API snippet above)

You'll generally see a plain text documentation of properties of which some may have complex context consisting of maps and arrays. The advantage is that these descriptions map nicely into all programming languages and platforms by ways of a skilled developer.

I realize this is controversial, so I'm looking forward to comments.

Like most teams at Microsoft we have a dedicated test organization. Very early on, when Service Bus was a one-machine open CTP running on a box under someone's desk (literally) and then on two machines in a test lab, we had some overinflated confidence on what it would take to ship a service. Then Cesar showed up with complete team and took the air out of that confidence balloon -- and then helped building confidence back up to the point that we can sleep very well while being on-call with 8 datacenters running key customer workloads on Service Bus.  

Test owns checking claims and aspirations against reality, and a lot more. Watch this one.

• Tag-based unicast and groupcast notifications
• Templates

I'm mostly covering the Windows 8 aspects like Tiles as Windows' notification model is richer as that of iOS, but the concepts also apply in the same way to iOS.

For info on how to realize these concepts in code go to http://aka.ms/pushtowin8 (Win8) and http://aka.ms/pushtoios (iOS) 

We also talk about how new members of the team get going and how senior developers help new hires. That said, we have open positions

https://careers.microsoft.com/search.aspx#&&p4=US&p0=%22Service+Bus%22&p5=2134&p1=3%2c20&p2=1093&p3=1014

In this clip I'll walk you through the basic principles of the brand-new Windows Azure Service Bus Notification Hubs in a somewhat more formal and serious fashion than in my chat with Elio.

Recapping from the other post,Service Bus Notification Hubs are an intrinsic feature of Windows Azure Service Bus and are different from other push notification services in four key areas:

• Complete client registration management. Your backend application (if you even have one) does not need to worry at all about device-ids or channels or other particulars of push notifications and doesn't need to cooperate in management. It doesn't even have to be a web app that's publicly accessible.  
• Platform independence. Service Bus Notification Hubs allow cross-platform push notifications so that iOS Alerts and Windows Live Tiles can be targeted with a single event message. 
• Broadcast and tag-based Multicast - Service Bus Notification Hubs are optimized around automatic notification broadcast to many thousand devices with low latency. One message in, thousands of notifications out.
• Mass customization - Notification Hub notification templates allow for customization of notification delivery for each individual registration, allowing each instance of a client App to choose how it wants to receive events.

In this preview, Notification Hubs are able to push notifications to Windows Store apps and iOS apps from .NET back-ends. Support for Android and Windows Phone, along with additional back-end technologies (including Windows Azure Mobile Services) will be added soon.

After the basic intro, I'm showing how to create and provision a iOS application from scratch, how to hook it up to a new Notification Hub, and send it a notification Alert using the portals and Visual Studio Express 2012 Web. (The equivalent Windows 8 walkthrough is here)

For those of you with a "TL;DW" attention span (too long; didn't watch), here's the whole extent of the code added to the stock iOS code template to enable Service Bus Notifications and that includes re-registering existing registrations at App startup. Decoration excluded it's 6 functional lines including one required by iOS - admittedly, without error handling.

- (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions
{
    NSString * connectionString =
    [SBConnectionString stringWithEndpoint:@"https://myhub-euw-ns.servicebus.windows.net/" listenAccessSecret:@"... secret..."];
    
    self.hub = [[SBNotificationHub alloc] initWithConnectionString: connectionString notificationHubPath:@"myhub"];
    
    [[UIApplication sharedApplication] registerForRemoteNotificationTypes:
     UIRemoteNotificationTypeAlert|UIRemoteNotificationTypeBadge|UIRemoteNotificationTypeSound];
     ...
    return YES;
}


- (void)application:(UIApplication *)application didRegisterForRemoteNotificationsWithDeviceToken:(NSData *)deviceToken {
    
    [self.hub refreshRegistrationsWithDeviceToken:deviceToken completion:
       ^(NSError *error) {
         if ( error == nil )
         {
             [self.hub defaultRegistrationExistsWithCompletion:^(BOOL exists, NSError *error2) {
                 if ( error2 == nil && !exists )
                 {
                     [self.hub createDefaultRegistrationWithTags:nil completion:
                      
                      ^(NSError *error3) {
                          if ( error3 != nil) {
                          }
                      }];
                 }
             }];
         }
     }];
}

The event-source code, written on the server-side for a Windows Azure Web Site in ASP.NET, is similarly terse:

var cs = ServiceBusConnectionStringBuilder.CreateUsingSharedAccessSecretWithFullAccess( "myhub-euw-ns", "...secret...");
var nh = NotificationHubClient.CreateClientFromConnectionString( cs, "myhub");

nh.SendAppleNativeNotification( AppleNotificationJsonBuilder.Create(TextBox1.Text).ToJsonString());

 3 lines. Three lines. No management of device ids. No public endpoint for the phone to talk to. Service Bus does all that. It really is worth playing with. 

And here are all the key links ....

• Feature guide (Windows Store Apps) - http://go.microsoft.com/fwlink/?LinkID=275828
• Feature guide (iOS) -  http://go.microsoft.com/fwlink/?LinkId=275829
• Fundamentals - http://go.microsoft.com/fwlink/?LinkId=277072
• Tutorial (Windows Store Apps) - http://go.microsoft.com/fwlink/?LinkId=277073
• Tutorial (iOS) - http://go.microsoft.com/fwlink/?LinkId=277074

 SDKs:

• Windows 8 Managed Client Library - http://go.microsoft.com/fwlink/?LinkID=277160
• iOS Client Library - http://go.microsoft.com/fwlink/?LinkID=277161
• Preview client NuGet - http://nuget.org/packages/ServiceBus.Preview

Recapping from the other post,Service Bus Notification Hubs are an intrinsic feature of Windows Azure Service Bus and are different from other push notification services in four key areas:

• Complete client registration management. Your backend application (if you even have one) does not need to worry at all about device-ids or channels or other particulars of push notifications and doesn't need to cooperate in management. It doesn't even have to be a web app that's publicly accessible.  
• Platform independence. Service Bus Notification Hubs allow cross-platform push notifications so that iOS Alerts and Windows Live Tiles can be targeted with a single event message. 
• Broadcast and tag-based Multicast - Service Bus Notification Hubs are optimized around automatic notification broadcast to many thousand devices with low latency. One message in, thousands of notifications out.
• Mass customization - Notification Hub notification templates allow for customization of notification delivery for each individual registration, allowing each instance of a client App to choose how it wants to receive events.

In this preview, Notification Hubs are able to push notifications to Windows Store apps and iOS apps from .NET back-ends. Support for Android and Windows Phone, along with additional back-end technologies (including Windows Azure Mobile Services) will be added soon.

After the basic intro, I'm showing how to create and provision a Windows 8 application from scratch, how to hook it up to a new Notification Hub, and send it a notification "Toast" using the portals and Visual Studio 2012. (The equivalent iOS walkthrough will follow later this week)

For those of you with a "TL;DW" attention span (too long; didn't watch), here's the whole extent of the code added to the stock Windows Store Grid template to enable Service Bus Notifications and that includes re-registering existing registrations at App startup. 5 lines without cosmetic wrapping and some massaging of XML for the template:

public App()
{
    var cn = ConnectionString.CreateUsingSharedAccessSecretWithListenAccess(
            "sb://clemensv1.servicebus.windows.net",
            "{{secret-key}}");
    this.notificationHub = new NotificationHub("myhub", cn);
    ...
}
async Task InitNotificationsAsync()
{
    await notificationHub.RefreshRegistrationsAsync();

    if (!await notificationHub.RegistrationExistsForApplicationAsync("myToast"))
    {
        await notificationHub.CreateTemplateRegistrationForApplicationAsync(
            CreateTemplate(), "myToast");
    }
}
        
XmlDocument CreateTemplate()
{
    var t = ToastNotificationManager.GetTemplateContent(ToastTemplateType.ToastText01);
    var n = t.SelectSingleNode("//text[@id='1']") as XmlElement;
    if (n != null)
    {
        n.InnerText = "$(msg)";
    }
    return t;
}

The event-source code is similarly terse:

var cn = ServiceBusConnectionStringBuilder.CreateUsingSharedAccessSecretWithFullAccess(
    "clemensv1", "{{{secret key}}");

var hubClient = NotificationHubClient.
    CreateClientFromConnectionString(cn, "myhub");

hubClient.SendTemplateNotification(new Dictionary<string, string>{
    { "msg", TextBox1.Text }});

3 lines. Three lines. No management of device ids. No public endpoint for the phone to talk to. Service Bus does all that. It really is worth playing with. 

And here are all the key links ....

• Feature guide (Windows Store Apps) - http://go.microsoft.com/fwlink/?LinkID=275828
• Feature guide (iOS) -  http://go.microsoft.com/fwlink/?LinkId=275829
• Fundamentals - http://go.microsoft.com/fwlink/?LinkId=277072
• Tutorial (Windows Store Apps) - http://go.microsoft.com/fwlink/?LinkId=277073
• Tutorial (iOS) - http://go.microsoft.com/fwlink/?LinkId=277074

 SDKs:

• Windows 8 Managed Client Library - http://go.microsoft.com/fwlink/?LinkID=277160
• iOS Client Library - http://go.microsoft.com/fwlink/?LinkID=277161
• Preview client NuGet - http://nuget.org/packages/ServiceBus.Preview

Fast forward to the present day, and Iguazu has become a real thing, thanks to an enormously engaged team who built a brand new service from scratch and plugged it into the Service Bus infrastructure. Today we're taking the wraps off the Service Bus Notification Hubs preview.

Service Bus Notification Hubs are an intrinsic feature of Windows Azure Service Bus and are different from other push notification services in four key areas:

• Complete client registration management. Your backend application does not need to worry at all about device-ids or channels or other particulars of push notifications and doesn't need to cooperate in management. It doesn't even have to be a web app that's publicly accessible.  
• Platform independence. Service Bus Notification Hubs allow cross-platform push notifications so that iOS Alerts and Windows Live Tiles can be targeted with a single event message. 
• Broadcast and tag-based Multicast - Service Bus Notification Hubs are optimized around automatic notification broadcast to many thousand devices with low latency. One message in, thousands of notifications out.
• Mass customization - Notification Hub notification templates allow for customization of notification delivery for each individual registration, allowing each instance of a client App to choose how it wants to receive events.

In this preview, Notification Hubs are able to push notifications to Windows Store apps and iOS apps from .NET back-ends. Support for Android and Windows Phone, along with additional back-end technologies (including Windows Azure Mobile Services) will be added soon.

If you can't wait to see teh codez check out the docs and bits below or watch my walkthrough.

And here are all the key links (if they're not yet active/current they will soon be)....

• Feature guide (Windows Store Apps) - http://go.microsoft.com/fwlink/?LinkID=275828
• Feature guide (iOS) -  http://go.microsoft.com/fwlink/?LinkId=275829
• Fundamentals - http://go.microsoft.com/fwlink/?LinkId=277072
• Tutorial (Windows Store Apps) - http://go.microsoft.com/fwlink/?LinkId=277073
• Tutorial (iOS) - http://go.microsoft.com/fwlink/?LinkId=277074

 SDKs:

• Windows 8 Managed Client Library - http://go.microsoft.com/fwlink/?LinkID=277160
• iOS Client Library - http://go.microsoft.com/fwlink/?LinkID=277161
• Preview client NuGet - http://nuget.org/packages/ServiceBus.Preview

We talk about how the Halo 4 game uses Windows Azure Service Bus in various ways, including the flow of all multiplayer game statistics from the console game into Waypoint, and how Service Bus helped the Halo team dealing with the massive traffic peak occurring on and within a few days of the game title's release.