Be nice if this had a High Quality PC download. Will it be available soon?
Most of the Use Cases you gave are floating point & fairly easy to scale out to massively parallel operations. Ideal for the GPU's & their associated data libraries, Nvidia (CUDA) or AMD (Bolt C++)
eg: One workstation with four Nvidia GTX Titan X cards provides 23,040 cores, I would need 2,880 A4(8 core) VM's to reach a similar number.
Clearly you are targeting the GPU based "super-computer" sweet spot with Azure batch.
Has Azure or Azure Batch team looked at offering GPU accelerated VM's?
Nice, Seems we've come full circle.
Like SQL 2000's SQL Notification Services, it uses SQL Queries for rapid development. But takes advantage of Azure framework to overcomes the clunky setup the plagued SQL NS.
I installed this a while ago but it doesn't display in the intellisense menu.
I believe there is an issue with Resharper. It adjusts the focus on the intellisense menu to suggest the most likely option based on what you are doing / have do before.
Be nice if they could co-exist together.
Constraining the images to be contained in an image library seems quite a limitation.
Personal Libraries lock you into a single User.
Public Libraries solve that issue but are limited to a single drive. Often the C Drive, often quite full.
Much more flexible to have an option to point to any file location. That way I could access images from a large cheap external USB drive or fileserver.
Will this be part of next release of Web Essentials? Or do you plant to keep it separate?
Aug 28, 2013 at 9:33 PM
We'll miss you. I'll look forward to the new enhancements you'll be involved in making to the Azure platform.
PS: It would be REALLY nice if you could get some kind of Event Notification / Event Handler feature on the Azure Storage queues. Polling queues sucks. SQL's Service Broker is a significantly more efficient model & much easier to program too.
James I am very disappointed by your omissions re the NN environment. You comment above about your experience with SQL's NN. Yet your talk made no mention of that as an alternative.
That work & related DM algorithms were developed by MSR. The papers they wrote on this topic were seen by the global DM research community as significant breakthroughs. They worked closely with the dev teams to incorporate all those learning's into the SQL Data Mining engine.
Unlike the core algorithm you present here, They scale massively & solved many of the "limited by memory" & model training issues most DM algorithms / products have.
They also provide / solve a ton of other issues you'd need to think about when embedding a NN system into your code. eg: They have a pluggable interface that works with most standards (ADO, OLDDB). A language, DMX, that makes it easy to enhance, configure & use with no code change. Tools that automate the training & evaluation of your model, Ability to tweak your model's parameters.
AND as it is a platform is it easy to extend the DM experience by embedding your own algorithm into that platform to create new mining models.
(Given you prefer your own algorithm, consider writing it as a DM plug-in & compare your perf with what already exists. It frees you from the plumbing & allows you to focus on the bit you do best.)
Yet you turned your back in it all. Suggesting NN wasn't well documented (which is is) & talking about "the only thing going was a java app" (there are heaps of products). Now you've encouraging attendees to completely reinvent the wheel. Instead of a project that could take hours/days (6 lines of code to embed into their app, a sql report, etc) they will take weeks or months doing everything from scratch & still unlikely to get close to the multi-core performance, the parallel model training, the scale nor the benefit of the insight outlined in the many research papers published on this subject by the MSR folks.
In isolation. Nice talk you covered the technical details & background of NN well.
But as a representative of MSR &/or Microsoft you failed to accurately brief these folks. And did your audience a disservice by implying that their only option was to start at square one.
You are a smart man & do write great articles. You have great influence. Please be more responsible in the future.
I enjoyed your 2 Azure series. Nice, quick, informative & saves time reading a ton of pages. I look forward to your next instalment.
Thanks Clemens. Very comprehensive is good. Many people are betting their business (& maybe their home) on building apps using this technology. It is vital to know as much of the low level details as you can.
I'd love to see other PM's deliver the same level of detail to the public that they do at TechReady.
The index is handy. But if you feel it is too long. You could always split the recording into 2 or 3 parts. (may also make it easier to download).