A Studio in the Palm of Your Hand: Developing Audio and Video Creation Apps for Windows 10

Sign in to queue

Description

Existing Windows APIs for media manipulation are powerful and flexible, but they require deep domain knowledge and are not easily accessible to C#/JS developers. Windows 10 introduces a powerful creative media platform that makes your original audio/video scenarios simpler to implement. Windows 10 brings the features that are important to you for creative audio like low latency, an easier way to connect all audio paths you need, and a built-in, improved MIDI experience. Our creative video platform builds on the foundation of Windows.Media.Editing and makes it even easier to implement advanced features like custom video effects, photo/video/surface overlays, and interfacing with other APIs like Win2d and Windows.Media.Effects to take your apps to the next level. Come see how all of this ties together into our most powerful creative media platform to-date and explore building creative media apps for Windows 10.

Tag:

Media

Day:

1

Session Type:

Breakout

Code:

3-634

Room:

Room 2004

Embed

Download

The Discussion

  • User profile image
    balasiv

    Hey everyone,
    You can download the SDK samples with more code and scenarios for the topics covered in the talk at the links below:

    AudioGraph: [Github Link]
    MIDI: [Github Link]
    MediaComposition: [Github Link]

    If you have any questions or comments, please don't hesitate to reach out to Jason and me.

  • User profile image
    tellingmach​ine

    Hi Bala,

    will the MIDI sample code work with the Launch Pad Midi controller? And will it work on the Raspberry Pi with Windows 10?

    Thanks

  • User profile image
    Psychlist19​72

    @tellingmachine

    It works with Launchpad S and Launchpad Mini. Yes, it works with Raspberry Pi with Windows 10. 

    If you want the actual demo Bala used, I'll post that. It's very close to this:

    http://10rem.net/blog/2015/01/06/interfacing-the-novation-launchpad-s-with-a-windows-store-app

     

    Pete

  • User profile image
    balasiv

    @tellingmachine: Like Pete said, the MIDI sample code will work with all USB MIDI controllers. It will also work with the Raspberry Pi 2 with Windows 10 (or any device that supports the Universal Windows Platform).

    We will be releasing the code that was used for the actual demo in the talk on the GitHub link mentioned in the end. I'll post a link in the comments once it has been posted.

  • User profile image
    tellingmach​ine

    @pete, @bala,

    Very cool stuff.

    Thanks

  • User profile image
    Koshdukai

    Seems that we're getting closer to get (someday) into Windows a user level AudioGraph style rerouting between apps, hardware and virtual ports all as connectable nodes.

    Been hoping for something like that for years, once low-latency and multi-client MIDI was done properly, which looks like we're finally getting with Windows 10! :)

    Speaking of virtual ports, (audio) interface aggregation would be something useful too, btw, to group and present various interfaces as a unique multi-channel one to use with apps only capable of dealing with 1 interface.

    Would be interesting to see all the above on... say Windows 10.1 ;) (i.e. or in my lifetime at least, eh)

  • User profile image
    balasiv

    @Koshdukai:Thanks for the useful feedback. We are always looking for our customers to tell us what you want to see in the platform.Those are interesting scenarios that we will certainly look into enabling! Keep the feedback coming!

  • User profile image
    VBurel

    AudioGraph sounds like old DirectSound concept! Remember IFilterGraph and Mapper, isn't it ? Anyway it seems you are still missing the audio aggregation and applications interconnection... to get inspiration on this, i invite you to take a serious look on Voicemeeter.

  • User profile image
    Psychlist19​72

    Here's a brief article and the source for the simple drum demo from the Day 2 keynote. 

    http://blogs.msdn.com/b/johnshews_blog/archive/2015/05/08/build-2015-and-audio-effects.aspx

    Pete

  • User profile image
    DownUnderDan

    Jason & Bala,
    I don't know if the audience had been having a long day or something, but let me just say that while watching this recording from Tasmania Australia, I was cheering.

    Some amazing stuff in your demo. Thank you to the Microsoft team for all the work.

    Dan

  • User profile image
    balasiv

    @DownUnderDan:Thank you for the kind words, Dan! Please do not hesitate to reach out to us with feedback regarding the talk, scenarios or questions.

  • User profile image
    labs

    What are the chances of getting the samples to run on Windows Server 2012 in Azure?

  • User profile image
    balasiv

    @labs: The samples require Windows 10 to work since they use new functionality only available in Windows 10. Loading a Windows 10 Preview build in an Azure VM might work.

  • User profile image
    Gert

    @bala

    what are the chances of getting the samples to run on Windows 10 preview using „VS2013 Express for Windows“ (and not „VS2015 Community RC“). I ask this question because I temporarily had installed „VS2015 Community RC“ under Windows 8.1 and it crashed when calling create app package. Before (and now again) I was using (and am using again) „VS2013 Express for Windows“ for my Windows Store app GrafiCompo and „VS2013 Express for Windows Desktop“ for my other Windows desktop apps both parallel and it works fine. In my app I want to send MIDI commands using „Windows.Devices.Midi“
    Gert

  • User profile image
    balasiv

    @Gert:Sorry about the late response Gert. I would not recommend using VS2013 to run these samples. The recommended and supported way to run the samples is only with VS2015. I recommend sticking to VS2015 on the Windows 10 preview release.

  • User profile image
    Keith

    When will the white paper about low latency be released?

    Keith

  • User profile image
    balasiv

    @Keith:We're still working on the whitepaper, finalizing details about various hardware configurations that could impact latency. Is there something specific I can help answer in the meanwhile? Please feel free to email me at balasiv_at_microsoft_dot_com

  • User profile image
    Flow

    Hi

    Will it be possible to have the audio layer of a video be an input node of an audiograph?

    In short: can you play a video and do things with its audio track?

    Thanks

  • User profile image
    balasiv

    @Flow: Great question. Yes - you can set the FileInputNode to take a video file as input and AudioGraph will handle the audio track of the file. All container/codec formats supported by MediaFoundation can be used as input.

    If you encounter issues, do not hesitate to let me know.

  • User profile image
    balasiv

    Hi all,
    I wanted to let everyone know that the whitepaper on audio latency in Windows 10 is now live here: [MSDN page].

    The whitepaper contains more details and goes into greater depth into the topic than the talk at //build 2015. If you have any questions, comments or suggestions, please do not hesitate to reach out to me. I can be reached at balasiv_at_microsoft_dot_com

Add Your 2 Cents