Can you let us know which of the pieces that we talked about here you were interested in?
In the video, we showed some UWP APIs like SpeechRecognizer and SpeechSynthesizer and we also showed some RESTful APIs from Microsoft's Cognitive Services. We also showed some pieces on Android.
The RESTful services you can use from anywhere including WPF and 'Xamarin' code (i.e. I would expect you to be able to code this into a portable class library although recording audio might be a challenge that you have to overcome).
For the UWP APIs...In general, there are a lot of UWP APIs that you can use from 'the desktop' and MSDN has a guide to that here and some work has been done here to make that more accessible.
I'm not sure that UWP's SpeechRecognizer and SpeechSynthesizer are part of that list of APIs but then the .NET Framework already has speech APIs within System.Speech.Synthesis and System.Speech.Recognition so I'd expect that you can get equivalent functionality that way.
In terms of using UWP API's 'from Xamarin' - I'd expect that you would define an interface to abstract out the functionality that you want and then you'd implement it once for UWP, once for iOS and once for Android and as long as all 3 platforms have what you need in terms of speech then you would be good to go. That work may well already exist out there somewhere.