WEBVTT

00:00:00.000 --> 00:00:01.200
>> Welcome back. We're in

00:00:01.200 --> 00:00:03.200
the Microsoft booth live at GDC.

00:00:03.200 --> 00:00:04.485
I am Adam Tuliper,

00:00:04.485 --> 00:00:05.810
I'm a Tech Evangelist and

00:00:05.810 --> 00:00:07.360
Software Engineer
with Microsoft,

00:00:07.360 --> 00:00:08.905
living in Sunny,
Southern California.

00:00:08.905 --> 00:00:10.930
And I'm here interviewing
Jesse McCullough.

00:00:10.930 --> 00:00:12.680
Jesse has been instrumental

00:00:12.680 --> 00:00:14.435
in the Mixed Reality Community.

00:00:14.435 --> 00:00:16.275
Jesse, can you tell us
a little bit about yourself.

00:00:16.275 --> 00:00:18.010
>> Yes. I've been doing

00:00:18.010 --> 00:00:20.630
Mixed Reality development
ever since HoloLens came out.

00:00:20.630 --> 00:00:22.610
I was wave one developer,

00:00:22.610 --> 00:00:25.115
and jumped right into it.

00:00:25.115 --> 00:00:29.675
And I've kind of gone
headfirst into it,

00:00:29.675 --> 00:00:32.390
you could say, and stirred
up a community around it.

00:00:32.390 --> 00:00:34.310
It's been super successful

00:00:34.310 --> 00:00:35.320
and having a lot of fun with it.

00:00:35.320 --> 00:00:36.460
>> So, what wave were you

00:00:36.460 --> 00:00:37.520
when you got
your first HoloLens?

00:00:37.520 --> 00:00:38.010
>> I was Wave one.

00:00:38.010 --> 00:00:38.730
>> Wave one.

00:00:38.730 --> 00:00:39.100
>> Yes.

00:00:39.100 --> 00:00:41.570
>> So, you're hardcore
into it, ready to

00:00:41.570 --> 00:00:43.220
>> Actually, that's kind

00:00:43.220 --> 00:00:45.120
of a really
interesting story is,

00:00:45.120 --> 00:00:50.660
when I filled out my stuff
for the developer edition,

00:00:50.660 --> 00:00:52.290
there was like a bunch
of questions on that.

00:00:52.290 --> 00:00:53.710
Like, "What are you
going to build with this?

00:00:53.710 --> 00:00:54.950
What kind of cool technology?"

00:00:54.950 --> 00:00:56.160
I'm like, "I have no

00:00:56.160 --> 00:00:57.370
idea what I'm going to
do with this thing."

00:00:57.370 --> 00:00:59.245
And that's literally
what I wrote down.

00:00:59.245 --> 00:01:01.315
Like there was
five questions, and I wrote,

00:01:01.315 --> 00:01:03.385
"I don't know. I have no idea."

00:01:03.385 --> 00:01:04.740
And I'm like, "I'm going to be

00:01:04.740 --> 00:01:06.730
the last person on earth to
get one of these things."

00:01:06.730 --> 00:01:08.310
Then all of a sudden, I
get this email that says,

00:01:08.310 --> 00:01:09.880
"Hey, you're Wave one."

00:01:09.880 --> 00:01:11.190
And I was like, "Really?"

00:01:11.190 --> 00:01:12.790
>> Wow. That's pretty cool.

00:01:12.790 --> 00:01:15.740
I think I got mine at Wave
four or so. That's interesting.

00:01:15.740 --> 00:01:18.660
So, you had never done
development before then,

00:01:18.660 --> 00:01:19.550
or you had never done

00:01:19.550 --> 00:01:20.640
Mixed Reality
Development before then?

00:01:20.640 --> 00:01:21.840
>> I'd never done
Mixed Reality Development.

00:01:21.840 --> 00:01:25.750
I was working for an enterprise
company doing JavaScript.

00:01:25.750 --> 00:01:29.375
I've always been kind of a fan
of Microsoft Technologies,

00:01:29.375 --> 00:01:33.420
but I hadn't really
dove into any 3D stuff.

00:01:33.420 --> 00:01:35.440
>> Now, if I recall,

00:01:35.440 --> 00:01:38.940
were you a bit newer to
development technologies?

00:01:38.940 --> 00:01:41.360
In other words, how long have
you been a developer for?

00:01:41.360 --> 00:01:44.450
>> I've been a developer
now for eight years.

00:01:44.450 --> 00:01:45.170
>> Eight years, okay.

00:01:45.170 --> 00:01:45.470
>> Yes.

00:01:45.470 --> 00:01:47.400
>> Interesting. So,
Mixed Reality came out,

00:01:47.400 --> 00:01:48.450
you're like, "I have
no idea what this is

00:01:48.450 --> 00:01:49.610
all about. Let me
get into this."

00:01:49.610 --> 00:01:50.780
>> Yes. When I jumped into it,

00:01:50.780 --> 00:01:52.235
I had to learn Unity.

00:01:52.235 --> 00:01:54.420
I had a pretty good
baseline on UWP,

00:01:54.420 --> 00:01:56.170
but never in

00:01:56.170 --> 00:01:58.890
a professional sense, just
kind of it as a hobby.

00:01:58.890 --> 00:02:01.900
But, yes. All the 3D stuff
was brand new to me.

00:02:01.900 --> 00:02:03.000
>> That's really cool.

00:02:03.000 --> 00:02:05.680
>> It's a whole different
paradigm to think in that way.

00:02:05.680 --> 00:02:06.850
>> Is that your Wave one device?

00:02:06.850 --> 00:02:07.320
>> It is.

00:02:07.320 --> 00:02:08.680
>> Very cool. So, you got

00:02:08.680 --> 00:02:10.850
the device and you
started up a community.

00:02:10.850 --> 00:02:11.975
You started the Slack Channel.

00:02:11.975 --> 00:02:12.455
>> Yes.

00:02:12.455 --> 00:02:15.250
>> And that's
really grown to be,

00:02:15.250 --> 00:02:17.140
I think something that
you spend a lot of

00:02:17.140 --> 00:02:18.900
time helping the community grow.

00:02:18.900 --> 00:02:20.510
And so, from Microsoft,
thank you for that.

00:02:20.510 --> 00:02:20.690
>> Yes.

00:02:20.690 --> 00:02:22.080
>> I think that's
awesome you've done that.

00:02:22.080 --> 00:02:24.430
But developers kind
of often wonder,

00:02:24.430 --> 00:02:26.150
the device has been on

00:02:26.150 --> 00:02:27.780
the market for now,
it's establish.

00:02:27.780 --> 00:02:29.350
And so the developer
community has grown,

00:02:29.350 --> 00:02:31.210
and is really instrumental

00:02:31.210 --> 00:02:32.720
for folks like you to
help that community.

00:02:32.720 --> 00:02:37.145
And even at times when we're
quieter on future details,

00:02:37.145 --> 00:02:38.490
the community is still extremely

00:02:38.490 --> 00:02:39.820
strong and helping into that.

00:02:39.820 --> 00:02:41.280
Can you talk about what goes

00:02:41.280 --> 00:02:43.580
on in that developer community
and what that's like?

00:02:43.580 --> 00:02:46.540
>> Yes. So, it's
interesting because you say

00:02:46.540 --> 00:02:49.585
that this is a kind of
an established device,

00:02:49.585 --> 00:02:51.315
but it's still such a new market

00:02:51.315 --> 00:02:52.935
that nobody's really
cornered it yet.

00:02:52.935 --> 00:02:54.875
So, if you're out there

00:02:54.875 --> 00:02:58.180
and you're interested
in it, don't wait.

00:02:58.180 --> 00:03:01.495
This market it's
not cornered at all.

00:03:01.495 --> 00:03:03.160
So, in the Slack Community,

00:03:03.160 --> 00:03:04.370
we get people that come in

00:03:04.370 --> 00:03:06.550
and have never done
HoloLens Development,

00:03:06.550 --> 00:03:07.880
we get people who
come in who have been

00:03:07.880 --> 00:03:10.205
doing it since
the early days of it.

00:03:10.205 --> 00:03:12.150
And we've got all kinds
of channels in there.

00:03:12.150 --> 00:03:13.470
We've got channels about Unity.

00:03:13.470 --> 00:03:14.720
So, if you have
questions about doing

00:03:14.720 --> 00:03:17.790
Unity development in
relationship to HoloLens,

00:03:17.790 --> 00:03:19.430
or the Mixed Reality
immersive headset.

00:03:19.430 --> 00:03:20.130
>> Cool.

00:03:20.130 --> 00:03:22.665
>> We've got the help
channel where you're like,

00:03:22.665 --> 00:03:25.525
"I'm struggling with
this, I need some help."

00:03:25.525 --> 00:03:28.530
We've got specific channels

00:03:28.530 --> 00:03:31.180
about the different conferences.

00:03:31.180 --> 00:03:32.500
We've got to build channels for

00:03:32.500 --> 00:03:35.505
the build conference
coming up here in May.

00:03:35.505 --> 00:03:37.945
So, kind of a little
bit everything.

00:03:37.945 --> 00:03:38.905
>> That's interesting.

00:03:38.905 --> 00:03:42.370
>> And we try and just keep
the news as open as we can.

00:03:42.370 --> 00:03:43.700
>> So, do you find that

00:03:43.700 --> 00:03:45.555
developers help each
other on that channel?

00:03:45.555 --> 00:03:46.230
>> Oh, yes.

00:03:46.230 --> 00:03:46.805
>> Right.

00:03:46.805 --> 00:03:48.660
>> It's almost
entirely developer led.

00:03:48.660 --> 00:03:50.430
>> Okay. I've got
an issue with this.

00:03:50.430 --> 00:03:51.640
Can you guys help me out figure

00:03:51.640 --> 00:03:52.770
out how to do Spatial Mapping

00:03:52.770 --> 00:03:55.270
and Spatial Understanding,
and that's phenomenal?

00:03:55.270 --> 00:03:56.980
>> Yes, and people will
drop code in there,

00:03:56.980 --> 00:03:58.855
and say, "Oh, this
is how I solved it."

00:03:58.855 --> 00:04:02.150
Or, that's not
really possible yet.

00:04:02.150 --> 00:04:03.995
We're hoping that it'll come

00:04:03.995 --> 00:04:06.900
through in the next
build update and whatnot.

00:04:06.900 --> 00:04:09.010
>> And so, when I mentioned
established device,

00:04:09.010 --> 00:04:10.620
so there we're talking
about, the device

00:04:10.620 --> 00:04:12.640
has been out for a bit
now, API is established.

00:04:12.640 --> 00:04:13.890
We're growing over time, and

00:04:13.890 --> 00:04:16.595
Mixed Reality Toolkit
has gotten well.

00:04:16.595 --> 00:04:17.870
Initially, the HoloToolkit,

00:04:17.870 --> 00:04:19.540
right now it's
the Mixed Reality Toolkit,

00:04:19.540 --> 00:04:21.400
and we've been adding
features over time.

00:04:21.400 --> 00:04:22.975
You have been a contributor

00:04:22.975 --> 00:04:24.290
to the Mixed Reality
Toolkit as well.

00:04:24.290 --> 00:04:26.450
>> Yes. I sit in on the team

00:04:26.450 --> 00:04:28.650
that does the weekly
shift room calls for that.

00:04:28.650 --> 00:04:33.070
So, we look at the issues
that are open out there,

00:04:33.070 --> 00:04:34.920
what we have time to work on,

00:04:34.920 --> 00:04:36.415
kind of guiding new features.

00:04:36.415 --> 00:04:36.690
>> Okay.

00:04:36.690 --> 00:04:39.220
>> And then trying to pull
more people into being

00:04:39.220 --> 00:04:42.250
contributors because it
really is a community effort.

00:04:42.250 --> 00:04:44.700
>> It's kind of interesting
because folks will say, "Hey,

00:04:44.700 --> 00:04:47.020
you're not running
a Windows phone or something,

00:04:47.020 --> 00:04:48.320
aren't you all only

00:04:48.320 --> 00:04:50.200
Microsoft only kind
of Windows ecosystem?"

00:04:50.200 --> 00:04:52.490
Is like, "No. We try to
enable developers everywhere."

00:04:52.490 --> 00:04:53.570
And so a lot of
that has been the

00:04:53.570 --> 00:04:54.760
open source movement
that we have.

00:04:54.760 --> 00:04:55.240
>> Yes.

00:04:55.240 --> 00:04:57.580
>> And so, the Mixed
Reality Toolkit

00:04:57.580 --> 00:04:59.755
being an open source
community project,

00:04:59.755 --> 00:05:01.535
you would probably know best.

00:05:01.535 --> 00:05:03.990
There's a lot of
community effort

00:05:03.990 --> 00:05:05.640
that goes into maintaining
that tool. But yes,

00:05:05.640 --> 00:05:07.615
we have folks that work
at Microsoft that do that,

00:05:07.615 --> 00:05:08.960
but we absolutely rely on

00:05:08.960 --> 00:05:10.780
community support and
folks like yourself, right?

00:05:10.780 --> 00:05:12.290
>> Yes. I'd say at this point,

00:05:12.290 --> 00:05:15.530
it's equally as
many community developers

00:05:15.530 --> 00:05:17.745
are putting time in and
as are Microsoft people.

00:05:17.745 --> 00:05:18.380
>> Wow.

00:05:18.380 --> 00:05:19.920
>> Which is really awesome.

00:05:19.920 --> 00:05:21.500
We've got a couple
of people who are

00:05:21.500 --> 00:05:22.830
really super strong in there,

00:05:22.830 --> 00:05:25.325
and then they're making up
a large number of changes.

00:05:25.325 --> 00:05:27.740
But a lot of people will
say, "Hey, I built this."

00:05:27.740 --> 00:05:29.930
And were like, "Hey,
could we put it in there?"

00:05:29.930 --> 00:05:31.250
>> Sounds good. Let's put it
in the toolkit, that's neat.

00:05:31.250 --> 00:05:31.450
>> Yes.

00:05:31.450 --> 00:05:33.160
>> That's neat. So, you have

00:05:33.160 --> 00:05:35.100
a talk that you're going
to do, is it today?

00:05:35.100 --> 00:05:35.300
>> Yes.

00:05:35.300 --> 00:05:36.750
>> What time, four O'clock?

00:05:36.750 --> 00:05:37.555
>> Four o'clock, yes.

00:05:37.555 --> 00:05:39.000
Over here in
the Azure Club Theatre.

00:05:39.000 --> 00:05:39.740
>> Tell us about that talk.

00:05:39.740 --> 00:05:42.830
>> So, I'm going to be
talking about the AR Cloud.

00:05:42.830 --> 00:05:45.370
So if you're not aware
of what the AR Cloud is,

00:05:45.370 --> 00:05:47.125
it's this idea that

00:05:47.125 --> 00:05:51.260
our physical world needs
a digital representation.

00:05:51.260 --> 00:05:53.710
And once we have a digital
representation in that world,

00:05:53.710 --> 00:05:55.690
we can start sharing
digital content.

00:05:55.690 --> 00:05:58.000
>> Now, when you say, "Our
physical world needs a

00:05:58.000 --> 00:06:00.775
digital representation,"
are you talking purely maps?

00:06:00.775 --> 00:06:01.910
>> To start with, yes.

00:06:01.910 --> 00:06:02.300
>> Okay.

00:06:02.300 --> 00:06:05.910
>> So, maps of the area.
Maps get kind of interesting

00:06:05.910 --> 00:06:07.660
because you've got features in

00:06:07.660 --> 00:06:10.220
mapping that are
permanent features,

00:06:10.220 --> 00:06:13.470
like the columns in this
building, never going to change.

00:06:13.470 --> 00:06:15.120
And then you've got

00:06:15.120 --> 00:06:16.910
features of objects that

00:06:16.910 --> 00:06:18.390
are not necessarily
so permanent,

00:06:18.390 --> 00:06:20.740
tables and some stuff
that can move around.

00:06:20.740 --> 00:06:22.770
So, that's one of
the challenges,

00:06:22.770 --> 00:06:24.310
is being able to differentiate

00:06:24.310 --> 00:06:26.410
things that are
going to stay where

00:06:26.410 --> 00:06:28.760
they're at versus things
that could get moved

00:06:28.760 --> 00:06:32.790
around and change the environment
that you're working at.

00:06:32.790 --> 00:06:34.000
>> Would things that
are going to get

00:06:34.000 --> 00:06:35.360
moved around just not the maps,

00:06:35.360 --> 00:06:37.400
not included in
this presentation?

00:06:37.400 --> 00:06:38.820
>> If they're not maps, or we

00:06:38.820 --> 00:06:40.100
need a way of identifying them.

00:06:40.100 --> 00:06:40.330
>> Okay.

00:06:40.330 --> 00:06:42.945
>> Right now,

00:06:42.945 --> 00:06:44.890
it's really hard
just because the

00:06:44.890 --> 00:06:46.860
technology's not there
as far as the hardware,

00:06:46.860 --> 00:06:48.795
from being able to look
at something and say,

00:06:48.795 --> 00:06:51.130
"I recognize that this is
probably a table because

00:06:51.130 --> 00:06:53.370
it's got a flat surface
between a certain height,

00:06:53.370 --> 00:06:56.290
and it's got legs that
go down to the ground.

00:06:56.290 --> 00:06:58.040
So, I can reason this table,

00:06:58.040 --> 00:07:00.405
and at some point it
could probably move."

00:07:00.405 --> 00:07:00.850
>> Right.

00:07:00.850 --> 00:07:02.090
>> But we'll get there.

00:07:02.090 --> 00:07:04.860
>> We'll get there. Right.
We see the advent of

00:07:04.860 --> 00:07:07.890
modern GPS has really
enabled AI to take off,

00:07:07.890 --> 00:07:09.730
and we have cognitive services.

00:07:09.730 --> 00:07:11.335
You can look at
a picture of a table,

00:07:11.335 --> 00:07:12.830
and know it's a table
in that picture,

00:07:12.830 --> 00:07:14.480
but you might not know in
space what's that, right?

00:07:14.480 --> 00:07:14.660
>> Right.

00:07:14.660 --> 00:07:16.240
>> And on the flip side, we have

00:07:16.240 --> 00:07:17.380
the HoloLens which does its

00:07:17.380 --> 00:07:18.840
Spatial Understanding
and Mapping.

00:07:18.840 --> 00:07:20.950
And it doesn't know
what a table is

00:07:20.950 --> 00:07:23.435
unless you define
a surface, right?

00:07:23.435 --> 00:07:26.115
A half meter off the ground,
and two square meters.

00:07:26.115 --> 00:07:26.340
>> Right.

00:07:26.340 --> 00:07:27.740
>> It doesn't know it's
a table, it's knows there's

00:07:27.740 --> 00:07:29.840
a mesh and a surface there.

00:07:29.840 --> 00:07:32.310
>> Right. And we can reason
that it's something sitable,

00:07:32.310 --> 00:07:34.020
or what we call sitable.

00:07:34.020 --> 00:07:35.780
Which means, it's
probably a chair,

00:07:35.780 --> 00:07:37.050
or a couch, or some surface

00:07:37.050 --> 00:07:38.220
that somebody could sit on it.

00:07:38.220 --> 00:07:39.375
>> So by sitable, you mean,

00:07:39.375 --> 00:07:41.120
I've defined
something to the API,

00:07:41.120 --> 00:07:43.105
I've said it's a half meter
off the ground,

00:07:43.105 --> 00:07:46.905
it has a flat surface area,
we'll call that sitable?

00:07:46.905 --> 00:07:47.335
>> Right.

00:07:47.335 --> 00:07:49.010
>> And therefore
a 3D character come

00:07:49.010 --> 00:07:50.910
walking up and sit down on
there, or something like that?

00:07:50.910 --> 00:07:51.750
>> Yes, exactly.

00:07:51.750 --> 00:07:53.230
>> There was kind of
where two end of the spectrum.

00:07:53.230 --> 00:07:54.760
Computer vision
can understand it,

00:07:54.760 --> 00:07:55.990
the HoloLens can map it,

00:07:55.990 --> 00:07:57.540
then we kind of need to
meet in the center there.

00:07:57.540 --> 00:07:57.860
>> Yes.

00:07:57.860 --> 00:07:59.410
>> And so, once that
happens like so,

00:07:59.410 --> 00:08:01.810
when you talk about the AR
Cloud, what does this mean?

00:08:01.810 --> 00:08:03.115
Like what can we
do with that then?

00:08:03.115 --> 00:08:05.460
>> So, once we can do that,

00:08:05.460 --> 00:08:06.915
developers could put out,

00:08:06.915 --> 00:08:10.130
say their application,
puts digital content into

00:08:10.130 --> 00:08:11.890
the real world either
through HoloLens

00:08:11.890 --> 00:08:14.300
or through an ARKit
phone or whatnot.

00:08:14.300 --> 00:08:14.560
>> Okay.

00:08:14.560 --> 00:08:16.810
>> Once they have a shared
understanding of the world,

00:08:16.810 --> 00:08:20.355
that it can place that
digital content in the world,

00:08:20.355 --> 00:08:21.920
and know where it is in

00:08:21.920 --> 00:08:23.775
relationship to
the physical world.

00:08:23.775 --> 00:08:25.080
And now you could
have somebody with

00:08:25.080 --> 00:08:27.570
a different device,
say a HoloLens,

00:08:27.570 --> 00:08:31.135
or ARKit, or whatever come in,

00:08:31.135 --> 00:08:33.230
and look at it
through their device.

00:08:33.230 --> 00:08:35.220
And you guys can look
at the same content

00:08:35.220 --> 00:08:37.210
in the same world space.

00:08:37.210 --> 00:08:39.500
And the reason this is
important is because right

00:08:39.500 --> 00:08:41.795
now AR is a very
lonely experience.

00:08:41.795 --> 00:08:44.545
If I'm wearing my HoloLens
and my girlfriend comes home,

00:08:44.545 --> 00:08:46.020
I could be wandering around

00:08:46.020 --> 00:08:47.875
doing this and all
over the house,

00:08:47.875 --> 00:08:49.170
and she has no idea what

00:08:49.170 --> 00:08:50.910
I'm doing because
she can't see it,

00:08:50.910 --> 00:08:53.815
because it's only a
personal experience for me.

00:08:53.815 --> 00:08:56.000
And there are shared experiences
where you can have

00:08:56.000 --> 00:08:58.735
multiple HoloLenses
looking at the same stuff,

00:08:58.735 --> 00:09:02.440
but we're not really at a point
where that's common yet.

00:09:02.440 --> 00:09:03.460
>> It's not generalized, right?

00:09:03.460 --> 00:09:06.320
We have third parties that
make sharing services.

00:09:06.320 --> 00:09:08.920
We have open sourced
collaboration features.

00:09:08.920 --> 00:09:10.580
But you're right, there's

00:09:10.580 --> 00:09:12.940
an application that
has to be baked in.

00:09:12.940 --> 00:09:13.270
>> Yes.

00:09:13.270 --> 00:09:15.070
>> So, do you see
this becoming like

00:09:15.070 --> 00:09:18.260
a standard API
where folks across

00:09:18.260 --> 00:09:21.150
any AR type experience
can just pull in

00:09:21.150 --> 00:09:24.040
this information and then it
gets rendered on the device,

00:09:24.040 --> 00:09:25.180
and they have
some understanding of it?

00:09:25.180 --> 00:09:27.270
>> Yes. And one of the things
that we're working on,

00:09:27.270 --> 00:09:29.310
the company I work for
is called PracticalVR.

00:09:29.310 --> 00:09:30.610
One of the things
we're working on

00:09:30.610 --> 00:09:33.765
is cloud source in
the data gathering.

00:09:33.765 --> 00:09:37.500
So, having people
out there who are

00:09:37.500 --> 00:09:39.530
doing these different
applications

00:09:39.530 --> 00:09:41.020
to implement
our mapping experience.

00:09:41.020 --> 00:09:42.890
So what they do is, they

00:09:42.890 --> 00:09:44.710
put our mapping experience
at the front end,

00:09:44.710 --> 00:09:46.000
when somebody maps a space,

00:09:46.000 --> 00:09:47.850
it gets uploaded to
our cloud service.

00:09:47.850 --> 00:09:49.280
And so the idea being,

00:09:49.280 --> 00:09:51.185
that if somebody
else comes in later,

00:09:51.185 --> 00:09:52.950
we can say, and they

00:09:52.950 --> 00:09:54.855
pull up an app that uses
our mapping service.

00:09:54.855 --> 00:09:57.015
We can say, "We
recognize the space."

00:09:57.015 --> 00:09:58.775
Rather than making you map it,

00:09:58.775 --> 00:10:01.305
we'll just download
it to your device.

00:10:01.305 --> 00:10:04.950
>> So then you build
out this database of

00:10:04.950 --> 00:10:06.600
understanding of
all these spaces

00:10:06.600 --> 00:10:08.730
inside and outside
as well, right?

00:10:08.730 --> 00:10:09.120
>> Yes.

00:10:09.120 --> 00:10:11.730
>> And somebody else
comes to the space.

00:10:11.730 --> 00:10:13.610
Indoors, how would this work?

00:10:13.610 --> 00:10:14.990
How would somebody know that

00:10:14.990 --> 00:10:17.190
this space has already
been mapped, for example?

00:10:17.190 --> 00:10:18.440
>> So what they do is,

00:10:18.440 --> 00:10:20.880
they launch whatever application
is they're going to use.

00:10:20.880 --> 00:10:25.245
And if that application
implements our mapping service,

00:10:25.245 --> 00:10:28.550
then they come in and
they've started up,

00:10:28.550 --> 00:10:29.810
our mapping service would start

00:10:29.810 --> 00:10:31.325
at the front end of their app.

00:10:31.325 --> 00:10:35.560
And once we get
enough reference points,

00:10:35.560 --> 00:10:36.660
we can say, "Oh,

00:10:36.660 --> 00:10:38.665
I recognize this based on

00:10:38.665 --> 00:10:41.945
the Wi-Fi signature that
you're connected to."

00:10:41.945 --> 00:10:44.160
And then some initial
point cloud data

00:10:44.160 --> 00:10:45.250
that we start gathering.

00:10:45.250 --> 00:10:46.600
>> I got you. So there's

00:10:46.600 --> 00:10:48.950
different methods to
understand the space.

00:10:48.950 --> 00:10:50.580
I look around
the hall that's here,

00:10:50.580 --> 00:10:52.695
and the chairs and
the features in here.

00:10:52.695 --> 00:10:56.485
So you look at the Wi-Fi
fingerprint of the area,

00:10:56.485 --> 00:10:59.470
and Spatial Mapping features.

00:10:59.470 --> 00:11:00.980
There's walls and certain areas,

00:11:00.980 --> 00:11:03.915
so the algorithms detect
that. That's very interesting.

00:11:03.915 --> 00:11:08.170
Do you suspect that places
are unique enough to do that,

00:11:08.170 --> 00:11:10.725
or will there be
some confusion initially?

00:11:10.725 --> 00:11:14.050
>> I think we're going to
have a mixture of both.

00:11:14.050 --> 00:11:15.380
There are some places
that you're going to go

00:11:15.380 --> 00:11:17.090
into that are absolutely

00:11:17.090 --> 00:11:18.680
unique and some people

00:11:18.680 --> 00:11:20.845
when you go into
a conference room,

00:11:20.845 --> 00:11:23.580
every conference room in
that office building is exactly

00:11:23.580 --> 00:11:24.800
the same and it's
going to be really

00:11:24.800 --> 00:11:26.425
hard to differentiate them.

00:11:26.425 --> 00:11:29.575
Folks say, "Hey, so HoloLens
can do Spatial Mapping?"

00:11:29.575 --> 00:11:31.900
What happens if I
go halfway round

00:11:31.900 --> 00:11:33.740
the world and I

00:11:33.740 --> 00:11:35.300
put it in a room that looks
just like the room I came in,

00:11:35.300 --> 00:11:36.380
and like well, it's
going to think you're in

00:11:36.380 --> 00:11:38.310
the same room halfway
round the world.

00:11:38.310 --> 00:11:40.210
That is I guess a common thing,

00:11:40.210 --> 00:11:41.220
and we might not have

00:11:41.220 --> 00:11:42.850
the information to be
able to get around that,

00:11:42.850 --> 00:11:45.270
especially if there's
no Wi-Fi signals

00:11:45.270 --> 00:11:46.870
around or some other difficult

00:11:46.870 --> 00:11:48.727
issues to solve on that front.

00:11:48.727 --> 00:11:51.075
How do you think we'll deal

00:11:51.075 --> 00:11:53.085
with different
device capabilities?

00:11:53.085 --> 00:11:56.205
For example, I'm on
a HoloLens and I'm rendering

00:11:56.205 --> 00:11:59.355
something at whatever poly level

00:11:59.355 --> 00:12:01.045
and maybe a hundred
thousand polygons.

00:12:01.045 --> 00:12:03.085
Then, whereas some other device
that's maybe a little

00:12:03.085 --> 00:12:05.285
bit older and doesn't have
that same running capability,

00:12:05.285 --> 00:12:07.530
how do you think that
that would share across?

00:12:07.530 --> 00:12:09.080
>> That's going to be

00:12:09.080 --> 00:12:12.225
a matter of developers
building their experience

00:12:12.225 --> 00:12:14.465
with totally different levels of

00:12:14.465 --> 00:12:18.160
detail similar to
what you can do now.

00:12:18.160 --> 00:12:20.035
You can build
a game that works on

00:12:20.035 --> 00:12:24.505
PC and XBOX all the way
down to a mobile phone.

00:12:24.505 --> 00:12:26.650
You just have to make sure
that it's set up right so

00:12:26.650 --> 00:12:29.865
that it looks at the GPU
and the CPU and says, Hey

00:12:29.865 --> 00:12:31.025
>> This is what I'm looking at.

00:12:31.025 --> 00:12:32.415
>> This is what I'm capable of.

00:12:32.415 --> 00:12:35.185
This is what I'm
going to sacrifice,

00:12:35.185 --> 00:12:36.785
basically, for
a better experience.

00:12:36.785 --> 00:12:38.765
>> For the platform-
The platform

00:12:38.765 --> 00:12:40.095
that you guys are working
on then is really going

00:12:40.095 --> 00:12:41.255
to be understanding with

00:12:41.255 --> 00:12:44.550
the spatial mapping around
us, understanding space.

00:12:44.550 --> 00:12:46.615
Me, as a developer, I
would go in and say,

00:12:46.615 --> 00:12:48.610
'Okay, I understand where I am.

00:12:48.610 --> 00:12:50.055
I understand somebody else has

00:12:50.055 --> 00:12:52.130
instantiated a gremlin here.'

00:12:52.130 --> 00:12:54.095
My other application is going to

00:12:54.095 --> 00:12:56.480
clear that API and say,
'Hey, there's a gremlin.'

00:12:56.480 --> 00:12:59.340
It's up to the application
to say this 3D model.

00:12:59.340 --> 00:13:02.135
You already know
about it. If somebody

00:13:02.135 --> 00:13:03.645
else is running
a different application

00:13:03.645 --> 00:13:04.565
in the same space,

00:13:04.565 --> 00:13:06.015
there is a separation
there, right?

00:13:06.015 --> 00:13:09.070
>> Interesting. What's
some interesting used cases

00:13:09.070 --> 00:13:11.945
that you foresee this type
of technology being used in?

00:13:11.945 --> 00:13:15.895
>> The further you
get into the future,

00:13:15.895 --> 00:13:17.790
the more interesting
the used cases get, right?

00:13:17.790 --> 00:13:19.245
The technology gets better,

00:13:19.245 --> 00:13:23.145
and gets smaller, more
lightweight, and better battery.

00:13:23.145 --> 00:13:24.505
Eventually, this is going to

00:13:24.505 --> 00:13:26.400
be a pair of glasses that just,

00:13:26.400 --> 00:13:28.605
you wear all the time, right?

00:13:28.605 --> 00:13:31.115
Once we have a reasonable
understanding of

00:13:31.115 --> 00:13:32.905
the world and
the objects in the world,

00:13:32.905 --> 00:13:35.870
you can actually start
to infer actions.

00:13:35.870 --> 00:13:38.935
Maybe the city of
San Francisco is on

00:13:38.935 --> 00:13:42.815
a kick for cleaning
up their parks,

00:13:42.815 --> 00:13:46.045
and they can infer that.

00:13:46.045 --> 00:13:48.435
You reach down and grab
something off the ground,

00:13:48.435 --> 00:13:50.195
and you put it in a receptacle

00:13:50.195 --> 00:13:52.005
that they know is a trash can.

00:13:52.005 --> 00:13:54.660
Now, they can offer
some sort of reward base.

00:13:54.660 --> 00:13:55.865
>> Fascinating.

00:13:55.865 --> 00:13:56.670
>> To you.

00:13:56.670 --> 00:13:57.705
>> Interesting.

00:13:57.705 --> 00:14:01.815
>> Once we get
past maps and objects,

00:14:01.815 --> 00:14:03.585
we can start inferring actions.

00:14:03.585 --> 00:14:05.895
>> That's fascinating.
We can use AI in

00:14:05.895 --> 00:14:08.455
the future to understand
user actions, incentivize them,

00:14:08.455 --> 00:14:10.215
reward them with this

00:14:10.215 --> 00:14:13.365
opt-in behavior.
That's fascinating.

00:14:13.365 --> 00:14:15.505
I really like the idea that we

00:14:15.505 --> 00:14:17.635
can share experiences
right now or that

00:14:17.635 --> 00:14:18.605
we'll be able to
have a better way

00:14:18.605 --> 00:14:19.665
to share experiences between

00:14:19.665 --> 00:14:22.340
mobile devices and the HoloLens.

00:14:22.340 --> 00:14:23.505
Back to the HoloLens
here, because

00:14:23.505 --> 00:14:25.430
you've been so active
in the community,

00:14:25.430 --> 00:14:29.265
do you have a favorite kind
of application,

00:14:29.265 --> 00:14:32.255
application of a device
that you've seen so far?

00:14:32.255 --> 00:14:34.145
>> It's interesting,

00:14:34.145 --> 00:14:35.915
because I've been
playing with it for so

00:14:35.915 --> 00:14:37.295
long and I use the word

00:14:37.295 --> 00:14:41.095
'play' because sometimes
that's what it feels like.

00:14:41.095 --> 00:14:42.715
It's such amazing technology

00:14:42.715 --> 00:14:44.875
but I also forget
how amazing it is,

00:14:44.875 --> 00:14:46.075
because I made it every day.

00:14:46.075 --> 00:14:47.595
One of my favorite
things to do is put

00:14:47.595 --> 00:14:49.530
it on people who've
never used it before.

00:14:49.530 --> 00:14:52.245
All the time, I'm in coffee
shops, working or what not,

00:14:52.245 --> 00:14:54.715
people like, 'Hey, it used
to be like, Hey what's that?

00:14:54.715 --> 00:14:55.475
Now it's like, Hey,

00:14:55.475 --> 00:14:57.325
is that a HoloLens?'
We're getting further.

00:14:57.325 --> 00:14:59.265
>> People recognize
in devices now. Sure.

00:14:59.265 --> 00:15:01.145
>> People recognize
for what it is.

00:15:01.145 --> 00:15:02.915
I love to put it on
people and just show

00:15:02.915 --> 00:15:04.865
them, play some holograms.

00:15:04.865 --> 00:15:07.085
Whenever I go into a Starbucks
and I start working,

00:15:07.085 --> 00:15:08.755
the first thing I
do is I fire it up,

00:15:08.755 --> 00:15:10.065
and I go into the holograms app.

00:15:10.065 --> 00:15:11.785
I start putting
holograms around,

00:15:11.785 --> 00:15:13.105
because eventually
somebody's going

00:15:13.105 --> 00:15:14.145
to come up and want to see it.

00:15:14.145 --> 00:15:14.985
>> It's all set there.

00:15:14.985 --> 00:15:15.980
>> It's already set up.

00:15:15.980 --> 00:15:16.915
>> You've set the trap.

00:15:16.915 --> 00:15:20.295
>> Exactly. Polytours is

00:15:20.295 --> 00:15:23.050
a great experience
to draw people into,

00:15:23.050 --> 00:15:24.720
for if you haven't done it,

00:15:24.720 --> 00:15:27.885
it basically is
a 360 video that you're in,

00:15:27.885 --> 00:15:30.780
of either Rome or Machu Picchu.

00:15:30.780 --> 00:15:34.420
It's stunning that's what
they've done with it.

00:15:34.420 --> 00:15:36.835
Fragments, if you got
a good long time to

00:15:36.835 --> 00:15:38.775
go play it, it's
such a fun game.

00:15:38.775 --> 00:15:41.415
I remember the first
time I put it on.

00:15:41.415 --> 00:15:43.245
If you're not familiar with it,

00:15:43.245 --> 00:15:45.595
it's kind of a detective
game where you're

00:15:45.595 --> 00:15:47.835
the detective and you have

00:15:47.835 --> 00:15:49.715
virtual counterparts that kind

00:15:49.715 --> 00:15:51.185
of beam into your living space.

00:15:51.185 --> 00:15:52.755
At one point, somebody,

00:15:52.755 --> 00:15:56.745
one of the virtual
characters- I have part in

00:15:56.745 --> 00:15:59.035
my house and they put

00:15:59.035 --> 00:16:00.395
their hands up and
hike themselves

00:16:00.395 --> 00:16:01.755
up, and sat on the bar.

00:16:01.755 --> 00:16:03.370
When they did that, I was like,

00:16:03.370 --> 00:16:04.955
this is the most amazing
technology in the world.

00:16:04.955 --> 00:16:06.535
>> My mind has been blown.

00:16:06.535 --> 00:16:07.000
>> Right.

00:16:07.000 --> 00:16:08.750
>> I tell folks,

00:16:08.750 --> 00:16:10.545
Fragments is hands down,

00:16:10.545 --> 00:16:13.470
my favorite experience,
because it really integrates.

00:16:13.470 --> 00:16:14.965
It overlays a new floor.

00:16:14.965 --> 00:16:16.995
It's raining out, you
see rats running around,

00:16:16.995 --> 00:16:19.705
and these characters integrate
with your environment.

00:16:19.705 --> 00:16:20.855
We were talking before about,

00:16:20.855 --> 00:16:22.310
how do you understand space?

00:16:22.310 --> 00:16:25.455
Well, we can define
a suitable surface to

00:16:25.455 --> 00:16:26.955
the HoloLens and understand

00:16:26.955 --> 00:16:28.605
that we can take a 3D character
and sit down there.

00:16:28.605 --> 00:16:30.335
In fact, that that is here now,

00:16:30.335 --> 00:16:32.295
we can have 3D characters that

00:16:32.295 --> 00:16:34.825
integrate with our real
world, is phenomenal.

00:16:34.825 --> 00:16:36.815
I mean, it changes how we can

00:16:36.815 --> 00:16:39.795
interact with
applications entirely.

00:16:39.795 --> 00:16:43.325
For me, going forward with bots.

00:16:43.325 --> 00:16:45.985
We talk about conversation
as a platform is

00:16:45.985 --> 00:16:48.625
this new way of
apps to communicate

00:16:48.625 --> 00:16:50.635
with the users but
imagine when we have

00:16:50.635 --> 00:16:52.705
these 3D assistants that

00:16:52.705 --> 00:16:54.565
can be shared even
across spaces, too.

00:16:54.565 --> 00:16:56.655
Our context, aware and can

00:16:56.655 --> 00:16:58.915
hang out in our
living rooms and talk.

00:16:58.915 --> 00:17:00.545
I think it's going to
be a whole new frontier

00:17:00.545 --> 00:17:01.835
on there, and it's
super exciting.

00:17:01.835 --> 00:17:04.390
>> One of my favorite videos
that Mike put out

00:17:04.390 --> 00:17:06.875
is this experience where

00:17:06.875 --> 00:17:09.450
a woman is trying to
design a shoe store.

00:17:09.450 --> 00:17:11.170
She's using [inaudible]
do it and she's

00:17:11.170 --> 00:17:12.855
got a little bot that
hangs out with her.

00:17:12.855 --> 00:17:16.595
She brings her other co-workers
in from remote locations,

00:17:16.595 --> 00:17:19.570
then they all collaborate
on designing the space.

00:17:19.570 --> 00:17:21.365
It's interesting to
watch, because all

00:17:21.365 --> 00:17:25.430
that technology is there
in its own individual form.

00:17:25.430 --> 00:17:28.395
We just haven't tied it all
together yet and once we do,

00:17:28.395 --> 00:17:30.185
it's going to be amazing.

00:17:30.185 --> 00:17:32.545
>> We're talking
about that excitement

00:17:32.545 --> 00:17:33.735
when you put a device on

00:17:33.735 --> 00:17:37.085
somebody's head for
the first time in the same way.

00:17:37.085 --> 00:17:39.095
It is hands down, one of
my favorite things to do.

00:17:39.095 --> 00:17:40.935
A couple of weeks ago,
I had the opportunity

00:17:40.935 --> 00:17:43.440
to put a device on

00:17:43.440 --> 00:17:46.925
a 98-year-old World War II vet

00:17:46.925 --> 00:17:49.795
who was one of the
remaining Tuskegee airmen.

00:17:49.795 --> 00:17:51.435
We were in an aviation museum

00:17:51.435 --> 00:17:52.960
and he put a device on his head.

00:17:52.960 --> 00:17:54.395
He starts walking
around looking at

00:17:54.395 --> 00:17:56.215
holograms like,
'This is amazing.

00:17:56.215 --> 00:17:58.625
This just made my year, sire.'

00:17:58.625 --> 00:17:58.940
>> Right.

00:17:58.940 --> 00:18:00.045
>> It's phenomenal.

00:18:00.045 --> 00:18:01.420
It is one of the
best things about it.

00:18:01.420 --> 00:18:04.475
Well, Jesse, what kind of

00:18:04.475 --> 00:18:06.155
timeframe do you think
that we're looking

00:18:06.155 --> 00:18:08.280
at going forward
in this AR Cloud?

00:18:08.280 --> 00:18:12.905
>> Practicals getting
started with it now.

00:18:12.905 --> 00:18:15.035
We're about ready to release

00:18:15.035 --> 00:18:18.625
our STK out a little bit
later this year.

00:18:18.955 --> 00:18:21.445
The sooner we start building it,

00:18:21.445 --> 00:18:24.735
the sooner it will be
available in a wider form.

00:18:25.175 --> 00:18:27.955
It's one of those technologies
you can look at.

00:18:27.955 --> 00:18:29.345
You can say, 'This
is 10 years down

00:18:29.345 --> 00:18:30.835
the road' or you can
look at it and say,

00:18:30.835 --> 00:18:32.155
'We're going to start
on it right now and

00:18:32.155 --> 00:18:33.905
just do what we can.'

00:18:33.905 --> 00:18:36.175
Iterate on it and
make it better.

00:18:36.175 --> 00:18:37.690
That's what we've decide
we're going to do.

00:18:37.690 --> 00:18:40.225
>> How do folks find
more information about the STK?

00:18:40.225 --> 00:18:43.795
>> Go to our website,
experience.practicalvr.com.

00:18:43.795 --> 00:18:47.500
We can sign up for
the mailing list.

00:18:47.500 --> 00:18:48.815
We go a little bit
about the company.

00:18:48.815 --> 00:18:50.860
We also do analytics
for HoloLens,

00:18:50.860 --> 00:18:53.815
so you can find out about that.

00:18:53.815 --> 00:19:00.745
Follow me on Twitter,
@jbmcculloch, M-C-C-U-L-L-O-C-H.

00:19:00.745 --> 00:19:02.325
I'm more than happy to

00:19:02.325 --> 00:19:04.295
answer questions and
get people involved.

00:19:04.295 --> 00:19:06.515
>> Very cool. You've been
incredibly helpful in

00:19:06.515 --> 00:19:07.880
the HoloLens
develop instruction.

00:19:07.880 --> 00:19:09.065
How do people find that?

00:19:09.065 --> 00:19:12.140
>> The website's really long,

00:19:12.140 --> 00:19:14.095
message me on Twitter
and I'll send it to you.

00:19:14.095 --> 00:19:14.735
>> Okay.

00:19:14.735 --> 00:19:17.245
>> Very good. Well,
thank you very much.

00:19:17.245 --> 00:19:18.535
It's been a pleasure
having you today

00:19:18.535 --> 00:19:20.265
and I'm sure we'll talk
more mixed reality.

00:19:20.265 --> 00:19:20.985
>> Of course, thank you.

00:19:20.985 --> 00:19:22.785
>> Thank you very much.