The story of the WorldWide Telescope
- Posted: Jun 20, 2008 at 5:18AM
Right click “Save as…”
The WorldWide Telescope was first shown to the public at TED 2008, in a joint presentation by project leader Curtis Wong, manager of Next Media Research for Microsoft, and Roy Gould, a science educator with the Harvard-Smithsonian Center for Astrophysics. In this interview they discuss how -- and why -- the WorldWide Telescope combines many sources of astronomical data and imagery to create a seamless view of the night sky.
CW: I was interested in astronomy as a kid, but when you grow up in Los Angeles, the odds of seeing the Milky Way are pretty slim. I think the only time it happened recently was during the quake when the whole city lost power.
It wasn't until much later that I actually got to see the Milky Way, and other objects I'd seen pictures of, and it was really quite a transformative experience. I always wanted to figure out how to recreate and share that experience.
Early on, in the 80s, I made a little HyperCard stack called MacTelescope, which was my attempt to create the experience of looking at the sky, and then -- if you could manage to see the Milky Way -- to zoom into a section where there are all these interesting globular clusters and nebulae, if you know where to look.
JU: So there was already the idea of taking people on a guided tour.
CW: Exactly. Later I moved from the Voyager company to a company called Continuum, a little think-tank organization started by Bill Gates. The company was thinking about authoring tools and media. The project I wanted to do was called John Dobson's Universe.
John Dobson was a physical chemist at UC Berkeley who got drafted to work on the Manhattan Project. He was there to do the chemistry for tuballoy, which was the code word for uranium. He became a conscientious objector, left the project -- which was pretty hard to do -- and became a Vedantan monk. Then he became interested in looking at the sky but, being a monk, he didn't have any money for a telescope.
He knew that San Francisco shipyards were sources of glass discs, but they were too thin. Conventional wisdom said that you need thick glass to be able to grind a mirror. But he defied wisdom and found a way to use round porthole glass. He also came up with an ingenious way of mounting the telescope, using cardboard concrete form tubes. His design is now one of the most common designs for telescope mounts in the world.
He also created an organization called San Francisco Sidewalk Astronomers, where people who have telescopes are encouraged to take them out into the public and show people what's up in the sky, and explain what's going on.
John spent a lifetime in national parks, and in San Francisco, talking about astronomy to the general public. He was really good at taking complex ideas and conveying them to the public.
I remember once when he was showing the Andromeda galaxy, there was a picture with the galaxy in the background which looks like a kind of fog, with a lot of stars in front, and he said: "The stars you're looking at in front are like raindrops on your window, looking at a distant cloud."
So anyway, we started that project, got about halfway, then other things happened and it got cancelled.
JU: It's a great story!
CW: So I've thought about astronomy for a long time. At Microsoft, I heard a talk by Jim Gray who was applying his database expertise to astronomy.
JU: I looked up a paper that Jim published with a guy from Johns Hopkins, Alex Szalay, and it's called The World-Wide Telescope, but interestingly, the title also includes the phrase: An Archetype for Online Science.
The idea is that, not just in astronomy but in all of science, we're getting to the point where there's less direct observation, and more collection and analysis of data on a really large scale, happening in ways that are computationally assisted, and also assisted by the collaborative properties of the Internet.
CW: Exactly. We're getting this deluge of data. The challenge then becomes how you process, how you utilize. Bringing computer technologies -- SQL, visualization -- can really help.
So Jim had written that paper with Alex in 2002, and he'd given a talk on some of the work he'd been doing with the Sloan Digital Sky Survey at that time.
JU: Yeah, I've listened to that talk, and I was hoping you could help me connect the dots between the work that was done there and the federated virtual observatory which, for him, became a case study in the use of XML web services to create an Internet telescope that was a federation of radio astronomy services.
CW: Right. Alex and several other research scientists came together to create the National Virtual Observatory, which establishes common protocols for accessing astronomical data and imagery.
When Jim told me they were starting that project, I told him I wanted to help. Part of the pitch was a PowerPoint I made about SkyServer that showed how you could embed that data and imagery in a virtual environment.
What they were thinking was that astronomers would be querying the database, and pulling out objects. But I thought that to make this an interesting educational resource, we would need to build an environment in which people could create and share stories about the objects, and could connect those stories to original source information.
Later he came back to me and said that the SkyServer data was released, and he wanted to redesign the website to make the data more accessible to the public. So I helped with that. SkyServer DR2 (data release 2) was the redesigned website. And I used that to make the case for building the learning environment that became WorldWide Telescope.
JU: Under the covers at SkyServer, there was a lot of work done to correlate observations from different sources of data.
CW: Exactly. Alex Szalay did a lot of that work, in collaboration with people from other universities.
JU: And that's the foundation for what we see now in WWT?
CW: That was sort of the first case. It was the Sloan Data, which is just the northern sky. Then at TechFest in 2006, we were working with Alex to take the SkyServer images -- he has an image server that'll give you an image from coordinates, and from that you can get data about other objects in the field of view. For our TechFest demo, our developer Jonathan Fay -- who had been fiddling around with building his own hierarchical multi-resolution tile-browsing engines -- put together an engine that would pull the images from SkyServer and assemble them into a mosaic that you could browse and zoom in and out of. That was the first manifestation.
Meanwhile, we'd gotten some interest from Harvard. Alyssa Goodman found an intern for us who was passionate about education, and she came to work with us in the fall of 2006.
And then in January, as I was emailing back and forth with Jim about our plans, he disappeared after sailing out in San Francisco bay to spread his mother's ashes.
So when TechFest came around again in 2007, we decided to rename it WorldWide Telescope in honor of Jim. We started building it in March. That summer was a big effort to secure image sets and data from lots of different sources, as well as building the engine and defining the authoring environment. We showed a rough prototype at the Astronomical Society of the Pacific meeting ... Roy, was that in Chicago?
CW: Roy, do you want to pick up the story from there?
RG: Sure. Let me rewind the tape on my end all the way back. I was smiling when Curtis told his childhood story about being interested in stars he couldn't see. I have the east coast version of that, looking up in New York City, hounding my parents for a telescope, and then when I finally got one, we went up on the roof and there was nothing to see.
JU: Just a little too much ambient light!
RG: Exactly. We thought we saw a star, but it was a plane.
My career has been in science education, and I was familiar with Curtis' work long before the WorldWide Telescope. When I heard his talk in Chicago, and at some subsequent conferences, I saw that it was addressing two of my passions. One is astronomy, what's out there in the night sky, and having a unified view of it. But also, as somebody who communicates science, I've always been interested in the learning interface. It's not just about using the resource, it's about learning from it.
It was clear there were lots of things that could be done, but it took a long time to see what all of them might be. It was only after I used it for a while that I realized what a great breakthrough it is.
First, there's the seamless experience of the night sky. It's true that all these images are accessible in principle, if you know where to find them, and of course many us in the field do that when we prepare curricula or museum exhibits or planetarium shows.
But when you do that, you see the universe in a disconnected way. Once you go on WorldWide Telescope, it's a different experience. It's like you could look up with perfect vision.
JU: And with complete contextual awareness.
RG: Exactly. And I think we take the night sky for granted. We don't realize how important it used to be. When we have floods in the midwest, we call them disasters. Well, disaster literally means against the stars. Catastrophe is the Greek for against the stars. Romeo and Juliet were the star-crossed lovers. It's all through literature, it's part of common speech.
Then you fast-forward to the modern day, and few of us have even seen the stars, let alone have that relationship to them. For me that's number one about WorldWide Telescope. It's really inviting us to take a long look at the night sky again.
Of course you can do that through the WorldWide Telescope, but then you can also look at the night sky from your back yard, or from a dark location, and have taht dual relationship. So you can both see the night sky and, in WWT, you can explore it.
JU: That's one of my favorite things to do. At night, of course, laptops create their own illumination. My first astronomy program ran under MS-DOS, amazingly enough, and I'd take my laptop from that era out in the backyard and use it as a guide. This is the latest and greatest incarnation of that tradition.
RG: And from the educational point of view, this pays enormous dividends. We've done research here about what students know when they graduate from high school about the night sky, and about the universe in general. From that, we know that the majority of high school students graduate placing the stars inside the orbit of Pluto.
JU: C'mon. Really?
RG: It's true. About 52 percent, and that's based on surveys of thousands of students in 37 states.
There are many reasons for that, but certainly one of them is this lack of good images of the sky. It's hard enough to see a picture of the solar system, let alone its context within the galaxy. That's one beautiful feature of WorldWide Telescope, you get a sense of where things are.
We also know that students think galaxies are closer than stars, because they tell us that stars are just point sources, and no matter what your magnification or telescope they remain points, so they must be very far away. Whereas galaxies, whatever they are, are big, and so they must be closer.
But if you go on WorldWide Telescope, and look at the Andromeda galaxy, the nearest big galaxy to us, and the furthest thing you can see with your naked eye, you get a physical feeling. You see all the stars in our Milky Way that are the veil of stars we look through, and you really get a sense that the Andromeda galaxy is vast and distant.
Another thing that's useful is the ability to look at the universe in different wavelengths of light. We see only the visible light our eyes can see, but that's like listening to one instrument in the orchestra.
You can download a Chandra image, an X-ray image of the sky, but what's that about? You can't really figure it out. Or one of the infrared images. In WorldWide Telescope, you can seamlessly crossfade back and forth between images taken at various wavelengths. So you see what's going on in the visible wavelengths, but you also see what's going on that's emitting these other forms of light. And that's important, because most of the action in the universe happens at wavelengths of light we can't see. The WorldWide Telescope automatically aligns these different images.
JU: Can you say a bit about how that's done?
CW: Yes, it was a challenge to register all these different surveys so you can do that kind of cross-comparison. There are some emerging standards from the National Virtual Observatory and others. The AVM (Astronomy Visualization Metadata) standard provides metatags at high precision within objects, to give exact position and scaling and orientation for images in the sky.
JU: Does that work by reprocessing existing survey data?
CW: Right. Generally the data exists. But for example, when Hubble makes these beautiful press-release images that they send out in color, the metadata is usually lost, because the images have been post-processed in Photoshop and other programs. So a lot of these images that we want to put out there for the public needed to have that metadata reintegrated. And sometimes they're composites of many Hubble images to create a large mosaic. So that composite image needs to have metadata put into it.
I think a source image is about 400 megabytes. So while it's technically in the public domain, they don't release it because it's way too much data for most people to download. They've released low-resolution versions, but we have the full-resolution image of the Crab Nebula, and other things, in WorldWide Telescope, because we can enable people to use the high-resolution images without having to download all of them.
JU: So part of it was going back to sources that were notionally available, but not practically available.
JU: But part of it is about emerging standardization of how these images are described.
CW: Yes. The whole NVO group is working toward common standards and common protocols for image metadata, so they can all be used by everybody. There's AVM, part of the VAMP project -- Virtual Astronomy Multimedia Project -- leading the charge for annotation of imagery and other media in the context of the sky.
JU: And there are presumably many uses for those annotations.
CW: Yes. So we were one of the first guinea pigs for VAMP, and I think Google joined a bit later.
JU: So let's talk about the authoring aspect. I'm a huge fan of multimedia and audiovisual tools for educational and training purpose, and this is a wonderful example of that.
I was looking at what files get created when you author a slideshow in WorldWide Telescope, and it looks like the output is an XML file with coordinates and descriptions. To me that says two things.
First, it says that WorldWide Telescope presentations can be created using other tools, which is interesting.
It also says that presentations created inside WWT could potentially be played elsewhere, in other environments.
CW: Exactly. The whole idea with the authoring, and these guided experinces, is...let's go back to the educational intention. I spent a lot of years building instructional learning, where you bring in experts to tell you about subjects, in context, but also the self-directed discovery aspect of learning. Then there's a third part I wanted to bring in, constructive learning. There's always a duality between instructive and constructive learning.
JU: What do you mean by constructive?
CW: Basically, learning by doing. Where kids who don't know much about astronomy can take tours from experts, and be taken to unfamiliar places, but then pause those tours and explore on their own. In WorldWide Telescope you can always pause the tour, like stopping a tour bus and getting off to look around. At that point you can right-click to get more information, you can zoom into places that the tour didn't deeply explore because it had to keep moving, you can see other objects that are in the neighborhood.
If you notice down below, in the context menu, as you get to different objects you can not only see that object in different wavelengths, but you can also see other tours that intersect with that object.
The goal is that as you start to see more and more guided tours within this space, if you think of objects as nodes or intersections, there are more and more opportunities to cross over from one tour to another. Eventually we might see a kind of hypermedia web of learning where instead of hyperlinking among words, we're linking among stories and paths and ideas.
JU: And that'll include things I make for myself. If I make a narrative about a part of the sky, then the context surrounding the area I'm interested in will be available when someone else plays back that tour.
CW: Exactly. If you take a tour about stellar evolution, and learn about how stars get formed from nebulae, and you get to the planetary nebula stage, you might say, wow, those are really pretty, what's going on there? Then you might intersect with a tour about different planetary nebulae, where you dive deep into that category. Then you might find a tour just about the Ring Nebula, or the Helix Nebula.
Or conversely you may come across it in a different way. You're browsing the sky and you come across the Ring Nebula or the Helix Nebula, and you may say, what are these things? And you can see other things that relate categorically to them, which would then intersect with explanations of how planetary nebulae fit into the grand scheme of the origin of all the elements.
JU: Roy, what tours are you working on?
RG: We're working on two. One is a tour of black holes, in conjunction with a traveling exhibit the Smithsonian is producing.
Another is a tour of alien solar systems and their exoplanets, as they're known. There are more than 300 stars known to have planets orbiting them. Using a small educational telescope -- we have a network of five of these, they're called micro-observatories -- students in middle school and high school can take their own images and study these exoplanets. They can actually characterize them in a surprising amount of detail. They can figure out how large they are, how far away they are from their stars,
What's more, we're using the tour of exoplanets to get teachers who have never used the micro-observatory telescopes in this particular curriculum. You know, you can make a brochure, and send them some images of the night sky, but there's nothing more exciting than having a tour that gives you a sense of the context, the depth of the sky, where things are.
JU: Do you think there will be a citizen science aspect to this?
RG: Absolutely. I think that's going to be a major use. Astronomy is probably unique among all the sciences, in there are more amateurs than professionals. But this is a third category. You've got the amateurs, you've got the professionals, but now WorldWide Telescope makes possible the blossoming of citizen science. Especially given the flood of images we're going to have in the next few years, more than researchers can ever look at. You'll have images that have never been seen by humans, and that opens up a huge possibility.
JU: So part of it's about getting more eyeballs on this flood of imagery. Potentially another part is citizen-driven analysis of data. I wonder if SkyQuery will become part of the suite, so people can start to ask questions, like how many fast-moving objects are in this part of the sky, which is one of the queries Jim Gray mentioned in his 2002 paper.
RG: Yes, I'd love to see that. There are several ways the public can contribute. One is to look for things, and make serendipitous discoveries. With some of the NASA solar missions, for example, where we have so many images of the sun coming back, you can see comets that were never seen before as they get close to the sun. And ordinary citizens have discovered comets that nobody knew existed.
But to me the most important thing is what you just alluded to: Asking questions that nobody had thought to ask, even the professionals.
So for example, what's the volume of a black hole? How big is it inside? If you go on the web and look at the standard references, you'll find answers all over the place, all at odds with one another.
There are questions that researchers just haven't gotten around to asking, that many of the public will ask, and we don't know what those are yet.
In a way, although there's all this wonderful technology in the WorldWide Telescope, but in a sense it's the modern incarnation of a campfire that you sit around and trade stories. Our organization has telescopes in Australia and Chile and elsewhere, and when I go to those countries, I find that the native cultures have all sat around campfires and developed incredible stories about the night sky.
JU: So how does the collaboration work? I've made a little slideshow, it's stored as a file on my computer, how do I share that?
CW: We're trying to encourage the development of communities. Sky and Telescope is forming one, Astronomy magazine is forming one, Meade -- a telescope maker -- is forming one.
JU: So the unit of sharing is the WWT file which gets created when you make a slideshow. It's a bundle of XML and images of thumbnails and maybe audio if there's voiceover. In a lot of cases, those will want to live out on the web where people can link to them. If I post that file, and somebody clicks on it, and WorldWide Telescope is installed, then it'll launch and play the slideshow when you click on the file?
CW: Yes. I was talking to a storyteller from a local Snoqualmish tribe, a lot of their stories happen to be about the sky. I wanted to try to capture some of those as examples of what you can do.
By the way, if you're playing the tour, you can pause and go into edit mode, and it's all open. You can change destinations, you can drop in your own audio narration, music, text, and images.
JU: So you've forked the thing you've downloaded, and at that point you can...
CW: ... put your own interpretation on it, exactly. It's just like View Source. Except easier, because you don't know have to program in HTML.
JU: Yeah, it's quite straightforward.
Do you think that there's a need -- I suspect that there is -- for some sort of universal player that wouldn't require the full application, and wouldn't even require Windows, but would just be a way for anybody to play these things?
CW: Absolutely. That's a really good idea.
So by the way, I want to highlight one citizen science story for you. It relates to Galaxy Zoo, a website that allows the public to help the Sloan digital sky survey catalog and tag the hundreds of millions of galaxies that were covered in the various data releases. In one case, a teacher from Amsterdam was looking at a galaxy and it looked really blue. She reported that to Galaxy Zoo, and they looked at it, and it was something they'd never seen before. So they retargeted the Very Large Array Radio Telescope to take a look at it. And based on those results, they've now secured Hubble time to study that galaxy in great detail.
That's a case where putting the data out there, letting the public look at it, dividing up the sky, and having that feedback mechanism can really advance science. Because when you think about it, as we start to get telescopes like LSST and PanSTARS and these other large telescopes that will generate many terabytes of imagery and data every night, it's going to be impossible for any one person or group to see what's up there. Image recognition's good, but nothing's as good as the human eye and human brain.
JU: Some guidance on what to look for is really useful. If someone's found something interesting, and that justifies spending the resources to take a closer look, that's beautiful. That's exactly how things ought to work.
CW: Right. And I think a lot of these telescope projects are thinking, how do we make this much data available to people? And how do we make that accessible in a simple way? So they've had conversations with us about how WorldWide Telescope might be able to help.
Also, NASA is very interested in how realtime data feeds from them would enable the public -- at the same time -- to have access to mission data.
JU: You start to think about what's possible, and you quickly realize there's an infinite number of interesting possibilities. It'll be great to see how this plays out over the next few years. Thanks!