Comment Feed for Channel 9 - Singularity Revisited

Re: Singularity Revisited

Sven Groot — Fri, 02 Dec 2005 01:01:36 GMT

I'd just like to point out the spelling error in the thread title...

posted by Sven Groot

Re: Singularity Revisited

DuNuNuBatman — Fri, 02 Dec 2005 01:01:55 GMT

I don't know why it is, but this thing is just cool...
Any chance you guys are going to open up a little bit of the source to the rest of the world? Or are you planning on making this into something?

posted by DuNuNuBatman

Re: Singularity Revisited

Charles — Fri, 02 Dec 2005 01:02:18 GMT

Thanks, Sven. Fixed...
C

posted by Charles

Re: Singularity Revisited

Manip — Fri, 02 Dec 2005 01:25:33 GMT

OMG!!! I haven't watched it yet.. But THANK YOU CHANNEL 9 TEAM!

You guys rock! No.1

I get WHY they don't want dynamic loading; but it is a shame, dynamic loading is amazingly powerful.. For one example plugins, how would you implement an extension mechanism for your programs without the ability to dynamically load?

Another thing; there is lots of discussion about compiler checks and with no JIT running what is from stopping someone from writing raw x86 (or whatever it is compiled to) and using pointers to gain admin rights on the system? In fact if you can't trust the running code what is stopping anyone from gaining greater privileges on the system or over-writing the kernel with their own one?

Or do you assume local code is safe to run on the system?

Edit: All of this was answered later in the video... Good questions Charles, thanks.

posted by Manip

Re: Singularity Revisited

Tonatiúh — Fri, 02 Dec 2005 01:51:16 GMT

What a wonderful welcome!... Thank you Charles and Singularity Project Team... I have to left my home about 3:00 a.m. (GMT-6) to meet my love for dinner and I just have arrived full of happiness... And what I see when I loged in to Channel 9...

Thank you again.

Tonatiúh

posted by Tonatiúh

Re: Singularity Revisited

Charles — Fri, 02 Dec 2005 02:03:56 GMT

You're welcome, Tonatiúh

Keep in mind that at the end of the video we did talk about a part 3

I hear you, the Singularity folks are doing some of the most innovative OS work in the world (though they'd never agree with that). The possible uses of Singularity, and what's learned from it, will take security and reliability to a new level some day...

Much thanks to the Singularity folks who took an hour out of their schedule to talk to Channel 9.

Glad your questions were answered in the video, Manip!

C

posted by Charles

Re: Singularity Revisited

jmacdonagh — Fri, 02 Dec 2005 02:35:27 GMT

Charles wrote:

You're welcome, Tonatiúh

Keep in mind that at the end of the video we did talk about a part 3

I hear you, the Singularity folks are doing some of the most innovative OS work in the world (though they'd never agree with that). The possible uses of Singularity, and what's learned from it, will take security and reliability to a new level some day...

Much thanks to the Singularity folks who took an hour out of their schedule to talk to Channel 9.

Glad your questions were answered in the video, Manip!

C

Wow, an hour of awesome content and there's still so much more to talk about!

Any ideas when you're going to film the next installment?

posted by jmacdonagh

Re: Singularity Revisited

amotif — Fri, 02 Dec 2005 02:45:32 GMT

Wow, nice vid, Charles. And nice question:

Charles wrote:

...of course on Windows I can run applications, right?

A few comments:

The file system run for benchmarks--was it FAT/FAT32/NTFS?

Closed processes: no dynamic code loading?! This seems like a weakness from the developer's perspective. E.g., does this prohibit the ability to load plug-ins? DLLs? Configuration-defined type loading? (Seems like a weak-ness...)

On choosing threads over event-driven architecture:
"You lose the structure of the code if you write it as a bunch of event handlers. You know, each handler makes sense, but you don’t see the big picture. You don't see the forest for the trees. And moreover there's a lot of experience in, you know, windowing code, for instance, that says that this is extremely difficult code for people to maintain, because the first person who wrote it may have had a clear picture in their mind but it goes away… and the next person coming along has no idea what it is."
~~Amen. But shouldn't good design and design artifacts (comments, documents, drawings, etc.) take care of some of that? Is that too much to ask of developers?~~

ExHeap: very interesting.

Any chance Bartok and stand-alone compiled-to-native-code apps will happen on XP/Vista in the next five years. (Gratuitous, feel-good question. )

posted by amotif

Re: Singularity Revisited

ploe — Fri, 02 Dec 2005 03:26:04 GMT

<3 the singularity videos

os research is just plain fascinating...my favorite CS class

posted by ploe

Re: Singularity Revisited

AIM48 — Fri, 02 Dec 2005 03:34:25 GMT

amotif wrote:

Any chance Bartok and stand-alone compiled-to-native-code apps will happen on XP/Vista in the next five years. (Gratuitous, feel-good question. )

NGEN ?

posted by AIM48

Re: Singularity Revisited

Wells — Fri, 02 Dec 2005 04:09:18 GMT

I'm 02:38 in.

Don't look at the camera!

I don't know why, but I just can't stand it.

edit: ah good, they've stopped

edit2: this is an excellent video

posted by Wells

Re: Singularity Revisited

Tonatiúh — Fri, 02 Dec 2005 06:43:44 GMT

Charles wrote:

You're welcome, Tonatiúh [... ... ...]

Wow!!!... Niners like sharing my rants and disappointments with them... From some days a go up to now, I was feeling quite guilty because the way I have engaged in some discussions... Indeed I was pity about my self, looking how my awareness have spoted in front of my eyes the faulty manner I have behaved in my last posts... Furtermore, I have thought that the last "Suspension idea from Minh have been implemented and applied to me"... So I comforted my heart telling to my self: "Well, at least you have not been banned; so you may get a lot of profit reading every one's posts and following those whorthwile links many Niners are so kind to share here"... I know for sure, I shouldn't endure to lost those.

Thank you Charles
Thank you Singularity Team
Thank you Eagle

Thank you Niners... I beg pardon if any body have been offended by me, specially tou you: Leighsword, mot256 and Cornelius Ellsonpeter

Tonatiúh

posted by Tonatiúh

Re: Singularity Revisited

sakisp — Fri, 02 Dec 2005 10:55:34 GMT

Great video !!!
Thanks to Singularity team, there is still some hope for managed code. Even though we have to wait a few years for a true managed -> native compiler.

Thank you all for the info.

posted by sakisp

Re: Singularity Revisited

mwirth — Fri, 02 Dec 2005 11:32:59 GMT

first of all: a very worthy installment in the going deep series!! this one is a really interesting and fun to watch video. nice work, charles!

I always can't help it but think of who might be interested in using singularity (or some deviation) in their own domain.. does a system structured like singularity lend itself to applications that are mission critical? i'm especially thinking of very specialized devices in avionics, healthcare etc; systems with a very special purpose that have a very limited set of hardware devices they' are running on.
i suppose the mix of fully verifiable code on the one hand and the rest of the code being unsafe but trusted might have a great appeal to many people in the need of specialized mission critical systems. has there been any interest so far? maybe the ADA standardizing organizations considering c# in the future? ( i don't mean to bash ADA. i do like it quite a bit myself. )

anyways, great video and i'm really looking forward to seeing part 3!

posted by mwirth

Re: Singularity Revisited

ALyman — Fri, 02 Dec 2005 15:37:04 GMT

Wow, great video. Cant wait for #3 -- gotta see a demo of it. Gotta see more content in the (unfortunatly named) "Going Deep" show.

With the new detail on Channels and the ExHeap, I can imagine that a owner-draw GUI might have some preformance issues running on this. With ExHeap buffers (as mentioned durring the whiteboard when talking about how disk chunks would be passed) changing ownership when passed as a message, you would have to have a double-buffer thing going on, to make sure the video system had a screen buffer it owned whenever it had a vsync.

I'd love to play around with that kind of thing -- any ideas from the Singularity Team [when/if] they might release [binaries/code] to the outside world and let us do some [research/play] with it?

posted by ALyman

Re: Singularity Revisited

amotif — Fri, 02 Dec 2005 16:08:39 GMT

AIM48 wrote:

amotif wrote:

Any chance Bartok and stand-alone compiled-to-native-code apps will happen on XP/Vista in the next five years. (Gratuitous, feel-good question. )

NGEN ?

It's not the same, you're still dependent on the presence of the framework.

posted by amotif

Re: Singularity Revisited

Cornelius Ellsonpeter — Fri, 02 Dec 2005 17:12:17 GMT

Tonatiúh wrote:

Charles wrote: You're welcome, Tonatiúh [... ... ...]

Wow!!!... Niners like sharing my rants and disappointments with them... From some days a go up to now, I was feeling quite guilty because the way I have engaged in some discussions... Indeed I was pity about my self, looking how my awareness have spoted in front of my eyes the faulty manner I have behaved in my last posts... Furtermore, I have thought that the last "Suspension idea from Minh have been implemented and applied to me"... So I comforted my heart telling to my self: "Well, at least you have not been banned; so you may get a lot of profit reading every one's posts and following those whorthwile links many Niners are so kind to share here"... I know for sure, I shouldn't endure to lost those.

Thank you Charles
Thank you Singularity Team
Thank you Eagle

Thank you Niners... I beg pardon if any body have been offended by me, specially tou you: Leighsword, mot256 and Cornelius Ellsonpeter

Tonatiúh

I can't think of any reason why you would be banned. Has anybody been truly banned from this site anyway?

I've never been offended by your posts. Although I have thought about taking out a window or two with a piece of flying furniture.

posted by Cornelius Ellsonpeter

Re: Singularity Revisited

JChung2006 — Fri, 02 Dec 2005 17:31:21 GMT

Cornelius Ellsonpeter wrote:

I can't think of any reason why you would be banned. Has anybody been truly banned from this site anyway?

orbit86 comes to mind.

posted by JChung2006

Re: Singularity Revisited

barlo_mung — Fri, 02 Dec 2005 19:18:19 GMT

Wow. Great stuff.

Bring on #3!

posted by barlo_mung

Re: Singularity Revisited

piersh — Fri, 02 Dec 2005 19:37:53 GMT

very interesting stuff.

i have a question, though: what was the reason for choosing to have a separate GC for each SIP? it would seem to me that in a fully trusted environment like this you could have a single GC for the whole system, integrated with the page manager, would eliminate the need for the exchange heap. memory in one SIP would automatically be 'available' to another process, the consistency of the channel being checked by the verifiable 'handing-off' mechanism you described.

posted by piersh

Re: Singularity Revisited

IRenderable — Sat, 03 Dec 2005 03:54:56 GMT

These guys are really really smart.

posted by IRenderable

Re: Singularity Revisited

piersh — Sat, 03 Dec 2005 04:09:46 GMT

(dupe)

posted by piersh

Re: Singularity Revisited

Tonatiúh — Sat, 03 Dec 2005 05:35:07 GMT

Cornelius Ellsonpeter wrote:

[...] I can't think of any reason why you would be banned. Has anybody been truly banned from this site anyway?

I've never been offended by your posts. Although I have thought about taking out a window or two with a piece of flying furniture.

I haven't looked at this thread from yesterday, so I thank you quite later because the kindiness of you response... Really, I have gone scared when you have said: "peace out"... Nice to meet you time to time at least.

As for those windows... Please, throw to me only one... That capable of enabling me to stand stared behind... Looking the road ahead, far away over the horizon... Anyhow, if it is an stone caming from your hand, be sure I will happily recive it in the middle of my chest over my heart.

Tonatiúh

posted by Tonatiúh

Re: Singularity Revisited

David Tarditi — Sat, 03 Dec 2005 06:31:00 GMT

Having a separate GC per SIP lets you choose the an appropriate garbage collector for each application. For example, some GCs are real-time (they have low pause times), whereas other GCs emphasize throughput, at the expense of higher pause times. You might want to use a real-time collector for streaming video and a GC with higher throughput for a compute-intensive application.

In addition, it allows each process to be separately garbage collected , increasing the scalability and robustness of the system. For example, you could run multiple garbage collections at the same time. You could also avoid a denial of service attack where someone is allocating lots of data, causing other processes to slow down because one garbage collector for the whole system can't keep up.

Finally, it simplifies tracking resource usage and reclaiming resources when a process ends.

posted by David Tarditi

Re: Singularity Revisited

Tonatiúh — Sat, 03 Dec 2005 06:40:05 GMT

piersh wrote:

very interesting stuff.

i have a question, though: what was the reason for choosing to have a separate GC for each SIP? it would seem to me that in a fully trusted environment like this you could have a single GC for the whole system, integrated with the page manager, would eliminate the need for the exchange heap. memory in one SIP would automatically be 'available' to another process, the consistency of the channel being checked by the verifiable 'handing-off' mechanism you described.

Please, don't take me as any kind of literated guy about this matter... But as far as I've understood from the documents available at the Singularity Project site, Each SIP has its own GC because the goal of making Singularity an highly reliable OS... That is, and again, as far as I understood: When a process is runing in a SIP (aka Software Isolated Process), it is totally isolated in its own space and time frames; by design definition, it can't share its space or even its timming amongst the kernel or other SIPs... So if it becomes unstable or corrupted in any manner, or there is the need to shoot down it because any kind of malfunctioning, there will not be left more unrecovered memory than the still not released by the affected process, and that, only for the time the kernel gets aware of and empties the SIP... OS corruption and unstability arises mainly beacuse the resonance of a chain effect of non-released memory.

I hope this may help.

Tonatiúh

EDIT: Resonance of a chain effect of non-released memory is only posible in an OS running with a single GC... I wonder after observing the way Windows XP SP2 (under 256 KB RAM) behaves in serendipitous scenarios prone for unstability, as many of those I am quite used to set up in my daily work on design, programming and research (keeping several heavy applications opened up to the end of the day and not few times thru a three or four days span, without any problems); if such version wasn't a worthwile feed for the design requirements of the Singularity Project, as for to the point I've reached on the Spec# specification papers, there are many underlaying paradigms whose are, conceptually, very similar.

posted by Tonatiúh

Re: Singularity Revisited

Tonatiúh — Sat, 03 Dec 2005 06:50:31 GMT

Thank you David... I got blushy when read your name and suddenly became aware that you are one of the Singularity Project researchers... I've enjoyed this interview and the papers I have studied untill now.

Care tell me who of the four researchers in the video are you?... Perhaps you want spot me your shirt's color... That will suffice.

Tonatiúh

posted by Tonatiúh

Re: Singularity Revisited

David Tarditi — Sat, 03 Dec 2005 07:12:21 GMT

I'm the person on the far right, wearing the dark long-sleeve shirt. Manuel is sitting to the left, then Galen, then Jim.

posted by David Tarditi

Re: Singularity Revisited

Tonatiúh — Sat, 03 Dec 2005 11:50:31 GMT

David Tarditi wrote:

I'm the person on the far right, wearing the dark long-sleeve shirt. Manuel is sitting to the left, then Galen, then Jim.

Thank you a lot for such a rapid reply to a very superficial and personal question, derived only from my inability to hear English... Every time I've started the video, I've lost your names at the very presentation of your selves.

David Tarditi wrote:

[...] You might want to use a real-time collector for streaming video and a GC with higher throughput for a compute-intensive application. [...]

From the above quoted fragment, I understand that there is a mechanism at the implementation side as for tunning the timer of a GC... If it is true, it is there one of the discrete kind (i.e.: a fixed set of types amongst whose the implementor of the process targeted to run inside a SIP may choose), and/or it is there one of the opposite (i.e.: a value between a given range, which could be passed, or settled before calling, to the appropriate GC's method for timing)... Furthermore, I wonder if it is there, in addiition to one or both of the aforementioned types of mechanisms, one of an automatic type; as I think the GC timer could be tied to a relevant SIP runtime feature, which shall adjust cybernetically the former (i.e.: dynamically, as in a file's downloaded amount counter, by using as the GC's cybernetic feedback feature, the frecuency of the memory chunks left unused by the process at the SIP, been collected by the GC itself).

Makes this any sense?

Tonatiúh

posted by Tonatiúh

Re: Singularity Revisited

scarz — Sun, 04 Dec 2005 06:37:34 GMT

Thanks for doing these interviews guys, I am really enjoying them. As a CS student I haven’t gotten to learn much about where we are going with future OS’s, only where we've been. This work is extremely intriguing and makes me confident I'm in the right field (I mean come on - an OS research project has me excited). I can't wait to see part 3 of the video. My hopes for this project would be to push it as the successor to Vista. Cut the cord with legacy systems once and for all. With the new Intel and AMD virtualization technology coming out we can do some awesome stuff with virtualization, so why not run all our legacy apps (dos, 95, 98, XP, etc.) in a Virtual PC under Singularity. It seems to me that the transitioning from Vista to something as drastic as Singularity would not be as hard as in the past with this kind of technology. Most of the important applications would be rewritten in safe code within a few years, and anything not worth rewriting can just be run under the virtual machine. Anyways I know that’s not the point of the project I just hope people see the value in the system your building. Keep it up guys; I’ll be following your work.

posted by scarz

Re: Singularity Revisited

staceyw — Mon, 05 Dec 2005 16:44:38 GMT

Thanks again Charles. Was waiting for this one. Can't wait for #3. Maybe a quaterly installment could be put on the agenda. Maybe you could do a whole show on Transactional Memory locks and invariants, and compare/contrast with traditional lock symantics.

I do have a question. You keep asking "why threads" as if you know of some alternative method. TMK, a process always has at least one thread to get anything done. So if you just use process like old unix systems did (before PThreads), your back to forking and issues with that. Threads are great from a logic flow perspective, but the locks for shared resources become the issue. This transactional invariant memory deal would seem to really help here. As well as any kind of compile time and/or run time tools that can help verify correctness of syncronization in our programs.

It would be cool to have a simple SQL server SIP (naturally managed) with all managed types as proof of concept. Layered ontop of the sql api is the TSQL block and another CLR block that would allow any new SQL type language to directly call down to SQL spi without overhead of the ADO layer - so it would be a peer to the TSQL block. Then you could have a MSH Provider block as another peer. So you could "CD" into any instance of SQL and CD into databases and tables to list and manage table data.

Love it! Cheers.
--
William Stacey [MVP]

posted by staceyw

Re: Singularity Revisited

kidzi — Mon, 05 Dec 2005 18:01:29 GMT

scarz wrote:

Thanks for doing these interviews guys, I am really enjoying them. As a CS student I haven’t gotten to learn much about where we are going with future OS’s, only where we've been. This work is extremely intriguing and makes me confident I'm in the right field (I mean come on - an OS research project has me excited). I can't wait to see part 3 of the video. My hopes for this project would be to push it as the successor to Vista. Cut the cord with legacy systems once and for all. With the new Intel and AMD virtualization technology coming out we can do some awesome stuff with virtualization, so why not run all our legacy apps (dos, 95, 98, XP, etc.) in a Virtual PC under Singularity. It seems to me that the transitioning from Vista to something as drastic as Singularity would not be as hard as in the past with this kind of technology. Most of the important applications would be rewritten in safe code within a few years, and anything not worth rewriting can just be run under the virtual machine. Anyways I know that’s not the point of the project I just hope people see the value in the system your building. Keep it up guys; I’ll be following your work.

Since its still a reserch project, you probably have a few years before its production ready Even then, I'd think if it does eventually make it somewhere, it would first need to start in an area where the software can be controlled: Embedded, xbox, and maybe later pocket software. After it gains traction and vs2010 provides building things natively, maybe we'll see something. But I find it unlikely it would be on the desktop within the immediate future with MS's committment to backwards support without having a whole native API programming model on top of singularity.

posted by kidzi

Re: Singularity Revisited

reinux — Tue, 06 Dec 2005 01:30:30 GMT

Nice!

I'm curious... would Monad run on this, or would cmdlets cause a problem?

posted by reinux

Re: Singularity Revisited

reinux — Tue, 06 Dec 2005 02:34:33 GMT

Man, I'd so love to see my web server running on Singularity

Heck, it'd even be worth Spec#ing the whole thing if I could see that happen.

posted by reinux

Re: Singularity Revisited

jachymko — Tue, 06 Dec 2005 03:19:20 GMT

reinux wrote:

Nice!

I'm curious... would Monad run on this, or would cmdlets cause a problem?

no, you would have to rewrite at least the win32 console host.. and the runspace probably too, etc...

posted by jachymko

Re: Singularity Revisited

pacowski — Tue, 06 Dec 2005 03:35:04 GMT

I thought the exchange heap was very neat, but I was wondering how a particular buffer can be marked to be within the exchange heap. Is there a particular method for marking buffers to be used within a channel? There was a description of the contract that is used to communicate across a channel with specific message formats. I was wondering if in the next video someone could address how the exchange heap and channels interract with two communicating processes. More specifically, is there anything special that needs to be done from a programming stand point or is it all taken care of in the message sending code?

Thank you.

posted by pacowski

Re: Singularity Revisited

staceyw — Tue, 06 Dec 2005 03:48:11 GMT

"no, you would have to rewrite at least the win32 console host.. and the runspace probably too, etc..."

Actually MSH would be perfect as msh is written in managed code. Naturally they would need to write a SIP managed console and port the msh console calls to the new console abstraction. Actually, I would be very suprized if they don't have this working already or at least working in some way for their research purposes.

--wjs

posted by staceyw

Re: Singularity Revisited

reinux — Tue, 06 Dec 2005 04:14:58 GMT

staceyw wrote:

"no, you would have to rewrite at least the win32 console host.. and the runspace probably too, etc..."
Actually, I would be very suprized if they don't have this working already or at least working in some way for their research purposes.

They did mention that dynamically loading code is a no-no in Singularity... MSH scripted cmdlets are interpreted (I think) so those would work fine, but I bet the compiled ones wouldn't work, unless I'm misunderstanding what they said.

posted by reinux

Re: Singularity Revisited

staceyw — Tue, 06 Dec 2005 04:32:21 GMT

"They did mention that dynamically loading code is a no-no in Singularity... MSH scripted cmdlets are interpreted (I think) so those would work fine, but I bet the compiled ones wouldn't work, unless I'm misunderstanding what they said."

But MSH is not dynamic from the OS SIP perspective. MSH.exe and its dlls are compiled and verified like any SIP. It happens to be able to take a script and run it. But, I think, this is not the same as the process loading dynamic code at runtime. This is the script driving verified logic in the MSH exe. So the script could not drive anything that was already not verified. This is really not much different then your exe loading a config file and running different logic based on what is in the xml file - similar idea I think. Running sql scripts in a SQL SIP is the same kind of idea.

Compiling cmdlets may be a different story as msh needs to dynamically load those. Not sure what the story there would be, but they would somehow need to support that kind of thing. Maybe things like MSH SIP would get special handling or something or there would be some way to verify the cmdlet dll so it could be trusted and loaded. Not sure, but a good question and I would be interested in this as well.
--wjs

posted by staceyw

Re: Singularity Revisited

kentcb — Wed, 07 Dec 2005 11:01:17 GMT

Thanks for such an interesting video, and kudos to the singularity team for their outstanding effort so far (and their awesome ability to explain their work). Looking forward to part III.

One thing I've been thinking a lot about lately is how useful it might be to have a CPU that understands IL. In other words, a specific CLR version is integrated directly into hardware instead of being a software abstraction.

I'm not sure whether this is even feasible but it seems to me that it would have a number of benefits:

Security. The security measures currently enforced by a software layer would instead be enforced at the lowest level - in the CPU itself.
Performance. I'm no expert in hardware but I imagine that performance could be greatly improved (not that it's bad now).

Essentially, unsafe code would be non-existant. This is kind of what you're trying to enforce with singularity but at an even lower level. I can imagine such a CPU being really useful in, for example, mobile phones.

Anyway, just ranting . . . thanks again,
Kent

posted by kentcb

Re: Singularity Revisited

dhi — Wed, 07 Dec 2005 18:13:46 GMT

In the second video, I saw that you are doing message passing over channels and run everything in its own SIP.
I did a lot of programming for µnOS which is an OO OS written in C++ including a GC.
It also uses channels for message passing (without message contracts) just like QNX does.
So if every driver runs in it's own SIP, can I write a SIP by myself sending messages to the NIC SIP
to tell it to send raw ethernet packets to the net?
Or has the TCP/IP SIP exclusive rights to communicate to the NIC SIP?
In µnOS we dicided to compile device drivers to DLLs.
So the network service, which is a seperate process in user mode creates an instance of the device driver object(s)
in its own process space and uses a well-defined interface to access these driver object.
Only the network service has a channel over which the other applications can create and use network connections.

How about modularity and the extension of a certain system service?
Let's say your TCP/IP SIP currently supports IP, ICMP, ARP, TCP and UDP.
How would I extend it to support SCTP as well?
Can I write an extension module (e. g. in form of a C# class) that is loaded by the TCP/IP SIP at startup to support SCTP?
Or do I have to write the extension in form of another SIP?

I also saw that you are doing process creation, channel management and security inside the microkernel.
Did you thaught of implementing a SIP in user mode for doing stuff like that?
E. g. the QNX process manager does some of these things in user mode.

At all, you did a great job with Singularity!
I hope Microsoft stays tuned with it!

posted by dhi

Re: Singularity Revisited

Manuel Fahndrich — Fri, 09 Dec 2005 22:37:19 GMT

pacowski wrote:

I thought the exchange heap was very neat, but I was wondering how a particular buffer can be marked to be within the exchange heap. Is there a particular method for marking buffers to be used within a channel? There was a description of the contract that is used to communicate across a channel with specific message formats. I was wondering if in the next video someone could address how the exchange heap and channels interract with two communicating processes. More specifically, is there anything special that needs to be done from a programming stand point or is it all taken care of in the message sending code?

Thank you.

Since data in the exchange heap is completely separate from GC data and has different types, it is also allocated with a variation:

byte[] in ExHeap buf = new[ExHeap] byte[512];

would allocate a 512 byte buffer in the ExHeap. Given a message

message Data(byte[] in ExHeap buf);

and a channel endpoint e supporting this message, you would use

e.SendData(buf);

to transfer the buffer. Note that buffers are not associated with a particular channel. e.g., the receiver of the Data message above can turn around and send the buffer to whoever it wants to; or alternatively, it could free the buffer explicitly via:

delete buf;

The type system/static analysis makes sure that programs don't touch data in the ExHeap they don't own (e.g., after send or delete).

posted by Manuel Fahndrich

Re: Singularity Revisited

mluschas — Tue, 27 Dec 2005 19:02:55 GMT

Looking at this OS, it sortta seems like its attempting to go back to the old days with the mainframes. I wasnt alive then (im 19 and a CS student), but i've heard alot.
   I believe that IBM used to have something similar. Remember the old BASIC that was on a chip in the computer. You could buy the computer, but without that chip, it would do nothing. If using singularity as an OS, you were to redesign the hardware, you could do something of the same thing.
   Now, dont get me wrong, I like developing and I think it would be wrong to charge anyone that wants to develop on a machine a lot of money. But its an idea.
   Redesign the BIOS of a machine (probally have to have more than 512 bytes for the BIOS), to do the normal POST, but have it have a CLR in it. This would enable you then to write all of your code in a managed (SAFE) language. This would eliminate the need for any unsafe code in either the kernel or the bootloader. Yes you would need unsafe code to exist in the heart of the machine for the same reasons you do now, direct hardware access. However, if you have the BIOS able to allocate pages in this safe runtime enviroment, you would not need to have the kernel have unsafe code; same goes for the HAL.
   Then by doing this you would be able to sell a complete packaged solution, increasing profit as well as end user stability.
   Also, I like the idea of a transactional filesystem and page manager. I was actually thinking of creating that type of system. I was working with that idea when i came across the videos here. I thought it was interesting that I had thought up something and you guys were already trying it.
   Is there any chance of the singularity source being released anytime soon ? Or even Bartok (not so much the source, but the binaries for it). I do beleive there are many open ideas for it, just remember: Dont be greedy if you do a package solution, IBM failed and so will you if you charge too high of a price for development licenses.
The development licenses that I am speaking of, are not for the compiler, or the tools. You said all code is either "Trusted" or "Verified". Taking that to the next level of "Is this code signed" if so allow it to run, if its not, display a message and dont allow execution. Thats the type of license I'm refering to.
Oh btw, you guys wouldnt happen to have a job opening for a 19 yr old CS student, whos been programming since 5 would ya ? lol. Figured I'd atleast ask.

posted by mluschas

Re: Singularity Revisited

Steve411 — Mon, 09 Jan 2006 11:24:51 GMT

I bet its UI sucks more than linux's.

- Steve

posted by Steve411

Re: Singularity Revisited

Rockweiler — Sun, 05 Feb 2006 20:37:18 GMT

In the beginning of the video it is said that you use the Bartok Compiler to compile the Runtime System into a specialize form of an executable that also contains a runtime to run the Runtime for the rest of the system. How does that all work?

posted by Rockweiler

Re: Singularity Revisited

Calum — Tue, 18 Apr 2006 11:49:18 GMT

kentcb wrote:

Thanks for such an interesting video, and kudos to the singularity team for their outstanding effort so far (and their awesome ability to explain their work). Looking forward to part III.

One thing I've been thinking a lot about lately is how useful it might be to have a CPU that understands IL. In other words, a specific CLR version is integrated directly into hardware instead of being a software abstraction.

I'm not sure whether this is even feasible but it seems to me that it would have a number of benefits:

Security. The security measures currently enforced by a software layer would instead be enforced at the lowest level - in the CPU itself.
Performance. I'm no expert in hardware but I imagine that performance could be greatly improved (not that it's bad now).

Essentially, unsafe code would be non-existant. This is kind of what you're trying to enforce with singularity but at an even lower level. I can imagine such a CPU being really useful in, for example, mobile phones.

Anyway, just ranting . . . thanks again,
Kent

I believe that the Singularity guys are thinking of something along these lines, at least on some level. They mention the concept of "typed assembly language", which implies that there is essentially type information associated with the native code that the MSIL for programs is compiled to. This type information essentially allows one to have "safe" assembly language. I believe there is currently some work into incorporating this into hardware, but even if that does not happen, superimposing type information (and I'm not sure if this is possible in an effective way, but bear with me) onto the native (x86 or whatever) code for the system would allow the OS to check the code it is provided for safety, even though the code is precompiled. Since the code in a SIP cannot be changed, the type information could be stripped and the strong guarantee of safety provided by the check would stay intact.

Hope I'm not reiterating anything here...

I do love this project, when I stumbled upon it at first it was like something I had been thinking about myself, except with far, far more thought put into it. Good stuff.

posted by Calum

Re: Singularity Revisited

skyde — Fri, 30 Jun 2006 12:37:02 GMT

It is often said that c# is more type-safe than c++.
Does the type-safety is provided by the c# to msil conversion or the msil to x86 conversion.
Because if it is the msil to x86 conversion you can ensure the typesafety of c++ code that have been converted to msil.

To write high performance code in java you should write a librairi in C++ and call it from java and when c++ is not enough you write assembler in your c++ code.
This is because the java Jit-compiler dont optimize as much as c++ compiler, also when doinng ahead of time compiling.
you said you where using assembler in c# code.
So my question is, does the bartok ahead of time msil compiler do as much optimization as c++ compiler.

So i understand that you choose c# over c++ because it is a more high-level language giving more simpler code to read.
But you said that writing a device driver in visual basic can cause some problem because of is dynamic feature.
What do you mean .. (Bartok compiler can't convert msil code produced by VisualBasic to x86) or (the type-safety is hard to ensure in dynamic feature)
Is it possible to create a Ruby to msil compiler that keep the dynamic feature of ruby and can be converted to native x86 code like you do for c#.
This way you would have a highly object-oriented dynamic type-safe language

posted by skyde

Re: Singularity Revisited

rysus — Wed, 12 Mar 2008 02:36:25 GMT

BTW,where can i download it?

posted by rysus

Re: Singularity Revisited

beats — Sat, 06 Sep 2008 15:48:44 GMT

Is this video still available?

It does not seem to play or is available for download

Thanks

posted by beats