Comment Feed for Singularity: A research OS written in C# (Going Deep on Channel 9)

Re: Singularity: A research OS written in C#

louis vuitton — Fri, 07 Aug 2009 08:41:29 GMT

Louis Vuitton Louis Vuitton Replica Louis Vuitton outlet Louis Vuitton Louis Vuitton Replica Louis Vuitton Outlet

Re: Singularity: A research OS written in C#

PhilP — Tue, 18 Dec 2007 12:15:09 GMT

Most impressive - quick and stable. I like it a lot.

Re: Singularity: A research OS written in C#

Yankee — Sun, 16 Dec 2007 19:12:15 GMT

OK, the question of all questions :)

Is there any chance to get our hands on Serenity in the future? Will it be the foundation of the next generation Windows OS?

Re: Singularity: A research OS written in C#

magicalclick — Tue, 12 Jun 2007 04:44:19 GMT

Hello guys, just got this interview link from MSDN forum. The project is quite interesting. I am only undergrad degree, so this is off my league. But I have some concern about the flexibility of the OS.

What kind of restrictions will be intriduced with managed envirnment? I know there will be restrictions, but is there a list of possible restrictions? Maybe an example. And will there be work around? How hard is it to introduce a new hardware device driver to the OS?

I know managed envirnment is cool, but as a programmer used both C++ and Java, I would rather choose C++ for its flexibility and expressiveness over weak Java. I have looked into C# a bit, it is more flexible than Java, so it is nice, but it is still not as flexible as C++. I am not sure how many people will be turned off by its restrictions.

I am more focused on the impact to 3rd party software providers. About scurity, I don't care about the virus because I use PC Backup, but I care about worns and trojens. Hopefully you guys can come up with a better solution integrated in the OS.

Best of luck and keep up the good work.

Re: Singularity: A research OS written in C#

RichardRudek — Sun, 28 May 2006 02:03:16 GMT

Wow. Even though this thread is really old, I just had to respond.

Years ago, I had these exact same thoughts, the genesis of which was probably triggered by Helen Custer’s Inside Windows NT, and probably Gordon Letwin’s Inside OS/2 - tripping over my old, disused PC’s (22 at last count) also provided constant reminders, until I eventually made some shelves and moved them out of the way. The hoarder’s stubbed toe theory… :)

Back then, apart from processor speed and memory constraints (only ?!), the biggest problem was that the processors didn’t have any, or the appropriate hardware support to allowed “protected” operations, and I really loved strongly-typed languages – I simply built better code.

Then I’d heard about something called Pseudo-Code, and I quick realised that combining the two ideas may lead to solutions for the lack-of-protection-hardware problem. Actually, I was already familiar with the idea of Pseudo-Code, but I knew it by the it’s implementation: BASIC (tokens, etc).

Anyway, at that time, the tools (IL, compiler, code analysis, etc) were obviously not around, and I was not able to build the tools – well if I’d won the lottery, I’d probably just be getting to this point, now. So, it’s great to see a lot of this stuff is coming into fruition. It’s even better to see that you have the compiler people firmly entrenched in the team.… maybe I should have gone to Uni…

Anyway, just like with Java, I’ve been meaning to learn and actually use C#. I see that C# has one the things I ranted about, at one point: structured source code commenting, which aids the programmer in producing self-documenting source code. Though I’m not sure that I like it’s HTML-like “tagging” implementation.

Keep up the good work.

Re: Singularity: A research OS written in C#

s_jetha — Tue, 13 Dec 2005 15:21:57 GMT

Wow. Charles, I have new found respect for you. Going on about AI base, and Homeostatic OSes, just made my jaw drop.

This video was really interesting, especially coming from someone that has practically no knowledge of what ACTUALLY goes on inside an operating system. The idea of managed code inside an OS is quite intriguing.

Can't wait to see the second part!!

Re: Singularity: A research OS written in C#

dhi — Tue, 06 Dec 2005 21:35:18 GMT

In the second video, I saw that you are doing message passing over channels and run everything in its own SIP.
I did a lot of programming for µnOS which is an OO OS written in C++ including a GC.
It also uses channels for message passing (without message contracts) just like QNX does.
So if every driver runs in it's own SIP, can I write a SIP by myself sending messages to the NIC SIP to tell it to send raw ethernet packets to the net?
Or has the TCP/IP SIP exclusive rights to communicate to the NIC SIP?
In µnOS we dicided to compile device drivers to DLLs.
So the network service, which is a seperate process in user mode creates an instance of the device driver object(s)
in its own process space and uses a well-defined interface to access these driver object.
Only the network service has a channel over which the other applications can create and use network connections.

How about modularity and the extension of a certain system service?
Let's say your TCP/IP SIP currently supports IP, ICMP, ARP, TCP and UDP.
How would I extend it to support SCTP as well?
Can I write an extension module (e. g. in form of a C# class) that is loaded by the TCP/IP SIP at startup to support SCTP?
Or do I have to write the extension in form of another SIP?

I also saw that you are doing process creation, channel management and security inside the microkernel.
Did you thaught of implementing a SIP in user mode for doing stuff like that?
E. g. the QNX process manager does some of these things in user mode.

At all, you did a great job with Singularity!
I hope Microsoft stays tuned with it!

Re: Singularity: A research OS written in C#

Ignatich — Fri, 18 Nov 2005 21:13:55 GMT

What about scalability of such system? I suspect transition from single to multi CPU enviroment will be quite painful perfomance wise? Did you do some testing n multiprocessor enviroments?

Re: Singularity: A research OS written in C#

Tonatiúh — Sat, 05 Nov 2005 01:55:19 GMT

Beer28 wrote:
[...]
I'm guessing it's linked to undisclosed low level C or ASM libraries since it's not libc. You can't write low level code with C#, it's impossible because it won't let you break free into the instruction set you need to reach the bios and or service interupts, such as faults, device interupts, or system interupts of any kind.
[...]

Beer I have read several of your comments while studying this thread in order to enhance my background about the Singularity Proyect of Microsoft Research on Operating Systems... While reading your post I held my self thinking on what I have learned from one recent paper from MSR Titled "
That paragraph have finally broken my self-restrainment to reply you and wait until I have read every thing... Sorry... Just I could not avoid to be fired up for such an event procedure call interrupt... triggered directly to my brain and bypassing the nutshell my microkernel awareness operating system is...

Would you be so kind as for reading in full the above linked for download document (277 Kb in PDF format)... I will greatly appreciate it, specially if you do it before any further comment about Singularity.

Thank you a lot for reading me Beer...

Tonatiúh

Re: Why?

Charles — Fri, 04 Nov 2005 19:59:57 GMT

chris31 wrote:
They mentioned that the current NT kernel can't really be changed that much and I'm wondering why? Can't you just change the nt kernel and just make sure the interface between kernel and userland is the same? As long as the kernel interface is the same the app wouldn't care how the kernel did something. I would think that the app wouldn't even have to know it changed?

The NT kernel can (and does) change, but the fundamental architecture remains static. Singularity represents a completely different operational model than NT. For one thing, there is no notion of shared memory and processes are truly independent. Also, the Singularity kernel is closed (read impossible to rootkit an OS like Singularity).

The Vista kernel is a great piece of engineering and represents an evolution of the NT kernel (the Vista kernel is not a modified XP kernel, for example. It's new (for a client OS anyway)).

Please stay tuned for more on Vista (and Singularity) in the DotNetCPU, but... is it gone?

Re: Why?

chris31 — Mon, 20 Jun 2005 15:05:58 GMT

They mentioned that the current NT kernel can't really be changed that much and I'm wondering why? Can't you just change the nt kernel and just make sure the interface between kernel and userland is the same? As long as the kernel interface is the same the app wouldn't care how the kernel did something. I would think that the app wouldn't even have to know it changed?

Re: Singularity: A research OS written in C#

chemendonca — Sun, 19 Jun 2005 01:18:20 GMT

To Mr. Larus:

I'm currently finishing the Computer Engeneering course at Universidade de São Paulo (Sao Paulo University -- http://www.usp.br) and ever since i started graduation, i've been studying C# and some CLR which eventually allowed me to take the MS Student Consultant position in our university lab (http://www.usp.br/labms).

I was planning to write my end-of-course monography based on managed-code kernels and ever since i decided that, i've gathering material about the subject (which, as you may know, is not an easy task). Eventually i found your project and since it is much like the topic i chose, i was wondering if you could help me by pointing out any papers, books or any other research materials you may or may not have used.

I'm really willing to learn and would appreciate very much this kind of input, being more than happy to later share my research. Thanks in advance.

Carlos

Re: Singularity: A research OS written in C#

artificer — Thu, 02 Jun 2005 20:47:28 GMT

Any Singularity alpha that we can download just to play around? Any .PPT explaining the architecture?

Re: Singularity: A research OS written in C#

purevoid — Wed, 25 May 2005 22:24:13 GMT

How do you extend the libraries of code to allow reuse by various applications? For example, adding a SOAP library.

What I meant by the static linking question is: How do you a) build the app, and b) load it into the system such that it can call the kernel library functions? This also builds on the previous question of extending the kernel library...

I'm confused how you get by without a JIT, and how this precompiling process works. Especially how you create a SIP, and put code into it. Is that not dynamic code loading?

Re: Singularity: A research OS written in C#

larus — Wed, 25 May 2005 17:41:07 GMT

These are good questions, and the answers probably aren't clear from the video.

1. We don't have DLLs, in the sense of dynamically loaded libraries, but we do have libraries of code that can be reused in various applications.

2. I'm not sure what you mean by "app/drive statically linked against the code"? Code in two processes is only related by the channels between them, which have contracts expression the data that is transfered and the legal message patterns. Beyond those, there is not linking and the code in each process is entirely independent.

3. We don't have a JIT. We precompile everything before executing it. If you don't have dynamic code loading, you don't need a JIT.

/Jim

Re: Singularity: A research OS written in C#

larus — Wed, 25 May 2005 17:40:45 GMT

an unsound assumption that a program isn't violating language semantics with dirty tricks, such as converting an integer to a pointer.

I never thought of using pointers as integers a dirty trick....
in x86 they're the same width so it doesn't really matter in the compiled code nor asm which is which type. I guess the same is true on 64 with long.

Do you have any plans for embedded systems, 8051/2, arm?
what about just arm? I don't forsee this going to 8 or 16 bit in short retrospect.

Sorry to offend old-time C programmers, but it is a dirty trick to treat pointers as integers. Moreover, in any version of C after K&R C, it is also unnecessary, except in a few obvious circumstances, such as referencing a literal memory address such as a device register: *((int*)0xffffeeee)

The fact that it works on a particular machine is just a coincidence that makes your code more difficult to read, more difficult to port, and aviolation of the C language semantics (that why they introduced void* a long time ago).

But, in context, I was refering to the difficulty of analyzing code with static tools. Once you start confusing pointers and integers, the tools (including optimizing compilers) pretty much give up and assume the worst.

Types serve a purpose, both to make code clearer and easier for humans to read, but also to tell compilers and other tools that the universe of items accessible through a variable are distinct from the items accessible from another variable and that only a limited set of operations can be applied to that variable.

And, yes, the lowest level of our runtime system obviously manipulates raw memory pointers. But, we call them memory pointers (UInt_Ptr), not integers.

We haven't looked at 8/16 bit systems, but that does not seem like a likely direction for our work. 64 bits, on the other hand, opens a lot of interesting possibilities.

/Jim

Re: Singularity: A research OS written in C#

AndyC — Wed, 25 May 2005 09:37:01 GMT

Beer28 wrote:

Also you can write out a function from the code like JIT, and call that new entry point, say on win, you virtualalloc pages with +rwx and write out some opcodes, you can then cast that address to the function pointer with the params and return type, then funcptr(arg1, arg2);
So in otherwords you can do dynamic code and call it after marking t he heap pages executable.

I believe the managed equivalent of that is using the System.Reflection.Emit namespace and it's associated classes, which can generate IL on the fly.

Haven't ever tried actually using it though, it's mostly used by compilers/script engines, but I guess you could do if it you really wanted to.

Beer28 wrote:

There are just tons of great uses for addressable memory. C# and Java are pretty limited in my opinion. It sounds like the Microsoft guys may be using some dirty tricks with that "bartok" compiler though

"Safe C#" and Java are admittedly more limited in this respect than say C or C++. The advantage they give you, however, is much better code verifiabilty as larus mentioned.

"Unsafe C#", i.e. code that is marked as unverifiable, however is easily as capable as C/C++ and performing all manner of dirty operations like this, at this expense of requiring Full Trust code access security permissions to run.

You really should try C# out Beer, even if only under Mono. I think, if you ignored you're preconceptions about Microsoft and .NET behind, you'd actually really like it as a language.

Re: Singularity: A research OS written in C#

Buzza — Wed, 25 May 2005 05:15:18 GMT

Beer28 wrote:

Buzza wrote:
I class it as a dirty trick as a pointer is just that - it points to data - but it is not the actual data - the data is what is pointed at that memory location. An integer is data - not a pointer. Keep the 2 seperate.

in assembly it doesn't matter, C is shorthand for assembly

it's 32 bits either way. All those types in windef.h are all typedefs, they are not compiler standard types. WORD, DWORD, WPARAM, all that stuff.
A pointer to a type and an int are compiler types, they happen to be the same size on x86, long and * are the same size on x86_64.

That's not cheating, that's programming. If you ever do small MCU's with limited addresses, you will quickly see, that you need addressability.

Managed code like java is a whole other story. You can't say C or C++ code is dirty because it doesn't behave like java. It's not supposed to.

They key word here is 'happen to be the same size on x86' thats the dirty word as its just a coincidence.

Could you give me a sample of this 'addressability', as being a c and c++ dev - i have never needed it.

Re: Singularity: A research OS written in C#

Buzza — Wed, 25 May 2005 02:56:41 GMT

Beer28 wrote:

larus wrote: an unsound assumption that a program isn't violating language semantics with dirty tricks, such as converting an integer to a pointer.

I never thought of using pointers as integers a dirty trick....
in x86 they're the same width so it doesn't really matter in the compiled code nor asm which is which type. I guess the same is true on 64 with long.

Do you have any plans for embedded systems, 8051/2, arm?
what about just arm? I don't forsee this going to 8 or 16 bit in short retrospect.

I class it as a dirty trick as a pointer is just that - it points to data - but it is not the actual data - the data is what is pointed at that memory location. An integer is data - not a pointer. Keep the 2 seperate.

Re: Singularity: A research OS written in C#

purevoid — Tue, 24 May 2005 22:49:22 GMT

How do you work around the issue of DLLs, and loading reusable code? Is that possible at all?

Also, with regards to channels as the only communication method between two separate entities in the system, is the app/driver just statically linked against that code?

I'm also working on a project that parallels a small number of your goals/achievements, and not knowing much of the internals of linking, etc., I'm curious how it all works. Is it the job of the JIT to link in static library code at runtime?

Hope I'm not too vague, as I'm not quite sure what it is exactly I want to ask ;-)

Re: Singularity: A research OS written in C#

larus — Fri, 20 May 2005 16:54:15 GMT

This is a very interesting project, mostly because of the original ideas on the OS rather than the fact it's written in C# - I see C# as just an enabler for the new process model concept as it allows for process seperation without the need for heavy-weight hardware protection.

It's also nice to hear some validation of my claim that the security model on conventional operating systems is essentially broken in the network age.

Your observation is correct. C#, because it is a type-safe, memory-safe language enables us to explore new architectures for an OS. The key factor is the safety, rather than C#, though I like the language, particularly v2 with generics.

C# (or another safe language) has an additonal advantage as well. Programs written in safe language are easier to analyze completely and accurately. My research group SPT (research.microsoft.com/spt) has build many tools for finding defects in programs. Analyzing C/C++ is difficult and always has an unsound assumption that a program isn't violating language semantics with dirty tricks, such as converting an integer to a pointer. In a safe language, these assumptions are enforced by the type system and run time system, so tools can rely upon them.

Re: Singularity: A research OS written in C#

sytelus — Fri, 20 May 2005 16:53:03 GMT

Amazing. Just few days before this video came out, I wrote a small essay on architectural form a managed OS might take and some rational behind that.

The post is at:
http://tinyurl.com/dy8b7

Re: Singularity: A research OS written in C#

larus — Fri, 20 May 2005 16:48:37 GMT

The video was great, C9 isn't about PR and marketroid stuff. It's about what you're up to and I think that's one of the most thought provoking videos I've seen here.

You guys are super lucky to do that and get paid for it.

You don't have teenage kids, do you?

I really enjoyed the Channel 9 interview, since it let us talk directly to a technical audience, without having to worry about it getting edited into a narrow perspective.

And, yes, we are lucky. MSR is a great place to work. The rest of the company is pretty good too :-).

/Jim