Entries:
Comments:
Posts:

Loading User Information from Channel 9

Something went wrong getting user information from Channel 9

Latest Achievement:

Loading User Information from MSDN

Something went wrong getting user information from MSDN

Visual Studio Achievements

Latest Achievement:

Loading Visual Studio Achievements

Something went wrong getting the Visual Studio Achievements

Discussions

Bass Bass Knows the way the wind is flowing.
  • It is time... Move the filesystem off of disks

    fknight said:
    Bass said:
    *snip*

    See --- this long post by Bass right here.

     

    This is why I want Channel 9 forums to switch to something with a little bit more maturity like SMF or phpBB or something. The delays in bringing up posts, the times when you can't get to page 2 of a thread, and the over-AJAXification of the current Channel 9 forums interrupts and distracts from a steady stream of posts like this one which are both highly entertaining, yet educational. 

     

    If I had succumbed to the urge to just not visit the site a few minute ago (an urge fueled mainly by the usability of these forums), I would not have read this post.

     

    Thanks.

     

     

    Thank you for the kind words. Smiley

  • It is time... Move the filesystem off of disks

    Sven Groot said:
    Bass said:
    *snip*

    So you have information cached in memory. Unfortunately, you can't have everything cached in memory, because you just don't have that much memory. So at some point, the DBMS is going to have to read something from disk, and, most importantly, something else is going to have to be removed from the cache to make room from the new information. Deciding what piece of information to remove from the cache when space runs out is probably the hardest and most performance critical part of cache algorithms.

     

    Now ask yourself: who has more information by which to decide what should be dropped from the cache? The OS kernel, or the DBMS itself?

    The OS kernel has a lot of information here.


    (1) It can figure out where the most commonly read information is, and store those pages in memory. I know this may seem silly but the most commonly read information is wait for it... exactly what you want to be in memory. Smiley Thus the kernel does a pretty good job of finding exactly what needs to be in memory. The DB's data structures MUST be written to be automatically cached anyway. You can not use unpredictably spread out data structures (see why heap sort performs poorly on x86), you are quite limited in how you structure your data for performance due to the CPU cache. But seriously, I in all honestly can not see what a DB's domain specific caching algorithm could do better. Since you are the so called DB expert perhaps you could enlighten me?

     

    (2) It has access to a lot of information about the persistent store (eg: the position of the head). So it knows exactly when it can pull some piece of data off the disk (often measurable in nanoseconds), while at at best the DB could query the kernel for this information (zomg cache miss / context switch), but more likely simply guesses and then has blocking stuff all nicely queued for unpredictable times on the kernel's I/O scheduler.

     

    (3) It has access to an array of modern caching technologies like DMA. Remember what I said about cache misses and all that? Well you are passing data in and out of the CPU like an idiot and that's BAD BAD BAD for performance. Unfortunately this is the only realistic way to implement a user-mode file cache is to pass data in and out of the CPU like an idiot. You really want to do that? Be my guest. I'll take DMA over that any day.

     

    At best, if you want to implement a cache in a DB you should be subservient to the kernel, passing it the right information so it can do it's job via it's sheer awesome kernel powers. As I said before, there is ways to ask the kernel to cache something for you and there are ways to structure your data that is conductive to caching. But one thing you should never do the kernel's job for it, because you will never do it as well.

  • Zune for Mac, YEESSS

    W3bbo said:
    Bass said:
    *snip*

    It's because Apple makes money on hardware sales, not software (well, it does, but it's a pittance compared to the billions of dollars the iMac has brought in). OS X is the "pull" they use to get people to buy the hardware (never mind that the hardware is very desirable in itself) so if people can install it on non-Apple hardware they wouldn't be making anywhere near the amount of money they would have done if those people bought Mac hardware.

     

    Remember, boxed retail OS X copies are really Upgrade editions. To compare the $199 price that OS X 10.5 Leopard commanded to $199 for Windows 7 Professional is unfair to Windows.

     

    Apple could sell OS X at retail for installation on non-Apple hardware, but with a few caveats:

    • They would have to charge at least $500 for it to make up for the earnings lost by having not sold hardware
    • They lose control over the hardware experience, which is one of the reasons the Apple platform is so highly rated by critics; it's like going into a McDonalds fast-food outlet, paying $500, and getting a Michelin-starred meal. It's nice, sure, but you'd rather be sitting on proper chairs in a calm environment, not hard plastic with screaming children running amok.
    • And Apple would lose money through support. Fact is, most users are idiots and would come running to Apple for support issues with their hardware which are outside of their concern; even if Apple fobs off the users they'd lose money by having to answer the call, and then lose PR as angry idiots start posting their stories online.

    Apple, however, doesn't care what you run on Macs. That's why they talk-up Boot Camp to win over Microsoft's customers, even if their hardware is running Windows 7 they've still made money on the hardware sales, and that's what matters. However they're careful not to hype it too much, lest they become "just another" boutique PC manufacturer like Sony Vaio or Alienware.

     

    It's impossible for another company to rise up and be the next Apple because they'd need their own OS, which at this point would be stupid to start from scratch, look at what happened to Be and SGI. Not only do you have the problems of actually writing an OS, but also the chicken/egg situation for software: no-one will use your platform if MS Office and Photoshop aren't available for it, and Microsoft nor Adobe will port their applications if no-one uses it. So the hypothetical company's only choice is to take either Windows or Linux and add some new UX layer on top of it (some do it by using Stardock's Object Desktop) but this is ultimately a gimmick and invariably poorly implemented, leaving a sour taste in the mouth.

     

    My prediction: we're in a stalemate for now. I do predict Linux (possibly in the form of ChromeOS) will become more popular thanks to Wine and will take more territory away from Windows, especially in embedded and low-cost applications. Microsoft doesn't really have a hardware strategy so they'll have to come up with something to survive.

    Regarding Mac OS X:

     

    I understand the the business reason for Mac OS X being restricted to Apple. And I mostly agree with your explanation of it.

    -----


    Regarding Linux:

    As so far as Linux is concerned it's pretty obvious Microsoft is going to dominate the desktop for a long time to come. Wine while it has potential and is already quite useful (and I use Wine to run the game "Guild Wars" with surprisingly good effect), still has many problems running Windows apps (I'm looking at you Office 2007), is generally slow, and has been in development for like 15 years. It might take another 15 years for them to get to Windows 7 level, and you know Microsoft will introduce 500 new API libraries in that time frame. Smiley I do think if Linux is ever to disable Windows, Wine is be a big part of that, because very few people are willing to drop the apps they are used to.

     

    -----

    Regarding the original topic:

     

    I am not sure why I understand the business reason for Zune being restricted to Windows. I just know that Microsoft is intentionally restricting the Zune to Windows. The only valid business reason I could see if the Zune is some kind of loss leader when Microsoft loses money on each Zune sale they hope to recoup through the marketplace that comes with the mandatory Zune software. Which is the infamous video game console business model. Although even a "it's our DAP, suck it bitches" would be respectable in my eyes then a canned PR response about some supposed technical difficulty of Microsoft implementing a Microsoft created standard. Wink 

     ----


    Regarding the hamburger that I am eating:

     

    Tasty.

  • Zune for Mac, YEESSS

    encyclopedia said:
    Bass said:
    *snip*

    Could you give us some links to your posts on an Apple forum conversing with an Apple employee, questioning why OSX and or its software is bared from running on non Apple hardware?... Much appreciated.

    They actively make it both legally difficult (see W3bbo), and technically difficult (by certain DRM-like things in the kernel which makes an unmodified Mac OS X simply not work on a regular PC). There is people who got around those technical restrictions, but Apple isn't too happy about things like that.

     

    Well I think the answer is quite obvious. Apple does not want PCs to run Mac OS X. But I've never seen a bogus technical explanation from Apple regarding this. They are pretty upfront with their rational which can be summarized as thus: "it's our OS, suck it bitches".

  • It is time... Move the filesystem off of disks

    Dexter said:
    Bass said:
    *snip*

    Feel free to continue rambling. I think I'll go drink my pan galactic gargle blaster now, thank you.

    You just don't like admitting you are wrong huh? I noticed that in Tech Off too. You have to try to "correct" everyone. You can't correct what is already correct though, buddy. Smiley

  • It is time... Move the filesystem off of disks

    AndyC said:
    Bass said:
    *snip*

    You have to write your database caching algorithms with the kernel and CPU/hardware behaviour in mind. You certainly don't just "leave it to the kernel" unless you're expecting performance not to be important. That's true of any bit of coding.

    You want to leave "as much as possible" to the kernel though. Notice that's exactly what I said. Heh heh heh.

     

    You guys are a riot.

  • It is time... Move the filesystem off of disks

    Dexter said:
    Bass said:
    *snip*

     

    Yeah, you're right... about a completly different and unrelated problem. What problem? Ah, it was branch prediction. Ah no, sorry, that was about CPU cache. Ah no, wrong again it was about optimizing compilers. Oops, I've missed again. No, I bet it was about 42.

    A system memory cache is a just a much slower and much bigger CPU cache.

     

    In fact this is how the CPU sees the world:

     

    Page fault: Geographical time

    Cache miss: Snail time

    In cache: Ferrari Enzo

     

    You want your CPU to be riding that Ferrari as much as possible. Smiley That means NO DIRECT MEMORY ACCESS IF POSSIBLE. A big optimization thing (at least big as autovectorization) is to try to keep your working data on the cache, and never make the CPU explicitly access a piece of memory. Does that sound familiar? Isn't that exactly what you want to do with memory caching in general?

     

    Oh wait, I can even make a fill in the blank:

    The point of X caching is to keep as much working data in a fast memory store (in this case: Y) as possible.

     

    I'm going to guess the reply: "irrelevant". LOLWUT?

  • It is time... Move the filesystem off of disks

    Dexter said:
    Bass said:
    *snip*

     

    And how exactly does this relate to what I said? Or more generally, how exactly does this relate to caching?

     

    Ha ha, look who's talking. You're twisting my words yet you claim "losing argument".

     

    Well I see you want to be on an equal footing with me on this. So I'll call you an insane person also. Insane person.

     

    And how exactly does this relate to what I said? Or more generally, how exactly does this relate to caching?

     

    This is the original point I am addressing:

    Seriously, do you really want/expect a database system to move gigabytes or terrabytes of data around just to keep the kernel happy?

     

    Okay I'm going to say this again. Please no more "NUH UHs" on this. I'm quite obviously right.

     

    You can not write any explicit caching code to efficently cache things on an x86 processor. This algorithm is hardcoded into the control unit of the actual processor. The only way to get important data into cache (which is of course, what you want) is to "suggest" it to the proccessor by your data structures. This means working with the quirks of the branch predictor, and also localizing commonly accessed data.

     

    Of course this is to keep the CPU happy and you are probably going to be like "well I didn't say CPU, so I am somehow correct, and you are wrong". Well buddy, you have to keep the CPU happy. A unhappy CPU is a cache missing CPU, and you MUST avoid this to have any reasonable performance.

     

    And interestingly, by keeping the CPU happy, you also tend to make the kernel happy. Because the kernel isn't using some magical caching algorithm that Intel doesn't know about.

     

    Kapeesh?

  • Google founders to cash in $5.2 billion dollars worth of stock

    figuerres said:
    Bass said:
    *snip*

    Bass:  I was asking a very real and very valid question.

     

    lose the silly reply and lets think about it....

     

    that's a lot of funds to go do something....  if they for example want to fund some VC stuff we could all start making some cool plans.

    or if they plan on starting a new corp what are they going to do?

    hardware? software? services? Games? ????

     

    Why do you think my reply wasn't serious? If I owned everything money could buy, at that point I'd want to acquire the things that are not "for sale", IE. total domination of the world.

  • It is time... Move the filesystem off of disks

    Dexter said:
    Bass said:
    *snip*

     

    Except the characteristics of the hardware are likely to be simpler than the characteristics of the data structures and access patterns. Which set of characteristics do you think will be easier to communicate to the other party?

    Are you insane?  I've said that I don't think the database should move data around to keep the kernel happy. That's not the same thing as "localizing" important data so it fits the CPU cache (or the harddrive).

     

    And why not? The kind of caching algorithm the Linux kernel uses is not all that different from what Intel microcode uses. You kill two birds with one stone. PS: Ad homiem is evidence of a losing argument.

     

    Except the characteristics of the hardware are likely to be simpler than the characteristics of the data structures and access patterns. Which set of characteristics do you think will be easier to communicate to the other party?

     

    Well a kernel can safely assume that important/commonly used data in memory will be near each other, because that's how optimizing compilers and high performence databases tend to structure their data. True story.