Entries for Sven Groot

Does this code have a race condition?

Sven Groot — Fri, 20 Nov 2009 08:46:31 GMT

Ok, this is a tough one. I basically spent all of yesterday attacking this from every angle I could think of, but I'm still not entirely convinced that there's a race condition here. I think there is one. One that's not easy to circumvent without introducing some other issues. I would however like to get some additional thoughts on the issue.

The situation is this: we have what on the surface is a basic producer-consumer pattern. Items are added to a queue and processed by one or more worker threads. However, the number of worker threads is flexible; if more are needed they will be started (up to a limit). If a worker thread sits idle for too long, it will quit.

Here's the code (it's not my code, but I've sanitized it and removed anything not relevant to the problem):

[code language="C#"]public class ItemProcessor
{
    private const int _maxWorkers = 100;
    private const int _workerWaitTime = 20000; // ms
    private readonly Queue<string> _work = new Queue<string>();

private int _runningThreads;
private int _availableThreads;

    public bool AddItem(string item)
    {
        if( item == null )
            throw new ArgumentNullException("item");

        lock( _work )
        {
            if( _availableThreads > 0 )
            {
                --_availableThreads;
                _work.Enqueue(item);
                Monitor.Pulse(_work);
                return true;
            }
            else if( _runningThreads < _maxWorkers )
            {
                _work.Enqueue(item);
                Thread worker = new Thread(WorkerThread);
                worker.IsBackground = true;
                ++_runningThreads;
                worker.Start();
                return true;
            }
        }

return false;
}

    private void WorkerThread()
    {
        while( true )
        {
            string work = null;
            lock( _work )
            {
                do
                {
                    if( _work.Count > 0 )
                        work = _work.Dequeue();
                    else
                    {
                        ++_availableThreads;
                        if( !Monitor.Wait(_work, _workerWaitTime) )
                        {
                            if( _work.Count > 0 )
                            {
                                // Work added as we timed out
                                work = _work.Dequeue();
                            }
                            else
                            {
                                --_availableThreads;
                                --_runningThreads;
                                return;
                            }
                        }
                    }
                } while( work == null );
            }

            // Code to process the item goes here; omitted for brevity.
        }
    }
}[/code]

Two somewhat interesting decisions were taken here. First, _availableThreads is decremented in the AddItem method; this is done to ensure that no other call to AddItem can think there are available workers while there are items waiting in the queue. Second, the worker threads will pick up work, even if the Monitor.Work call timed out. This is done to make sure that if there is only one thread and it timed out just as an item was being added this item isn't "lost" (it would sit in the queue until some other item comes in and starts a new thread).

Now here's the problem: suppose some thread gets the lock in AddItems. It sees _availableThreads > 0 and calls Monitor.Pulse. A thread receives the signal and is moved to the ready queue (for that thread, Monitor.Wait will return true). However, before that thread runs, another thread had just timed out and manages to acquire the lock first. This timed out thread will pick up the work, process it, and continue. The thread that was pulsed will see the queue is empty, loop around and continue. End result: _availableThreads got decremented once (in AddItem), and incremented twice (in both the timed out and pulsed thread) and is no longer correct.

However, we can't just decrement _availableThreads if the Wait call timed out, since if Monitor.Pulse didn't wake up any thread, you wouldn't want to do that. There is no way to tell if Monitor.Pulse actually woke up some thread.

There are probably some ways to correct this problem, but for now I'm simply interested in this: am I right there is a problem? Can this scenario (a timed out thread running in between a call to Monitor.Pulse and the pulsed thread getting the lock) actually happen? I can find nothing in the documentation of the Monitor class that says it couldn't, but maybe I'm missing something.

Any insight appreciated.

History in the making: Ares I-X [History in the making: Ares I-X]

Sven Groot — Tue, 27 Oct 2009 13:32:37 GMT

Currently it looks like we're about 10 minutes away from the launch of Ares I-X, the first test launch of the Ares rocket that will carry men back to the moon and then hopefully on to Mars. Watch it live on NASA TV.

EDIT: Apparently there is a ship in the launch danger area, to it looks they're going to delay at least 90 minutes. :(

EDIT2: Ok, looks like they got the ship out of the way and are resuming the countdown soon.

in reply to History in the making: Ares I-X

Mysterious local disk Q: [Mysterious local disk Q:]

Sven Groot — Tue, 13 Oct 2009 12:19:56 GMT

For no reason that I can think up, I suddenly have a local disk Q: in Explorer. There is no hardware associated with it, it doesn't show up in Disk Manager, it's not a network drive, and everything I try to do with it results in "access denied". Yet it stubbornly persists.

Any idea how I can find out what it is, where it came from, and how to get rid of it?

EDIT: Windows 7 x64 Ultimate RTM

in reply to Mysterious local disk Q:

Command line argument parser for .Net [Command line argument parser for .Net]

Sven Groot — Sun, 06 Sep 2009 09:04:33 GMT

I recently needed to do parse command line arguments for a whole host of different scenarios. Rather than do each individually, I've created generalized command line argument parser. Since I figured others might find that useful too, I've now made it public.

Simply create a class, and the constructor parameters and properties marked with a special attribute will define the arguments your application takes. It can even generate usage text and format it for display on the console.

In case anyone's interested, you can get the class library, documentation, samples and source code on my site.

in reply to Command line argument parser for .Net

Refresh button gets disabled after post or refresh [Refresh button gets disabled after post or refresh]

Sven Groot — Tue, 01 Sep 2009 15:54:42 GMT

Suddenly I'm having a problem with the refresh button on the post list on a thread. After you make a new post or refresh the list once, the button stays disabled so you can't refresh again. Because about 90% of the time your new post doesn't show up immediately this is extremely annoying.

in reply to Refresh button gets disabled after post or refresh

Find all audio files with DRM? [Find all audio files with DRM?]

Sven Groot — Thu, 13 Aug 2009 09:00:59 GMT

Is there any way to find all audio files in a certain folder (and subfolders) that have DRM? I don't think Windows Search can do this.

Anyone have any ideas?

in reply to Find all audio files with DRM?

Oh crap, external HDD dead [Oh crap, external HDD dead]

Sven Groot — Fri, 24 Jul 2009 15:05:44 GMT

My external hard disk fell over just now (literally), and it appears it didn't take it well. It started making some weird noises and now it won't start anymore (it just makes weird noises and doesn't get picked up by Windows).

That sucks, especially since I can't really afford to replace it right now. Hopefully they'll replace it under warranty.

There was nothing truly irreplacable on the disk, just a lot of ISOs from MSDN and things of that nature. All in all a few 100GB I'll have to get back somehow. And I just lost one my backups, so if my main hard disk fails now I'm in deep *.

I am not happy. :(

in reply to Oh crap, external HDD dead

Measure .Net object size [Measure .Net object size]

Sven Groot — Tue, 21 Jul 2009 15:58:04 GMT

I'm storing a lot of objects in a List<T>. The number of objects can be very, very large, and is unknown beforehand. I also want to limit the memory usage of my application. Essentially the final goal is to sort the items.

What I want to do is this: when the total in memory size of the List<T> exceeds a certain size, I want to sort what I have so far, save them to disk, and continue with an external sorting algorithm. However, in order to do that, I need to know how much memory all the objects in the List<T> are using. Is that even possible in .Net? How would you solve this problem?

in reply to Measure .Net object size

Let's Play [Let's Play]

Sven Groot — Tue, 14 Jul 2009 13:43:06 GMT

Time for something completely off topic. You may know what "Let's Play" videos are. They're videos on YouTube where somebody plays an entire game, often providing commentary while doing it. Many of these are funny and/or interesting, though some of them are boring depending on the game and the person providing the commentary. :)

I quite enjoy watching these, when done well, and I figured I could jump onto this bandwagon and make a Let's Play of my own. The game I chose for this endeavour is Riven: The Sequel to Myst. This is one of my favourite games, with an atmosphere and depth that has been unmatched by anything produced before or since.

So I've begun uploading videos for this Let's Play, and currently have the first six available on YouTube. I intend to continue adding two a day; in total it will be thirty videos to play the game start to finish. While playing, I provide commentary on the puzzles, the story, and the locations. So yeah, it's probably pretty boring, but anyway. :P

This is the first part:

Note there is some slight audio distortion in my narration, especially in the beginning; this doesn't happen in any of the other parts though.

The remaining videos can be found in this playlist. I will add new videos to that playlist as well as I upload them.

Enjoy! :D

(Note: it wasn't originally my intention to post about this on C9, since it's wildly off topic, but Littleguru liked the videos and insisted I do so anyway ;) )

in reply to Let's Play

Standard user compatibility [Standard user compatibility]

Sven Groot — Tue, 16 Jun 2009 03:22:05 GMT

There's been a lot of talk with this UAC stuff about applications that are compatible with standard users.

It's true that the number of applications that can work without strange workarounds in a limited rights environment has increased since Vista's release. I know this because I'd been running XP as a limited user for years before that.

But as someone who still runs as a real limited user in Vista/7, I still regularly come across applications that don't work. Quite often these apps work fine with the limited administrator token, but they don't work in a real limited user account.

The primary reason is this: too many applications still assume that whoever installs the application will run the application. This manifests in installers writing stuff to %AppData% or HKCU, and then when you try to run under a different account the app complains that data is missing (if you're lucky; if you're not it'll just crash). This is such a stupid thing and it's really Windows installations 101 that an installer should never write user-specific data, and it annoys me to no end. UAC does nothing to force developers to fix this either, because admin accounts are still the default.

Probably the worst offender is ActiveSky X, a weather enhancement add-on for Flight Simulator X. ASX is not only guilty of the above, they also force you to elevate it by means of an embedded manifest. The reason they want you to elevate it is because ASX needs to modify some files belonging to FSX which are usually in Program Files so only an elevated user can write to them. Even if you do have write access to those files (and on my machine that's the case because FSX is on another drive because it's so huge), ASX still forces you to elevate it. That wouldn't even be so bad, except that if ASX is running in a different account than FSX, the two can't connect, so I'm forced to elevate FSX too! Here I had to manually copy files and registry data from one user to the other, and use a resource editor to change the manifest so I can run the app in my own account. That's real user friendly right there.

FSX is often a target for issues like this btw; it uses a dll.xml file to register add-on DLLs which is located in %AppData%. So whenever you install a new add-on that needs to load DLLs into FSX, they'll end up being registered only for the user account that installed the add-on, and you have to manually edit dll.xml to get them working in other accounts.

So, if you write applications, please don't do any of the above. :)

in reply to Standard user compatibility

NASA photosynths [NASA photosynths]

Sven Groot — Fri, 08 May 2009 04:51:35 GMT

NASA released several photosynths of the interior and exterior of the International Space Station, as well as one of a future Mars Rover.

Nice to see NASA embracing Silverlight. Now if only they'd switch the NASA TV player (which uses a WMP control) with Silverlight and switch to a higher quality stream... :P

in reply to NASA photosynths

Priority queue for .Net [Priority queue for .Net]

Sven Groot — Wed, 29 Apr 2009 06:34:27 GMT

This post is borderline spam, but it might be of interest to some people here. :P In the past I would've used the Sandbox for this, but I don't think anyone uses that anymore.

I was recently in need of a priority queue data structure, and discovered that .Net unfortunately lacks one. So I wrote my own, and decided to make this available online. In case you're interested, it uses a binary heap implementation which means that adding an element to the queue is O(log n), finding the smallest element is O(1), and removing it is O(log n).

You can read more, and download it, here.

in reply to Priority queue for .Net

Sandbox is (nearly) empty [Sandbox is (nearly) empty]

Sven Groot — Wed, 29 Apr 2009 06:27:41 GMT

It seems all the old posts were removed from the Sandbox. Why?

in reply to Sandbox is (nearly) empty

So I bought an iPhone... [So I bought an iPhone...]

Sven Groot — Sat, 11 Apr 2009 10:48:48 GMT

Yep, I've gone over to the dark side. Or something.

My contract was up, so time to get a new phone. And my provider (Softbank) happens to be the Japanese provider for the iPhone. I thought long about whether it was a good idea, but in the end every other phone I liked was more expensive (they had a special deal making the iPhone completely free, which meant even the scrappiest phone they had wasn't any cheaper).

I'm sure some of you are wondering why not a Windows Mobile phone. The answer to that is simple: here in Japan phones and contracts are tightly coupled. It is as good as impossible to buy a phone without a contract or vice versa. So if I wanted a WM phone I'd have to buy it here. And unfortunately, you cannot change the UI language in Windows Mobile, so I'd be stuck with Japanese which I didn't want.

It's sorta nice, I now use Microsoft, Apple and Linux in my daily life. :)

What I wonder about most is whether I will completely replace my Dell Axim X51v. It's two main uses were reading e-books and Japanese dictionary. I already installed eReader on the iPhone, and the UI seems a lot nicer than the WM version (strange since it's the same application, but anyway). I do prefer the way text looks on the Axim though, what with ClearType and all. I also still have quite a large number of books in .lit format, I'll have to use the Axim if I want to re-read those. I haven't looked for a Japanese dictionary yet, I suspect they exist, but I don't see how I can do write kanji without a stylus, and the built-in Japanese keyboard is only useful if I know how to pronounce a character, which is often not the case.

in reply to So I bought an iPhone...

Layout broken on IE8 in Win7 beta [Layout broken on IE8 in Win7 beta]

Sven Groot — Tue, 31 Mar 2009 14:53:28 GMT

I notice the site's X-UA-Compatible attribute has been changed to force IE8 Standards Mode. That may be fine for IE8 RTM, but those of us using Windows 7 beta 1 are stuck with an older version of IE8 and the layout is totally broken. The post list in the forums appears way too narrow. I've also encountered the missing text bug. And because this option is set, there is also no easy way to select compatibility mode anymore.

I would recommend you stick to compatibility mode, or don't specify the attribute at all so at least we get the choice, until the Win7 RC is released to the public.

In the mean time, I'm forced to use another browser just for C9, which I thoroughly hate.

in reply to Layout broken on IE8 in Win7 beta

Life in 2019 according to MS [Life in 2019 according to MS]

Sven Groot — Sat, 28 Feb 2009 12:22:55 GMT

I'm guessing most of you will have seen the video Long posted. It shows some interesting concepts, although I highly doubt the future will look anything like that. ;)

The classroom bits seem the most realistic to me (well, with the exception of automatic translation and the drawings becoming animations by themselves). Smartboards are already used in many classrooms and this seems like a logical next step, a merge between smartboards, touch screen and video conferencing technology that doesn't look too far fetched to me.

The aviation nerd in me couldn't help but notice that the aircraft shown in the video appears to be a Boeing 787 (compare the image here, it's identical except for the screens in the seats). So MS is predicting that Boeing will have actually managed to finish the aircraft by then :P (it's already more than two years behind schedule). The question on my mind is: will we get screens that big in economy class? I doubt it. ;)

in reply to Life in 2019 according to MS

It's getting crowded [It's getting crowded]

Sven Groot — Fri, 13 Feb 2009 09:57:23 GMT

This image from NASA shows just how much junk has been put into orbit since the sixties. Obviously it's not to scale, but it gives a good idea of how much is out there.

NASA tracks not just working and derelict satellites, but every piece of debris in orbit that could be a potential danger to a space craft or an astronaut during EVA is carefully tracked.

in reply to It's getting crowded

Profile feed updating [Profile feed updating]

Sven Groot — Sat, 03 Jan 2009 12:04:00 GMT

Checking my statistics, it seems that the C9 feed updater is checking my site's RSS feed every 30 seconds. Isn't that a bit excessive?

in reply to Profile feed updating

40 years ago today... [40 years ago today...]

Sven Groot — Wed, 24 Dec 2008 04:59:30 GMT

Today it's exactly forty years ago that three humans celebrated Christmas from one of the most extraordinary locations imaginable, 385,000km from here.

On December 24th 1968, NASA astronauts Frank Borman, Jim Lovell and Bill Anders on Apollo 8 became the first human beings to orbit the Moon. On Christmas Eve they made a historical, although controversial, broadcast. At the time it was the most watched TV program ever.

Their reading of Genesis was considered very controversial, but I still think it's a very powerful moment, and I'm an atheist.

With this mission NASA proved that the giant Saturn V booster and the Apollo command module (redesigned after the Apollo 1 disaster) could cross the distance to the moon, safely. Apollo 8 was initially meant to be a test of the command and service module and lunar module in Earth orbit, but the mission was changed as it was determined that Grumman Aerospace wouldn't be able to have the lunar module ready in time, and because NASA feared that the Russians were preparing a flight to circumnavigate the moon, after having sent an unmanned capsule around the moon earlier that year.

The original Apollo 8 mission to test the lunar module was given to Jim McDivitt, Dave Scott and Rusty Schweickart on Apollo 9 in March 1969, and Thomas Stafford, John Young and Eugene Cernan tested the lunar module in lunar orbit on Apollo 10 in May 1969, paving the way for Apollo 11 to finally land on it in July of that year.

Fast forward 40 years to today, and the expedition 18 crew on the International Space Station wishes you a merry Christmas:

PS: I want a floating Christmas tree too. :P

in reply to 40 years ago today...

Layout bugs in Opera [Layout bugs in Opera]

Sven Groot — Sun, 21 Dec 2008 03:33:34 GMT

There's a huge gap above the post list in the forum index pages in Opera 9.63.

And every time you switch between regular and HTML view in the editor, it gets bigger. :S

in reply to Layout bugs in Opera

The death of a familiar voice [The death of a familiar voice]

Sven Groot — Fri, 19 Dec 2008 07:38:39 GMT

Majel Barrett-Roddenberry passed away yesterday. She played the Enterprise's first officer in the original, un-aired Star Trek pilot, nurse Chapel in the original series, and Lwaxana Troi in The Next Generation and Deep Space Nine. She was also the voice of the computer in all series and films since TNG, probably the best known computer voice after HAL9000.

A sad day for Star Trek fans. :(

in reply to The death of a familiar voice

R.I.P.: The Sandbox 2004-2008 [R.I.P.: The Sandbox 2004-2008]

Sven Groot — Fri, 12 Dec 2008 10:10:57 GMT

Ever since C9v4 was released the Sandbox has been nearly completely dead. Almost nobody ever posts in there anymore. I've no idea why that is, but I'm guessing the removal of the images has something to do with it.

in reply to R.I.P.: The Sandbox 2004-2008

"Your password has to meet stricter criteria" ["Your password has to meet stricter criteria"]

Sven Groot — Wed, 26 Nov 2008 11:02:00 GMT

I got a message from Leiden University, where I still have an e-mail address, that starting next year they will have more stringent criteria. They've increased the minimum length, require the use of at least one capital and one number, and it can't contain your login name. So far so good. It also doesn't affect me because my password is already stronger than that.

The list of criteria goes on: the password must be at most 13 characters and may not contain special characters such as ë, & or #.

WTF!!?

What are they attempting to achieve here? My current password is more than 13 characters and naturally it contains "special characters". So I will actually have to make my password weaker to meat their stricter criteria.

(I should add that this is the University-wide e-mail service, it isn't managed by the CompSci department who hopefully wouldn't make such stupid mistakes.)

in reply to "Your password has to meet stricter criteria"

Building a 737 [Building a 737]

Sven Groot — Thu, 13 Nov 2008 06:32:03 GMT

I came across this video, thought it was neat and felt like sharing. :)

We all know how to build software, and I'd wager most of you have seen a car assembly line in action. But how do you build something as big as an aircraft? For one thing, it takes more people. At normal capacity, the Boeing assembly lines put out one 737 every day!

in reply to Building a 737

Windows 7 UAC crippled - I broke it already! [Windows 7 UAC crippled - I broke it already!]

Sven Groot — Fri, 31 Oct 2008 16:09:41 GMT

I managed to get hold of the PDC build of Windows 7 via methods that I will not mention here. I installed it in Virtual PC and am pretty impressed on the whole. But this new UAC default setting makes it completely and utterly useless.

Windows 7's default UAC setting lets you change Windows settings without needing a prompt. In practice this means that any app that is part of Windows can elevate without a prompt. Even when another app is asking them to do it! Three lines of C# code is all I need to have my app write to parts of the registry that an untrusted app shouldn't be able to write to without getting a prompt.

Here's the code:
[code language="C#"]ProcessStartInfo p = new ProcessStartInfo("cmd.exe", "/k reg add HKLM\\Software\\malicious");
p.Verb = "runas";
Process.Start(p);[/code]

Run that code on Win7 with default UAC settings and it will not prompt and allow the registry write!

This is rediculous. They're going way too far in the opposite direction now. You might as well not have UAC at all and put up a sign "malware welcome!" or something. :(

in reply to Windows 7 UAC crippled - I broke it already!