Tech Off Thread

14 posts

Forum Read Only

This forum has been made read only by the site admins. No new threads or comments can be added.

Timing APIs - This seems like an oversight, or am I missing something?

Back to Forum: Tech Off
  • User profile image
    BitFlipper

    I am currently experimenting with creating a "DAW" type application that can record and play back MIDI and audio data. It has MIDI and audio tracks, and can host VST plugins (which themselves can consume and produce both MIDI and audio data). This is written in C# so I'm P/Invoking into the unmanaged multimedia APIs, but this should not be too relevant to my issue...

    So when recording both MDI and audio data, and having the requirement to sync them up, I have to use some sort of timing reference:

    For audio (at least for an ASIO driver - I have not wrapped other audio APIs yet), it will tell me the system time in milliseconds for each audio buffer it presents to me. This time is supposedly derived from the timeGetTime call. This returns the number of milliseconds since the system was booted. OK fine, I can work with that.

    For MIDI, each MIDI message that arrives will also be time-stamped, also derived from timeGetTime. OK, that is also good, since at least there will be no drift between the audio and MIDI input. However, the MIDI time stamp is not system time, it is time since my app called midiInStart. So now I have to keep record of the exact time I called midiInStart. This can be done with somethin like:

    midiInStart(hMidiIn);
    m_midiStartTime = timeGetTime();

    Then I can align the MIDI events with audio buffers by adding m_midiStartTime to the timestamp for each MIDI message.

    OK, here is my problem: What if the thread was pre-empted between the two calls? Now I will have an invalid start time reference. It might not sound like much, but a few milliseconds could make a difference, and I would really like to know what the "right" way is to handle this case.

    Is this an oversight in the multimedia API? If there was an additional API that allowed me to retrieve the true reference time that the MIDI driver will use, it would solve my problem. Something like midiInGetStartTime or similar.

    BTW, I set the timer resolution (and hence OS resolution) to 1ms by calling timeBeginPeriod. I can get down to 1ms resolution when using APIs like timeSetEvent (verified that timing accuracy is < 2ms).

    Any ideas on how to handle this situation?

  • User profile image
    dentaku

    Not that I can help but I just wanted to say I'll be interested to see what you're working on when there's something to show. I guess you could call me a VST collector Smiley

    I was just at the Windows Phone 7 - App Hub Forums and there are still people talking about MIDI tones and MIDI sounds. It's like after all these years people haven't figured out the MIDI is NOT sound and really has nothing to do with sound other that being a good way of controlling instruments which make sound.

  • User profile image
    Sven Groot

    MIDI, yes, but General MIDI does define a standard set of sounds that MIDI instruments should provide. Tongue Out

    Anyway, getting an accurate timer on a non-realtime OS is going to be impossible in the general case.

     

  • User profile image
    teslaBytes

    see, Make.com many Midi, hacks.

  • User profile image
    BitFlipper

    I posted the same question on the MSDN forums and someone suggested the following solution:

    while (true)
    {
        var timeBeforeCall = TimeGetTime();
        MidiInStart(...);
        var timeAfterCall = TimeGetTime();
        
        if (timeBeforeCall == timeAfterCall)
        {
            m_startTime = timeBeforeCall;
            break;
        }
    
        MidiInStop(...);
    }

    It is a bit...inelegant. Although I guess with these ancient APIs you have to make do with what you get. My only concern is, how can one be sure that the MidiInStart call will able to complete within 1 ms? What if the driver does some intense initialization that always take longer than 1ms? Then you have an infinite loop. One can guard against that by looping some fixed number of times, and if it still hasn't exited the loop, introduce an offset that is added into the equality comparison that gradually increments over time. Eventually you will hit an iteration where it will satisfy the equation and exit. Shudder...

    BTW I implemented it this way and it works. Well, on my system it does...Scared

  • User profile image
    Sven Groot

    I suppose you could try this:

    Thread.Sleep(0);
    midiInStart(...);
    var timeAfterCall = timeGetTime();

    The Thread.Sleep call will yield execution to a different thread, which means that it will have its entire time slice available when midiInStart is called making it extremely unlikely the thread will be interrupted between the two calls.

    Note that you'll never get this done completely accurately because you can't account for the time that midiInStart itself takes after it resets the time stamps (which is also the problem that might cause the looped version to hang).

  • User profile image
    BitFlipper

    @Sven Groot:

    I actually did it like that initially (with the Sleep) but it still felt like it might not be 100% accurate because who knows what goes on inside the midiInStart call, as you mention. You just don't know for sure...

    I think I like the solution with the loop, and I guess it is unlikely that the midiInStart call will take longer than 1ms.

  • User profile image
    Sven Groot

    Personally, I'd go with the simple way, then through testing determine whether the accuracy is actually a problem, and only when it turns out that it is do something as convoluted as that loop.

  • User profile image
    BitFlipper

    @Sven Groot:

    I think the more complex solution makes sense for the same reason that we would use locks around resources in a multithreaded app, use Interlocked.Increment, volatile, etc. It isn't that the situation will arise very often where we could run into problems in these scenarios, but we do want to ensure we cover the 0.1% of the time where we would be in in trouble otherwise.

    I mean, it not really that much more complex. It's 5 extra lines of code and it is pretty easy to see what is happening.

  • User profile image
    Sven Groot

    Except it's code that can, by design, potentially hang your application. If you do go this route, you must have some way to break the loop if the values never become equal, which already makes it more complex.

    Locks are not comparable. Those are used because in their absence, race conditions create incorrect programs. Here you have a situation where the original code is not incorrect, it just potentially has slightly less accurate timing, and as I said accurate timing is impossible anyway on a non-realtime OS.

    Additionally, every programmer probably has at some point in their career encountered a race condition. We know these exist and that they are a problem. You have not yet observed that there is an actual problem, you have just theorized there might be, and are going to introduce what I can only describe as a really ugly hack based on that theory. You're doing something ugly without actually knowing for sure that it's necessary. To me, that's the same kind of thing as premature optimization, which is why I say you should test first to see if your code actually needs this hack.

     

  • User profile image
    BitFlipper

    @Sven Groot:

    The reason I mention locks is because it follows a very similar thought process: Are you going to leave out a lock (...) around your shared resource because even though multiple threads might access the same resource at the same time, it would be a rare case if it did? Well if you haven't done testing proving that two threads can indeed access the same resource at the same time, why bother putting the lock in? The point is, you don't need to do testing to know that it can happen.

    I can sit here doing 1000 test runs and never experience the problem. That does not mean that the thread cannot be pre-empted somewhere along the line (most likely inside one of the two calls). I already know it can happen because that is what pre-emptive multitasking does.

    Now the question is, in the rare cases that it does happen, what are the consequences? Well, I just tested how long a thread is pre-empted in this case and it is around 6ms when things get even just a little bit busy. Now you might think 6ms isn't much, but even Microsoft's MIDI API documentation states that MIDI resolution needs to be down to 1ms. In the commercial DAW software I use, I can easily tell the difference if MIDI events are shifted by 6ms compared to other events in the song. It just sounds off. Either dragging or rushing. Couple this with the fact that if the thread was pre-empted during the two calls, then it will be out of sync for the whole time while the application is running. That means that all recording that is done during that time will be out of sync.

    I also believe things should be made as simple as possible, but sometimes to get the desired results (or guard against unlikely but possible edge-cases), you do need to make the code a bit more complex. Caching is another example of this. Any sort of caching makes code more complex, but the end usually justifies the means and results in a better end-user experience.

    BTW, I followed your suggestion and did some testing. I tested how long the midiInStart call takes to complete, and it is around 15us - 19us. That is well below 1ms.  I have 4 unique MIDI drivers in my systems and they are all in that range for some reason (maybe this call is a system-only call, and the system only sets some state for each driver implementation? Not sure). Whether other MIDI drivers could potentially take longer than 1ms is not known, but if this software ever runs on other systems (it is after all just a pet project), this code will be made more robust to guard against that situation. I think the end justifies the means.

  • User profile image
    dentaku

    It's good to know some people here actually understand MIDI Smiley I hope WP7 gets some good quality audio apps soon. It's very important if people are to take the OS seriously. IOS 4.2 even has Core MIDI now.

    As for accurate timing, I actually had some experience dealing with this in WP7 this Christmas break too (and I'm not a programmer). http://channel9.msdn.com/Forums/TechOff/hey-look-an-accurate-metronome-ticker-on-WP7

    , Sven Groot wrote

    MIDI, yes, but General MIDI does define a standard set of sounds that MIDI instruments should provide. Tongue Out

    Anyway, getting an accurate timer on a non-realtime OS is going to be impossible in the general case.

  • User profile image
    AndyC

    All that looping doesn't actually gain you anything. It does, however introduce an unnecessary race condition.

    Consider, for example, what happens if the thread gets pre-empted right after the 'if'' condition has tested true and your thread doesn't get rescheduled for a very long time. You can't avoid this on a general purpose, non-realtime operating system.

     

     

     

     

  • User profile image
    BitFlipper

    @AndyC:

    Sorry, I don't follow your logic. Once timeBeforeCall == timeAfterCall, it doesn't matter whether the thread gets pre-empted or not, since I know exactly what reference time the MIDI driver will be using. Which is the whole point of the loop.

    Maybe you are missing the fact that the value that is ultimately assigned to m_startTime is also the value that was used to test whether the thread was pre-empted or not, and that value will not change no matter how long the thread gets pre-empted after the point where timeAfterCall is retrieved? The only time your logic makes sense is if I assigned the value returned by yet another call to TimeGetTime to m_startTime, which is not what I'm doing.

    Put another way... At some point within the MidiInStart call, the driver calls TimeGetTime to establish its reference time. If I can put a TimeGetTime call before MidiInStart and one after MidiInStart, and determine that those two calls return exactly the same value, then I am guaranteed that the value that MidiInStart retrieved internally is also that exact same value. Which is exactly the value I wanted to determine.

    Am I missing something?

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.