UST/MSC: Using the Frontier MSC to Detect Errors

In most of Introduction to UST and UST/MSC, we carefully assumed that:

• On input, your AL or VL buffer never overflows. This means that you read data out of your buffer frequently enough that the device always has room to deposit new data, so no data is ever dropped.

• On output, your AL or VL buffer never underflows. This means that you write data to your buffer frequently enough that the device will never starve for data, so the output will never glitch (audio click, video flash or pause).

You can also use the frontier MSC to detect overflow and underflow conditions, and determine their precise length. To understand how this works, you have to remember that the MSC "slots" we described earlier are entries in the input or output signal and not necessarily your data. Usually your signal perfectly matches your data:

If overflow occurs on input, the timeline of your signal and data tear apart and you are left with slots (MSCs) of the input signal that never appear in your data:

If underflow occurs on output, the timelines again tear apart and you are left with slots (MSCs) of the output signal that contain pad data:

The pad may consist of black/silence, colorbars, or a duplicate of earlier data, depending on the device.

In the next sections, we'll show you how the tearing effect above affects the frontier MSC, and how you can use the frontier MSC to detect overflow and underflow.

Underflow on Input or Overflow on Output?

• Underflow on input is not possible because the calls that read data (alReadFrames(), vlGetNextValid(), vlEventRecv()) will block or return failure until your requested amount of data becomes available.

• Overflow on output is not possible because the calls that allocate space to write data (alWriteFrames(), vlGetNextFree(), dmBufferAllocate()) will block or return failure until your requested amount of space becomes available.

Detecting Underflow on Audio Output

This diagram is like the AL output diagrams in Introduction to UST and UST/MSC, except:

• We have removed USTs, since we can detect underflow using only MSCs.

• We have split the audio frames within the audio subsystem into two groups:
  • The part shown as "AL ringbuffer" represents the internal AL buffer which you access with alReadFrames() (input ports) and alWriteFrames() (output ports). This is the buffer you must consume on input to prevent underflow, and fill on output to prevent underflow. You create the buffer with alSetQueueSize() and alOpenPort(), and query its current state with alGetFilled() or alGetFillable(). In the diagram above we show an AL buffer with 6 entries, 4 of which are currently filled.

    The VL has an analogous buffer. For the classic VL API, you create the buffer with vlCreateBuffer(), access it with vlGetNextValid(), vlGetNextFree(), vlPutValid(), and vlPutFree(), and query its current state with vlGetFilled(). For the O2 and cross-platform VL API, you create the buffer with dmBufferSetPoolDefaults() and dmBufferCreatePool(), access it with vlEventRecv()/vlDMBufferGetValid(), vlDMBufferSend()/vlDMBufferPutValid(), and other DMbuffer calls, and query its current state with vlGetFilledByNode()/vlDMBufferGetFilledByNode() and dmBufferGetPoolState(). See What Are the SGI Video-Related Libraries? for more info.

    For typographical convenience, the AL examples in this document will use 6 frame AL buffers and 2-5 frame reads and writes, and underflows and overflows will last 1 frame. Typical programs use buffers and transfers of at least 10ms, and errors can easily exceed 1 frame in duration.

  • The part shown as "Internal processing delay" represents other buffering going on within the subsystem. This delay may vary while your system is running, and it may be a non-integral number of sample periods.

Now the program writes the frame:

After writing one frame, the AL ringbuffer now has 5 filled entries instead of 4. The program sees that the frontier MSC has gone up by 1. This makes sense since alWriteFrames() now addresses the next slot in the output signal. If there is no underflow, then every time we put N items into the buffer, we would expect the frontier MSC to go up by exactly N.

Now say the program goes off and does something else, and during this time the audio system pulls out 5 frames:

Even though the buffer is now empty, the audio system is still happy: every time it has needed a frame from the buffer, one was available, so there is no underflow.

Note that the frontier MSC is unchanged. This makes sense because alWriteFrames() still addresses the same slot in the output signal. If there is no underflow, and we do not write any data, then the frontier MSC should remain unchanged.

Now say the program dallies a little longer and the audio subsystem unsuccessfully tries to pull out one more audio frame:

Now the buffer is underflowing. Interestingly, the frontier MSC has gone up, even though we did not write any data. This follows directly from the underflow timeline picture. When the timeline of your signal and your data are torn apart by an underflow, MSCs track the signal, not your data. If we were to repeatedly get the frontier MSC without writing any more data, we would see it increment once per audio frame period. The frontier MSC will return to a stable state as soon as we write some more data.

You can use this behavior to detect underflow:

{     
  stamp_t newmsc, oldmsc=-1;
  
  /* this is your main data-writing loop, not a special one */
  while (1)
    {
      alWriteFrames(port, buf, nframes);
     
      alGetFrameNumber(port, &newmsc);
      if (oldmsc > 0)
        {
          stamp_t M = (newmsc-oldmsc) - nframes;
          if (M != 0)
            printf("we underflowed for %lld MSCs!\n", M);
        }
      oldmsc = newmsc;
    }
}

Some applications will simply abort if they ever see M > 0. Other applications will use the value of M to implement the fastest possible recovery scheme.

Detecting Overflow on Audio Input

{     
  stamp_t newmsc, oldmsc=-1;
  
  /* this is your main data-reading loop, not a special one */
  while (1)
    {
      alReadFrames(port, buf, nframes);
     
      alGetFrameNumber(port, &newmsc);
      if (oldmsc > 0)
        {
          stamp_t M = (newmsc-oldmsc) - nframes;
          if (M != 0)
            printf("we overflowed for %lld MSCs!\n", M);
        }
      oldmsc = newmsc;
    }
}

If your application cares about accurately determining the UST of the samples which did make it into the buffer, or if you want a pictorial example of how the frontier MSC behaves on input overflow, read on.

First, your program opens an AL input port:

This diagram is like the AL input diagrams in Introduction to UST and UST/MSC, except:

• We've removed USTs and separated the frames within the audio subsystem as described in Detecting Underflow on Audio Output above.

• We have labeled each slot (MSC) of the signal with a letter:

• We have also labeled each data location in the audio subsystem and your program with the letter of the slot whose signal it contains.

After reading 3 frames, the AL ringbuffer has 1 filled entry instead of 4. The program sees that the frontier MSC has gone up by 3. This makes sense since alReadFrames() now addresses a slot which is 3 slots later in the input signal. If there is no overflow, then every time we take N items out of the buffer, we would expect the frontier MSC to go up by exactly N.

Now say the program sits around, and during this time the audio system puts in another 5 frames:

Even though the buffer is now full, the audio system is still happy: every time it has needed space to put a new frame in the buffer, it was available, so there is no overflow.

Note that the frontier MSC is unchanged. This makes sense because alReadFrames() still addresses the same slot in the input signal. If there is no overflow, and we do not read any data, then the frontier MSC should remain unchanged.

Now say your program delays even longer, and during this time the audio system unsuccessfully tries to put in one frame:

Now the buffer is overflowing. The frontier MSC has gone up, even though we didn't read any data. Your application can immediately tell that the buffer is overflowing. If we were to repeatedly query the frontier MSC without reading any more data, we would see it increment once per audio frame period. The frontier MSC will return to a stable state as soon as we read some more data.

Notice in the diagram that the labels A-I for the data on the right do not match the labels for the MSCs B-J on the left. While the buffer is overflowing, the 6 frames of data in the buffer (D-I), and the 3 frames we read earlier (A-C) stay the same. Yet the frontier MSC tells us that the 3 frames we read earlier have MSC 55-57 (B-D), and that the next frame we read from the buffer will have MSC 58 (E). An overflow causes the frontier MSC to "mislabel" the MSC of data that arrived before the overflow. If we were to use a UST/MSC pair to compute the UST of our 3 frames, we would get the wrong UST.

This is why it is crucial to use the frontier MSC to compute USTs (with the UST/MSC pair) only when the buffer is not underflowing and not overflowing, and why extra caution is required even after an overflow.

How long will this mislabeling of MSC last? Fortunately, only the audio frames that arrived before the overflow are affected. Say we relieve the overflow by reading 2 frames with alReadFrames(), and then the audio system deposits 2 new frames into the buffer:

Notice that the labels for the data and MSC of slots K and L again line up. When you read frame K, L, and all subsequent frames up to the next overflow, you will be able to compute their MSC correctly using the frontier MSC. So if you do nothing special, then your program will recover to a state of computing correct USTs based on correct MSCs within an amount of time roughly equal to your AL ringbuffer size.

Since you can use the frontier MSC to compute the exact length of each overflow, it turns out that you can compute the correct MSC of every frame in your buffer, even during overflows. In the worst case, this requires that you store as many stamp_t's as there are entries in your AL ringbuffer, since repeated overflows could generate that many discontinuities in the MSC of items in your buffer. So far, no audio developer has needed to do this, so we won't bother going into details.

Because video MSCs are relatively infrequent compared with audio MSCs, video applications can solve this "MSC mismatch during overflow" problem more easily. The VL provides input timestamping mechanisms, described in Introduction to UST and UST/MSC, which allow your program to extract a UST (and in some cases an MSC) for every piece of incoming data. Because the UST and MSC/sequence become associated with each VLInfoPtr or DMbuffer as soon as the data enters the computer, successfully captured data never becomes mislabeled. Video developers might consider using these timestamping mechanisms instead of UST/MSC for programs that only do video input.