The VL source code examples we ship demonstrate several possible ways to write one's main event dispatching loop, none of which are sufficient for typical real applications. This is because typical real applications:
In additon, there are some outright bugs in some of our sample programs that happen to surface only very rarely in those programs, but may surface more commonly in real app code.
This document shows you how to roll your own main event loop in a way which will navigate you around the most common pitfalls of VL programming, and in a way which will not yank control of the main select() loop away from you. Then it shows you how to deal if you are unfortunate enough to be stuck with XtMainLoop().
This document will assume that your app creates a VL path involving memory. Much of what is said here is useful even if that is not the case.
Some of the tips described here depend on whether you are using the classic VL buffering API or a DMbuffer-based VL buffering API. Check out What Are the SGI Video-Related Libraries? for information on what that means.
The VL offers two different file descriptors:
You should generally select() on both file descriptors. VL events are useful for notifying your app of path-global events such as transfer failures, control changes, GPI triggers, and VLPath sharing events. However, when you need something to wake you up as soon as possible when data or space becomes available, the buffer fd is far superior. The buffer fd tends to fire much sooner than the corresponding VLTransferComplete event would arrive. This is partially because the UNIX kernel event that makes the VL buffer fd fire goes right from the video driver to the app, whereas the VLTransferComplete event from the driver has to take a trip through video daemon (on some systems) before it gets to the app.
On some systems such as O2, the event fd and the buffer fd may be the same. If you write code that assumes the file descriptors are different, it should work on all implementations (FD_SET and poll() will do the right thing).
If you are using a DMbuffer buffering API in a memory-to-video program, you may also be interested in the fd returned by dmBufferGetPoolFD(). See below for more information.
Because the buffer fd is a superior source of data/space available events, there really isn't any reason to include VLTransferComplete events in your VLEventMask (unless of course you have no VLBuffer in your path). Some code we have seen uses VLTransferComplete and VLSequenceLost events to try and detect exactly which items were dropped or duplicated when the VLBuffer overflows or underflows. This method will not work on all devices---VLTransferComplete and VLSequenceLost are only indications that one or more items transferred or failed to transfer. See the section on "detecting dropped or duplicated data" below for a cleaner and more portable way to detect dropped or duplicated data.
If you are using the O2 buffering API for a video-to-memory path, you receive data as part of VLTransferComplete events. Because there is no video daemon on O2, event reception is as efficient as using the buffer file descriptor on other systems. So it's a good idea to field VLTransferComplete events for transfers to and from memory.
On some systems, the file descriptor returned by vlGetFD() has a bizarre and undocumented behavior when used in a select() loop. This is the same behavior exhibited by the IRIS GL file descriptor returned by and documented in qgetfd() (qtest() is vlPending() in this context):
...to ensure the proper behavior of select, call qtest before and after you call select. When select indicates data is available on the file descriptor, there may not be GL event data to read. Similarly, there is a condition where select would block, but data is available. The use of qtest before and after select guards against this...Below, we will present a code example which heeds this warning.
Classic and Cross-Platform buffering API:
We have seen VL code which assumes that it will receive one VLTransferComplete event for each incoming item of data. We have seen other code that assumes that exactly one item of data/space is available every time the buffer fd unblocks. Neither of these strategies will yeild correct code on all VL devices. Both VLTransferComplete and the buffer fd are indications that one or more items of data/space are available.
An example: say you're doing video-to-memory, and you process 1 item per VLTransferComplete event. You may get a VLTransferComplete event which "counts" more than one input item, but you have only processed one input item. Each time this happens, the fill level of your input buffer will grow, until eventually your input buffer is permanently overflowing.
O2 buffering API:
For memory-to-video, there is always one item of data for each incoming VLTransferComplete event. The video-to-memory situation is as with the classic buffering API: each VLTransferComplete could indicate one or more free spaces available in the pool. Either way, since other events (VLTransferComplete or not) can fill the event queue, you still cannot assume that exactly one item of data or space is available each time you wake up on the buffer/event fd.
All buffering APIs:
Therefore, every time you wake up, your N-item buffer may contain anywhere between 0 and N valid (video-to-memory) or free (memory-to-video) items. It is a good idea to process all of those items before going to sleep again. Here are the ways you can do that:
N-vlGetFilled() will not do the trick. So move on to the
next method.
The way around this is simply to impose some finite limit on the number of video items the application processes per loop. A good limit to choose is N.
Here is some sample code using the classic buffering API which illustrates these points in action:
{
int vlFD, vlBufferFD;
/* open VL */
/* set up VL path */
/* create and associate VLBuffer of size "buffer_size" items */
/* vlSelectEvents: mask out VLTransferComplete/VLSequenceLost events */
/* vlBeginTransfer */
vlFD = vlGetFD(vid->vlSvr);
vlBufferFD = vlBufferGetFd(vid->rb);
while (1)
{
int i;
/* first check to see if VLEvents are available */
while (vlPending(vid->vlSvr))
{
VLEvent ev;
VC(vlNextEvent(vid->vlSvr, &ev));
switch (ev.reason)
{
case VLTransferFailed:
... /* transfer has now stopped. deal with that. */
break;
/* ... other events ...*/
}
}
/* now check to see if we have some data/space available */
if (path_is_video_to_memory_path)
{
int nfilled = vlGetFilled(vid->vlSvr, vid->buf);
for(i=0; i < nfilled; i++)
{
VLInfoPtr info;
info = vlGetNextValid(vid0>vlSvr, vid->rb);
assert(info); /* filled count sez there must be data */
...; /* use the data! */
vlPutFree(vid->vlSvr, vid->rb);
}
}
else if (path_is_memory_to_video_path)
{
/* classic VL buffering API has no "vlGetFillable()" operation */
/* process at most N items */
int nfillable = buffer_size;
for(i=0; i < nfillable; i++)
{
VLInfoPtr info;
info = vlGetNextFree(vid0>vlSvr, vid->rb);
/* we're not sure exactly how many times we'll succeed */
if (!info)
break;
...; /* use the space! */
vlPutValid(vid->vlSvr, vid->rb);
}
}
/* now do the select() */
{
fd_set readset, writeset, exceptset;
FD_ZERO(&readset);
FD_ZERO(&writeset);
FD_ZERO(&exceptset);
/* video */
FD_SET(vid->vlFD, &readset);
FD_SET(vid->vlBufferFD, &readset);
/* do FD_SETs for other agents: audio, serial, X, gfx, ... */
if (-1 == select(getdtablehi(), &readset, &writeset, &exceptset, NULL))
...;
}
/* now service the events that woke up the select() */
{
/* at this point, we don't even have to call FD_ISSET() on any
* of the video fd's if we don't want to---vlPending and
* vlGetFilled will tell us all FD_ISSET() would and more.
*/
/* act on FD_ISSET for other agents: audio, serial, X, gfx, ... */
}
}
One thing to notice is that upon entering this main loop for the first time, we call vlPending() before we select() for the first time. This is crucial. As explained by the obscure paragraph from qgetfd(3G) quoted above, there are circumstances where select() could hang forever even though VL events are available. We chose to also put the call to vlGetFilled() before select() just because it was convenient. There could very well be data/space in the buffer before the first select(), but in this case the buffer fd would unblock immediately, so it's not a big deal.
As any Motif programmer knows, Xt owns the main loop for your
application, and it calls select() for you. You have
little control over that. However, it is still possible for an Xt
program to respond to VL events. It is also possible to handle VL
events which may already be in the queue before
XtMainLoop() calls select(). Finally, you
can even do all of this in a ViewKit program, without munging
VkApp:run() (ViewKit's wrapper around the Xt event
loop).
To respond to VL events, you must register Xt callbacks for the VL event queue, and also for the transfer buffer, if your application uses one. For each, you provide a callback handler function. Here is some sample code to set up the callbacks.
void setupVLEvents()
{
//
// register an Xt callback for the VL's event queue.
// EventHandlerFunc is an Xt input callback which will
// process VL events.
//
videoInputId = XtAppAddInput( appContext,
vlConnectionNumber(svr),
(XtPointer) XtInputReadMask,
&EventHandlerFunc,
(XtPointer) clientData);
//
// Register an Xt callback for the file descriptor associated
// with the VL buffer. We do this instead of asking for
// TransferComplete events.
//
// BufferFullHandlerFunc is an Xt input callback which will
// take the next frame from the buffer and display it on the
// screen.
//
videoBufferId = XtAppAddInput(appContext,
vlBufferGetFd(buf),
(XtPointer) XtInputReadMask,
&BufferFullHandlerFunc,
(XtPointer) clientData);
//
// Choose the VL events we want to listen to. Your application
// might have a different set of events it cares about than the
// ones shown in this example.
//
vlSelectEvents(svr, displayPath,
VLDeviceEventMask |
VLDefaultSourceMask |
VLStreamPreemptedMask |
VLStreamAvailableMask |
VLControlChangedMask |
VLControlPreemptedMask |
VLControlAvailableMask |
VLTransferFailedMask
);
}
void removeVLEvents()
{
if (videoInputId != NULL) {
XtRemoveInput(videoInputId);
videoInputId = NULL;
}
if (videoBufferId != NULL) {
XtRemoveInput(videoBufferId);
videoBufferId = NULL;
}
}
A sample callback handler for VL events might look something like this:
void EventHandlerFunc(XtPointer client_data, int*, XtInputId*)
{
EventHandler();
}
void EventHandler()
{
VLEvent ev;
while (vlPending(svr)) {
vlNextEvent(svr, &ev);
switch (ev.reason) {
case VLTransferComplete:
break;
case VLTransferFailed:
transferFailed(ev);
break;
case VLDeviceEvent:
deviceEvent(ev);
break;
case VLDefaultSource:
defaultSourceChanged(ev);
break;
case VLStreamPreempted:
streamPreempted(ev);
break;
case VLStreamAvailable:
streamAvailable(ev);
break;
case VLControlChanged:
controlChanged(ev);
break;
case VLControlPreempted:
controlPreempted(ev);
break;
case VLControlAvailable:
controlAvailable(ev);
break;
default:
printf("unhandled VL event, reason %d\n", ev.reason);
break;
}
}
}
The callback handler for the VL transfer buffer looks something like this:
void BufferFullHandlerFunc(XtPointer client_data, int*, XtInputId*)
{
BufferFullHandler();
}
void BufferFullHandler()
{
VLInfoPtr info = NULL;
info = vlGetLatestValid(svr, buf);
if (info != NULL) {
// do something with the video data. Here, we draw it to the
// screen.
void* dataPtr = vlGetActiveRegion(svr, buf, info);
glRasterPos2i(originX, originY);
glPixelZoom(1.0, -1.0);
glDrawPixels(sizeW, sizeH,
GL_ABGR_EXT, GL_UNSIGNED_BYTE, dataPtr);
vlPutFree(svr, buf);
}
}
To make sure you properly handle VL events which may already queued before select is called, you should explicitly call the Xt event handlers immediately after starting the transfer on the path:
void startVideo()
{
// ... other VL setup code ...
setupVLEvents();
// start video transfer.
vlBeginTransfer(svr, displayPath, 0, NULL);
// do a first pass check to handle any
// queued events from the VL or the transfer buffer
BufferFullHandler();
EventHandler();
}
The caveat: say your app wants to wait until there are N (N>1) items in an input buffer, and then dequeue them all at once. For example, it may want to coalesce buffers for disk writing or interleaving. Say your app selects on the buffer or event fd. After the first item arrives in the VL buffer, these fds will always unblock immediately, and your app will begin to spinloop. At the moment, the only recourse offered to a VL app is not to select() on the buffer fd when there are already items available (which you can tell from vlGetFilled()), and instead to provide select() with a timeout on the order of one field time. This is rather gross. Another gross solution is to call vlGetNextValid() or vlGetNextFree() as soon as any data or spaces become available, keeping track of the resulting VLInfoPtrs yourself. To the VL, each time you call select(), there is no data or no space, so your app will block for a reasonable amount of time waiting for the next data/space. When you have accumulated N VLInfoPtrs, you act on the data.
A similar thing happens if your app wishes to wait until there are N spaces in an output buffer. Since there is no vlGetFillable(), you have to use the second technique above---grab free data immediately and keep track of VLInfoPtrs yourself.
The real solution to this problem has been in the AL and other libraries for some time---fillpoints (see ALsetfillpoint(3)). Perhaps someday we will have fillpoints in the VL.
This caveat also applies to the DMbuffer buffering APIs. The buffer/event fd and the dmBufferGetPoolFD() fd will unblock immediately if their condition is true. The dmBufferSetPoolSelectSize() function lets you set the threshold of free DMpool space for unblocking, but this function was not implemented as of 12/16/97 (the select size was fixed at one item).
The VL events VLTransferComplete and VLSequenceLost indicate that one or more items of video data transferred or failed to transfer. There is not (for all VL devices) a one-to-one correspondence between these events the individual fields/frames that transferred or failed to transfer. Therefore, these VL events are not useful for determining exactly which items were dropped or duplicated.
For video to memory paths with the classic buffering API, the easiest way to detect dropped data is to look at the "sequence" field of the DMediaInfo struct returned by vlGetDMediaInfo() for a particular item of video data (a particular VLInfoPtr). Every VL device keeps a field counter which increments by one every time a field crosses an input jack of the machine (whether or not that field is successfully transferred into memory). When a transfer is successful, the VL will snap the current value of the sequence value and place it in the DMediaInfo struct of that particular VL item (for frames you get the DMediaInfo struct for the first field of the frame).
For video to memory paths with the DMbuffer buffering APIs, you can retrieve similar timestamp information from a DMbuffer using dmBufferGetUSTMSCpair(). In this case the counter value has the units of MSC, which may be fields or frames depending on VL_CAP_TYPE (see vlGetFrontierMSC(3dm)).
Therefore, by watching for jumps in the "sequence" or MSC values of successive pieces of video data, you can detect all failed transfers, you can tell exactly where they fell in the sequence of data, and you can tell exactly how many fields were dropped. This is the recommended method for detecting drops on input.
At the moment, there is no equivalent queue of DMediaInfo or USTMSCpair structures coming back to the application in a memory to video path. Perhaps one day there will be.
But it turns out that there is a second mechanism (part of the UST/MSC implementation in the VL), called vlGetFrontierMSC(), which can be used to determine all the same statistics for both input and output in a symmetric manner. The mechanism works identically for both buffering APIs. See the Lurker Page Introduction to UST and UST/MSC and the man page vlGetFrontierMSC(3dm) to find out how to detect all the failed transfers and the length of all the failures. You can also use vlGetFrontierMSC(3dm) to determine exactly which video items were dropped or duplicated, but this is more tricky and is explained in UST/MSC: Using the Frontier MSC to Detect Errors.