Gcc Atomic Operations Causing SEGV

Questions

I ve been using a gcc atomic operation for quite a while in a multi-threaded application, and yesterday, ran into an interesting scenario that I cannot explain. These atomic functions are overloaded and can use datatypes that are 1, 2, 4, or 8-bytes wide. In particular, I use the bool_compare_and_swap (CAS) operation succesfully a lot. The issue that arises here is repeatable, and only occurs when I compile optimized code (O3). The issue does not arise when I compile unoptimized (O0). Keep this in mind, since I believe the optimizer is doing something funky in the case I m presenting here.

What I typically do is create a union of a struct containing named types (chars, shorts, etc.), and data type that is the appropriate size that "fits" that struct into one object (i.e. a long long), In this case, I have an 8-byte (long long) in a union whose struct counterpart contains bitfields. The example data type definition is shown below. The thought is that bitfields can be changed by assignment statements, and once all assignments are done, the 8-byte datatype is the one that is then CAS d. Specifically, I m using:

bool __sync_bool_compare_and_swap (type *ptr, type oldval type newval, ...)

wrapped in a macro as such:

define THD_CAS(ptr, oldVal, newVal) __sync_bool_compare_and_swap(ptr, oldVal, newVal)

I have a struct defined as follows:

typedef union _TSynchro {
    struct {
        int          *pFirstSynchWork;
        unsigned short   idTransaction;
        unsigned short       fNewTrans  :1,
            fFileBad        :1,                 
            fOpComplete :1,                 
            fCancelWork :1,                     
            fPurged     :1,
            fStatRequired   :1;
    } Data;
    // above struct is overlayed by this struct so we can CAS all values with a single 64 bit cas
    long long           n64;
} TSynchro;

So, I have a concurrency loop (while(1)) in code that grabs a "snapshot" of the current data value and store in Old, sets bits in a new copy (New) of the data, and then trys the CAS operation. If the CAS succeeds, I m the thread that changed the data and I break out of the loop. If the CAS failed, then some other thread changed the data under me, and I retry, grabbing another "snapshot" of the current data.

void NewSynchro(TSynchro *pSynchro)
{
    volatile TSynchro   New;
    volatile TSynchro Old;

    while (1) // concurrency loop
    {
        Old.n64 = pSynchro->n64;
        New.n64 = Old.n64;
        New.Data.fOpComplete = 1;
        New.Data.fStatRequired = 0;
        if (fFileBad)
        {
            New.Data.fFileBad = 1;
        }
        else
        {
            New.Data.fReleased = 0;
            New.Data.fFileBad = 0;
        }

        if (THD_CAS(&pSynchro->n64, Old.n64, New.n64))
            break;  // success
    }
}

Now, here is what s interesting...see that I m declaring the Old and New as volatile? Well, if BOTH Old AND New do not have the volatile modification, I get a SEGV when I step into the very next function call after the call to NewSynchro(). If EITHER OLD or NEW or BOTH have the volatile modifier, the application code never SEGVs. In development, I m only running 1 thread right now (no real threat of contention for changing the value), so I also tried getting rid of the CAS and replacing with a simple assignment (i.e., pSynchro->n64 = New.n64), and the application runs fine, too.

I ve been using the 8-byte CAS in other spots and it appears to be working fine. One difference here is that I think this is the first time I m using bitfields in a struct.

Thoughts?

Answers

Let me give a few thoughts: A union is normally intended to be used either-or: Either long long, or struct. In this case you use both, and it just works like on most simple processors. However, if you have a complex pipelined processor, you might run into necessities for memory barriers or similar. More concrete: Setting a bit in a bit-field is a read-modify-write operation. The problem might occur when you perform these operations in such an order:

New.n64 = Old.n64;
New.Data.fOpComplete = 1;

Because of the union, the read to set the bit can be started before the write of n64 is finished. This pipelining can be countered by the compiler inserting pipeline-flushes or memory barriers. But using a union, the compiler can assume the elements to be separate: "A union can contain only one of its component values at a time." (H&S5, 5.7.1) It will not be inclined to insert any flush/barrier, especially when using aggressive optimization like -O3. And using volatile for New will also direct the compiler (even with -O3) to make sure the read/write are properly separated. Why declaring Old volatile prevents your problems, I cannot say. Then I d have to see and compare the generated assembly code.

Source

License : cc by-sa 3.0

http://stackoverflow.com/questions/7586747/gcc-atomic-operations-causing-segv

Related