There are many useful cases where checks done at compile time can prevent runtime errors. Consider the following (non-compiling) code:

struct {
    int x;
    int y;
} a;
#if (sizeof(a) != 8)
#error "Struct a is not the right size!"
#endif

Unfortunately this code doesn’t compile. The ‘#if’ statement is evaluated at precompile-time, but sizeof is a compilation instruction. The solution is compile time assertion. A little hack that forces the compiler to evaluate an expression during compilation follows.

CCASSERT – Compile Time Assert Macro
In order to evaluate the expression, the macro will try to define an array with size calculated by predicate. Note that this is legitimate ANSI-C code, so most compilers should support it.

#define CCASSERT(expr) \\
           typedef char ASSERT_CONCAT(constraint_violated_on_line_, __LINE__) \
                               [2*((expr)!=0)-1];
#define ASSERT_CONCAT(a, b) ASSERT_CONCAT_(a, b)
#define ASSERT_CONCAT_(a, b) a##b

Example usage:

#define TOTAL 256
#define SIZE1 122

int arr1[SIZE1]
int arr2[TOTAL - SIZE1];

CCASSERT(sizeof(arr2) == 134 * sizeof(int))

In case the validation fails, the following compile time error will occur:

error: size of array ‘assert_on_line_8’ is negative

When approaching the string comparison optimization problem, what we would like to do is to provide effective and efficient ways to rule out most of the candidate strings. We may refer to it as a “disqualifying comparison” – it lets us move faster down the search tree or move faster along the hash bucket linked list, until reaching the final string comparison in the search, keeping in mind that even the most efficient hash structure would probably waste a substantial amount of its time and cycles in string comparison.

Note that this article assumes all strings in the world are composed of 1 byte characters, which is not true (unfortunately for our community, someone invented Unicode…). However, to demonstrate the principals let’s assume they do.

A. Different lengths is sometimes enough

When the only need is to find out whether two strings are identical or not, checking that string lengths differ is enough for ruling out most of the compared candidates. Of course, that doesn’t mean calculating the lengths of strings for each comparison – In common practice, usually, all that needs to be done is storing the string lengths along with the actual strings in the data structures, and designing the software so that the query will also contain both the actual string and its length. Therefore, in runtime, there will be no need for calculating lengths for most comparisons.
Only when the string lengths are the same, there is a need to actually compare the strings themselves.
Obviously, the above optimization would be very effective only for data repositories that contains strings with different lengths, but in these cases,for each string, we minimize all the comparison operations to only one. When comparing user names, URLs, or other human readable resource names, this little optimization would prove itself.

B. Why compare byte by byte anyway?

It turns out that some (if not most) implementations of libc use a byte by byte comparison for strcmp and other string comparison functions. For example, a snippet from OpenBSD’s libc implementation (found with Google Code Search):

int
strcmp(const char *s1, const char *s2)
{
        while (*s1 == *s2++)
                if (*s1++ == 0)
                        return (0);
        return (*(unsigned char *)s1 - *(unsigned char *)--s2);
}

A much more efficient implementation would compare elements according to the processor’s block size, size_t bytes usually. A 64 bit architecture is able to compare 8 ASCII characters in one cycle – why not use it?
In many cases, a simple solution would be to use memcmp instead of strcmp. In many platforms, memcmp is implemented very efficiently – some by using a block size comparison in C as in this glibc implementation, and some even implement it very carefully in assembly, such as Sparc64 linux kernel implementation). When you know your memcmp is not as efficient, it’s just implementing your own function, relying on implementations such as these from the newest glibc you can find.

C. Direction counts

In some cases, all strings have similar characteristics, specifically – similar prefixes or suffixes. For example, a repository of phone numbers is likely to have many similar prefixes, whether the same country code for USA or even the same area code prefixes for all the phone numbers in the same state. In our example, the length comparison optimization described above would not be so effective since all the phone numbers in the same state have the same lengths.

However, an effective optimization here is to implement reverse order string comparison function. This would rule out most of the strings is much faster than using regular comparison methods.
Going forward with the same optimization method, looking at the string characteristic may imply the optimal comparison function to use; As an example, let’s examine a repository with picture file names. Each file name is likely to begin with, let’s say, the prefix “pic” and end with the file extension, which is probably “.jpg”. Therefore, an effective disqualifying comparison would probably be a reverse order comparison starting four characters from the end.

D. Boost your case insensitive comparisons

Some string based repositories are required to be case insensitive. Therefore, given a search string, a naive implementation would first transform the string to lower case letters and only then search for it. However, this means copying the query string. Given that most of the operations to be done are queries, there must be a way to avoid these string copy operations.
A nice solution found on one GameDev forum is using the magic constant 0xDF. It relies on the fact that the difference between lower case and upper case ASCII characters is only in the 6th bit. Therefore, a simple bitwise operation for each character comparison could make this comparison case insensitive. So a single case-insensitive character comparison would look like this, assuming your repository entries are already in lower case:

query[i] & 0xDF == dbString[i]

When expanding this method to be used in conjunction with the optimization in section B above, a case insensitive comparison of 4 characters on a 32 bit architecture would be with a bitwise operation of & 0xDFDFDFDF !

query32bitValue[i] & 0xDFDFDFDF == dbString32bitValue[i]

The optimal string comparison function

Perhaps the most important conclusion here, is that there is no such thing as an optimal string comparison function to copy-paste from this article. When implementing a strings repository, special consideration may be made to find your specific optimal string comparison function – suited to your own specific needs and your own specific data characteristics.

Variadic Functions in C
A classic usage of a function that accepts a variable number of arguments is:

#include <stdarg.h>
#include <stdio.h>

enum {
    ERROR_LOG,
    WARN_LOG,
    INFO_LOG,
    VERBOSE_LOG,
    DEBUG_LOG
} log_levels;

const char *log_levels_str[] = {
    "Error",
    "Warning",
    "Info",
    "Verbose",
    "Debug",
};

int applog(int level, const char *fmt, ...)
{
    int ret;
    FILE *stream;
    va_list ap;
    va_start(ap, fmt);

    (level <= WARN_LOG) ? (stream = stderr) : (stream = stdout);
    fprintf(stream, "%s: ", log_levels_str[level]);
    ret = vfprintf(stream, fmt, ap);
    va_end(ap);
    return ret;
}

The call to va_start will initialize the variable ap right after fmt. The caller can then use ap to access variables via va_arg, or pass it to other functions.

GCC's 'format' __attribute__
GCC provides a nice protection against misuse of variadic functions in compile time. Just like warnings issued on the 'printf' family of functions, it is possible to protect your own implementation. For example:

//If the compiler does not support attributes, disable them
#ifndef __GNUC__
#   define  __attribute__(x)
#endif

int applog(int level, const char *fmt, ...) 
    __attribute__ ((format(printf, 2, 3) ));

The attribute will preform a compile time test for a printf style call, and will try to match argument #2 (fmt) as the format string. Argument #3 of the function will mark the beginning of the variadic arguments.

The format attribute also support the following validation methods: scanf, strftime and strfmon.

This attribute is supported in all GCC versions 2.x and up. For more information about this attribute (and others) see [1].

C++ quirks
When writing variadic functions in C++ you might think the following is correct:

class A {
...
    int myprintf(const char *fmt, ...)
        __attribute__((format(printf, 1, 2)));
};

The format attribute should use the first parameter as the format string and the second as the variable part. Compiling this in GCC will output the following error:

error: format string argument not a string type

This is of course due to the fact that C++ has an extra argument in all class members, this. So the correct declaration should be:

class A {
...
    int myprintf(const char *fmt, ...)
        __attribute__((format(printf, 2, 3)));
};

Accessing Variables Manually
Besides passing the va_list struct to various supporting functions, it is possible to iterate on the function's variables manually. This can be done similar to:

int sum_many(int first, ...)
    __attribute__((sentinel(0)));

int sum_many(int first, ...)
{
    int num, ret = first;
    va_list ap;
    va_start(ap, first);
    while ( (num = va_arg(ap, int)) != NULL) {
       ret += num;
    }
    va_end(ap);
    return ret;
}

Each call to va_arg will return the next argument. The va_arg function requires a typename (or a pointer to a typename) to cast the resulting argument.
The sentinel(0) attribute enforces a compile time check that the argument at position 0 (last argument of the function) is an explicit NULL.

Variadic Macros
Sometimes it can be very useful to have a macro with unknown number of variables. While this seems simple enough, it was only added as part of C99. Example usage follows:

#define LOG_DEBUG(fmt, ...) (applog(DEBUG_LOG, "[%s at %s:%u]: " fmt,  \
__FUNCTION__, __FILE__, __LINE__, __VA_ARGS__))

Variadic macros are supported by any C99 compliant compiler. Namely GCC 3.3 and above, Microsoft Windows Visual Studio 2005 and up (see [3]).

References:
[1] http://unixwiz.net/techtips/gnu-c-attributes.html
[2] http://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html
[3] http://msdn2.microsoft.com/en-us/library/ms177415(VS.80).aspx

BINARY_HEADER 	*pHeader		= NULL;

//pCurByte points to the last byte of the previous strcture.

pCurByte+= sizeof(DWORD) – (pCurByte % sizeof(DWORD) );

pHeader = pCurByte;

However I came across this really cool code in some MS sources I was looking at. They add an Align() method to the structure which returns a pointer to the next DWORD, but the coolest thing about it, is this function implementation (!).

structure BINARY_HEADER

{

    byte mField[2];

    byte mAnotherField[10];

    const  BINARY_HEADER* Align()

    {

        return( (BINARY_HEADER*) ( ( ( (UINT_PTR) this) + 3)  & ~3) );

    }

};

All you have to do now is –

pHeader = pCurByte;

pCurByte = pCurByte->Align();

Much nicer! :)

This post will present the subject and discuss design considerations, code examples will be presented in a later post.

Multi-Core vs. Multi Processor

Multi-core environments tend to be a bit different than multi-processor ones. Here are the two major processor-specific factors we address to in this article:
Cache – Multi core CPUs usually have smaller L1 cache, and the L2 cache is shared between cores. This calls for small and specialized code. Large code will cause a lot of instruction cache misses, and will degrade performance.
Bus – Inter core bus is an important factor. If the bus is slow, data transfer between cores will become a huge bottleneck.

The Dispatcher

The dispatcher model assumes that there is an incoming queue of messages (or tasks) that needs some processing. Each processing unit (from here on referred simply as ‘core’) can be used in parallel to the others. Usually in the end the message is transported to a location. Prime examples of the dispatcher model are packet processing devices (router, firewall etc.) and graphical processing units.

As an allegory we can see the dispatcher as a kitchen, where waiters bring in orders, and the kitchen team prepares dishes and delivers them back to the waiters for serving.

Pipeline vs. Run-to-Completion

There are two major models for the dispatcher. The pipeline model assigns a specific task to each processing unit, where in the run-to-completion model each processing unit handles a single message from start to end.

The Pipeline Model

We would like to implement a pipeline in our restaurant. We teach each person a specific task. One will be in charge of sauces, another on garnish etc. When an order arrives, the plate goes from one cook to another, each performing its relevant work. Finally the plate is returned to the counter to be transported by the waiter.
In the packet processing world we can use one core to routes packet, another to enforce an ACL (access list) etc.

Analysis

The pipeline model has several advantages:

• Specialization – Each processing unit specializes at a specific task. In the restaurant model this means we have to do less training, we train each person only with the appropriate task. In the multi core model this means that less code runs on each core. When the size of the code is small, it can be optimized to fit in the instruction cache, increasing overall performance.
• Flexibility – The pipeline model is very flexible, if we see that a certain task slows down the entire process, we can assign another core to that task. For example if all the dishes are waiting for garnish for a long time, we assign another person to garnish from another task.
  Note: This is a very expensive task to do at runtime. It so must not be done too often or we lose other benefits (see strong affiliation below).
• Shared Data Locks – This model usually needs only a few locks on shared data structures, since not all the cores access all the data structures.

The pipeline model has several limitations:

• Data duplication – Data should travel between different processing units. If the data is large, this will clog the bus, causing messages to wait on the bus most of the time.
  An important rule of thumb in the pipeline model is to transfer as little data as possible – preferably just a pointer between the cores.
• Strong affiliation – Since each task is assigned to a specific processing unit, it is said that the code is affiliated with this core. If we decide during runtime to change the task of a core, for a significant amount of time we lose all the instruction and data cache.

  In the restaurant model this is similar to a person that is trained to cook fish, and now we need to train the person to prepare sauces.

• Message Locks – Since we need to exchange data between cores, if we write on the message itself, we will almost certainly need to lock the data in transport. This calls for multiple locks for each message.
• Robustness – What happens if one of the processing units gets stuck? If no control is done, this will cause the entire process to fail. If data gets corrupt in one core, this will cause the entire message to get corrupt. The pipeline therefore requires a strong watchdog to act when something goes wrong. See more on the control section below.
• Unfair Work Division – Let’s assume that in our kitchen one person is in charge for fish, and another for desserts. If there is a slow fish day, our fish person is mostly unemployed. In a multi-core system this means that some cores might be idling while the system runs on full load. This can be handled by dynamically allocating cores to task, but as explained earlier, with some penalty.

Control Point

The pipeline model requires a strong controlling process to make sure nothing goes wrong. The control will usually have a dedicated core to the task, or even an entire dedicated CPU for extra robustness (in case we need to restart an entire processor).
Some of the control point roles are:

• Message Handling Time Limit – Putting some upper limit to the amount of time a message can spend in a core is usually a good idea. This can help detect deadlocks and non uniform performance.
• Core Reassignment – A watchdog must be prepared to remove a core from the pipeline or changing its task. This helps dealing with major faults and fair task division.
• Command and Control Central Point – The control point is the central place where all control and configuration commands are processed. It is usually a bad idea giving direct user commands to processing cores. User commands can be errornous and cause system instability. The control point must assure the commands are safe, and track for complete execution of the control commands. In case the control command failed or caused system instability, the watchdog must re-stabilize the system and notify the user of the error.

The Run-to-Completion Model

Let’s assume that in our restaurant we chose a different model. Every person will handle a dish from beginning to end. Every person is well trained to do all the tasks that are involved, and from the moment an order arrives, that person prepares it with no interruptions until it’s finished.

This is very similar to the thread-pool model, but here we have a guarantee that a dedicated core runs from beginning to end uninterrupted.

Analysis

Let’s go over the advantages of the run-to-completion model:

• Independence – Every processing unit is independent, no data is transferred between cores, when there are no interruptions, it is easy to measure how much time each processing unit takes to complete the task and provide real-time assurances.
• Scalability – Adding more processing units is an easy task. Since all the cores are symmetric, adding a core to the game will just add another worker to the pool.
• Message Locks – The message does not need to be locked. From start to end it is accessed only by a single core.

The model has several shortcomings:

• Large code – All the processing units run all the tasks thus every processor needs to run a lot of code. If the instruction caching is not good enough, this will cause a lot of cache misses and performance penalty.
• Shared Data Locks – We will almost always need to share resources between all cores. When these resources are modified, they need to be locked, causing performance penalty.

Control Point

The control point in the run-to-completion model should perform similar tasks to the ones in the previous model. Controlling the cores is usually easier, since there is no difference between the cores and there are less scenarios to deal with.

Common Issues

• Core affiliation – In both modes it is imperative that code will run on the same core, to benefit from processor cache and better control on the process. If the controller does not knows that a specific task runs on core X it will have hard time tracking its status.
• The Input Queue – The design of the input queue has a major influence the overall process. The input queue is usually designed as a FIFO queue or a priority queue if QoS is required. The queue is a single entry point to the system, so it creates a natural bottleneck. Inefficient queue will limit the number of messages entering the system even before a single processing instruction occurred.
• Bus efficiency – Inter-core bus and I/O busses can create a bottleneck if the cores transfer data between them, once again, data should be moved as little as possible while processing. Fast busses can sometimes compensate for little or no data cache.
• Instruction and data cache – Each processing unit usually have far less cache than a full fledged CPU. Code that runs on each core should be optimized to get as many cache hits as possible, or performance will suffer. Measuring the performance of the instruction and data cache is an important system dependent factor.
• I/O and memory allocation – I/O operations and memory allocations are problematic in two factors.
  First, it is obvious that if the processing unit spends a lot of time waiting for I/O and memory, it is idling.
  A second but not less important factor is the real-time factor. I/O access and memory managers are not deterministic. We would like to be able to measure the amount of processing time for each message as accurately as possible, and I/O infer with our goal.
  As usual it is recommended to pre-allocate all the memory required for the message processing, and avoid any I/O operations while processing a message.

Summary

When designing a complete system, you will probably need to mix-and-match the two methods to get best performance. Depending on implementation, some subsystems should have run-to-completion properties, while others should use the pipeline model as a whole.

It is a good idea to profile your requirements and split the work at hand to micro tasks. Once you defined all the tasks and inner relationships between them, a decision can be made.

If you are bound to a specif processor architecture and OS, it is imperative to research all the processor advantages and shortcomings to reach the best decision. On the other hand, on multi-platform systems you must decide and enforce a set of basic requirements, and be flexible on others you can’t control (like strong affiliation, real-time scheduler priority or fast locks, which are OS/hardware specific).

Take special notice to the control point. Do not satisfy designing the data data handling path. The control point is just as important!

Basic SHM setup

As mentioned before, our goal is writing a program that shares a struct in C with all running instances of the software, no matter when they run. The implementation will be something in the lines of:

#include <fcntl.h>
#include <sys/stat.h>
#include <errno.h>
#include <sys/mman.h>
#include <sys/types.h> //shm_open
#include <stdio.h>  //printf
#include <stdlib.h> //exit
#include <unistd.h> //close
#include <string.h> //strerror

/* This will be created under /dev/shm/ */
#define STATE_FILE "/program.shared" 
#define  NAMESIZE 1024
#define   MAXNAMES 100

/* Define a struct we wish to share. Notice that we will allocate 
 * only sizeof SHARED_VAR, so all sizes are constant              
 */
typedef struct
{
  char name[MAXNAMES][NAMESIZE];
  int flags;
}  SHARED_VAR;

int main (void)
{
int first = 0;
int shm_fd;
static SHARED_VAR *conf;

  /* Try to open the shm instance with  O_EXCL,
   * this tests if the shm is already opened by someone else 
   */
  if((shm_fd = shm_open(STATE_FILE, (O_CREAT | O_EXCL | O_RDWR), 
                       (S_IREAD | S_IWRITE))) > 0 ) {
          first = 1; /* We are the first instance */
  }
  else if((shm_fd = shm_open(STATE_FILE, (O_CREAT | O_RDWR), 
                        (S_IREAD | S_IWRITE))) < 0) {
   /* Try to open the shm instance normally and share it with 
    * existing clients 
    */
    printf("Could not create shm object. %s\n", strerror(errno));
    return errno;
  } 

  /* Set the size of the SHM to be the size of the struct. */
  ftruncate(shm_fd, sizeof(SHARED_VAR));

  /* Connect the conf pointer to set to the shared memory area,
   * with desired permissions 
   */
  if((conf =  mmap(0, sizeof(SHARED_VAR), (PROT_READ | PROT_WRITE), 
                   MAP_SHARED, shm_fd, 0)) == MAP_FAILED) {

    return errno;

  }
  if(first) {
   /* Run a set up for the first time, fill some args */
   printf("First creation of the shm. Setting up default values\n");
   conf->flags = 4;
  }
  else
  {
    printf("Value of flags = %d\n", conf->flags);
  }

/* Do some work... */

  close(shm_fd);
  exit(0);
}

Notice the code should be compiled against librt like so: gcc -lrt -o shm shm.c

Once the shared memory has been set up, every time the program loads (until reboot or deletion of the virtual file), it can connect to the same shared memory location, get and store it’s data.

When running the binary, you will notice the shm object was created in /dev:

ami@codemaestro:~$ ls -l /dev/shm/
-rw------- 1 ami ami 102404 Jul 27 09:34 program.shared

Access Control and Synchronization

It is clear that the shm mechanism provides bare-bones tools for the user. All access control must be taken care of by the programmer. Locking and synchronization is being kindly provided by the kernel, this means the user have less worries about race conditions. Note that this model provides only a symmetric way of sharing data between processes. If a process wishes to notify another process that new data has been inserted to the shared memory, it will have to use signals, message queues, pipes, sockets, or other types of IPC.

Shared Kernel memory

Although it’s frighteningly more dangerous, sharing memory can also be done between kernel space and user space. For those of you wondering why on earth will they want to do this, the answer is simple – Performance ; Why copy memory when you can share it? For example, imagine the performance boost for a network appliance if its network interface doesn’t have to copy each and every packet’s data to user space, but only send its memory address to a user space network application that can read from the shared kernel memory section.

Defining a shared memory space between kernel space and user space is not much different from the usual user space shared memory, but as mentioned before, the possible damage is far greater in case of a bug.

Describing how to write a kernel module that shares some kernel memory is outside the scope of this article. However, a complete detailed example of exposing shared memory from the kernel for writing a network driver can be found in an article “Network Buffers and memory management” [3].

The Registry Model

SHM allows a simple management of a knowledge base to be shared between processes. The facilities supplied with SHM make it easy to implement the following model:
Registry manager process loads the registry from file and maps it to SHM.
Child / sibling processes can read / write entries with managed permissions.
Registry manager saves desired entries to file (on every write for example).

Via SHM we can detect if we are the first running instance and manage permissions. If we combine this with a notification mechanism, we can create an efficient way to share structured data between processes. More on this in the referenced article.

A More Advanced Model

Sachin Agrawal and Swati P. Udas from IBM published an article that provides an infrastructure for shared memory in high level languages like C++. The referenced article [2] shows the use of SHM in object oriented environment, as well as a system for IPC.

References

• Using mmap
• Handle synchronous events from shared objects in Linux
• Network Buffers and Memory Management, By Alan Cox

Let’s first look at a naive Instance static member function:

MyClass* MyClass::Instance() 
{ 
    if (!pInstance) { 
        pInstance = new .... 
    }
    return pInstance; 
}

Obviously, it is not thread safe. Two threads can cause the instance to be created twice. A natural solution is just to use a mutex:

MyClass* MyClass::Instance() 
{ 
    // Mutex is freed when the local lock object is destructed
    mutex lock("Mutex0");
    
    if (!pInstance) { 
        pInstance = new .... 
    }
    return pInstance; 
}

Assuming that MyClass::Instance() is called very frequently, the last implementation will slow down the execution, since locking (especially a mutex) can be very very slow. However, locking is only needed for the instance’s creation, and not for every time we are trying to get the instance. Following is a very simple improvement:

MyClass* MyClass::Instance() 
{ 
    if (!pInstance) {
        mutex lock("Mutex0");
    
        if (!pInstance) { 
            pInstance = new .... 
        }
    }
    return pInstance; 
}

Every caller who enters the function after pInstance was allocated will not lock the mutex. Callers who enter the function before the instance was created are still synchronized with the mutex.

References

• An article about this issue by Douglas Schmidt, 1996

It’s not surprising that callback functions will usually look ugly in C++ code. Pointers to member functions are usually a mess. It is mainly because they need the “this” pointer as a parameter. Observe the following example:

class A  {
public:
    int func ();
};

...

int (A::*pmf)();
pmf = &A::func;
A a;
A *pa = &a;
(a.*pmf)();

C++ was meant to be nicer than this, wasn’t it?

Implementing Delegates in C++

If you’re familiar with C# programming, one of the new bright ideas in C# is “Delegates”. If you get to the bottom of it, delegates can actually be easily implemented in C++ using regular classes, as shown below.

As a rule of thumb, use the ideas of the following code whenever you want to implement a callback function in C++ and you won’t get disappointed.

Implementing a C++ delegates consist of a few simple tasks. The following example illustrates how to create and use a delegate of a function that takes string and returns void.

1. Declare a prototype of a “pointer to a function that takes string and returns void” as a pure virtual class with one member function:

class StringDelegate
{
public:
      virtual void runTheFunction(string params) = 0;
};

2. Implement your specific callback function

The callback function’s implementation is a class that inherits from StringDelegate. Optionally, it would be nicer if it contained some kind of a “Runner” class that is responsible to handle the received data. The code follows:

class OurDelegate : public StringDelegate
{
public:
   void runTheFunction(string data); // Implementation!
   OurDelegate(Runner& runner); // The constructor should get the runner
 private:
   OurDelegate(); // No default constructor
   Runner m_runner;
};

// The constructor
OurDelegate::OurDelegate(Runner& runner):m_runner(runner)
{
}

// The actual implementation
void OurDelegate::runTheFunction(string data)
{
   m_runner.run(data);
}

Now we can write the code that's calling our "callback function":

void callme(StringDelegate sd)
{
     sd.runTheFunction("Tralala");
}

Running the delegates is simple:

callme(OurDelegate(runner));

No pointers needed! Now that’s C++.