The point with a compiler is obviously that it lets us write code in language that’s easier to read than assembly code, and modern optimizing compilers can make it more readable still by saving us from obfuscating hand-optimization. For instance, in the 1980s, it may have been good for the performance of your C code to replace x/2 with the slightly cryptic (x >> 1), but modern compilers will probably generate the same code for both, so better to keep the short and intuitive form.
But capable as modern compilers are, they can’t know all the conditions under which the code will execute, and won’t come up with specific optimizations for them the way you can. Take a look at this C function, which moves audio samples from a channel-separated buffer to an interleaved buffer:
void sep_to_in(const int32_t * const *source, // channel separated void *restrict dest, // interleaved unsigned ss, // sample size, bytes unsigned channels, // how many channels unsigned frames) // buffer size { int8_t *w = dest; for (int i = 0; i < frames; i++) { for (int j = 0; j < channels; j++) { memcpy(w, source[j] + i, ss); w += ss; } } }
The somewhat cryptic type of the source parameter can be read as array of arrays of integers: the samples belonging to different channels are stored in separate arrays of int32_t, and source is an array of pointers to them. The function moves the lower ss bytes of each sample (assuming that the representation is little endian, which it seems to be pretty much everywhere these days) into the one-dimensional dest buffer, lining up samples of different channels in the same frame next to each other.
The function is written in a general way to work with all possible values of channels and ss, which is nice but comes with a performance cost: lots of loop bounds checking and a call to memcpy for every single sample. What we can know and the compiler can’t is that the number of channels is usually just one or two (mono or stereo), the sample size never more than four, and a common format is 16-bit stereo where ss and channels are both 2. We can write a more efficient version for that format like this:
void sep_to_in_16_stereo(const int32_t * const *source, void *restrict dest, unsigned frames) { int16_t *w = dest; for (int i = 0, j = 0; i < frames; i++) { w[j++] = source[0][i]; w[j++] = source[1][i]; } }
That should be significantly faster for large buffers, so even if we want to keep the function working for all possible formats it’s probably a good idea to test for this case and use the optimized code for it, like this:
void sep_to_in(const int32_t * const *source, // channel separated void *restrict dest, // interleaved unsigned ss, // sample size unsigned channels, // how many channels unsigned frames) // buffer size { if (ss == 2 && channels == 2) { int16_t *w = dest; for (int i = 0, j = 0; i < frames; i++) { w[j++] = source[0][i]; w[j++] = source[1][i]; } } else { int8_t *w = dest; for (int i = 0; i < frames; i++) { for (int j = 0; j < channels; j++) { memcpy(w, source[j] + i, ss); w += ss; } } } }
We can continue on this track and add more special cases, with a nested switch to choose between them.
But wait! Before we rush off to write special code for another thirteen cases, winding up with a big lump of code that takes the rest of the day to test, let’s try an experiment. First, we compile the specialized sep_to_in_16_stereo to assembly code, using gcc -O -S. On my Raspberry Pi, the code comes out like this:
sep_to_in_16_stereo: cmp r2, #0 bxeq lr ldr ip, [r0] sub ip, ip, #4 mov r3, r1 ldr r0, [r0, #4] sub r0, r0, #4 add r1, r1, r2, lsl #2 .L3: ldr r2, [ip, #4]! strh r2, [r3] ldr r2, [r0, #4]! strh r2, [r3, #2] add r3, r3, #4 cmp r3, r1 bne .L3 bx lr
I don’t actually speak ARM machine code, so I don’t get all the details, but it looks like I expected: a tight little loop, with a bne, branch-if-not-equal to jump back to .L3 until done. Compiling the original general sep_to_in produces more than twice as many lines of assembly code with a double loop that calls memcpy for every sample.
Now the interesting part: let’s try writing the function like this instead:
void sep_to_in_16_stereo(const int32_t * const *source, void *restrict dest, unsigned frames) { unsigned ss = 2; unsigned channels = 2; int8_t *w = dest; for (int i = 0; i < frames; i++) { for (int j = 0; j < channels; j++) { memcpy(w, source[j] + i, ss); w += ss; } } }
So that’s the same as the general sep_to_in, except that ss and channels are set inside the function instead of passed as parameters. This is what comes out of gcc -O -S for the new version:
sep_to_in_16_stereo: cmp r2, #0 bxeq lr push {r4, lr} ldr r4, [r0] ldr lr, [r0, #4] mov r3, #0 .L3: lsl r0, r3, #2 ldrh ip, [r4, r0] strh ip, [r1] ldrh r0, [lr, r0] strh r0, [r1, #2] add r1, r1, #4 add r3, r3, #1 cmp r2, r3 bne .L3 pop {r4, pc}
That’s 19 lines just like the other version, a single loop, and no call to memcpy. Surprising? Not really, because now GCC knows the values of ss and channels, and it knows what memcpy does, so it can optimize the code analogously to how we did. The hand-optimized version might have a small edge (I’m not sure), but this looks perfectly acceptable, and we can write the general version like this:
void sep_to_in(const int32_t * const *source, // channel separated void *restrict dest, // interleaved unsigned ss, // sample size unsigned channels, // how many channels unsigned frames) // buffer size { # define COPY_LOOP \ for (int i = 0; i < frames; i++) { \ for (int j = 0; j < channels; j++) { \ memcpy(w, source[j] + i, ss); \ w += ss; \ } \ } int8_t *w = dest; if (ss == 2 && channels == 2) { COPY_LOOP; } else { COPY_LOOP; } }
So if the condition holds, we run the loop, and if it doesn’t we do the same thing? Sure, it looks a bit strange, but the compiler generates different code from the same source code in the different cases! In the first case it knows what ss and channels are and can use something like the 19-line version, while in the second case it has to resort to an inefficient general version.
There’s no way the compiler would have come up with this on its own, because from its point of view the 16-bit stereo format is just one out of billions of different possibilities.
To handle more of the common audio formats efficiently, we can expand to something like this:
void sep_to_in(const int32_t * const *source, // channel separated void *restrict dest, // interleaved unsigned ss, // sample size unsigned channels, // how many channels unsigned frames) // buffer size { # define COPY_LOOP \ for (int i = 0; i < frames; i++) { \ for (int j = 0; j < channels; j++) { \ memcpy(w, source[j] + i, ss); \ w += ss; \ } \ } # define SS_CASES \ switch (ss) { \ case 1: COPY_LOOP break; \ case 2: COPY_LOOP break; \ case 3: COPY_LOOP break; \ case 4: COPY_LOOP break; \ default: COPY_LOOP; \ } int8_t *w = dest; switch (channels) { case 1: SS_CASES break; case 2: SS_CASES break; default: SS_CASES; } }
Insanity? Well, it is a way to optimize for a bunch of common cases, and still have the function work for any other case, and all with just a single version of the core code to write, test, and maintain.