Embedded Clang

compiler-rt

As we're exploring bringing up a C/C++ runtime on our system, I'd like to share a very helpful resource for those using clang/llvm: compiler-rt.

Compiler-rt is an LLVM project that provides implementations of various builtin functions for a variety of architectures. This saves us a lot of heavy lifting when bringing up a new platform, as we can link compiler-rt instead of re-implementing these functions.

While most useful as a complete library, compiler-rt is also a useful source code resource if you need to implement these builtins with a different toolchain. Simply import the required builtin source into your project.

I'll let the compiler-rt project describe the builtins they provide:

builtins - a simple library that provides an implementation of the low-level target-specific hooks required by code generation and other runtime components. For example, when compiling for a 32-bit target, converting a double to a 64-bit unsigned integer is compiling into a runtime call to the __fixunsdfdi function. The builtins library provides optimized implementations of this and other low-level routines, either in target-independent C form, or as a heavily-optimized assembly.

builtins provides full support for the libgcc interfaces on supported targets and high performance hand tuned implementations of commonly used functions like __floatundidf in assembly that are dramatically faster than the libgcc implementations. It should be very easy to bring builtins to support a new target by adding the new routines needed by that target.

Pre-requisites

You will need the llvm-config binary on your platform. This binary is provided when you install llvm.

If you're using OSX, note that Apple does not provide llvm-config with Xcode, so you will need to install mainline llvm to get this binary. See my notes on installing clang/llvm on OSX.

Getting compiler-rt

You can checkout the compiler-rt source with svn:

svn co http://llvm.org/svn/llvm-project/compiler-rt/trunk compiler-rt

If you prefer git, check out the github mirror:

git clone git@github.com:llvm-mirror/compiler-rt.git

I'll leave the folder structure descriptions to the compiler-rt team:

include/ contains headers that can be included in user programs (for example, users may directly call certain function from sanitizer runtimes).
lib/ contains libraries implementations.
lib/builtins is a generic portable implementation of builtins routines.
lib/builtins/(arch) has optimized versions of some routines for the supported architectures.
test/ contains test suites for compiler-rt runtimes.

The lib/builtins/ folder contains the source for the various builtin functions. You can use these items piecemeal in your repository. This is useful if you just need to port specific functions or don't want to deal with installing clang or compiling compiler-rt.

Building compiler-rt

For those who are interested in the compiler-rt builtins library, let's continue our journey.

Once you have llvm-config on your system, you can build compiler-rt with the following commands:

$ mkdir build
$ cd build
$ cmake ../compiler-rt -DLLVM_CONFIG_PATH=/path/to/llvm-config
$ make

The build diectory is important - it's where cmake will place the resulting files.

For those following with homebrew, you could use this command:

$ cmake ../compiler-rt -DLLVM_CONFIG_PATH=$(brew --prefix llvm)/bin/llvm-config
$ make

By default, make will build everything. If you want to build a limited subset, you can run make help and pick the specific items you want to build.

If you want to install libraries, run this additional command:

$ make install

I usually do not run the make install step.

Finding the Right Library

After your build is completed, change to the lib/builtin directory in your build folder. There you will likely see a massive list of files. Here's my example output from compiling with Apple Clang on OSX:

libclang_rt.builtins_arm64_ios.a
libclang_rt.builtins_armv7_ios.a
libclang_rt.builtins_armv7k_ios.a
libclang_rt.builtins_armv7s_ios.a
libclang_rt.builtins_i386_10.4.a
libclang_rt.builtins_i386_iossim.a
libclang_rt.builtins_i386_osx.a
libclang_rt.builtins_x86_64_10.4.a
libclang_rt.builtins_x86_64_iossim.a
libclang_rt.builtins_x86_64_osx.a
libclang_rt.builtins_x86_64h_osx.a
libclang_rt.cc_kext_arm64_ios.a
libclang_rt.cc_kext_armv7_ios.a
libclang_rt.cc_kext_armv7k_ios.a
libclang_rt.cc_kext_armv7s_ios.a
libclang_rt.cc_kext_i386_osx.a
libclang_rt.cc_kext_x86_64_osx.a
libclang_rt.cc_kext_x86_64h_osx.a
libclang_rt.hard_pic_armv7_macho_embedded.a
libclang_rt.hard_pic_armv7em_macho_embedded.a
libclang_rt.hard_pic_i386_macho_embedded.a
libclang_rt.hard_pic_x86_64_macho_embedded.a
libclang_rt.hard_static_armv7_macho_embedded.a
libclang_rt.hard_static_armv7em_macho_embedded.a
libclang_rt.hard_static_i386_macho_embedded.a
libclang_rt.hard_static_x86_64_macho_embedded.a
libclang_rt.soft_pic_armv6m_macho_embedded.a
libclang_rt.soft_pic_armv7_macho_embedded.a
libclang_rt.soft_pic_armv7em_macho_embedded.a
libclang_rt.soft_pic_armv7m_macho_embedded.a
libclang_rt.soft_static_armv6m_macho_embedded.a
libclang_rt.soft_static_armv7_macho_embedded.a
libclang_rt.soft_static_armv7em_macho_embedded.a
libclang_rt.soft_static_armv7m_macho_embedded.a

You only need one of these for your system, likely. Which one do you pick?

Here's a quick decoder:

  • hard vs soft: this is floating point. Is your platform configured to support hard or soft floating point operations?
  • static vs pic: is the code compiled as a static library, or are you compiling with position-independent-code? (PIC)
  • i386 vs armv7x: this will be dependent upon your platform's processor. You need to pick the instruction set to match.
  • The last portion of the name is the library format.

Generally, I end up picking this library for my purposes if I am compiling and linking on OSX:

libclang_rt.hard_pic_armv7_macho_embedded.a

Note the "macho embedded" format - this requires special parsing to use with your embedded system. We will investigate MACHO files further in a future article.

Using compiler-rt in your project

Since compiler-rt builtin libraries do not regularly need updates, I recommend pre-compiling compiler-rt into a library file that can be linked against in your project. It may be worth it to build compiler-rt on your build machine so you have a known source to retrieve updates from.

Once you have built compiler-rt, you can copy the desired library to your project's repository.

You will need to add the -L linker flag to get the location into the library search path. The -l linker flag can be used to include the library itself: -lcompiler_rt.

If you built compiler-rt on OSX, you ended up with a bunch of macho libraries. The macho format will require additional handling that will be described in a future article.

Further Reading

Installing LLVM/Clang on OSX

If you are a developer using clang or gcc on OSX, the odds are good that you are using the Xcode build tools.

These tools work well when compiling for your host machine. You can even get the iPhone SDK to work well enough for embedded development using ARM. However, I have encountered many cases where using the Apple clang compiler has resulted in odd behavior or unexpected errors after updating Xcode.

As a result, I recommend using the mainline clang for your embedded development. Using mainline clang on my system proved to take a little more work than I expected, so I hope my notes make setup easier for you.

Manual Compile + Install

If you would like to go the manual compile+install route, please check this guide from the LLVM group. The manual compile+install instructions are also applicable for those not using OSX.

Installing with Homebrew

I use Homebrew to manage extra packages, and I highly recommend it.

Using Homebrew, we can install llvm/clang very easily. Simply run this command:

$ brew install --with-toolchain llvm

It may be worth running the following command first if you already have an existing homebrew installation:

$ brew info llvm

Seeing llvm v3.6 listed? I recommend updating homebrew, as the new taps have llvm v3.9.1. llvm v3.6 is still lacking in many of the more recent C and C++ features that we'll want to use.

To update homebrew, simply run:

$ brew update
$ brew upgrade

After that, you should be able to check the llvm information again and see v3.9.1 (or later)

Where to find llvm/clang

When you install llvm with brew, the new binaries will not automatically be in the path. Note what the formula says:

OS X already provides this software and installing another version in parallel can cause all kinds of trouble.

You can find the binaries here:

$(brew --prefix llvm)/bin

If you want the Homebrew llvm/clang to show up in your PATH, run the following command:

$ echo 'export PATH="/usr/local/opt/llvm/bin:$PATH"' >> ~/.bash_profile

Building and Linking

Here are the compiler variables you should use for homebrew clang:

export CC := clang
export CXX := $(CC)++

You could also specify the path manually if you don't want to edit your PATH variable:

export CC := /usr/local/opt/llvm/bin/clang
export CXX := $(CC)++

Note that if your tools look for ld or ar, you will likely end up using the Apple tools. Make sure you change your compiler settings to llvm-ar and llvm-ld if you want to utilize the mainline tools.

You will need to add the following flags for compiling and linking:

LDFLAGS += -L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib
CPPFLAGS += -I/usr/local/opt/llvm/include -I/usr/local/opt/llvm/include/c++/v1/

You can also check whether brew clang is actually installed before adding these flags. This will help you support users who install clang by other means.

ifeq ($(shell brew info llvm 2>&1 | grep -c "Built from source on"), 1)
#we are using a homebrew clang, need new flags
LDFLAGS += -L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib
CPPFLAGS += -I/usr/local/opt/llvm/include -I/usr/local/opt/llvm/include/c++/v1/
endif

If you're getting errors about missing headers like assert.h, make sure to run xcode-select --install. This will populate /usr/include correctly.

Want to test your new clang installation? Checkout the embedded-resources github repo! If you look at the compiler.mk file, you will notice that I have implemented a very simple compiler detection scheme. I try to use, in order:

  1. Clang
  2. Apple Clang
  3. GCC

Give your new compiler a test drive by running make in the embedded-resources repo.

Further Reading

Clang: Manually Disable Warnings in a Specific Location

Utilizing public or generated code can often introduce warnings into your software builds. Many of these warnings are not serious and developers will often ignore them. This pollutes the warning report and prevents developers from noticing useful warnings in their own software.

Using clang, you can suppress warnings using a #pragma:

#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wfloat-equal -Wdeprecated"
#include <annoying_warning_creating_header.h>
#pragma clang diagnostic pop

I've often used this approach to silence acceptable warnings from external code.