Leveraging Your Toolchain to Improve Security

20 September 2023 by Phillip Johnston • Last updated 28 September 2023

Your toolchain is a useful place to start when incorporating security into your development process. There are several warnings and program augmentations that help harden your application.

This article focuses on GCC and Clang, as that’s what I primarily use. I’m happy to take suggestions from readers for other toolchains.

The flags in this option are present in GCC 12+ and Clang 18+. If you’re using an earlier toolchain version, you might find that some of these options are not available to you.

Table of Contents:

1. Compiler Warnings
   1. Turn up the Warning Level
   2. Integer Warnings
   3. Memory Safety
   4. Format Strings
   5. Thread Safety (Clang Only)
   6. Other Bug Catchers
2. Static Analysis
3. Program Augmentations
   1. Fortify Source
   2. Initialize all Uninitialized Variables to Zero
   3. Stack Buffer Overflow Protection
   4. Safe Stack (Clang Only)
   5. Sanitizers
4. Program Augmentations For Non-Microcontroller Projects
5. References

Compiler Warnings

A part of securing your system is not making errors. Compiler warnings are an excellent first step in this effort. This is not a comprehensive set of warning flags to enable on your projects, though it is a significant starting point.

GCC and Clang share many of the same toolchain flags. If I do not explicitly call out one or the other, a given flag works for both toolchains. Please note that I may have made an assignment error somewhere in this great jumble of compiler flags. Each toolchain is being frequently updated, so a flag that was previously only supported by one vendor may be added to the other without me noticing. Point out any errors in the comments and I will get them fixed!

Turn up the Warning Level

Many teams have been surprised when I share the list of warnings I typically use. On many projects, we see warning levels set too low or useful warnings disabled. This is a missed opportunity: warnings point out problematic, confusing, or incorrect language use.
Regardless of any additional flags you pick below, for both GCC and Clang, you need to reach beyond -Wall. It might have been “all” warnings at one point, but now it’s only a subset of the useful warnings that are out there. At a minimum, -Wextra should be your next step.

You will often see suggestions to use -Wpedantic. This flag is useful if you want to stick to the ISO language standards. If you’re making use of extensions in your code base, you will not want this flag.

Depending on your build structure, when you turn up your warning level, you are likely to see warnings in headers or sources for components that you don’t control (e.g., from an external library or vendor SDK). To address this, you need to:

1. Be intentional about specifying include directories as “system” includes or not (e.g., -I vs. -isystem).
   1. When you mark an include directory as a system include, warnings will not be shown for those headers. Note that it changes the include search order, with -isystem includes coming at the end of the search. In most cases, this is fine, but if your headers are using constructs like #include_next, you may need to make other build adjustments so headers are resolved properly.
2. Refine your build structure so that you can better control which files are compiled with tighter warning levels.
   1. The most common embedded build structure we see on projects is “build all the files as a single application target.” This limits your ability to control which files are compiled with which warnings (or other options). To regain control, focus on structuring your build so that different subsets of files are built as a library, which is then linked into the final application. Different library targets can be built with different compiler flags, allowing you to opt-out any vendor-supplied code from tighter warning levels.

We recommend at least reviewing the warnings for the vendor code before turning them off – there may very well be a bug that needs to be fixed. Whatever you do, though, don’t let a flood of warnings that you don’t intend to fix remain in the build output. This will cause you to become numb to warnings, and you’re likely to miss other important warnings.

Integer Warnings

Integer overflows and wraparounds are significant contributors to security vulnerabilities. Luckily, there are several warning options available.

• -Wconversion – warn for implicit conversions that might alter a value
• -Wsign-conversion – warn for implicit conversions that may change the sign of an integer
  • You can silence known-acceptable conversions with an explicit cast. This is preferable to disabling the warning entirely.
• -Wstrict-overflow=n – when signed overflow is undefined, the compiler can warn you about cases where the compiler will optimize based on the assumption that an overflow does not occur. There are potential false positives with this warning, so different levels ([1,5]) are provided. 2 is the default, though it’s useful to review at level 4 or 5, at least occasionally.
  • Clang only supports this flag for GCC compatibility; it does not enable any additional protection in Clang

GCC-specific:

• -Wshift-overflow=2: warn when left shift overflows into sign bit (causing the value to become negative), unless C++14 mode (or newer) is active

Clang-specific:

• -Wshift-sign-overflow: warn when left shift overflows into sign bit (causing the value to become negative)

Memory Safety

Memory safety problems are also significant contributors to security vulnerabilities. There are a efw additional warnings for memory safety that you can enable that aren’t in -Wall or -Wextra.

GCC-specific:

• -Warray-bounds=2 – warn about pointer arithmetic that may yield out-of-bounds values (GCC documentation notes this may give a larger number of false positives)
• -Wstringop-overflow=4 – improve overflow checking for string manipulation functions like memcpy and strcpy; may increase false positives (default level is 2)
• -Wuse-after-free=3 – warn about use-after-free problems, but increase the checks to include indeterminate pointers in equality expressions (i.e., checking that after realloc the object is different – this is common, but undefined behavior) (default level is 2)

Clang-specific:

• Warray-bounds-pointer-arithmetic – warn about pointer arithmetic that may yield out-of-bounds values

Format Strings

Format strings are prone to security problems, especially when functions like fprintf or snprintf are used with improperly sanitized user inputs. There are several warning options available that will alert you to potential problems with these functions

• -Wformat=2 enables additional format string warnings regarding cases like “may overflow”, “will always overflow”, “will always be truncated”, “size argument is too large”
  • We suspect that Clang includes some of the additional GCC checks in their implementation of this warning

GCC-specific:

• -Wformat-overflow=2 – warn for calls to the printf family of functions that might overflow the destination buffer; level 2 adds additional coverage
• -Wformat-truncation=2 – warn for calls to the printf family of functions that might result in output truncation; level 2 adds additional checks
• -Wformat-signedness – warn when there is a sign/unsigned mismatch between the format specifier and the supplied parameters.

Clang-specific:

• -Wformat-type-confusion – warn when format specifies type A but the argument has type B.

Thread Safety (Clang Only)

Clang supports Thread Safety Analysis for C and C++ programs with the -Wthread-safety and -Wthread-safety-beta flags. This analysis is occurs at compile-time; there is no run-time overhead involved. However, you would need to augment your program with annotations like REQUIRE(mutex_id) and GUARDED_BY(mutex_id) for this analysis to work. These annotations are implemented as #defines, so you can override them with empty expressions if you need to support multiple toolchains.

Certainly, catching thread safety problems at compile-time is an attractive option. However, we also think that the act of annotating your intention itself is valuable, as it makes it clearer which locks guard which variables, and which functions must be invoked while a lock is held.

Other Bug Catchers

The following warning flags are not specifically related to security checking, but they are useful to enable. After all, any logical problem can cause a security vulnerability.

• -Wuninitialized – warn when automatic variables are used without being initialized first (UB)
• -Wshadow – warn about cases where a local variable/type shadows another variable, parameter, type, class member, or built-in function (the problem is that when this is unintentional, you might not be using/updating the one you think you are)
• -Wcast-qual – warn when a cast discards qualifiers, like const and volatile
• -Wundef – warn if an undefined macro is used in an #if directive (safe to use in an #ifdef, but #if expects a value)
• -Wimplicit-fallthrough – warn when a switch case falls through (e.g., missing a break). If this is intentional, the end of the case must be annotated with __attribute__((fallthrough)), [[fallthrough]] (C++17 and later), or a comment resembling fallthrough/ fall through. Note that Clang only supports attribute annotations, not comments.
• -Wfloat-equal – warn about unsafe floating point comparisons using == or !=

GCC-specific:

• -Wlogical-op – warn about suspicious logical operators, when statements are always true/false, or when operands are the same
• -Wduplicated-cond– warn about duplicated conditions in an if/else if block
• -Wduplicated-branches – warn about duplicated branches in an if/else if block (i.e., both the if and the else do the same things)
• -Wswitch-default – warn when a switch statement does not have a default case
• -Wcast-align=strict – warn about pointer casts which increase alignment
• -Wjump-misses-init (C only, this is an error in C++) – warn if a goto or switch statement jumps across the initialization of a variable. This only warns about variables that are initialized when they are declared.

Clang-specific:

• -Wcast-align – warn about pointer casts which increase alignment
• -Wconditional-uninitialized – warn if a particular conditional branch would cause a variable to be used uninitialized
• -Wloop-analysis – warn about loop variable problems, like incrementing a variable in both the loop header and loop body
• -Wtautological-constant-in-range-compare – warn about comparisons that are always true/false based on the variable’s range (e.g., comparing an unsigned value < 0 is always false)
• -Wcomma – warn about possible misuse of the comma operator
• -Wassign-enum – warn about an integer constant that is not in range for an enumerated type
• -Wunreachable-code-aggressive – add additional checks for unreachable code

Static Analysis

Static analysis tools provide more extensive (and more expensive) checks than compiler warnings and will reveal additional issues. Using your toolchain’s static analysis capabilities is an easy first step into this world.

Clang has long provided a built-in static analysis tool, scan-build, which wraps your build process to compile your code and run static analysis. Reports are provided as HTML output. You can also use CodeChecker for a much better experience. Clang also ships with clang-tidy, a standalone static analysis tool. Clang-tidy and scan-build have some overlapping coverage. clang-tidy is nice in that it can automatically fix some problems, and it also supports writing custom rules.

GCC added built-in static analysis support with GCC 10 and has been improving it since, although we find it still lags behind Clang’s offering. You can enable static analysis with the -fanalyzer option. If you want to turn off particular checks, you can do so with the familiar -Wno-* construct. Other analyzer controls are defined in the documentation.

Program Augmentations

Your toolchain can also augment your program at runtime with additional checks to catch potentially catastrophic problems. Many of these augmentations can be used in your production program and should be enabled as long as your memory and processing overhead support it (measure that, don’t just assume it can’t because you have a microcontroller!). Also enable them for your off-target builds, like unit test suites and simulators.

1. Fortify Source
2. Initialize all Uninitialized Variables to Zero
3. Stack Buffer Overflow Protection
4. Safe Stack (Clang Only)
5. Sanitizers

Fortify Source

Some C standard library implementations, such as glibc and newlib, can provide alternative implementations of memory and string handling functions in the standard library that are prone to buffer overflows. The alternate versions leverage compiler built-ins to add in runtime bounds checking. If bounds checking fails, the program will terminate.

You can enable this support by compiling your program with the _FORTIFY_SOURCE=<n> preprocessor definition (n = 1, 2, or 3) and an optimization level > -O0. Higher n corresponds to more protection.

Building with _FORTIFY_SOURCE=2 has negligible overhead (it primarily adds length checking logic) and is a safe runtime default for your programs to use. The newer _FORTIFY_SOURCE=3 option adds additional checks, but there is also additional runtime code added for computing object sizes. This likely has more of a performance impact (which, as always, should be measured!). At the very least, the highest level should be considered for unit tests and simulations that don’t run on a micro.

Initialize all Uninitialized Variables to Zero

In C and C++, using an automatic variable without first initializing it is undefined behavior).

Both GCC and Clang provide a -ftrivial-auto-var-init=[uninitialized|pattern|zero] option to control the behavior in this situation. =uninitialized is the default C and C++. =pattern initializes these variables with the repeating byte 0xFE. And =zero initializes the memory to zero. There is overhead in the binary associated with initializing the previously uninitialized variables.

Using this option does not affect the -Wuninitialized warning output: offending variables will still be flagged by the warning. The optimizer will also be allowed to perform optimizations as if the variable was uninitialized. But the actual contents of the variable will be predictable, rather than whatever garbage is on the stack.

Our recommendation is to use =zero for your program runtime, as it is a more well-behaved default that is in line with default for common programming logic (you probably have more logical cases protecting against 0 than 0xFE). If you’re on a crash-finding mission, =pattern is useful as it is more likely to trigger crashes and be easily recognizable in the debugger.

Stack Buffer Overflow Protection

Clang and GCC implement stack smashing protection (SSP) using StackGuard. Most stack buffer overflows occur by writing past the end of a function’s stack frame. In order to detect this, a “canary value” is added to the stack before other values are declared. Before returning from the function, the stored canary value in the stack is checked. If it has been changed, a stack buffer overflow has occurred, a callback is invoked, and the program is terminated.

This feature is turned on with -fstack-protector. This adds basic protection, generally to funtions that contain a char array, constant sized calls to alloca greater than the size of ssp-buffer-size (usually 8 bytes), and variable sized calls to alloca. -fstack-protector-strong adds checks whenever a function includes an array of any size and type, any calls to alloca, or taking an address of a local variable. -fstack-protector-all adds checking to all functions.

If you want to see which functions aren’t protected, you can enable -Wstack-protector.

Our preference is to use -fstack-protector-strong by default from the start of development, only turning it down or off if absolutely forced to. The performance and space overhead is low for the gained protection, as the cost is writing a value onto the stack and reading it at the end of the function. If there is a measured hotspot impacted by the addition of stack smashing protection, you can turn the feature off in individual functions with __attribute__ ((no_stack_protector)).

Safe Stack (Clang Only)

Clang provides software support for splitting stacks into two distinct areas, “safe” and “unsafe” stacks, using -fsanitize=safe-stack. The “safe” stack stores return addresses and other data that might be subject to an attack, while the “unsafe” stack stores user variables. The purpose of this is to prevent a buffer overflow on the unsafe stack from smashing the return addresses on the safe stack. Of course, it does not prevent the buffer overflow from overwriting something else. The performance impact here is negligible. This can be safely combined with -fstack-protector.

Sanitizers

Both GCC and Clang can instrument your program with several sanitizers, such as the -fsanitize=safe-stack in the section above. These sanitizers use compiler instrumentation (and sometimes a supplemental runtime library) to add runtime checking to your program to identify issues. These are extremely valuable tools for debugging pernicious problems and augmenting automated testing workflows. They are also quite helpful in combination with fuzz testing, as they help you move beyond the basic check of “did the program crash?” to

Many of the most useful sanitizers will not work on a microcontroller, as your target toolchain likely does not have a properly ported runtime library for your desired sanitizer. The more of your system’s code you can run off-target, the better. At the very least, use these sanitizers with your unit tests suites. Even better coverage can be achieved by using them on a full application simulator that can run your development machine.

These are the most popular sanitizers:

• -fsanitize=address – AddressSanitizer, a memory error detector.
• -fsanitize=thread – ThreadSanitizer, a data race detector.
• -fsanitize=memory – MemorySanitizer, a detector of uninitialized reads. Requires instrumentation of all program code.
• -fsanitize=undefined – UndefinedBehaviorSanitizer (UBSan), a fast and compatible undefined behavior checker.
  Regarding security, other interesting sanitizers are:
• -fsanitize=signed-integer-overflow, which adds checks to ensure the result of +, *, and both unary and binary - do not overflow in the signed arithmetics. This also detects INT_MIN / -1 signed division.
• -fsanitize=bounds, which adds additional instrumentation to detect out-of-bounds accesses on arrays (GCC also has an additional bounds-strict variant)

Many other sanitizers are available. Check your compiler documentation to see the full suite:

• Clang: Controlling Code Generation and the complete list of options for UBSan
• GCC: Instrumentation Options

Program Augmentations For Non-Microcontroller Projects

Our primary interest is developing software for microcontrollers, but embedded is a great wide world. If you’re working on an Embedded Linux system (or something else that is much more “PC-like” in nature), you should also consider the following program augmentations:

• Stack Clash Protection==
  • -fstack-clash-protection in GCC and Clang
• Shadow stack (x86_64 only, works via hardware)
  • -mshstk in GCC and Clang
  • This operates like Clang’s Safe Stack support , but uses the x86_64 hardware shadow stack and Control-flow Enforcement Technology (CET)
• Address space layout randomization (ASLR)
  • You need to compile your program with the -fPIE flag and shared libraries with -fPIC to generate position-independent code, which is needed to enable ASLR via the loader at runtime.
• Control flow Integrity (CFI)
  • If you’re targeting x86_64, GCC and Clang can add support code for Intel CET with -fcf-protection=[full]
  • If you’re targeting AArch64, GCC and CLang can add support for Branch Target Identification (BTI) using -mbranch-protection=none|bti|pac-ret+leaf|pac-ret[+leaf+b-key]|standard
  • Clang can provide a software implementation of CFI using -fsanitize-cfi

If you’re using MacOS or Linux for your program, simulator, or unit tests with the stock standard libraries, you can look into glibc’s MALLOC_PERTURB_ or MacOS’s malloc debugging variables to identify potential use-after-free or uninitialized memory use issues.

Stack Limit (GCC Only)

GCC allows you to specify a stack limit. This adds runtime checking when dynamic stack allocations are involved (e.g., variable-length arrays or alloca()). A signal will be thrown if the stack is going to exceed its bounds. You can specify the limit using -fstack-limit-symbol=sym_name for a symbol (e.g., a __stack_limit variable supplied at link time). You can locally override stack limit checking by using the no_stack_limit function attribute.

We originally included this in another section above, but Billy pointed out the limitation in that checks are only added to functions with dynamic stack allocations. While this makes it much less useful for microcontrollers, the runtime check is still useful in programs with a single stack that is taking advantage of dynamic stack allocations.

Here is the comment:

A Guide to Using ARM Stack Limit Registers%%

Implementing Stack Smashing Protection for Microcontrollers (and Embedded Artistry’s libc)%%

• Warnings: -Weverything and the Kitchen Sink
• Security Technologies: FORTIFY_SOURCE
• GCC’s new fortification level: The gains and costs | Red Hat Developer
• Improve buffer overflow checks in _FORTIFY_SOURCE | Red Hat Developer

Toolchain vendor documentation:

• Warning Options (Using the GNU Compiler Collection (GCC))
• Instrumentation Options (Using the GNU Compiler Collection (GCC))
• Static Analyzer Options (Using the GNU Compiler Collection (GCC))
• Diagnostic flags in Clang — Clang 18.0.0git documentation

Other takes on security-related flags:

• Getting the maximum of your C compiler, for security: Clang
• Getting the maximum of your C compiler, for security: GCC
• Quarkslab Clang Hardening Cheat Sheet
• Use compiler flags for stack protection in GCC and Clang | Red Hat Developer