JTAG

OSX: Getting Pesky FTDI Debugging Devices to Work

Updated: 2018-12-19

Many useful FTDI-based debugging devices exist for those of us working in the embedded world. These devices are extremely useful and provide JTAG and SWD capabilities, as well as acting as I2C, SPI, or Serial protocol converters. With an FTDI debug tool, you can use your host machine to debug and interface with a variety of parts.

However, one fact that complicates things greatly is that most parts use their own USB product and vendor ID codes, which means that you have to manually modify the FTDI driver kext in order for your device to be detected. There are many guides on the internet for enabling custom FTDI device support, but most of them now seem out of date. This guide is valid as of OSX 10.12, which requires additional steps to disable OSX's "System Integrity Protection."

Testing a Connection

In order to judge whether an FTDI device is properly detected or not, I attempt to open a serial connection. Without the correct drivers, no usbserial devices show up. Your device will likely have a different name.

For example, here's how I connect to a TUMPA debugger on my host machine.

picocom -b 115200 /dev/tty.usbserial-TIM01416B

If you are unable to connect, you likely need to perform the steps below.

miniterm.py

If you are unsure of what serial program to use, run the following commands to enable miniterm.py:

$ sudo easy_install pip
$ sudo pip install pyserial

After installing pyserial, you can run miniterm.py, which will show you a list of connected serial devices:

$ miniterm.py
--- Available ports:
--- /dev/cu.Bluetooth-Incoming-Port n/a
--- /dev/cu.MuadDib-WirelessiAP n/a

Note above that I have no USB serial device connected, indicating a problem.

Try Your Device

Before doing anything, try plugging your device into the computer and make sure the AppleUSBFTDI driver doesn't already support your device.

If you don't see the FTDI serial device, there's still another step to try with the native OSX drivers. Try these steps, then attempt to connect:

sudo kextunload -p -b com.apple.driver.AppleUSBFTDI
sudo kextutil -b com.apple.driver.AppleUSBFTDI -p AppleUSBEFTDI-6010-1

If you see the FTDI serial device, stop reading!

Download the FTDI Drivers

If your device didn't work with the native FTDI driver, you will want to try the "official" FTDI drivers. [Download them for your machine here.][2]

Once installed, check your device again and see if it connects.

If you see the FTDI serial device, stop reading!

Unload the FTDI Driver and Reload Apple

When swapping between devices, you may have problems when using the FTDI kext, but not the Apple kext. The FTDI kext is often preferentially loaded over the Apple version.

You might need unload the FTDI kext and reload the Apple FTDI kext before your device works:

$ sudo kextunload /System/Library/Extensions/FTDIUSBSerialDriver.kext/
$ sudo kextload -b com.apple.driver.AppleUSBFTDI

Kext Modification

Find your PID/VID

At this point, it is likely that your FTDI device has a PID/VID combination that isn't listed in the FTDI kext. My TUMPA falls into this boat.

Connect your device, and issue the following command in the shell:

$ system_profiler SPUSBDataType

This command will print information for all USB devices connected to your computer. Look for your FTDI device entry:

TIAO USB Multi-Protocol Adapter:
    Product ID: 0x8a98
    Vendor ID: 0x0403  (Future Technology Devices International Limited)
    Version: 7.00
    Serial Number: TIM01416
    Speed: Up to 480 Mb/sec
    Manufacturer: TIAO
    Location ID: 0x14100000 / 5
    Current Available (mA): 500
    Current Required (mA): 450
    Extra Operating Current (mA): 0

Note down the Product ID and Vendor ID, and convert those numbers to decimal.

Modifying the FTDI kext

First, make a copy of the FTDI Info.plist so you have a backup:

$ sudo cp /System/Library/Extensions/FTDIUSBSerialDriver.kext/Contents/Info.plist /System/Library/Extensions/FTDIUSBSerialDriver.kext/Contents/Info.plist.orig

Open the FTDI kext Info.plist file in your favorite text editor, located at /System/Library/Extensions/FTDIUSBSerialDriver.kext/Contents/Info.plist

You will need to add a new device entry with the PID/VID for your product supplied. Copy one of the device entries, supply a new name corresponding to your device, and modify the VID/PID to match your product.

Here's an example for my TUMPA:

<key>TIAO USB Multi-protocol Adapter</key>
<dict>
    <key>CFBundleIdentifier</key>
    <string>com.FTDI.driver.FTDIUSBSerialDriver</string>
    <key>IOClass</key>
    <string>FTDIUSBSerialDriver</string>
    <key>IOProviderClass</key>
    <string>IOUSBInterface</string>
    <key>bConfigurationValue</key>
    <integer>1</integer>
    <key>bInterfaceNumber</key>
    <integer>0</integer>
    <key>idProduct</key>
    <integer>35480</integer>
    <key>idVendor</key>
    <integer>1027</integer>
</dict>
<key>TIAO USB Multi-protocol Adapter</key>
<dict>
    <key>CFBundleIdentifier</key>
    <string>com.FTDI.driver.FTDIUSBSerialDriver</string>
    <key>IOClass</key>
    <string>FTDIUSBSerialDriver</string>
    <key>IOProviderClass</key>
    <string>IOUSBInterface</string>
    <key>bConfigurationValue</key>
    <integer>1</integer>
    <key>bInterfaceNumber</key>
    <integer>1</integer>
    <key>idProduct</key>
    <integer>35480</integer>
    <key>idVendor</key>
    <integer>1027</integer>
</dict>

I have two entries here because my device has two FTDI interfaces. Whether you have one or two interfaces depends on your part - do some digging :)

Disabling SIP

Modern OSX enables "System Integrity Protection" by default, which will prevent our modified FTDI driver from loading since the signature no longer matches the driver contents. You will need to disable SIP to use the modified driver on your system.

  • Boot to Recovery OS by restarting your machine and holding down the Command and R keys at startup.
  • Launch Terminal from the Utilities menu.
  • Enter the following command to disable SIP:
    $ csrutil disable
  • Reboot your computer

If you want to enable SIP again, simply follow the steps and issue the following command:

$ csrutil enable

Loading/Unloading the Driver

After rebooting your system, you should be good to go. However, if you have already disabled SIP and are making modifications to the driver, you can manually load/unload in the shell:

$ sudo kextunload /System/Library/Extensions/FTDIUSBSerialDriver.kext/
$ sudo kextload /System/Library/Extensions/FTDIUSBSerialDriver.kext/

Further Reading

Change Log

  • 20181219:
    • Improved debug notes
    • Links open in external tabs
    • Added another link to Further Reading section

WFI and JTAG

JTAG and wfi do not place nicely together. You've probably noticed this yourself, wondering why JTAG can't connect or a halt/breakpoint doesn't work.

When the ARM core is executing the wait for interrupt operation, the core is placed into a low power mode and the core clock is gated. This wreaks havoc with JTAG, as the core clock is required to detect the JTAG clock transitions.

The simplest way to keep your debugging sanity is to provide hooks to disable wfi when JTAG is needed. One option is to use a #define to control whether your idle loop performs wfi or spins the processor. In quick pseudo-code:

#ifdef DISABLE_WFI
    tx_thread_sleep(1)
#else
    asm("wfi");
#endif

However, once you need to debug returned units, reliability units, or factory failures it's not always feasible to reflash a unit. A more complicated implementation is to incorporate an NVRAM library and set a software flag (more on that in the future). On boot, the software checks the flag and uses the correct idle loop. If you then need to debug a device but can't re-flash it, simply set the NVRAM flag, reboot, et voila!

Note that this strategy will have an effect on timing and power consumption, so it will not be the holy grail for all cases. But it will certainly make your life easier if you're running into the wfi wall.

Debugging Strategy: Finding Version from a Memory Dump

Every software product needs a version number. Once bug reports start rolling in, a version number is an absolute requirement for debugging and addressing issues.

In this guide, I describe the details I like to store in each build, how I capture these details, and how I make them available to the firmware.

Table of Contents:

  1. What to Capture
    1. Timestamp
    2. Build Machine
    3. Version
    4. Modifications to Support Windows
  2. Getting Version Into Source
  3. How do I generate my version automatically?
  4. Is there a simpler approach without git tags?
  5. Further Reading

What to Capture

Here are the details I like to log for each build:

  • Time/date of build
  • Machine that built it
  • Version (git tag number)
  • Number of commits away from the last tag
  • Commit hash
  • Dirty or clean build (dirty = local changes that are not committed)

With this set of information, I get a full picture of the build's history and can answer questions such as:

  • Was the build created by the server, or locally by Joe Smith?
  • Is it a clean build, or are there changes that weren't committed?
  • Is it a tagged build, or are there extra commits since the last build?

Timestamp

The easiest way to get the timestamp is to run the date shell command:

$ date
Thu Oct 27 10:27:40 PDT 2016

To include the string in a Makefile:

-DVERSION_BUILD_DATE=\""$(shell date)"\"

Build Machine

For build machine details, I use the whoami and hostname commands:

$ whoami
jenkins

$ hostname
buildbot

I take these two values and combine them into a single string:

jenkins@buildbot

To include the string in a Makefile:

-DVERSION_BUILD_MACHINE=\""$(shell echo `whoami`@`hostname`)"\"

Version

Conveniently, you can get the version information directly from git:

$ git describe --long --dirty --tags
0.1.64-2-gb27efef

Above, 0.1.64 is the latest version tag, 2 indicates that I am two commits ahead of the last versioned build, and the commit hash is at the end.

The simplest approach is to pass the entire git describe output into your version string, but I think it makes your version more confusing to customers, vendors, and CMs. The git describe output is also too verbose if you all you want is to display a marketing version, such as "1.2.64".

I store the following values separately:

  • BUILD_TAG (0.1.64)
  • COMMIT (gb27efef)
  • COMMITS_PAST (2)
  • DIRTY_FLAG (+ if dirty, empty if clean)

Here's some example Makefile code to make these values available:

version := $(subst -, ,$(shell git describe --long --dirty --tags))
COMMIT := $(strip $(word 3, $(version)))
COMMITS_PAST := $(strip $(word 2, $(version)))
DIRTY := $(strip $(word 4, $(version)))
ifneq ($(COMMITS_PAST), 0)
    BUILD_INFO_COMMITS := "."$(COMMITS_PAST)
endif
ifneq ($(DIRTY),)
    BUILD_INFO_DIRTY :="+"
endif

export BUILD_TAG :=$(strip $(word 1, $(version)))
export BUILD_INFO := $(COMMIT)$(BUILD_INFO_COMMITS)$(BUILD_INFO_DIRTY)

This will result in:

Build version: 0.1.64
Build info: gb27efef.2

If the build was dirty, you'd see this instead:

Build info: gb27efef.2+

Modifications to Support Windows

If you're using Windows, you will need to use different commands to get the BUILD_DATE and BUILD_MACHINE variables.

BUILD_DATE := $(shell python -c "from datetime import datetime; print(datetime.utcnow().strftime('%d/%m/%Y, %H:%M'))"
BUILD_MACHINE := $(shell echo %username%)@$(shell hostname)

You can support both Windows and OSX/Linux using shell detection:

ifeq ($(SHELL), cmd.exe)
BUILD_DATE := $(shell python -c "from datetime import datetime; print(datetime.utcnow().strftime('%d/%m/%Y, %H:%M'))"
BUILD_MACHINE := $(shell echo %username%)@$(shell hostname)
else
BUILD_MACHINE := $(shell whoami)@$(shell hostname)
BUILD_DATE := $(shell date -u +"%d/%m/%Y, %H:%M")
endif

Getting Version into Source

Now that we have all the information we want to include for each build available in MAKE variables, we can pass this information to the compiler using compiler definitions.

In your makefile, create a set of definitions for your version library and add them to your CFLAGS:

CFLAGS += -DVERSION_BUILD_DATE=\""$(shell date)"\" \
          -DVERSION_BUILD_MACHINE=\""$(shell echo `whoami`@`hostname`)"\" \
          -DVERSION_TAG=\"$(BUILD_TAG\" \
          -DVERSION_BUILD=\"$(BUILD_INFO)\"

In your source code, refer to these definitions:

void version_print()
{
    printf("%s %s %s (%s)\n", VERSION_BUILD_DATE, VERSION_BUILD_MACHINE, VERSION_TAG, VERSION BUILD);
}

And voilà! you have a version available.

How do I generate my version automatically?

If you're at the stage where you need version builds, you should also be using a CI server such as Jenkins to build your software. These automated build tools will give each build a unique version. You can use the version in your build process.

I create two parametric values for my builds:

  • MAJOR_VERSION
  • MINOR_VERSION

When combined with the Jenkins BUILD_NUMBER variable, you can form your typical version triple by combing them:

${MAJOR_VERSION}.${MINOR_VERSION}.${BUILD_NUMBER}

Resulting in a string like:

1.2.48

The versioning strategy outlined in the article relies on git tags. We'll use temporary tags during the build process to create a clean version number.

Before running the build, I make a temporary local tag:

git tag -a -f -m "Candidate build: ${MAJOR_VERSION}.${MINOR_VERSION}.${BUILD_NUMBER}" ${MAJOR_VERSION}.${MINOR_VERSION}.${BUILD_NUMBER}
make rtos
git tag -d ${MAJOR_VERSION}.${MINOR_VERSION}.${BUILD_NUMBER}

By creating an annotated tag before the build, git describe will work as expected and return a clean "0.1.64" instead of "0.1.63-8". After the build succeeds, push the tag to the host and so it will be available for the next build:

git tag -a -f -m "Successful nightly build ${MAJOR_VERSION}.${MINOR_VERSION}.${BUILD_NUMBER}" ${MAJOR_VERSION}.${MINOR_VERSION}.${BUILD_NUMBER}
git push origin ${MAJOR_VERSION}.${MINOR_VERSION}.${BUILD_NUMBER}

Is there a simpler approach without git tags?

You can pass the ${MAJOR_VERSION}.${MINOR_VERSION}.${BUILD_NUMBER} variables into your make command and utilize those definitions directly:

make rtos BUILD_TAG=${MAJOR_VERSION}.${MINOR_VERSION}.${BUILD_NUMBER}

And:

CFLAGS += -DVERSION_TAG=\"$(BUILD_TAG)\"

However, I always recommend pushing a tag for successful builds. Having each build tagged allows for easier debugging and investigation.

Triggering Version Updates Every Time We Run Make

This versioning scheme works well for build servers, since they are often configured to create versioned builds from a clean slate. However, if you are generating builds from the command line in an incremental manner, the version will not be updated each time you compile.

To trigger a recompilation at every step, we must get our build system to think the file is out of date. Modern build systems, such as Meson, provide options such as build_always_stale which can be used to create a target that is run each build, whether the file is changed or not.

With make, we must perform some trickery to get this to work.

One approach is to touch our version.c file each time we run make:

$(shell touch path/to/version.c)

Another approach is to set version.o to use a PHONY prerequisite which is not connected to a file:

version.o: FORCE

.PHONY: FORCE
FORCE:

Wolfram Roesler describes an alternative approach for setting the version when linking.

Further Reading