Exploiting Undocumented Hardware Blocks in the LPC55S69

The purpose of a secure boot process rooted in a hardware root of trust is to provide some level of assurance that the firmware and software booted on a server is unmodified and was produced by a trusted supplier. Using a root of trust in this way allows users to detect certain types of persistent attacks such as those that target firmware. Hardware platforms developed by cloud vendors contain a hardware root of trust, such as Google’s Titan, Microsoft’s Cerberus, and AWS’s Nitro. For Oxide’s rack, we evaluated the NXP LPC55S69 as a candidate for our hardware root of trust.

Part of evaluating a hardware device for use as a root of trust is reviewing how trust and integrity of code and data is maintained during the boot process. A common technique for establishing trust is to put the first instruction in an on-die ROM. If this is an actual mask ROM the code is permanently encoded in the hardware and cannot be changed. This is great for our chain of trust as we can have confidence that the first code executed will be what we expect and we can use that as the basis for validating other parts of the system. A downside to this approach is that any bugs discovered in the ROM cannot be fixed in existing devices. Allowing modification of read-only code has been associated with exploits in other generations of chips.

An appealing feature of the LPC55 is the addition of TrustZone-M. TrustZone-M provides hardware-enforced isolation allowing sensitive firmware and data to be protected from attacks against the rest of the system. Unlike TrustZone-A which uses thread context and memory mappings to distinguish between secure and non-secure worlds, TrustZone-M relies on designating parts of the physical address space as either secure or non-secure. Because of this, any hardware block that supports remapping of memory regions or that is shared between secure and non-secure worlds can potentially break that isolation. ARM recognized this risk and explicitly prohibited including their own Flash Patch and Breakpoint (FPB) unit, a hardware block commonly included in Cortex-M devices to improve debugging, in devices with TrustZone-M.

While doing our due diligence in reviewing the part, we discovered a custom undocumented hardware block by NXP that is similar to the ARM FPB but for patching ROM. While this ROM patcher is useful for fixing bugs discovered in the ROM after chip fabrication, it potentially weakens trusted boot assertions, as there is no longer a 100% guarantee that the same code is running each time the system boots. Thankfully, experimentation revealed that ROM patches are cleared upon device reset thus preventing any viable attacks against secure boot.

NXP also has a set of runtime APIs in ROM for accessing the on-board flash, authenticating firmware images, and entering in-system programming mode. These ROM APIs are expected to be called from secure mode and some, such as skboot_authenticate, require privileged mode as well. Since the ROM patcher is accessible from non-secure/unprivileged mode, an attacker can leverage these ROM APIs to gain privilege escalation by modifying the ROM API as follows:

This issue can be mitigated through use of the memory protection unit (MPU) or security attribution unit (SAU) which restricts access to specified address ranges. Not all code bases will choose to enable the MPU, however. Developers may consider this unnecessary overhead given the expected small footprint of a microcontroller. This issue shows why that’s a dangerous assumption. Even close examination of the official documentation would not have given any indication of a reason to use the MPU. Multiple layers of security can mitigate or lessen the effects of a security issue.

Part of building a secure product means knowing exactly what code is running and what that code is doing. While having first instruction code in ROM is good for measured boot (we can know what code is running), the ROM itself is completely undocumented by NXP except for API entry points. This means we had no idea exactly what code was running or if that code was correct. Running undocumented code isn’t a value-add no matter how optimized or clever that code might be. We took some time to reverse engineer the ROM to get a better idea of how exactly the secure boot functionality worked and verify exactly what it was doing.

Reverse engineering is a very specialized field but it turns out you can get a decent idea of what code is doing with Ghidra and a knowledge of ARM assembly. It also helps that the ROM code was not intentionally obfuscated so Ghidra did a decent job of turning it back into C. Using the breadcrumbs available to us in NXP’s documentation we discovered an undocumented piece of hardware related to a “ROM patch” stored in on-chip persistent storage and we dug in to understand how it works.

NXP’s homegrown ROM patcher is a hardware block implemented as an APB peripheral at non-secure base address 0x4003e000 and secure base address 0x5003e000. It provides a mechanism for replacing up to 16 32-bit words in the ROM with either an explicit 32-bit value or a svc instruction. The svc instruction is typically used by unprivileged code to request privileged code to perform an operation on its behalf. As this provides a convenient mechanism for performing an indirect call, it is used to trampoline to a patch stored in SRAM when the patch is longer than a few words.

NXP divides the flash up into two main parts: regular flash and protected flash. Regular flash is available to the device developer for all uses. The protected flash region holds data for device settings. NXP documents the user configurable parts of the protected flash region in detail. The documentation notes the existence of an NXP area which cannot be reprogrammed but gives limited information about what data exists in that area.

The limited documentation for the NXP area in protected flash refers to a ROM patch area. This contains a data structure for setting the ROM patcher at boot up. We’ve decoded the structure but some of the information may be incomplete. Each entry in the NXP ROM patch area is described by a structure. This is the rough structure we’ve reverse engineered:

There’s three different commands defined by NXP to program the ROM patcher: a group of single word changes, an svc change, and a patch to SRAM. For the single word, the addresses to be changed are written to the entry in the array at offset 0x100 along with their corresponding entries in the reverse array at 0xf0. The relative_address field seems to determine if the addresses are relative or absolute. All the patches on our system have only been a single address so the full use of relative_address may be slightly different. For an svc change, the address is written to the 0x100 array, and the instructions are copied to an offset in the SRAM region. The patch to SRAM doesn’t actually use the flash patcher but it adjusts values in the global state stored in the SRAM.

Nobody expects code to be perfect. Code for the mask ROM in particular may have to be completed well before other code given manufacturing requirements. Several of the ROM patches on the LPC55 were related to power settings which may not be finalized until very late in the product cycle. Features to fix bugs must not introduce vulnerabilities however! The LPC55S69 is marketed as a security product which makes the availability of the ROM patcher even riskier. The biggest takeaway from all of this is that transparency is important for security. A risk cannot be mitigated unless it is known. Had we not begun to ask deep questions about the ROM’s behavior, we would have been exposed to this vulnerability until it was eventually discovered and reported. Attempts to provide security through obscurity, such as preventing read access to ROMs or leaving hardware undocumented, have been repeatedly shown to be ineffective (https://blog.zapb.de/stm32f1-exceptional-failure/, http://dmitry.gr/index.php?r=05.Projects&proj=23. PSoC4) and merely prolong the exposure. Had NXP documented the ROM patch hardware block and provided ROM source code for auditing, the user community could have found this issue much earlier and without extensive reverse engineering effort.

NXP, however, does not agree; this is their position on the matter (which they have authorized us to share publicly):

At Oxide, we believe fervently in open firmware, at all layers of the stack. In this regard, we intend to be the model for what we wish to see in the world: by the time our Oxide racks are commercially available next year, you can expect that all of the software and firmware that we have written will be open source, available for view – and scrutiny! – by all. Moreover, we know that the arc of system software bends towards open source: we look forward to the day that NXP – and others in the industry who cling to their proprietary software as a means of security – join us with truly open firmware.