A Newlib upgrade mystrery story

Introduction

I was working on a DCC decoder project that I recently started.

Firstly, I found out a great library for DCC devices. It’s from ZIMO, actively maintained, so it’s possible that’s it’s what actually powers their products. It provides both command station building blocks and decoder building blocks. I quite like that it uses modern C++ and is hardware-agnostic, so users can feed it DCC data the way they like it. What I didn’t like is that it uses too modern C++, i.e. C++23 features as well as concepts, spans, z-suffixes etc. I ended up cleaning up the necessary parts to be mostly C++11 and ETL-compatible, but also looked at upgrading GCC to higher versions to use some of those fancy features myself.

Secondly, the devboard I happen to use for a prototype has a PY32F003x6 with 4Kb of RAM, and the percentage used is printed after each compilation. To my surprise, upgrading from GCC 9.2.1 GCC to 12.3.1 caused an increase of RAM usage from about less than a third to almost half! In absolute terms it’s going from 1200 to 1900B, but with this small total amount, every big jump is noticable! It’s not helping that some RAM must be free for stack and heap, so it definitely cannot go to 100%.

GCC 9.3.1:

RAM:   [===       ]  29.2% (used 1196 bytes from 4096 bytes)
Flash: [==        ]  20.0% (used 6540 bytes from 32768 bytes)

GCC 12.3.1:

RAM:   [=====     ]  46.3% (used 1896 bytes from 4096 bytes)
Flash: [==        ]  20.7% (used 6792 bytes from 32768 bytes)

In addition, compiling with newer compiler caused output of strange warnings from linker:

C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/../../../../arm-none-eabi/bin/ld.exe: C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/thumb/v6-m/nofp\libc_nano.a(libc_a-closer.o): in function `_close_r':
closer.c:(.text._close_r+0xc): warning: _close is not implemented and will always fail
C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/../../../../arm-none-eabi/bin/ld.exe: C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/thumb/v6-m/nofp\libc_nano.a(libc_a-lseekr.o): in function `_lseek_r':
lseekr.c:(.text._lseek_r+0x10): warning: _lseek is not implemented and will always fail
C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/../../../../arm-none-eabi/bin/ld.exe: C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/thumb/v6-m/nofp\libc_nano.a(libc_a-readr.o): in function `_read_r':
readr.c:(.text._read_r+0x10): warning: _read is not implemented and will always fail
C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/../../../../arm-none-eabi/bin/ld.exe: C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/thumb/v6-m/nofp\libc_nano.a(libc_a-writer.o): in function `_write_r':
writer.c:(.text._write_r+0x10): warning: _write is not implemented and will always fail

So, this post is about me investigating the source of this regressions.

RAM usage

I generated linker map files for both compilations with -Wl,-Map,output.map,--cref and inspected them in AMap viewer. Immediately it was visible that there are extra 312b used by __sf symbol, coming from _findfp.o. This was enough information to google, and I found out that some users have already noticed this problem:

on stackoverflow: https://stackoverflow.com/questions/78635407/gnu-tools-for-stm32-change-after-version-9-including-non-required-module-and-con

on newlib mailing list: https://inbox.sourceware.org/newlib/cee20c8a-7881-7cec-07da-ca5b41719b00@embedded-brains.de/t/#u

Mailing list says it was a regression from "splitting up the struct _reent into individual thread-local storage objects" between versions 4.2.0 and 4.3.0. The discussion is from 2023 and as nothing was done since, I assume that unfortunately it is not considered a problem by devs.

Studying the code reveals that __sf is an array of 3 FILE objects, which predictably are stdin, stdout and stderr handlers. Size of a file struct is around 100B.

Before version 4.3.0 __sf contained much smaller __sFILE_fake file objects, which were removed around June 2022.

Finding affected toolchain versions

Sources Newlib repo is hosted at https://www.sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git, but it’s also a repository of Cygwin, who host their repo (the same one) at https://cygwin.com/cgit/newlib-cygwin/ and maintain an official GitHub mirror here: https://github.com/cygwin/cygwin. In a typical ARM project setup, newlib is bundled with ARM gcc compiler. PlatformIO registry is populated by (some) versions from xPack GNU Arm Embedded GCC Project, those are populated from ARM GNU Toolchain releases. The version of newlib in ARM distribution is specified in arm-none-eabi/include/_newlib_version.h. However, what is written there is the latest release before the source was compiled, as ARM team simply grabs latest commit at the moment of release. This introduces a problem with one of the releases below. Each release on arm website offers a manifest file which says where the files were taken from, this allows to find a specific commit in newlib repo that was used to build the library.

Here are some GCC versions from ARM GCC downloads page and from platformio registry.

• GCC 10.3.1 ships with newlib 4.1.0 — ok

• GCC 11.2 has newlib 4.1.0 — ok.

• GCC 11.3.rel1 has newlib 4.2.0 — ok. (_newlib_version.h is broken in this release, so it must be found through commits)

• GCC 12.2.rel1 - this one is a bit problematic. The version clearly says "4.2.0". However, the source was taken from commit faac797, which is almost a year after version 4.2.0 was released, and roughly a month before 4.3.0. It already includes __sf symbol in the build — not ok!

• GCC 12.3.1 has newlib 4.3.0 — not ok!

Why is the symbol there in the first place?

It’s pulled by using vsnprintf function that I use to print to UART. Removing calls to vsnprintf removes the symbol too.

vsniprintf is a thin wrapper over _vsniprintf_r function. _r here means "reentrant", and as a first argument it takes a global _REENT struct.

The structure is essentially an array of all global objects, including errno and pointers to stdin/out/err file descriptors (the __sf entries we are concerned with!). The struct is also ~80 bytes large.

This function creates a fake file descriptor (but fully-sized) and calls _svfprintf_r that in turn does the actual formatting. These functions make frequent use of errno element of _REENT and maybe other file-related elements of it.

Seemingly vsnprint is the only place where _REENT is used in my program, so removing the call allows it, along with referenced __sf, to be garbage-collected by linker.

Linker warnings

Other reports and discussions about the problem:

• https://stackoverflow.com/questions/73742774/gcc-arm-none-eabi-11-3-is-not-implemented-and-will-always-fail

• https://community.st.com/t5/stm32cubeide-mcus/build-error/td-p/587816

• https://community.st.com/t5/stm32cubeide-mcus/linker-warnings-on-every-new-project/td-p/791910

• https://community.silabs.com/s/question/0D5Vm0000047to9KAA/linker-errors-on-update-to-gnu-arm-v1221?language=en_US

• https://community.arm.com/support-forums/f/compilers-and-libraries-forum/53519/arm-gcc-11-3-rel1-newlib-nano-linker-warning-_read-is-not-implemented-and-will-always-fail

It was also mentioned in the newlib mailing list, with no resolution: https://inbox.sourceware.org/newlib/CAAHv3KF9Gt0sFtm7Qsx4XfO0iaGHP2n9Z2qS8LdL2ytJcqR%3Deg%40mail.gmail.com

While I found no reliable and non-hacky solution, in the links above it is stated that the problem appeared in v11.3 of Arm GNU Toolchain. Indeed, after looking myself at the files, I found that starting with v11.3, file libnosys.a contains sections ".gnu.warning" (e.g. ".gnu.warning._write"), which cause the linker to produce the warnings (this feature of LD linker is described in its documentation). In 11.2 these sections are not present.

In newlib codebase, these warnings have been there for at least 27 years, so something in the process of building the library changed between 11.2 and 11.3. The warnings are still there in Arm GNU Toolchain v12 and v14.

The warnings are from linker, not from compiler, so they cannot be supressed with -W flags.

At the time of writing, my request for clarifications on ARM’s forum is not answered.

It seems that one workaround is to go ST’s way and supply stubs for these functions manually. However it somewhat defeats the purpose of using nosys in the first place (whose goal is to provide these stubs).

Outcome

Latest version in PlatformIO registry that is unaffected by both problems is 10 (10.3.1). It has moderate support for C++20 features, e.g. concepts, but almost no support for C++23.

For those who are not tied to platformio, GCC 12.2r1 should also work and not have these problems.

Also, since I use ETL extensively, a recently added etl::format is of interest as a potential replacement of sprintf.

It turns out, GCC 10 does not play well with debugging with blackmagic-debug. Its GDB complains about register mismatching packet. Older and later GCC versions, e.g. 14, do not show this problem. Which means I’m somewhat back to square one with selection of GCC version. I am now considering to use latest GCC, not use newlib printf functions family and use a 3rd party implementation, in particular nanoprintf. It seems to be feature-rich enough while still being small enough. It’s also actively maintained. However, the problem with linker warning would still be there.