Paul’s DIY blog

Reviving DCC command Station

2026-07-13T00:00:00+00:00

After several years of pause, I’ve decided to update my LocoNetControlStation project, both hardware and software side.

Hardware changes in new version:

Fit the Command station into a DIN rail enclosure.
Update DCC circuitry to modern ICs (instead of old L298N) to ease reading of current.
Use official ESP32-DevKitC board instead of Lolin32 Lite to avoid problems with availability and quality.
Modularize the design to allow adding new features without PCB redesign, for exmaple LCC or RailCom
Use USB-PD to power the board, use switching regulator instead of LM7805.
Add a small OLED display for status and diagnostics.
Limit current of LocoNet RailSync to better follow specs (to simplify things, it’s limited to 30mA, roughly 10 times less then the documentation mentions).

The PCB design will be released to public when finished, but I would consider it as an educational project for other people, rather than useful for recreating. I adapted it for my capabilities and needs, e.g. used components that I had in stock, will use full-color PCB because it’s cheaper with JLCPCB coupons, so it might not be the optimal to replicate 1-to-1. Same applies to firmware.

Software changes:

Actualize toolchains, update ESP32 SDK, libraries. The project was last actively worked on in 2021, so everything is rather encient. I was surprised that it still compiles, though.
Regularly send (refresh) function states to locomotives (old code only sends loco speed/direction packets, this was an issue I noticed while running on dirty tracks).
Display diagnostic info on OLED display (it shows number of WiThrottle/LbServer clients, track power state, number of running trains and some other)
Fast clock support on LocoNet, WiThrottle (and, in future, DCC)
Better processing of LocoNet turnout commands (also fixed some bugs with LocoNet slot ownership and dispatching)
WiThrottle has support for "steal" messages.

Upgrading code to ESP-IDF v5 proved to be a major challenge. New timer API caused the DCC bit timings to break, as the jitter in timer callbacks was too large in the new API for some reason. I was able to somewhat combat this by switching to RMT peripheral, but it’s also suffering from problems due to delays between RMT transmissions. In new SDK RMT peripheral has completely new API which doesn’t allow continous streaming. So, a legacy RMT driver was used for uninterrupted streaming. Because it’s completely removed in ESP-IDF 6, I decided to copy it from SDK 5 and store it as a library in repository (project still uses SDK 5 though). The implementation has limitations on DCC packet size, it’s limited to allocated RMT memory. ESP32 has 64 symbols (=DCC bits) per block, of which it it has 8, so it’s not a problem for original ESP32, but can be a major limitation for newer ESP32s.

Another major task was refreshing of function packets on DCC track. This required inventing some kind of a priority queue backed by table of alocated locomotives so that each locomotive regularly produces a speed/dir DCC packet or a function packet (3 groups of functions are cycled through, totalling 12 functions). Other, non-locomotive packets go through the queue directly and get picked up by DCC pulse generator according to their priority. For now priority system is rather simple, stop and emergency stop packets have higher priority than normal packets.

A lot of other work was done, I’ve spent a month in total on the software, I think. There are still some major improvements possible, like handling of various messages via a application-wide message bus and processing physical LocoNet packets in a separate task (to not block other tasks that initiate communication).

PS. While I was updating the PCB, easyEDA released a version 3 and the release process disappointed me a lot. Not only was the new version lacking features from EasyEDA v2 for no reason, it’s a major incompatible release, its v3 file format is incompatible with v2 software, so everyone needs to upgrade the projects, and the process is irreversible. Previous version was pulled from the website and is not available in browser, so users are forced to use the new one even if the new feature set is limiting for them. In addition, To add to the insult, support is ignoring feedback and giving responses like "The v3 effect has already been changed and will not be retracted based on a minority of opinions."

All in all, very frustrating experience. I will consider KiCAD in future.

Making scale lumber loads out of reed

2026-07-13T00:00:00+00:00

I decided to make a load for a log car that has been standing empty for a long time, waiting to be used for some purpose. The result turned out to look very realistic, if I can say so myself.

At first I had the idea of making a 3D model of a log pile and printing it, but then I decided that a model made from natural products would look better and take less time.

So, the creation process looked like this: I gathered dry reed outside, cut it into pieces of the required length, stuffed the ends with wood putty, smeared the tree trunks with it to have some rough texture, glued the logs together to form loads and piles, and painted them with acrylic paints.

Glueing first and painting later saved a lot of time, as otherwise I would need to do a lot more painting.

The trunks were painted with a mixture of Vallejo Burnt Umber and Vallejo Orange in various proportions.

The most tedious part turned out to be cutting all the trunks to required length so that the load fits into the wagon but is not too short.

The most difficult part however was making proper color for the cuts; I even had to repaint them a couple of times, because they looked either too bright, too dark, or too yellow. In the end I can’t even say exactly what mixture was used: something like "a lot of yellow, a bit of rust, a whole lot of water."

Photos of the process

Arduino libraries for advanced audio playback

2026-05-18T00:00:00+00:00

This is a follow-up to my old post about advanced audio effects on a diorama, where I described the capabilities of the ESP8266Audio library and how it can be used to create realistic sound effects.

I decided to have my own try on welding effect on a model train layout with sounds, and here I present my findings on the topic of audio libraries for audio playback.

I’ve found some more Arduino libraries that allow such tasks. Compared to ESP8266Audio, some of these solutions target AVR MCUs, so will work on a classic Arduino UNO or NANO.

Mozzi

The library is aimed at music synthesis, so polyphony comes as a default, as are some effects like frequency changes. Has built-in support for looping samples, with customizable loop points, which is a nice feature for engine sounds or similar long-lasting looping sounds.

Does not support WAV files, you’ll need to encode your samples as a Mozzi-specific C array. It’s designed to play small samples that are stored in MCU flash, so completely lacks features for streaming data from SD card, Internet or anything similar.

Targets AVR-based Arduinos, so can work with minimal amount of resources. Also supports a wide range or targets like STM32, RP2xxx, ESP32, ESP8266, Teensy and others.

The API allows to start/stop individual samples at any time, so it’s easy to play multiple samples at the same time and synchronize sounds with other events, like lighting effects. All in all, I quite liked the API and structure of the library.

TMRpcm

Plays WAV files from SD card (attached to Arduino via SPI bus). It’s written to play one file at a time, but can be used to play 2 files simultaneously by sending them to different PWM channels and mixing them in hardware (see MultiTrack examples). Works with AVR-based Arduinos, so needs minimal amount of resources.

I briefly looked at the internals of the library and ~~it's horrible~~ didn’t like the design decisions, for example those that restrict playing multiple files into a single output.

ESP32Synth

As the name suggests, works only with ESP32-based MCUs (and relies on its FreeRTOS for background processing). Has massive polyphony, can play samples from SPI SD card. Can output to I2S DACs, built-in DACs, PWM/PDM pins, Bluetooth sinks. I haven’t used it, as for this usecase it doesn’t add much compared to the ESP8266Audio library.

BackgroundAudio

Successor to the ESP8266Audio library, with limited feature set but better performance. Only supports ESP32 and RP2xxx MCUs, not AVR-based Arduinos. I haven’t used it, but may try in future.

On supercapacitor Stay-alive circuits

2026-05-07T00:00:00+00:00

Dirty tracks and power interruptions are a bane of model railroads. To combat this, stay-alive circuits are used, which usually consist of a bulky capacitor and some charge-discharge circuitry. Electrolytic capacitors of several 1000s of uF (micro Farads) are typically used.

In recent years, supercapacitors are becoming more and more popular. These state-of-the art devices with capacitances of several Farads(!) allow several seconds and several dozen centimiters of travel when power is interrupted.

The complications with supercapacitors come from the fact that they are usually low-voltage devices, 2.7-5.4V (5.4V are typically just 2 2.7V caps in series internally). Another issue with capacitors is that when empty, they take very high current, which to the command station can look as a short-circuit or an overcurrent situation, especially if there are a lot of locomotives on tracks and they all start to charge at the same time (which happens when power is turned on).

To regulate the second issue, RailCommunity standard RCN-530 requires that charging current be limited to 100mA.

So, the usual solution with supercapacitors is to charge them to their nominal voltage, and then to raise voltage back when discharging. First part is done with either a linear regulator or a step-down switching regulator, second part is performed by a step-up switching regulator.

For quite some time I was looking for examples of supercapacitor-based circuits, and finally found an open source design here: https://www.stummiforum.de/t171549f21-RE-SuperCapLader-im-Eigenbau-Goldcaps-als-Pufferspeicher.html (in German, and access to uploaded files requires an account) In fact, the forum thread contains several generations with different features.

All designs from original poster (schumo99) use a Diodes Inc. AP3211 step-down (buck) regulator to drop input to charging voltage, and an Olimex MT3608 step-up (boost) regulator for raise it back to 10V. Both regulators need their own inductors, one of which is the biggest component on the PCB.

Step-down regulator has an enable pin that is controlled by a 10V Zenner diode, it’s designed to enable charging when input voltage is above ~11.5V. Step-up regulator is always enabled, but switches itself off if input voltage is higher than 10V.

In the first generation of the design, charge current is limited by a 10-15 Ohm resistor in series with the supercapacitor. As a result, the resistor wastes a lot of energy at the beginning of charging cycle when the current is high. Wasted energy mandates that the resistor be big enough to dissipate it.

Second design (called "fast") improves on this by replacing current-limiting resistor with modified feedback loop for the regulator to limit the current as well as voltage. This effectively creates a CC/CV (constant current/constant voltage) charging circuit often found in Li-Ion chargers.

Another important characteristic of supercapacitor designs is minimum capacitor voltage that can still be used by boost converter. For example, when using 2.7V supercapacitor with a boost converter that works from 2.0V, only portion of energy from 2.7V to 2V can be used, and the rest will stay in the capacitor. For this reason, putting 2 2.7V capacitors in series to form a 5.4V capacitor is preferrable even though the capacitance of such connection is halved.

After studying the whole forum thread, I’ve come up with my own ideas for improvements.

Ideal solution: bi-directional buck-boost converter. Such a device replaces both charging and discharging havles of the circuit. It operates as a buck converter to charge the capacitor, and then operates as a boost converter to discharge it. Such a device would decrease component count by a lot, e.g. only only one inductor would be needed. Unfortunately, I could not find any part that fits this usecase perfectly, these are the ones that come close (but still unsuitable):
- SY9329 - CC/CV DC-DC, needs I2C to configure and switch, not found on LCSC
- BQ25690 - CC/CV DC-DC, needs 1 mosfet, seems to require I2C to switch
- TPS61289 - DC-DC, no CC mode, needs external mosfet
- ISL81601 - CC/CV DC-DC, needs 4 external mosfets
- LTC3110 - CC/CV DC-DC, 5V input only
- MAX38889 - DC-DC with CC/CV, only 5V, expensive, otherwise would be perfect
On top of lack of needed features, all these are rather specialized devices and are somewhat expensive.
CC/CV buck converters. The described design uses a CV buck converter with a modified feedback loop to acheive CC/CV functionality. In fact, ready-made CC/CV buck converters exist, so they can be used instead, reducing component count. Devices of this nature can be found in several categories like Switch-mode regulators (are usually CV-only, CC mode is rare), LED drivers (these are CC-mode regulators, and CV mode for them is rare), battery chargers (these are often tailored to specific voltages for specific chemistry), which complicates searching for them. Here are some potential devices that I’ve found, and most have some drawbacks.
- TPS92365x - suitable LED driver from TI, voltage is limited by over-voltage-protection feature.
- BQ24640 - CC/CV charger, proper input voltage, needs external MOSFETs. Comes in 3x3mm package. Might be used, needs testing, but price is not that attractive at ~$3 on LCSC.
- BQ25306 - CC/CV battery charger, up to 17V (but 28V abs max)
- LT3663 - CC/CV step-down regulator from LT, expensive, almost missing on LCSC
- A7431A - CC/CV step-down, big package, not found on LCSC.
Step-up regulators with low startup voltage. MT3608 used in the described design start up from 2V. This can be improved by replacing it with a boost converter with a lower startup voltage. Here I coundn’t find much, but this chip was suggested here:
- ME2149: 0.9V startup

Dumpcar and its updates

2026-03-09T00:00:00+00:00

For a couple months I’ve been working a new project, a model of a dumpcar wagon. The model needs motorized tipping and mechanized sideboards.

For full details, you can go to project page, while this post has small update for the project.

This is a video with the latest state of mechanical features. I’ve printed both sideboards with linkages, and assembled the full bed. The joints are free to rotate to make motors' work easier later.

As I only assembled one side before, it turned out that putting 2 sets of linkages into the bed creates a rather tight fit (especially after some shrinking and warping of the resin). So I had to redesign the linkages and their slot to make linkages thinner, slot larger, while still keeping things as thick as possible to retain rigidity. I also had to leave more empty space for protruding pieces of copper wire, as they turned out to stick out and scratch resin, creating friction and preventing free move. As a countermeasure I also try to clip them as flush as possible.

LEDs and magnet magic

2026-02-26T00:00:00+00:00

In 2024 I’ve made a small model of a construction site light tower and generator. I forgot to make a post about it here, so here it goes.

At first glance it looks like a rather simple scale model, just with fine details. However, the light tower is functional, and has a trick up its sleeve - it only turns on when you "attach" a generator to it.

The generator is, in fact, just a magnet, and the tower contains all the power and logic. The power supply is a small Li-Ion battery (from TWS headphones), and the "logic" is a reed switch that turns on when the magnet is close. The size of the magnet is chosen so that it triggers the switch when placed on one side of the tower, but on the other.

The idea is that these models should be transported by an RC truck across the diorama, unloaded on site, and then the operator could turn the lights on, all remotely, using only a manipulator crane.

Further idea is to use magnetic sensor IC instead of a reed switch, for footprint reduction. The sensors are available in SOT23 package, for example. You can read more about those in this post.

A memo about selection of hand-solderable miniaturized electronic components

2026-02-24T00:00:00+00:00

Part of my latest project is a custom a DCC decoder that needs to fit into frame of a dumpcar. This means it needs to be much smaller than a standard DCC decoder, with required width of mere 6 mm.

For this, I performed a search of the smallest electronic components possible, that would still do the job, with the requirement that they are also hand-solderable, i.e. have legs and are not too small. And they need to be available at LCSC. Being available for JLCPCB PCB assembly is a bonus.

Comparison of different small footprints (and corresponding devices)

Linear regulators

Usecase: drop track voltage (12-25V) to power MCU and other logic, i.e. 3.3V or 5V. For small footprints are are limited to low currect output (<50mA), but this should be fine for the usecase.

Suggested package: SC-70-5 aka SOT-353 or SOT323-5. It’s smaller than SOT23 while still being legged and solderable.

ST’s ST715C50R - 5V output, 24V, 85mA
ST715C33R is a 3.3V option. It is comparatively expensive and doesn’t have a lot of availability (<100 in stock at the moment)
TechPublic TPS71550DCKR-TP - 5V output, 24V input, 50mA current capacity.
TPS71533DCKR-TP - 3.3V variant, 24V, 50mA. Judging by specs and product number, these are a relatives to ST’s part. They are noticable cheaper and are easily available.

In my decoder I used TechPublic ones.

Dual transistors

Usecasae: control several high-power outputs with an MCU. For example, to have standatized decoder outputs for lamps, couplers, smoke machines etc.

Package: SOT-363 aka SC-70-6. Same size as SOT-353 before, but with 6 legs.

AO7800. 900mA, Vds=20V. These are values from Alpha&Omega, the device is available from many manufacturers.
NX3008NBKS-TP. 800mA, Vds=30V, works on 3.3V.
NTJD4001 from VBSemi. 2.6A Vds=20V. Same device from other manufacturers offer much smaller currents.
BJTs could be also useful, for example, if control voltage is not logic level, but full supply voltage, which might be outside allowable mosfet gate voltages. Seen in this project. It uses MMDT3904 in SOT-363 (200mA), available from a lot of manufacturers. MMDT2222 should also work and is seemingly better suited due to higher current capacity.

Diodes

Usecase: rectify DCC track voltage. Secondary usecase: to feed DCC signal to MCU (as part of a voltage conversion circuit).

Package: SOD-523 (SOD stands for Small Outline Diode). It is comparable to 0603 is size. There are 0603 (and 0805) diodes, but they have solder pads only at bottom (as compared to resistors and capacitors that have all sides as leads), so not solderable with an iron.

To build a rectifier, 4 of these diodes use less space than a single-chip rectifier. Smallest full recifier I found is in MBS package (~5x8mm footprint)

B5819WT Shottky diode, 40V, 1A, 610mV@1A. Many options are available by this name, with different characteristics.

Next diode size is SOD-323. It’s slightly larger.

Ordering metal 3D prints for the 1st (and 2nd) time

2026-02-24T00:00:00+00:00

JLCPCB’s 3D printing services offer metal 3D printing, which I wanted to try for some time. Since I got a coupon for it, I finally made an order. The model I wanted to print is a frame for a 1/87 scale truck. The model would benefit from extra weight and of course, compared to resin, metal frame is much more durable.

Requirements and review process

There are 2 metal printing processes offered by JLC3DP, SLM (selective laser melting) and BJ (binder jetting). The materials available are titanium and 316 steel.

BJ is a bit cheaper, but, as JLC technicians told me, is not suitable for such small and detailed models, so I switched to SLM.

Please kindly note that this model structure is not suitable for printing BJ-316 material, which is prone to breakage. It is recommended to switch to SLM process.

For metal SLM printing, the features of the model shall not be smaller than 1.5mm, 2mm preferrable, and the online viewer highlights the parts that are smaller/thinner (or at least it used to before, I don’t see it now for some reason). My model is full of features that are much finer, so during the review I got a request to accept possible risks that the model will come out imperfect.

Costs and duration

3D printing of this model costs around $8. Economy shipping discount covers 3DP orders too, so shipping can cost around $3 if the model is light enough. Around $3 is added on top of this as taxes and duties.

Stated manufacturing duration is 3 working days (72h), but in my case it took around 10. Interestingly, around 7 days were spent in QC step, and steps to actually print the model took promised 3 days. Seems QC was the bottleneck at the moment.

The package arrived in the same blue JLCPCB box delivered by same shipping company, as PCBs.

Review

The WOW effect is definitely there! It is immediately clear that this is a metal part, with noticeable weight and strength. No doubt that it can handle everything that a scale model truck can load it with, and much more.

However, cool factor, weight and strength are nearly the only advantages over resin printing.

Surface quality and detail representation is significantly worse then for resin 3D print. This is expected, but still disappointing. The surface is very rough.

Geometrically, the print is not very true to the model Holes came out smaller than designed; my typical 1.1mm hole was not large enough for an M1 screw. Out of 2 gaps of 0.3mm, one was completely jammed with metal bits, another was ok.

It’s clear that much looser tolerances must be designed in for metal printing, compared to resin printing. In general, all the features are 0.1—0.2mm larger, so this must be compensated in the model.

Some protruding parts were warped or bent. Luckily, this was easily fixed by bending them back with pliers and vise (which is in stark contrast to resin prints).

316 steel is rather difficult to work with. I broke nearly all my 1mm drillbits while trying to enlarge the holes. (but then, they were PCB-orientd drillbits, not intended for any steel, let alone 316 grade).

Second order

Because I was gifted with a coupon for 3D printing, I decided to order the model again, after making sure that the holes are bigger and tolerances looser. This time manufacturing went much faster, around 3 days as promised.

This time the coupon did not stack with economy shipping offer, so in total the order ended up costing more than the first one (but still cheap for what it offers). Seems this is the new change in 2026, as there was an announcement that coupons now can cover shipping costs (they didn’t before), and seemingly as a result, the ability to use economy shipping discount with coupons was sacrificed.

Surpringly, the quality of the print was much worse that the first print. There is a lot more warping, and the surface quality is poorer in general. It seems the engineers placed this model at ~45 deg angle, as is visible from the layer lines. The first order was printed flat, and came out much better. So, for posterity, during the review process, try to ask to put the model flat.

The screw holes are good this time, but out of 2 thin slots (which I made larger), one was again filled with metal. I was able to open it with a small slot screwdriver and a hammer.

A memo about LCSC orders with coupons

2026-02-05T00:00:00+00:00

A short note to myself about discount coupons on LCSC. Contrary to other JLC sites, on LCSC coupon amount must be lower than the cost of the shopping cart. On other sites, e.g. JLCPCB, you can apply a higher cost coupon to your order, unused coupon amount will be lost. LCSC factors in coupons on the fly, even before checking out, so after your cart gets more expensive then the coupon you have (only components are considered, not shipping), the price is reduced immediatey by the coupon amount. The coupon for components (e.g. th one you can get on Oshwlab) cannot be used to reduce shipping cost. There doesn’t seem to be a way to choose the coupon if you have more than one applicable.

At the time of the order there were some public coupons on shipping ($5 for order of $30, $8 for order of $50, etc). They stack with components coupons, and applying coupons to reduce cart cost does not remove the shipping discount.

Here I used $40 coupon from oshwlab, exchanged from points, and $5 shipping coupon that the system was giving to everyone at the time

A Newlib upgrade mystrery story

2026-02-04T00:00:00+00:00

Introduction

I was working on a DCC decoder project that I recently started.

Firstly, I found out a great library for DCC devices. It’s from ZIMO, actively maintained, so it’s possible that’s it’s what actually powers their products. It provides both command station building blocks and decoder building blocks. I quite like that it uses modern C++ and is hardware-agnostic, so users can feed it DCC data the way they like it. What I didn’t like is that it uses too modern C++, i.e. C++23 features as well as concepts, spans, z-suffixes etc. I ended up cleaning up the necessary parts to be mostly C++11 and ETL-compatible, but also looked at upgrading GCC to higher versions to use some of those fancy features myself.

Secondly, the devboard I happen to use for a prototype has a PY32F003x6 with 4Kb of RAM, and the percentage used is printed after each compilation. To my surprise, upgrading from GCC 9.2.1 GCC to 12.3.1 caused an increase of RAM usage from about less than a third to almost half! In absolute terms it’s going from 1200 to 1900B, but with this small total amount, every big jump is noticable! It’s not helping that some RAM must be free for stack and heap, so it definitely cannot go to 100%.

GCC 9.3.1:

RAM:   [===       ]  29.2% (used 1196 bytes from 4096 bytes)
Flash: [==        ]  20.0% (used 6540 bytes from 32768 bytes)

GCC 12.3.1:

RAM:   [=====     ]  46.3% (used 1896 bytes from 4096 bytes)
Flash: [==        ]  20.7% (used 6792 bytes from 32768 bytes)

In addition, compiling with newer compiler caused output of strange warnings from linker:

C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/../../../../arm-none-eabi/bin/ld.exe: C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/thumb/v6-m/nofp\libc_nano.a(libc_a-closer.o): in function `_close_r':
closer.c:(.text._close_r+0xc): warning: _close is not implemented and will always fail
C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/../../../../arm-none-eabi/bin/ld.exe: C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/thumb/v6-m/nofp\libc_nano.a(libc_a-lseekr.o): in function `_lseek_r':
lseekr.c:(.text._lseek_r+0x10): warning: _lseek is not implemented and will always fail
C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/../../../../arm-none-eabi/bin/ld.exe: C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/thumb/v6-m/nofp\libc_nano.a(libc_a-readr.o): in function `_read_r':
readr.c:(.text._read_r+0x10): warning: _read is not implemented and will always fail
C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/../../../../arm-none-eabi/bin/ld.exe: C:/Users/user/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/14.2.1/thumb/v6-m/nofp\libc_nano.a(libc_a-writer.o): in function `_write_r':
writer.c:(.text._write_r+0x10): warning: _write is not implemented and will always fail

So, this post is about me investigating the source of this regressions.

RAM usage

I generated linker map files for both compilations with -Wl,-Map,output.map,--cref and inspected them in AMap viewer. Immediately it was visible that there are extra 312b used by __sf symbol, coming from _findfp.o. This was enough information to google, and I found out that some users have already noticed this problem:

on stackoverflow: https://stackoverflow.com/questions/78635407/gnu-tools-for-stm32-change-after-version-9-including-non-required-module-and-con

on newlib mailing list: https://inbox.sourceware.org/newlib/cee20c8a-7881-7cec-07da-ca5b41719b00@embedded-brains.de/t/#u

Mailing list says it was a regression from "splitting up the struct _reent into individual thread-local storage objects" between versions 4.2.0 and 4.3.0. The discussion is from 2023 and as nothing was done since, I assume that unfortunately it is not considered a problem by devs.

Studying the code reveals that __sf is an array of 3 FILE objects, which predictably are stdin, stdout and stderr handlers. Size of a file struct is around 100B.

Before version 4.3.0 __sf contained much smaller __sFILE_fake file objects, which were removed around June 2022.

Finding affected toolchain versions

Sources

Newlib repo is hosted at https://www.sourceware.org/git/gitweb.cgi?p=newlib-cygwin.git, but it’s also a repository of Cygwin, who host their repo (the same one) at https://cygwin.com/cgit/newlib-cygwin/ and maintain an official GitHub mirror here: https://github.com/cygwin/cygwin.

In a typical ARM project setup, newlib is bundled with ARM gcc compiler. PlatformIO registry is populated by (some) versions from xPack GNU Arm Embedded GCC Project, those are populated from ARM GNU Toolchain releases. The version of newlib in ARM distribution is specified in arm-none-eabi/include/_newlib_version.h. However, what is written there is the latest release before the source was compiled, as ARM team simply grabs latest commit at the moment of release. This introduces a problem with one of the releases below.

Each release on arm website offers a manifest file which says where the files were taken from, this allows to find a specific commit in newlib repo that was used to build the library.

Here are some GCC versions from ARM GCC downloads page and from platformio registry.

GCC 10.3.1 ships with newlib 4.1.0 — ok
GCC 11.2 has newlib 4.1.0 — ok.
GCC 11.3.rel1 has newlib 4.2.0 — ok. (_newlib_version.h is broken in this release, so it must be found through commits)
GCC 12.2.rel1 - this one is a bit problematic. The version clearly says "4.2.0". However, the source was taken from commit faac797, which is almost a year after version 4.2.0 was released, and roughly a month before 4.3.0. It already includes __sf symbol in the build — not ok!
GCC 12.3.1 has newlib 4.3.0 — not ok!

Why is the symbol there in the first place?

It’s pulled by using vsnprintf function that I use to print to UART. Removing calls to vsnprintf removes the symbol too.

vsniprintf is a thin wrapper over _vsniprintf_r function. _r here means "reentrant", and as a first argument it takes a global _REENT struct.

The structure is essentially an array of all global objects, including errno and pointers to stdin/out/err file descriptors (the __sf entries we are concerned with!). The struct is also ~80 bytes large.

This function creates a fake file descriptor (but fully-sized) and calls _svfprintf_r that in turn does the actual formatting. These functions make frequent use of errno element of _REENT and maybe other file-related elements of it.

Seemingly vsnprint is the only place where _REENT is used in my program, so removing the call allows it, along with referenced __sf, to be garbage-collected by linker.

Linker warnings

Other reports and discussions about the problem:

It was also mentioned in the newlib mailing list, with no resolution: https://inbox.sourceware.org/newlib/CAAHv3KF9Gt0sFtm7Qsx4XfO0iaGHP2n9Z2qS8LdL2ytJcqR%3Deg%40mail.gmail.com

While I found no reliable and non-hacky solution, in the links above it is stated that the problem appeared in v11.3 of Arm GNU Toolchain. Indeed, after looking myself at the files, I found that starting with v11.3, file libnosys.a contains sections ".gnu.warning" (e.g. ".gnu.warning._write"), which cause the linker to produce the warnings (this feature of LD linker is described in its documentation). In 11.2 these sections are not present.

In newlib codebase, these warnings have been there for at least 27 years, so something in the process of building the library changed between 11.2 and 11.3. The warnings are still there in Arm GNU Toolchain v12 and v14.

The warnings are from linker, not from compiler, so they cannot be supressed with -W flags.

At the time of writing, my request for clarifications on ARM’s forum is not answered.

It seems that one workaround is to go ST’s way and supply stubs for these functions manually. However it somewhat defeats the purpose of using nosys in the first place (whose goal is to provide these stubs).

Outcome

Latest version in PlatformIO registry that is unaffected by both problems is 10 (10.3.1). It has moderate support for C++20 features, e.g. concepts, but almost no support for C++23.

For those who are not tied to platformio, GCC 12.2r1 should also work and not have these problems.

Also, since I use ETL extensively, a recently added etl::format is of interest as a potential replacement of sprintf.

It turns out, GCC 10 does not play well with debugging with blackmagic-debug. Its GDB complains about register mismatching packet. Older and later GCC versions, e.g. 14, do not show this problem. Which means I’m somewhat back to square one with selection of GCC version. I am now considering to use latest GCC, not use newlib printf functions family and use a 3rd party implementation, in particular nanoprintf. It seems to be feature-rich enough while still being small enough. It’s also actively maintained. However, the problem with linker warning would still be there.