Skip to content

Conversation

@maribu
Copy link
Member

@maribu maribu commented Dec 12, 2025

Contribution description

This changse the build system to expose the board name, the app name, the CPU name, and the RIOT version via dynamically generated headers rather than via CFLAGS += -D.

The intended goal is to to reduce the entropy of the input of ccache with the hope that for platform independent code, the compiler input to ccache will be often identical for related boards and similar apps.

The downside is that code that previously dependent on the macros being predefined by the environment has not to explicitly include the suitable dynamically created header(s).

One new macro has also been added: -DRIOT_OS=1. Previously a lot of code was checking for RIOT_VERSION to compile differently under e.g. Linux and RIOT. The code has been updated to check for RIOT_OS instead.

Testing procedure

  • A CI check has been added to static-tests find uses of previously global macros
  • Compilation should still work

Issues/PRs references

None

@github-actions github-actions bot added Platform: native Platform: This PR/issue effects the native platform Platform: ARM Platform: This PR/issue effects ARM-based platforms Area: network Area: Networking Area: tests Area: tests and testing framework Area: core Area: RIOT kernel. Handle PRs marked with this with care! Area: build system Area: Build system Area: pkg Area: External package ports Platform: AVR Platform: This PR/issue effects AVR-based platforms Area: drivers Area: Device drivers Area: tools Area: Supplementary tools Area: boards Area: Board ports Area: OTA Area: Over-the-air updates Platform: ESP Platform: This PR/issue effects ESP-based platforms Area: CoAP Area: Constrained Application Protocol implementations Area: cpu Area: CPU/MCU ports Area: USB Area: Universal Serial Bus Area: sys Area: System Area: examples Area: Example Applications labels Dec 12, 2025
@maribu maribu added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: full build disable CI build filter labels Dec 12, 2025
@maribu
Copy link
Member Author

maribu commented Dec 12, 2025

Note: This needs still an update of the Doxygen doc to account for the dynamically generated headers and an update of the riotbuild.h documentation.

@riot-ci
Copy link

riot-ci commented Dec 12, 2025

Murdock results

FAILED

634fdf0 build-system: Pass -D flags with @file argument

Build failures (21)
Application Target Toolchain Runtime (s) Worker
tests/drivers/my9221 arduino-mega2560 gnu 0.49 mobi7
tests/drivers/my9221 arduino-nano gnu 0.28 alien
tests/drivers/my9221 arduino-leonardo gnu 0.43 mobi6
tests/drivers/my9221 arduino-duemilanove gnu 0.53 mobi6
tests/drivers/my9221 arduino-uno gnu 0.31 alien
tests/drivers/sdcard_spi arduino-duemilanove gnu 0.53 mobi6
tests/drivers/sdcard_spi arduino-leonardo gnu 1.60 mobi6
tests/drivers/sdcard_spi arduino-nano gnu 0.29 tatooine
tests/drivers/sdcard_spi arduino-uno gnu 0.62 alien
tests/drivers/sdcard_spi atxmega-a1-xplained gnu 0.65 mobi6
tests/drivers/sdcard_spi atxmega-a1u-xpro gnu 0.61 mobi6
tests/drivers/sdcard_spi atmega328p-xplained-mini gnu 0.53 alien
tests/drivers/sdcard_spi atmega328p gnu 0.56 mobi6
tests/drivers/sdcard_spi atmega8 gnu 0.26 alien
tests/drivers/sdcard_spi nucleo-f031k6 gnu 1.82 mobi6
tests/drivers/sdcard_spi esp8266-esp-12x gnu 2.36 alien
tests/drivers/sdcard_spi nucleo-l011k4 gnu 3.15 mobi6
tests/drivers/sdcard_spi esp32s3-wt32-sc01-plus gnu 4.56 skyleaf
tests/drivers/sdcard_spi esp8266-sparkfun-thing gnu 1.85 tatooine
tests/drivers/sdcard_spi samd10-xmini gnu 1.19 mobi6

and 481 more build failures...

Artifacts

@maribu maribu force-pushed the build-system/buildinfo branch 2 times, most recently from 24800cc to 9f0eb1d Compare December 12, 2025 22:36
@github-actions github-actions bot added the Area: doc Area: Documentation label Dec 12, 2025
@maribu maribu force-pushed the build-system/buildinfo branch 2 times, most recently from 0996f97 to 956427e Compare December 12, 2025 22:49
@maribu maribu added the CI: no fast fail don't abort PR build after first error label Dec 12, 2025
@maribu maribu force-pushed the build-system/buildinfo branch 2 times, most recently from 7574fff to 978b778 Compare December 13, 2025 10:48
@maribu maribu removed Area: USB Area: Universal Serial Bus Area: sys Area: System Area: examples Area: Example Applications labels Dec 13, 2025
@maribu maribu force-pushed the build-system/buildinfo branch from e43fb84 to 742b6e8 Compare December 14, 2025 17:57
@github-actions github-actions bot added Platform: native Platform: This PR/issue effects the native platform Platform: ARM Platform: This PR/issue effects ARM-based platforms Area: network Area: Networking Area: doc Area: Documentation Area: tests Area: tests and testing framework Area: pkg Area: External package ports Platform: AVR Platform: This PR/issue effects AVR-based platforms Area: drivers Area: Device drivers Area: BLE Area: Bluetooth Low Energy support Area: boards Area: Board ports Area: OTA Area: Over-the-air updates Platform: ESP Platform: This PR/issue effects ESP-based platforms Area: CoAP Area: Constrained Application Protocol implementations Area: cpu Area: CPU/MCU ports Area: USB Area: Universal Serial Bus Area: sys Area: System Area: examples Area: Example Applications labels Dec 14, 2025
maribu and others added 6 commits December 14, 2025 20:57
External code may assume that RIOT's `sched.h` header is the same as
[POSIX's sched.h][1] header. This adds a slim wrapper to not have to
patch external code (which in case of glibcxx will be difficult to do).

[1]: https://pubs.opengroup.org/onlinepubs/009604299/basedefs/sched.h.html
CPU specific hacks never should go to `core`. This simplifies the hack
and moves it to `cpu/native` by providing a custom `sched.h` that adds
the workarounds for glibc and adds an `#include_next "sched.h"`.

It's a bit of a pity that RIOT's sched.h has a name conflict with
POSIX's sched.h - otherwise those hacks wouldn't really be needed.
This changse the build system to expose the board name, the app name,
the CPU name, and the RIOT version via dynamically generated headers
rather than via `CFLAGS += -D`.

The intended goal is to to reduce the entropy of the input of ccache
with the hope that for platform independent code, the compiler input
to ccache will be often identical for related boards and similar apps.

The downside is that code that previously dependent on the macros being
predefined by the environment has not to explicitly include the suitable
dynamically created header(s).

One new macro has also been added: `-DRIOT_OS=1`. Previously a lot of
code was checking for `RIOT_VERSION` to compile differently under e.g.
Linux and RIOT. The code has been updated to check for `RIOT_OS` instead.
This checks files that use previously globally defined macros without
having the correct header included.

Also populate GITHUB_RUN_ID in `github_annotate.sh` for compatibility
with `set -u`.
This drops the generation of `riotbuild.h` and instead just passes the
`-D...` / `-U...` flags via a command line options file to GCC/clang
instead.

The old `riotbuild.h` now is moved to `core/include` and only contains
deprecated aliases and documentation. It is still injected with
`-include riotbuild.h`, as otherwise the aliases for dynamic macros
injected with `-D` would not be visible.
This adds some `extern char *buildinfo_<NAME>;` to the `riotbuild.h`
as alternative to `RIOT_BOARD`, `RIOT_VERSION`, `RIOT_CPU`, and
`RIOT_VERSION`. Unlike those macros the object file generated from a
compilation unit using the C API do not depend on the actual values,
so making them globally available does not impact ccache in any way.
@maribu maribu force-pushed the build-system/buildinfo branch from 742b6e8 to d29c445 Compare December 14, 2025 20:11
@maribu
Copy link
Member Author

maribu commented Dec 14, 2025

With https://ci.riot-os.org/details/169027b4b6ff444fb115fedaf0f240bf were are up to 65% cache efficiency on the second build on building examples/basic/default twice for the exact same board under a different name:

rm ~/.cache/ccache bin -rf && make RIOT_CI_BUILD=1 CCACHE_DEBUG=1 BOARD=wemos-zero -j && ccache --zero-stats &&  make RIOT_CI_BUILD=1 CCACHE_DEBUG=1 BOARD=wemos-zero-copy -j && ccache -s
Building application "default" for "wemos-zero" with CPU "samd21".

mkdir -p "/home/maribu/Repos/software/RIOT/master/examples/basic/default/bin/wemos-zero/riotbuild/buildinfo"
   text	   data	    bss	    dec	    hex	filename
  24424	    128	   5080	  29632	   73c0	/home/maribu/Repos/software/RIOT/master/examples/basic/default/bin/wemos-zero/default.elf
Statistics zeroed
Building application "default" for "wemos-zero-copy" with CPU "samd21".

mkdir -p "/home/maribu/Repos/software/RIOT/master/examples/basic/default/bin/wemos-zero-copy/riotbuild/buildinfo"
   text	   data	    bss	    dec	    hex	filename
  24432	    128	   5080	  29640	   73c8	/home/maribu/Repos/software/RIOT/master/examples/basic/default/bin/wemos-zero-copy/default.elf
Cacheable calls:     79 /  79 (100.0%)
  Hits:              52 /  79 (65.82%)
    Direct:           0 /  52 ( 0.00%)
    Preprocessed:    52 /  52 (100.0%)
  Misses:            27 /  79 (34.18%)
Local storage:
  Cache size (GiB): 0.0 / 5.0 ( 0.43%)
  Hits:              52 /  79 (65.82%)
  Misses:            27 /  79 (34.18%)

For the more realistic test scenario we are still at 0% cache hits:

Cacheable calls:    2048 / 2048 (100.0%)
  Hits:                0 / 2048 ( 0.00%)
    Direct:            0
    Preprocessed:      0
  Misses:           2048 / 2048 (100.0%)
Local storage:
  Cache size (GiB):  0.4 /  5.0 ( 8.18%)
  Hits:                0 / 2048 ( 0.00%)
  Misses:           2048 / 2048 (100.0%)

include $(RIOTMAKE)/scan-build.inc.mk

export RIOTBUILD_CONFIG_HEADER_C = $(BINDIR)/riotbuild/riotbuild.h
export COMPILER_CLI_PARAMS_FILE = $(BINDIR)/ccparams.txt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where does this file come from? I can't find any reference to it anywhere

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: BLE Area: Bluetooth Low Energy support Area: boards Area: Board ports Area: build system Area: Build system Area: CoAP Area: Constrained Application Protocol implementations Area: core Area: RIOT kernel. Handle PRs marked with this with care! Area: cpu Area: CPU/MCU ports Area: doc Area: Documentation Area: drivers Area: Device drivers Area: examples Area: Example Applications Area: network Area: Networking Area: OTA Area: Over-the-air updates Area: pkg Area: External package ports Area: sys Area: System Area: tests Area: tests and testing framework Area: tools Area: Supplementary tools Area: USB Area: Universal Serial Bus Platform: ARM Platform: This PR/issue effects ARM-based platforms Platform: AVR Platform: This PR/issue effects AVR-based platforms Platform: ESP Platform: This PR/issue effects ESP-based platforms Platform: native Platform: This PR/issue effects the native platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants