Investigate picolibc, other newlib alternatives

[jepler]

We rely on newlib as implementation of the C standard library and some standard UNIX APIs. This is generally furnished by ARM with their GNU toolchain.

In the past, @tannewt has observed that Arch's newlib is built with -Os, and this saves some space compared to the newlib furnished by ARM.

I noticed that Debian has picolibc which has its home at https://keithp.com/picolibc/ -- I believe it would be suitable to incorporate in CircuitPython.

As Picolibc is a combination of code from both Newlib and AVR-libc, the licensing of the source code is fairly complicated, but the general result is similar to the BSD 3-clause or MIT license.

Investigate alternatives and see which one gains back code space with the least "other costs" associated.

[jepler]

[jepler]

I tried picolibc with the following change:

--- a/ports/atmel-samd/Makefile
+++ b/ports/atmel-samd/Makefile
@@ -197,9 +197,9 @@ CFLAGS += \
        -DSAM_D5X_E5X -DSAME51
 endif
 
+CFLAGS += -specs=/usr/lib/picolibc/arm-none-eabi/picolibc.specs
 
-
-LDFLAGS = $(CFLAGS) -nostartfiles -Wl,-nostdlib -Wl,-T,$(GENERATED_LD_FILE) -Wl,-Map=$@.map -Wl,-cref -Wl,-gc-sections -specs=nano.specs
+LDFLAGS = $(CFLAGS) -nostartfiles -Wl,-nostdlib -Wl,-T,$(GENERATED_LD_FILE) -Wl,-Map=$@.map -Wl,-cref -Wl,-gc-sections
 LDFLAGS += -flto=$(shell $(NPROC))
 LIBS := -lgcc -lc
 

However, it

• made the binary bigger (tested trinket_m0) -- reported free size 1396 bytes, down from 1644 bytes
• emitted some odd warnings during linking
• created a .bin file that was waaaayy too big (related to the above item?)

ld: warning: cannot find entry symbol _start; defaulting to 0000000000002000
ld: build-trinket_m0/firmware.elf: section `.data' can't be allocated in segment 2
LOAD: .ARM.exidx .data .stack
ld: build-trinket_m0/firmware.elf: section `.stack' can't be allocated in segment 2
LOAD: .ARM.exidx .data .stack

[bhav97]

Using picolibc 1.7.3
and gcc version 10.2.1 20201103 (release) (GNU Arm Embedded Toolchain 10-2020-q4-major)

I can build 4922e67b8c35ae3f87919c15ddf16efa62a2a4a5 with

--- a/ports/atmel-samd/boards/common.template.ld
+++ b/ports/atmel-samd/boards/common.template.ld
@@ -1,4 +1,5 @@
 /* Template for SAMD21/SAMD51 linking. dollar-sign-curly-bracket items are replaced with strings. */
+ENTRY(Reset_Handler);

 /* Specify the memory areas */
 MEMORY

--- a/lib/libm/libm.h
+++ b/lib/libm/libm.h
@@ -19,7 +19,7 @@
 #include <stdint.h>
 #include <math.h>

-#define FLT_EVAL_METHOD 0
+// #define FLT_EVAL_METHOD 0

 #define FORCE_EVAL(x) do {                        \
        if (sizeof(x) == sizeof(float)) {         \

in addition to your changes

[jepler]

Nice! Did your build work? Was the size any better than with newlib?

[bhav97]

I don't have any cp boards to test my builds right now (I was working on a new port when I saw this issue). In terms of size I saw a 0.12% increase in text, 16.78% increase in bss

~/circuitpython/ports/atmel-samd$ arm-none-eabi-size  build-trinket_m0/firmware.elf #picolibc
   text    data     bss     dec     hex filename
 185588     320   12584  198492   3075c build-trinket_m0/firmware.elf
~/circuitpython/ports/atmel-samd$ arm-none-eabi-size  baseline/firmware.elf #newlib
   text    data     bss     dec     hex filename
 185364     316   10472  196152   2fe38 baseline/firmware.elf

In case anyone has a trinket available for testing, here's the firmware.zip

[bhav97]

I tried a build with picolibc on my itsybitsy_m4_express. It doesn't boot. The segments are messed up and the exception table is not @ 0x4000

(I am also building with INTERNAL_LIBM = 0 instead of editing libm files)

The picolibc build doesn't have the exception table @ 0x4000 (tinyuf2 doesn't exit the programming session at all)

> * created a .bin file that was waaaayy too big (related to the above item?)

Adding the ENTRY(Reset_Handler) fixed this, I have seen this before when the linker assumed my code started @ 0x0 but some things were explicitly placed @ 0x8000000. Only the resulting binary was unnecessarily large.

> * emitted some odd warnings during linking

Adding the ENTRY(Reset_Handler) removed this as well, but that wasn't intended. I don't know why this happens, but if you check the segment sections using readelf -l, you'll see some missing segments

For the broken build with the ENTRY added to the linker script

$ readelf -l firmware.elf

Elf file type is EXEC (Executable file)
Entry point 0x54f1d
There are 4 program headers, starting at offset 52

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x004000 0x00004000 0x00004000 0x6d4f4 0x6d4f4 R E 0x10000
  LOAD           0x080000 0x20000000 0x000714f4 0x00650 0x0949c RW  0x10000
  LOAD           0x0000b4 0x00000000 0x00000000 0x00000 0x00000     0x10000
  TLS            0x000000 0x00000000 0x00000000 0x00000 0x00000     0x4

 Section to Segment mapping:
  Segment Sections...
   00     .text 
   01     .data .bss .stack 
   02     
   03     

$ nm -s build-itsybitsy_m4_express/firmware.elf  | grep exception_table
00071290 T exception_table

Same as above, built with newlib

$ readelf -l build-itsybitsy_m4_express/firmware.elf

Elf file type is EXEC (Executable file)
Entry point 0x77d1
There are 4 program headers, starting at offset 52

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  EXIDX          0x070454 0x00070454 0x00070454 0x00008 0x00008 R   0x4
  LOAD           0x000000 0x00000000 0x00000000 0x7045c 0x7045c R E 0x10000
  LOAD           0x080000 0x20000000 0x0007045c 0x00638 0x00638 RW  0x10000
  LOAD           0x000640 0x20000640 0x00070aa0 0x00000 0x085ec RW  0x10000

 Section to Segment mapping:
  Segment Sections...
   00     .ARM.exidx 
   01     .text .ARM.exidx 
   02     .data 
   03     .bss .stack  

$ nm -s build-itsybitsy_m4_express/firmware.elf  | grep exception_table 
00004000 T exception_table

[bhav97]

Some other interesting info in the map file (picolibc.ld)

Memory Configuration

Name             Origin             Length             Attributes
flash            0x0000000010000000 0x0000000000010000 axrl !w
ram              0x0000000020000000 0x0000000000008000 axw !rl
*default*        0x0000000000000000 0xffffffffffffffff
FLASH_BOOTLOADER 0x0000000000000000 0x0000000000004000 xr
FLASH_FIRMWARE   0x0000000000004000 0x000000000007a000 xr
FLASH_FILESYSTEM 0x000000000007e000 0x0000000000000000 r
FLASH_CONFIG     0x000000000007e000 0x0000000000000000 r
FLASH_NVM        0x000000000007e000 0x0000000000002000 r
RAM              0x0000000020000000 0x0000000000030000 xrw

Linker script and memory map

                [!provide]                        PROVIDE (__stack = (ORIGIN (ram) + LENGTH (ram)))

.init
 *(.text.init.enter)
 *(.data.init.enter)
 *(SORT_BY_NAME(.init) SORT_BY_NAME(.init.*))

.text           0x0000000000004000    0x6d290

vs when we actually build with the intended linker script

Memory Configuration

Name             Origin             Length             Attributes
FLASH_BOOTLOADER 0x0000000000000000 0x0000000000004000 xr
FLASH_FIRMWARE   0x0000000000004000 0x000000000007a000 xr
FLASH_FILESYSTEM 0x000000000007e000 0x0000000000000000 r
FLASH_CONFIG     0x000000000007e000 0x0000000000000000 r
FLASH_NVM        0x000000000007e000 0x0000000000002000 r
RAM              0x0000000020000000 0x0000000000030000 xrw
*default*        0x0000000000000000 0xffffffffffffffff

Linker script and memory map

                0x000000002002fff8                _estack = ((ORIGIN (RAM) + LENGTH (RAM)) - 0x8)
                0x000000002002fffc                _bootloader_dbl_tap = ((ORIGIN (RAM) + LENGTH (RAM)) - 0x4)

.vectors        0x0000000000004000      0x264
                0x0000000000004000                . = ALIGN (0x4)
                0x0000000000004000                _sfixed = .
 *(.vectors)
 .vectors       0x0000000000004000      0x264 firmware.elf.lto.o
                0x0000000000004000                exception_table

.text           0x0000000000004270    0x6c1f4
                0x0000000000004270                . = ALIGN (0x4)

[bhav97]

I got it working, with this. No other changes needed. (-T instead of -Wl,-T)

--- a/ports/atmel-samd/Makefile
+++ b/ports/atmel-samd/Makefile
@@ -199,7 +199,8 @@ endif
 
 
 
-LDFLAGS = $(CFLAGS) -nostartfiles -Wl,-nostdlib -Wl,-T,$(GENERATED_LD_FILE) -Wl,-Map=$@.map -Wl,-cref -Wl,-gc-sections -specs=nano.specs
+LDFLAGS = $(CFLAGS) -nostartfiles -Wl,-nostdlib -T$(GENERATED_LD_FILE) -Wl,-Map=$@.map -Wl,-cref -Wl,-gc-sections --specs=picolibc.specs
 LDFLAGS += -flto=$(shell $(NPROC))
 LIBS := -lgcc -lc

I don't know why the argument never propogated to the linker but because the picolibc.spec defines something like !T:-Tpicolibc.ld we just don't see the error.