include/nuttx: Add link-time iterable sections infrastructure

Documentation/components/index.rst

@@ -13,6 +13,7 @@ case, you can head to the :doc:`reference <../reference/index>`.
 
    binfmt.rst
    concurrency/index.rst
+   iterable_sections.rst
    drivers/index.rst
    nxflat.rst
    nxgraphics/index.rst

Documentation/components/iterable_sections.rst

@@ -0,0 +1,155 @@
+=================
+Iterable Sections
+=================
+
+Iterable sections provide **link-time registration** of ``struct``
+instances: an instance defined with :c:macro:`STRUCT_SECTION_ITERABLE` in
+any compilation unit is placed in a dedicated linker input section.  The
+linker collects all instances into a contiguous, name-sorted array
+delimited by ``_<type>_list_start``/``_<type>_list_end`` symbols, which
+the code can then iterate like a plain C array -- no runtime registration
+calls, no central list to maintain.
+
+This is the same mechanism used by the Zephyr RTOS ``STRUCT_SECTION_*``
+macros.  The first user of this infrastructure is the zbus message bus
+port (``apps/system/zbus``, from nuttx-apps).
+
+C API
+=====
+
+The macros are provided by ``include/nuttx/iterable_sections.h``:
+
+.. code-block:: c
+
+   #include <nuttx/iterable_sections.h>
+
+   struct my_entry
+   {
+     const char *name;
+     int value;
+   };
+
+   /* In any .c file (const places the instance in ROM): */
+
+   const STRUCT_SECTION_ITERABLE(my_entry, entry_foo) =
+   {
+     .name = "foo",
+     .value = 42,
+   };
+
+   /* Iterate over every instance collected by the linker: */
+
+   STRUCT_SECTION_FOREACH(my_entry, entry)
+     {
+       printf("%s = %d\n", entry->name, entry->value);
+     }
+
+Available macros:
+
+* ``STRUCT_SECTION_ITERABLE(type, varname)`` -- define an instance inside
+  the iterable section ``._<type>.static.<varname>``.  The variable name
+  is part of the input section name, so the linker's ``SORT_BY_NAME()``
+  defines the iteration order (instances may encode ordering in their
+  names).
+* ``STRUCT_SECTION_FOREACH(type, iterator)`` -- for-loop over all
+  instances.
+* ``STRUCT_SECTION_GET(type, i, dst)`` -- random access by index.
+* ``STRUCT_SECTION_COUNT(type, dst)`` -- number of instances.
+* ``STRUCT_SECTION_START/END/START_EXTERN/END_EXTERN`` -- direct access
+  to the boundary symbols.
+
+Linker integration
+==================
+
+The collection step needs linker script support.  Two mechanisms are
+available; both rely on the fact that the linker scripts listed in
+``ARCHSCRIPT`` are preprocessed with CPP (arm, arm64, risc-v, xtensa,
+x86_64 and tricore), so ``#include`` and ``#ifdef CONFIG_*`` work inside
+them.
+
+Board script include (first-class mechanism)
+--------------------------------------------
+
+The board linker script includes the central fragments, which expand to
+nothing unless a subsystem using iterable sections is enabled:
+
+.. code-block:: text
+
+   .text :
+   {
+       ...
+       *(.gnu.linkonce.r.*)
+   #include <nuttx/linker/common-rom.ld>
+       _etext = ABSOLUTE(.);
+   } > flash
+
+   .data :
+   {
+       _sdata = ABSOLUTE(.);
+       ...
+   #include <nuttx/linker/common-ram.ld>
+       . = ALIGN(4);
+       _edata = ABSOLUTE(.);
+   } > sram AT > flash
+
+* ``common-rom.ld`` collects the read-only (``const``) iterable sections
+  and must be included inside the read-only output section (typically
+  ``.text``, before ``_etext``).
+* ``common-ram.ld`` collects mutable *initialized* iterable sections and
+  must be included inside ``.data`` (between ``_sdata`` and ``_edata``)
+  so the startup FLASH-to-RAM copy initializes the entries.
+* Subsystems add their sections to these central files, guarded by their
+  Kconfig option (see ``include/nuttx/linker/common-rom.ld`` for the zbus
+  example).
+
+Supplementary INSERT script (zero-touch mode)
+---------------------------------------------
+
+Subsystems may alternatively provide a supplementary script using the GNU
+ld ``INSERT AFTER`` command so that **no board script modification is
+needed** (see ``include/nuttx/linker/zbus.ld`` and
+``CONFIG_ZBUS_LINKER_INSERT``).  The script is added through the
+``ARCHSCRIPT`` list in ``tools/Config.mk``.
+
+This mode has constraints, discovered the hard way and worth knowing
+before choosing it:
+
+* GNU ld only (``INSERT`` is not supported by the macOS ld64).
+* The INSERT script must come *before* the board script on the linker
+  command line.  Adding it via ``ARCHSCRIPT`` from ``tools/Config.mk``
+  guarantees that, because ``Config.mk`` is included by the board
+  ``Make.defs`` before it appends its own script.  (The reversed order
+  fails with ``.text not found for insert``.)
+* GNU ld assigns an INSERTed output section to a ``MEMORY`` region by
+  *attribute matching in declaration order*, not by inheriting the anchor
+  section's region.  The ROM/flash region must therefore be the first
+  region compatible with read-only sections.  Boards declaring a generic
+  ``rwx`` region at a lower address first (e.g. an ITCM at ``0x0``) are
+  incompatible with this mode and must use the board script include.
+* Giving the inserted section an explicit address is **not** a fix: a
+  section with an explicit address does not consume the memory region,
+  so the next region-allocated section overlaps it.
+
+Alignment rules
+===============
+
+Instances are aligned to the natural alignment of their type
+(``STRUCT_SECTION_ITERABLE`` adds ``__aligned__(__alignof__(type))``), and
+``sizeof`` is always a multiple of ``alignof``, so the collected section
+can be indexed as a plain array with no padding between entries from
+different compilation units.  The fragments additionally align the list
+boundaries to 4 bytes.
+
+Adding a new iterable type
+==========================
+
+1. Define the instances with ``STRUCT_SECTION_ITERABLE(mytype, name)``.
+2. Add ``ITERABLE_SECTION(mytype)`` to
+   ``include/nuttx/linker/common-rom.ld`` (const) or ``common-ram.ld``
+   (mutable initialized), guarded by the subsystem Kconfig option.
+3. Iterate with ``STRUCT_SECTION_FOREACH(mytype, it)``.
+
+Caveat on generated linker scripts: the preprocessed ``.ld.tmp`` files
+only depend on the board script and ``.config``; after editing the
+central fragments during development, remove the ``.tmp`` files (or run
+``make clean``) to force regeneration.

include/nuttx/iterable_sections.h

@@ -0,0 +1,117 @@
+/****************************************************************************
+ * include/nuttx/iterable_sections.h
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.  The
+ * ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
+ * License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+ ****************************************************************************/
+
+/* Iterable sections: link-time registration of struct instances.
+ *
+ * A struct instance defined with STRUCT_SECTION_ITERABLE() in any
+ * compilation unit is placed in a dedicated input section named
+ * "._<struct_type>.static.<varname>".  The board linker script collects
+ * these input sections (sorted by name) into a contiguous array delimited
+ * by the _<struct_type>_list_start/_<struct_type>_list_end symbols by
+ * including <nuttx/linker/common-rom.ld> (const data, inside the .text or
+ * .rodata output section) and <nuttx/linker/common-ram.ld> (mutable
+ * initialized data, inside the .data output section, so that the startup
+ * FLASH-to-RAM copy initializes it).
+ *
+ * The collection is only available on architectures whose linker scripts
+ * are preprocessed with CPP (arm, arm64, risc-v, xtensa, x86_64, tricore)
+ * and on boards whose scripts include the common-*.ld fragments.
+ */
+
+#ifndef __INCLUDE_NUTTX_ITERABLE_SECTIONS_H
+#define __INCLUDE_NUTTX_ITERABLE_SECTIONS_H
+
+/****************************************************************************
+ * Included Files
+ ****************************************************************************/
+
+#include <nuttx/config.h>
+
+/****************************************************************************
+ * Pre-processor Definitions
+ ****************************************************************************/
+
+/* Name of the input section for one instance.  The variable name is part
+ * of the section name so that the linker's SORT_BY_NAME() defines the
+ * iteration order (instances may encode ordering in their names).
+ */
+
+#define STRUCT_SECTION_NAME(struct_type, varname) \
+  "._" #struct_type ".static." #varname
+
+/* Define a struct instance inside an iterable section.  A "const"
+ * qualifier may be prepended at the point of use to place the instance in
+ * ROM.  Each instance is aligned to the natural alignment of its type so
+ * that the collected section can be indexed as a plain C array.
+ */
+
+#define STRUCT_SECTION_ITERABLE(struct_type, varname)                    \
+  struct struct_type varname                                             \
+  __attribute__((__used__,                                               \
+                 __aligned__(__alignof__(struct struct_type)),           \
+                 __section__(STRUCT_SECTION_NAME(struct_type, varname))))
+
+/* Start/end symbols provided by the linker script fragments */
+
+#define STRUCT_SECTION_START(struct_type) _##struct_type##_list_start
+#define STRUCT_SECTION_END(struct_type)   _##struct_type##_list_end
+
+#define STRUCT_SECTION_START_EXTERN(struct_type) \
+  extern struct struct_type STRUCT_SECTION_START(struct_type)[]
+#define STRUCT_SECTION_END_EXTERN(struct_type) \
+  extern struct struct_type STRUCT_SECTION_END(struct_type)[]
+
+/* Iterate over every instance of an iterable section.  "iterator" is the
+ * name of the loop pointer variable.
+ */
+
+#define STRUCT_SECTION_FOREACH(struct_type, iterator)                     \
+  STRUCT_SECTION_START_EXTERN(struct_type);                               \
+  STRUCT_SECTION_END_EXTERN(struct_type);                                 \
+  for (struct struct_type *iterator = STRUCT_SECTION_START(struct_type); \
+       iterator < STRUCT_SECTION_END(struct_type);                       \
+       iterator++)
+
+/* Get the i-th element of an iterable section (no bounds checking) */
+
+#define STRUCT_SECTION_GET(struct_type, i, dst)                \
+  do                                                           \
+    {                                                          \
+      STRUCT_SECTION_START_EXTERN(struct_type);                \
+      *(dst) = &STRUCT_SECTION_START(struct_type)[i];          \
+    }                                                          \
+  while (0)
+
+/* Number of elements in an iterable section */
+
+#define STRUCT_SECTION_COUNT(struct_type, dst)                            \
+  do                                                                      \
+    {                                                                     \
+      STRUCT_SECTION_START_EXTERN(struct_type);                           \
+      STRUCT_SECTION_END_EXTERN(struct_type);                             \
+      *(dst) = ((uintptr_t)STRUCT_SECTION_END(struct_type) -              \
+                (uintptr_t)STRUCT_SECTION_START(struct_type)) /           \
+               sizeof(struct struct_type);                                \
+    }                                                                     \
+  while (0)
+
+#endif /* __INCLUDE_NUTTX_ITERABLE_SECTIONS_H */

include/nuttx/linker/common-ram.ld

@@ -0,0 +1,41 @@
+/****************************************************************************
+ * include/nuttx/linker/common-ram.ld
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.  The
+ * ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
+ * License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+ ****************************************************************************/
+
+/* Mutable (initialized) iterable sections.  Boards opt in by adding,
+ * INSIDE their .data output section (between _sdata and _edata, so the
+ * startup FLASH-to-RAM copy initializes the entries):
+ *
+ *   #include <nuttx/linker/common-ram.ld>
+ *
+ * Every block below is guarded by its subsystem's Kconfig option, so this
+ * file expands to nothing on configurations that do not use iterable
+ * sections (zero binary impact).
+ */
+
+#include <nuttx/config.h>
+#include <nuttx/linker/iterable_sections.ld>
+
+/* zbus needs no RAM iterable sections: notification masks live in .bss
+ * and are initialized at runtime from ROM-preserved values.  This file is
+ * kept as the extension point for future subsystems that need initialized
+ * RAM iterable sections.
+ */

include/nuttx/linker/common-rom.ld

@@ -0,0 +1,45 @@
+/****************************************************************************
+ * include/nuttx/linker/common-rom.ld
+ *
+ * SPDX-License-Identifier: Apache-2.0
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.  The
+ * ASF licenses this file to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance with the
+ * License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.  See the
+ * License for the specific language governing permissions and limitations
+ * under the License.
+ *
+ ****************************************************************************/
+
+/* Read-only iterable sections.  Boards opt in by adding, INSIDE their
+ * read-only output section (typically .text, before _etext):
+ *
+ *   #include <nuttx/linker/common-rom.ld>
+ *
+ * Every block below is guarded by its subsystem's Kconfig option, so this
+ * file expands to nothing on configurations that do not use iterable
+ * sections (zero binary impact).
+ */
+
+#include <nuttx/config.h>
+#include <nuttx/linker/iterable_sections.ld>
+
+/* When CONFIG_ZBUS_LINKER_INSERT is selected the zbus sections come from
+ * the supplementary script <nuttx/linker/zbus.ld> instead (appended to
+ * the link via EXTRALINKCMDS), so they must not be emitted here too.
+ */
+
+#if defined(CONFIG_ZBUS) && !defined(CONFIG_ZBUS_LINKER_INSERT)
+ITERABLE_SECTION(zbus_channel)
+ITERABLE_SECTION(zbus_observer)
+ITERABLE_SECTION(zbus_channel_observation)
+#endif