Skip to content

Apache 2.4 adaptor: release the shared-memory mapping on restart (fixes ENOMEM after ~1 year of graceful restarts) - #1041

Open
mkondratov wants to merge 1 commit into
wocommunity:masterfrom
mkondratov:apache-adaptor-unmap-on-restart
Open

mkondratov wants to merge 1 commit into
wocommunity:masterfrom
mkondratov:apache-adaptor-unmap-on-restart

Conversation

@mkondratov

Copy link
Copy Markdown
Contributor

Problem

WOShmem_init() (Adaptor/shmem.c) creates the WOAdaptorState file, unlinks it and mmap()s it. Nothing in the adaptor ever unmaps it.

Apache unloads and reloads every DSO module on a graceful restart (apachectl graceful) or restart (SIGHUP). That resets the module's statics (initCalled), so WebObjects_post_config() runs init_adaptor() again in the same parent process, and each restart leaves one more mapped, nameless file behind.

On a 32-bit server this is a time bomb. Ours (AIX 7.1, httpd 2.4.62 built with -bmaxdata:0x80000000, one logrotate graceful restart a night) fails about once a year:

WOShmem_init(): couldn't map file: Not enough space
init_adaptor(): WOShmem_init() failed
WebObjects_post_config(): Adaptor initialization failed. All requests will be declined.
WebObjects_translate(): declining request due to initialization failure   (x 17,872)

Every WebObjects request answers 500 until Apache is fully stopped and started; apachectl restart keeps the parent process and does not help. svmon -P <apache parent pid> | grep -c "clnt /dev/hd3" shows the leaked mappings growing with every restart (78 after 57 graceful restarts on our box).

Fix

  • shmem.c / shmem.h: WOShmem_cleanup() — munmap() + close() + reset the statics. Safe to call when nothing is mapped; children that still hold the old mapping are unaffected.
  • Apache2.4/mod_WebObjects.c: after a successful init_adaptor(), register a cleanup on the configuration pool that calls WOShmem_cleanup(). Apache clears that pool before every restart, before the DSO is unloaded (cleanups run LIFO, so ours runs while the module code is still mapped). The configuration-check pass (ap_state_query(AP_SQ_MAIN_STATE) == AP_SQ_MS_CREATE_PRE_CONFIG, which also covers apachectl configtest) now skips initialisation, so it maps nothing; a failed init releases what it mapped.

Verification

Sandbox httpd (prefork, 127.0.0.1:8099) on the same AIX box, 10 × kill -USR1, counting mapped /tmp files in the parent with svmon:

module before after 10 graceful restarts requests still reach the application
stock (4.6.3 build and master code) 3 13 yes
this patch 2 2 yes

Built with gcc 10 against httpd-devel 2.4.62 / apr 1.7.4 headers. The Apache2 and Apache2.2 modules have the same structure and would need the same cleanup registration (ap_state_query is 2.4-only; the older ones would use the apr_pool_userdata idiom); I have only tested 2.4.

🤖 Generated with Claude Code

https://claude.ai/code/session_011Dioh1WMgfT1MjppjPeeXR

WOShmem_init() creates the WOAdaptorState file, unlinks it and mmap()s it, and nothing
ever unmaps it. Apache unloads and reloads the module on every graceful restart or
restart, so the module's statics are reset and init_adaptor() runs again in the same
parent process: each restart leaves one more mapped, nameless file behind. On a 32-bit
server (AIX 7.1, httpd 2.4.62 built with -bmaxdata:0x80000000) mmap() fails with ENOMEM
after a few hundred restarts - "WOShmem_init(): couldn't map file: Not enough space",
then "declining request due to initialization failure" for every request - and only a
full stop + start of Apache clears it. With one logrotate restart a night that is about
once a year.

- shmem.c/.h: WOShmem_cleanup() unmaps and closes.
- Apache2.4/mod_WebObjects.c: a cleanup registered on the configuration pool calls it;
  Apache clears that pool before every restart, before the DSO is unloaded. The
  configuration-check pass (AP_SQ_MS_CREATE_PRE_CONFIG, also "apachectl configtest") no
  longer maps anything, and a failed init releases what it mapped.

Measured in a sandbox httpd on the same AIX box, 10 graceful restarts (svmon -P <parent>,
mapped /tmp files): stock module 3 -> 13, patched module 2 -> 2; both still proxy
requests to the application.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011Dioh1WMgfT1MjppjPeeXR
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant