Skip to content

irrdb:update-prefix-db fails when AS-SET expands to large prefix list #1074

Description

@herver
ISSUE SUMMARY

Scheduled task irrdb:update-prefix-db memory usage exceeds PHP limits and gets killed resulting in partial prefix list updates (some customers have their prefix list always updated, others, after the "problematic" member, never get updated).

This affects members whose AS-SET expands to large prefix list.

OS

PRETTY_NAME="Debian GNU/Linux 13 (trixie)"
NAME="Debian GNU/Linux"
VERSION_ID="13"
VERSION="13 (trixie)"
VERSION_CODENAME=trixie
DEBIAN_VERSION_FULL=13.6
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

VERSION
APPLICATION_VERSION: 7.4.0
APPLICATION_VERDATE: 2026081700
MySQL server version: 8.4.8
Schema version: 2026_08_12_113213_remove_rs_prefixes_table

ENVIRONMENT
PHP 8.4.24 (cli) (built: Aug  5 2026 00:29:14) (NTS)
Copyright (c) The PHP Group
Built by https://github.com/docker-library/php
Zend Engine v4.4.24, Copyright (c) Zend Technologies
    with Zend OPcache v8.4.24, Copyright (c), by Zend Technologies

PHP extensions:
 - Core
 - date
 - libxml
 - openssl
 - pcre
 - sqlite3
 - zlib
 - ctype
 - curl
 - dom
 - fileinfo
 - filter
 - hash
 - iconv
 - json
 - mbstring
 - SPL
 - session
 - PDO
 - pdo_sqlite
 - standard
 - posix
 - random
 - readline
 - Reflection
 - Phar
 - SimpleXML
 - tokenizer
 - xml
 - xmlreader
 - xmlwriter
 - mysqlnd
 - bcmath
 - ds
 - exif
 - gd
 - intl
 - memcached
 - pcntl
 - pdo_mysql
 - rrd
 - snmp
 - sodium
 - yaml
 - zip
 - Zend OPcache

CONFIGURATION
APP_DEBUG=true
APP_URL=https:/xxxxxx.cern.ch
SESSION_SECURE_COOKIE=true
APP_TIMEZONE=UTC
LOG_LEVEL=debug
APP_NAME="IXP Manager"
IXP_RESELLER_ENABLED=true
IXP_AS112_UI_ACTIVE=false
IXP_FE_FRONTEND_DISABLED_LOGO=true
IXP_FE_FRONTEND_DISABLED_PEERING_MATRIX=true
GRAPHER_BACKENDS="mrtg"
GRAPHER_CACHE_ENABLED=true
GRAPHER_BACKEND_MRTG_DBTYPE=rrd
GRAPHER_BACKEND_MRTG_WORKDIR=/srv/mrtg
GRAPHER_BACKEND_MRTG_LOGDIR=/srv/mrtg
GRAPHER_ACCESS_IXP=3
GRAPHER_ACCESS_INFRASTRUCTURE=3
GRAPHER_ACCESS_VLAN=3
GRAPHER_ACCESS_SWITCH=3
GRAPHER_ACCESS_LOCATION=3
GRAPHER_ACCESS_TRUNK=3
VIEW_SKIN=cern
CACHE_DRIVER=memcached
IXP_IRRDB_BGPQ3_PATH=/usr/bin/bgpq4
[...]
IXP_FE_FRONTEND_DISABLED_LOOKING_GLASS=0
STEPS TO REPRODUCE
  • Add an AS-SET with a lot of prefix and tick the "RS Peer" box for a member.
EXPECTED RESULTS
  • Prefix list updated completely for all members
ACTUAL RESULTS
  • scheduled task gets killed because of memory_limit

Note: We operate with PHP's default memory_limit (128 MB), we could, of course increase it, but our testing shows that it would use more than 1 GB of memory.

IMPORTANCE

It is not impacting our production instance because it doesn't exist (yet).

We have patched our local instance of IXP Manager to not rely on json_decode to keep memory usage within boundaries and will propose the patch in a PR shortly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions