diff --git a/doc/man/zkg.1 b/doc/man/zkg.1 index ac05e74d..bb0ec887 100644 --- a/doc/man/zkg.1 +++ b/doc/man/zkg.1 @@ -295,7 +295,7 @@ usage: zkg refresh [\-h] [\-\-aggregate] [\-\-fail\-on\-aggregate\-problems] [\- .INDENT 7.0 .TP .B \-\-aggregate=False -Crawls the urls listed in package source zkg.index files and aggregates the metadata found in their zkg.meta (or legacy bro\-pkg.meta) files. The aggregated metadata is stored in the local clone of the package source that zkg uses internally for locating package metadata. For each package, the metadata is taken from the highest available git version tag or the default branch, like \(dqmain\(dq or \(dqmaster\(dq, if no version tags exist +Crawls the urls listed in package source zkg.index files and aggregates the metadata found in their zkg.meta files. The aggregated metadata is stored in the local clone of the package source that zkg uses internally for locating package metadata. For each package, the metadata is taken from the highest available git version tag or the default branch, like \(dqmain\(dq or \(dqmaster\(dq, if no version tags exist .TP .B \-\-fail\-on\-aggregate\-problems=False When using \-\-aggregate, exit with error when any packages trigger metadata problems. Normally such problems only cause a warning. @@ -721,7 +721,7 @@ state_dir = # A subdirectory named \(dqpackages\(dq is always created within the # specified path and the package manager will copy the directory # specified by the \(dqscript_dir\(dq option of each package\(aqs zkg.meta -# (or legacy bro\-pkg.meta) file there. +# file there. # If left blank or with \-\-user this defaults to /script_dir. # In Zeek\-bundled installations, it defaults to # /share/zeek/site. @@ -734,7 +734,7 @@ script_dir = # A subdirectory named \(dqpackages\(dq is always created within the # specified path and the package manager will copy the directory # specified by the \(dqplugin_dir\(dq option of each package\(aqs zkg.meta -# (or legacy bro\-pkg.meta) file there. +# file there. # If left blank or with \-\-user this defaults to /plugin_dir. # In Zeek\-bundled installations, it defaults to # /lib/zeek/plugins. @@ -766,7 +766,7 @@ default = https://github.com/zeek/package\-template [user_vars] # For any key in this section that is matched for value interpolation -# in a package\(aqs zkg.meta (or legacy bro\-pkg.meta) file, the corresponding +# in a package\(aqs zkg.meta file, the corresponding # value is substituted during execution of the package\(aqs \(gabuild_command\(ga. # This section is typically automatically populated with the # the answers supplied during package installation prompts diff --git a/doc/source.rst b/doc/source.rst index c19c374e..30099d84 100644 --- a/doc/source.rst +++ b/doc/source.rst @@ -22,12 +22,6 @@ usernames or some unique way of identifying the organization/person that maintains Zeek packages. However, a source is free to use a flat organization with a single, top-level :file:`zkg.index`. -.. note:: - - The magic index file name of :file:`zkg.index` is available :program:`since - zkg v2.0`. For compatibility purposes, the old index file name of - :file:`bro-pkg.index` is also still supported. - After creating a git repo for the package source and adding package index files to it, it's ready to be used by :ref:`zkg `. @@ -36,7 +30,7 @@ to it, it's ready to be used by :ref:`zkg `. Package Index Files ------------------- -Files named :file:`zkg.index` (or the legacy :file:`bro-pkg.index`) are used to +Files named :file:`zkg.index` are used to describe the :doc:`Zeek Packages ` found within the package source. They are simply a list of git URLs pointing to the git repositories of packages. For example:: @@ -75,9 +69,9 @@ Aggregating Metadata -------------------- The maintainer/operator of a package source may choose to periodically aggregate -the metadata contained in its packages' :file:`zkg.meta` (and legacy -:file:`bro-pkg.meta`) files. The :ref:`zkg refresh ` -is used to perform the task. For example: +the metadata contained in its packages' :file:`zkg.meta` files. +The :ref:`zkg refresh ` is used to perform the task. For +example: .. code-block:: console diff --git a/testing/baselines/tests.install-bro-pkg-warning/stderr b/testing/baselines/tests.install-bro-pkg-warning/stderr deleted file mode 100644 index 0676ed27..00000000 --- a/testing/baselines/tests.install-bro-pkg-warning/stderr +++ /dev/null @@ -1,2 +0,0 @@ -### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. -XXXX-XX-XX XX:XX:XX WARNING Package one/alice/foo is using the legacy bro-pkg.meta metadata file. While bro-pkg.meta still functions, it is recommended to use zkg.meta instead for future-proofing. Please report this to the package maintainers. diff --git a/testing/baselines/tests.refresh/agg.errout b/testing/baselines/tests.refresh/agg.errout index 0bcb3dec..77a5925b 100644 --- a/testing/baselines/tests.refresh/agg.errout +++ b/testing/baselines/tests.refresh/agg.errout @@ -1,8 +1,8 @@ ### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. XXXX-XX-XX XX:XX:XX DEBUG found source clone of "one" at <...>/one XXXX-XX-XX XX:XX:XX DEBUG refresh "one": pulling <...>/one -XXXX-XX-XX XX:XX:XX WARNING <...>/bro-pkg.meta: missing metadata file -XXXX-XX-XX XX:XX:XX WARNING skipping aggregation of <...>/bad_pkg: bad metadata: missing zkg.meta (or bro-pkg.meta) metadata file +XXXX-XX-XX XX:XX:XX WARNING <...>/zkg.meta: missing metadata file +XXXX-XX-XX XX:XX:XX WARNING skipping aggregation of <...>/bad_pkg: bad metadata: missing zkg.meta metadata file XXXX-XX-XX XX:XX:XX DEBUG metadata refresh: 8 additions (alice/bar, alice/baz, alice/foo, alice/i-have-no-scripts, alice/new_pkg, alice/qux, bob/corge, bob/grault), 0 changes, 0 removals XXXX-XX-XX XX:XX:XX INFO committed package source "one" metadata update XXXX-XX-XX XX:XX:XX DEBUG fetch package one<...>/foo diff --git a/testing/baselines/tests.refresh/agg.fail.errout b/testing/baselines/tests.refresh/agg.fail.errout index cbe7ff0f..43e80165 100644 --- a/testing/baselines/tests.refresh/agg.fail.errout +++ b/testing/baselines/tests.refresh/agg.fail.errout @@ -1,7 +1,7 @@ ### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. XXXX-XX-XX XX:XX:XX DEBUG found source clone of "one" at <...>/one XXXX-XX-XX XX:XX:XX DEBUG refresh "one": pulling <...>/one -XXXX-XX-XX XX:XX:XX WARNING <...>/bro-pkg.meta: missing metadata file -XXXX-XX-XX XX:XX:XX WARNING skipping aggregation of <...>/bad_pkg: bad metadata: missing zkg.meta (or bro-pkg.meta) metadata file +XXXX-XX-XX XX:XX:XX WARNING <...>/zkg.meta: missing metadata file +XXXX-XX-XX XX:XX:XX WARNING skipping aggregation of <...>/bad_pkg: bad metadata: missing zkg.meta metadata file XXXX-XX-XX XX:XX:XX DEBUG metadata refresh: 8 additions (alice/bar, alice/baz, alice/foo, alice/i-have-no-scripts, alice/new_pkg, alice/qux, bob/corge, bob/grault), 0 changes, 0 removals XXXX-XX-XX XX:XX:XX DEBUG fetch package one<...>/foo diff --git a/testing/baselines/tests.refresh/agg.fail.out b/testing/baselines/tests.refresh/agg.fail.out index e7f23ede..ea6e24ec 100644 --- a/testing/baselines/tests.refresh/agg.fail.out +++ b/testing/baselines/tests.refresh/agg.fail.out @@ -2,6 +2,6 @@ Refresh package source: one No membership changes WARNING: Metadata aggregated, but excludes the following packages due to described problems: - <...>/bad_pkg: missing zkg.meta (or bro-pkg.meta) metadata file + <...>/bad_pkg: missing zkg.meta metadata file Refresh installed packages No new outdated packages diff --git a/testing/baselines/tests.refresh/agg.out b/testing/baselines/tests.refresh/agg.out index 1d399d87..81f81633 100644 --- a/testing/baselines/tests.refresh/agg.out +++ b/testing/baselines/tests.refresh/agg.out @@ -2,7 +2,7 @@ Refresh package source: one No membership changes WARNING: Metadata aggregated, but excludes the following packages due to described problems: - <...>/bad_pkg: missing zkg.meta (or bro-pkg.meta) metadata file + <...>/bad_pkg: missing zkg.meta metadata file Pushed aggregated metadata Refresh installed packages No new outdated packages diff --git a/testing/tests/install-bro-pkg-warning b/testing/tests/install-bro-pkg-warning deleted file mode 100644 index bd94ede1..00000000 --- a/testing/tests/install-bro-pkg-warning +++ /dev/null @@ -1,3 +0,0 @@ -# @TEST-EXEC: (cd packages/foo; git mv zkg.meta bro-pkg.meta; git commit -m 'Use bro-pkg.meta') -# @TEST-EXEC: zkg install foo 2> stderr -# @TEST-EXEC: TEST_DIFF_CANONIFIER='sed -r "s/[0-9]{2}/XX/g"' btest-diff stderr diff --git a/zeekpkg/manager.py b/zeekpkg/manager.py index 1601a133..4b177b65 100644 --- a/zeekpkg/manager.py +++ b/zeekpkg/manager.py @@ -48,9 +48,6 @@ from .package import ( BUILTIN_SCHEME, BUILTIN_SOURCE, - LEGACY_METADATA_FILENAME, - LEGACY_PLUGIN_MAGIC_FILE, - LEGACY_PLUGIN_MAGIC_FILE_DISABLED, METADATA_FILENAME, PLUGIN_MAGIC_FILE, PLUGIN_MAGIC_FILE_DISABLED, @@ -195,18 +192,17 @@ class Manager: script_dir (str): the directory where the package manager will copy each installed package's `script_dir` (as given by its - :file:`zkg.meta` or :file:`bro-pkg.meta`). Each package gets a - subdirectory within `script_dir` associated with its name. + :file:`zkg.meta`). Each package gets a subdirectory within + `script_dir` associated with its name. plugin_dir (str): the directory where the package manager will copy each installed package's `plugin_dir` (as given by its - :file:`zkg.meta` or :file:`bro-pkg.meta`). Each package gets a - subdirectory within `plugin_dir` associated with its name. + :file:`zkg.meta`). Each package gets a subdirectory within + `plugin_dir` associated with its name. bin_dir (str): the directory where the package manager will link executables into that are provided by an installed package through - `executables` (as given by its :file:`zkg.meta` or - :file:`bro-pkg.meta`) + `executables` (as given by its :file:`zkg.meta`) source_clonedir (str): the directory where the package manager will clone package sources. Each source gets a subdirectory @@ -409,22 +405,12 @@ def _write_plugin_magic(self, ipkg: InstalledPackage) -> None: When the package doesn't include a plugin, or when the plugin directory already contains a correctly named magic file, this function does nothing. - - Until Zeek 6.1, the magic file was named __bro_plugin__. zkg implements - a fallback for recognizing the older name so that newer zkg versions - continue to work with older Zeek versions for some time longer. """ package_dir = pathlib.Path(self.plugin_dir) / ipkg.package.name - magic_paths_enabled = [ - package_dir / PLUGIN_MAGIC_FILE, - package_dir / LEGACY_PLUGIN_MAGIC_FILE, - ] + magic_paths_enabled = [package_dir / PLUGIN_MAGIC_FILE] - magic_paths_disabled = [ - package_dir / PLUGIN_MAGIC_FILE_DISABLED, - package_dir / LEGACY_PLUGIN_MAGIC_FILE_DISABLED, - ] + magic_paths_disabled = [package_dir / PLUGIN_MAGIC_FILE_DISABLED] for path_enabled, path_disabled in zip( magic_paths_enabled, @@ -3396,7 +3382,7 @@ def ignore(_: str, files: list[str]) -> list[str]: rval = [] for f in files: - if f in {".git", "bro-pkg.meta", "zkg.meta"}: + if f in {".git", "zkg.meta"}: rval.append(f) return rval @@ -3451,12 +3437,7 @@ def _get_package_metadata(parser: configparser.ConfigParser) -> dict[str, str]: def _pick_metadata_file(directory: str) -> str: - rval = os.path.join(directory, METADATA_FILENAME) - - if os.path.exists(rval): - return rval - - return os.path.join(directory, LEGACY_METADATA_FILENAME) + return os.path.join(directory, METADATA_FILENAME) def _parse_package_metadata( @@ -3466,9 +3447,7 @@ def _parse_package_metadata( """Return string explaining why metadata is invalid, or '' if valid.""" if not parser.read(metadata_file): LOG.warning("%s: missing metadata file", metadata_file) - return ( - f"missing {METADATA_FILENAME} (or {LEGACY_METADATA_FILENAME}) metadata file" - ) + return f"missing {METADATA_FILENAME} metadata file" if not parser.has_section("package"): LOG.warning("%s: metadata missing [package]", metadata_file) @@ -3521,19 +3500,6 @@ def _info_from_clone( default_branch=default_branch, ) - if ( - os.path.basename(metadata_file) == LEGACY_METADATA_FILENAME - and package.qualified_name() not in _legacy_metadata_warnings - ): - LOG.warning( - "Package %s is using the legacy bro-pkg.meta metadata file. " - "While bro-pkg.meta still functions, it is recommended to " - "use zkg.meta instead for future-proofing. Please report this " - "to the package maintainers.", - package.qualified_name(), - ) - _legacy_metadata_warnings.add(package.qualified_name()) - metadata = _get_package_metadata(metadata_parser) return PackageInfo( diff --git a/zeekpkg/package.py b/zeekpkg/package.py index ab2c570d..7fb6f129 100644 --- a/zeekpkg/package.py +++ b/zeekpkg/package.py @@ -14,7 +14,6 @@ #: The name of files used by packages to store their metadata. METADATA_FILENAME = "zkg.meta" -LEGACY_METADATA_FILENAME = "bro-pkg.meta" TRACKING_METHOD_VERSION = "version" TRACKING_METHOD_BRANCH = "branch" @@ -26,8 +25,6 @@ PLUGIN_MAGIC_FILE = "__zeek_plugin__" PLUGIN_MAGIC_FILE_DISABLED = "__zeek_plugin__.disabled" -LEGACY_PLUGIN_MAGIC_FILE = "__bro_plugin__" -LEGACY_PLUGIN_MAGIC_FILE_DISABLED = "__bro_plugin__.disabled" def name_from_path(path: str) -> str: @@ -309,7 +306,7 @@ class PackageInfo: packages metadata (dict of str -> str): the contents of the package's - :file:`zkg.meta` or :file:`bro-pkg.meta` + :file:`zkg.meta` versions (list of str): a list of the package's availabe git version tags @@ -323,10 +320,9 @@ class PackageInfo: invalid_reason (str): this attribute is set when there is a problem with gathering package information and explains what went wrong. - metadata_file: the absolute path to the :file:`zkg.meta` or - :file:`bro-pkg.meta` for this package. Use this if you'd like to - parse the metadata yourself. May not be defined, in which case the - value is None. + metadata_file: the absolute path to the :file:`zkg.meta` for this package. + Use this if you'd like to parse the metadata yourself. May not be + defined, in which case the value is None. """ def __init__( @@ -443,10 +439,9 @@ class Package: or if it's located in a top-level :file:`zkg.index` file. metadata (dict of str -> str): the contents of the package's - :file:`zkg.meta` or :file:`bro-pkg.meta` file. If the package has - not been installed then this information may come from the last - aggregation of the source's :file:`aggregate.meta` file (it may not - be accurate/up-to-date). + :file:`zkg.meta`. If the package has not been installed then this + information may come from the last aggregation of the source's + :file:`aggregate.meta` file (it may not be accurate/up-to-date). """ def __init__( diff --git a/zeekpkg/source.py b/zeekpkg/source.py index 7da02a07..aaf53f64 100644 --- a/zeekpkg/source.py +++ b/zeekpkg/source.py @@ -1,9 +1,9 @@ """ A module containing the definition of a "package source": a git repository -containing a collection of :file:`zkg.index` (or legacy :file:`bro-pkg.index`) -files. These are simple INI files that can describe many Zeek packages. Each -section of the file names a Zeek package along with the git URL where it is -located and metadata tags that help classify/describe it. +containing a collection of :file:`zkg.index` files. These are simple INI files +that can describe many Zeek packages. Each section of the file names a Zeek +package along with the git URL where it is located and metadata tags that help +classify/describe it. """ import configparser @@ -18,7 +18,6 @@ #: The name of package index files. INDEX_FILENAME = "zkg.index" -LEGACY_INDEX_FILENAME = "bro-pkg.index" #: The name of the package source file where package metadata gets aggregated. AGGREGATE_DATA_FILE = "aggregate.meta" @@ -112,8 +111,8 @@ def package_index_files(self) -> list[str]: pass for filename in files: - if filename in {INDEX_FILENAME, LEGACY_INDEX_FILENAME}: - rval.append(os.path.join(root, filename)) + if filename == INDEX_FILENAME: + rval.append(os.path.join(root, INDEX_FILENAME)) return sorted(rval) diff --git a/zkg b/zkg index e10879af..44998383 100755 --- a/zkg +++ b/zkg @@ -2841,8 +2841,8 @@ def argparser() -> argparse.ArgumentParser: "--aggregate", action="store_true", help="Crawls the urls listed in package source zkg.index files and" - " aggregates the metadata found in their zkg.meta (or legacy" - " bro-pkg.meta) files. The aggregated metadata is stored in the local" + " aggregates the metadata found in their zkg.meta" + " files. The aggregated metadata is stored in the local" " clone of the package source that zkg uses internally for locating" " package metadata." " For each package, the metadata is taken from the highest available" diff --git a/zkg.config b/zkg.config index e7e78bcc..2506a9b3 100644 --- a/zkg.config +++ b/zkg.config @@ -34,7 +34,7 @@ state_dir = # A subdirectory named "packages" is always created within the # specified path and the package manager will copy the directory # specified by the "script_dir" option of each package's zkg.meta -# (or legacy bro-pkg.meta) file there. +# file there. # If left blank or with --user this defaults to /script_dir. # In Zeek-bundled installations, it defaults to # /share/zeek/site. @@ -47,7 +47,7 @@ script_dir = # A subdirectory named "packages" is always created within the # specified path and the package manager will copy the directory # specified by the "plugin_dir" option of each package's zkg.meta -# (or legacy bro-pkg.meta) file there. +# file there. # If left blank or with --user this defaults to /plugin_dir. # In Zeek-bundled installations, it defaults to # /lib/zeek/plugins. @@ -79,7 +79,7 @@ default = https://github.com/zeek/package-template [user_vars] # For any key in this section that is matched for value interpolation -# in a package's zkg.meta (or legacy bro-pkg.meta) file, the corresponding +# in a package's zkg.meta file, the corresponding # value is substituted during execution of the package's `build_command`. # This section is typically automatically populated with the # the answers supplied during package installation prompts