Dev to main v1.6.0

.codacy.yaml

@@ -0,0 +1,29 @@
+---
+# Codacy configuration for wifi_db
+#
+# Exclusions below remove noise and false positives so the dashboard reflects
+# real issues only.
+exclude_paths:
+  # Local virtualenv committed in the repo: not project source code.
+  - 'wifi_db_env/**'
+  # Generated / cache directories.
+  - '__pycache__/**'
+  - '**/__pycache__/**'
+  - '.pytest_cache/**'
+  # Empty package marker (0 statements). Codacy renders its 0/0 coverage as a
+  # misleading 0%; there is nothing to analyse or cover, so exclude it.
+  - 'utils/__init__.py'
+  # Vendor data dump (OUI MAC vendor list), not hand-written code.
+  - 'utils/mac-vendors-export.csv'
+  # pip requirements file, not Python. Codacy routes it to Bandit, which
+  # AST-parses it and fails on the version specifiers ("syntax error while
+  # parsing AST from file", e.g. `cryptography>=48.0.1`). Dependency/CVE
+  # scanning is a separate Codacy feature and is unaffected by this exclusion.
+  - 'requirements.txt'
+  # Static, trusted SQLite schema/view definitions. These are plain SQLite DDL
+  # shipped with the project and executed via sqlite3.executescript(). Codacy's
+  # SQL analysis parses them with a SQL Server (T-SQL) dialect, which produces
+  # false positives ("syntax error near NOT/CONSTRAINT", "SET ANSI_NULLS",
+  # "queries must target RAC_* tables", etc.) that do not apply to SQLite.
+  - 'utils/view.sql'
+  - 'utils/wifi_db_database.sql'

.coveragerc

@@ -0,0 +1,32 @@
+# Coverage.py configuration. `pytest --cov` (pytest-cov) reads this and, with
+# --cov-report=xml, writes a Cobertura coverage.xml -- the format Codacy expects
+# for Python (https://docs.codacy.com/coverage-reporter/).
+[run]
+branch = True
+# Store repo-root-relative file paths (utils/foo.py, wifi_db.py) instead of
+# absolute ones, so Codacy can match the report to the repository files.
+relative_files = True
+# Measure from the repo root so paths stay repo-relative, then omit everything
+# that is not application code (tests, helper scripts, the local virtualenv).
+source = .
+omit =
+    tests/*
+    scripts/*
+    conftest.py
+    wifi_db_env/*
+    */__pycache__/*
+    # Empty package marker: 0 statements. coverage.py calls that vacuously
+    # 100%, but Codacy computes covered/total and renders 0/0 as a misleading
+    # 0%, so keep it out of the report entirely.
+    utils/__init__.py
+
+[report]
+show_missing = True
+# Non-executable lines that should never count against coverage.
+exclude_lines =
+    pragma: no cover
+    if __name__ == .__main__.:
+    raise NotImplementedError
+
+[xml]
+output = coverage.xml

.dockerignore

@@ -0,0 +1,44 @@
+# Keep the build context (and the COPY layer) small. Anything not needed to
+# run wifi_db inside the container is excluded here. Tests are not run during
+# the build; they run against the finished image (see test_docker.sh).
+
+# VCS / CI
+.git
+.github
+.gitignore
+
+# Local virtualenv and Python caches
+wifi_db_env
+__pycache__
+**/__pycache__
+*.pyc
+.pytest_cache
+
+# Coverage artifacts (regenerated inside the image when running the suite)
+.coverage
+.coverage.*
+coverage.xml
+.coverage_out.*
+htmlcov
+
+# Test fixtures: not baked into the image. The release pipeline / test_docker.sh
+# mounts this directory into the built container to run the suite against it.
+test_data
+
+# README image assets and the roadmap/changelog are not needed at runtime.
+# README.md itself is kept: the test suite hashes it as a sample input file.
+resources
+ROADMAP*.md
+CHANGELOG.md
+
+# Local databases / editor / tooling
+*.sqlite
+*.SQLITE
+*.sqlite-journal
+.vscode
+.codacy.yaml
+.flake8
+
+# Docker files themselves
+Dockerfile
+.dockerignore

.github/workflows/docker-image-dev.yml

@@ -10,45 +10,77 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v1
-
+        # v3 ships a recent QEMU; old QEMU emulation crashes arm64 apt-get
+        # (exit code 100). Required for the linux/arm64 build below.
+        uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3.7.0
+
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
+        uses: docker/setup-buildx-action@f7ce87c1d6bead3e36075b2ce75da1f6cc28aaca # v3.9.0
 
       - name: Checkout
-        uses: actions/checkout@v3   
+        uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3
+
+      # Build the amd64 image locally first and run the test suite against it.
+      # The image is test-free; test_docker.sh mounts the fixtures and runs
+      # pytest with the image's own Python/tshark/hcxtools. Nothing is pushed
+      # until this passes, so a failing build never reaches a registry.
+      - name: Build image for testing
+        uses: docker/build-push-action@ac9327eae2b366085ac7f6a2d02df8aa8ead720a # v2
+        with:
+          context: .
+          file: ./Dockerfile
+          platforms: linux/amd64
+          load: true
+          push: false
+          tags: wifi_db:test
+
+      - name: Test the built image
+        run: |
+          chmod +x ./test_docker.sh
+          ./test_docker.sh wifi_db:test
+
+      # test_docker.sh leaves the Cobertura report at ./coverage.xml. Upload it
+      # so the Codacy dashboard reflects the coverage of the code just built,
+      # instead of freezing on whatever was last sent by hand. This workflow
+      # only runs on push to dev, so the CODACY_PROJECT_TOKEN secret is always
+      # available here.
+      - name: Upload coverage to Codacy
+        uses: codacy/codacy-coverage-reporter-action@89d6c85cfafaec52c72b6c5e8b2878d33104c699 # v1.3.0
+        with:
+          project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
+          coverage-reports: coverage.xml
 
       # manager docker
       - name: Extract metadata (tags, labels) for Docker
         id: meta
-        uses: docker/metadata-action@v5
+        uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5
         with:
-          images: ${{ github.actor }}/wifi_db
+          images: ${{ github.repository }}
 
       - name: Login to DockerHub
-        uses: docker/login-action@v1 
+        uses: docker/login-action@dd4fa0671be5250ee6f50aedf4cb05514abda2c7 # v1 
         with:
           username: ${{ github.actor }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
       - name: Build and push docker
         id: docker_build
-        uses: docker/build-push-action@v2
+        uses: docker/build-push-action@ac9327eae2b366085ac7f6a2d02df8aa8ead720a # v2
         with:
           context: .
           file: ./Dockerfile
           platforms: linux/amd64,linux/arm64
           push: true
-          tags: ${{ secrets.DOCKERHUB_USERNAME }}/wifi_db:dev
+          tags: ${{ github.repository }}:dev
 
       - name: Login to GitHub Container Registry
         run: echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
 
       - name: Build and push docker
         id: docker_build_ghcr
-        uses: docker/build-push-action@v5
+        uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5
         with:
           context: .
           file: ./Dockerfile
           platforms: linux/amd64,linux/arm64
           push: true
-          tags: ghcr.io/${{ github.actor }}/wifi_db:dev
+          tags: ghcr.io/${{ github.repository }}:dev

.github/workflows/docker-image.yml

@@ -12,29 +12,61 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Set up QEMU
-        uses: docker/setup-qemu-action@v1
-
+        # v3 ships a recent QEMU; old QEMU emulation crashes arm64 apt-get
+        # (exit code 100). Required for the linux/arm64 build below.
+        uses: docker/setup-qemu-action@c7c53464625b32c7a7e944ae62b3e17d2b600130 # v3.7.0
+
       - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
+        uses: docker/setup-buildx-action@f7ce87c1d6bead3e36075b2ce75da1f6cc28aaca # v3.9.0
 
       - name: Checkout
-        uses: actions/checkout@v3   
+        uses: actions/checkout@f43a0e5ff2bd294095638e18286ca9a3d1956744 # v3
+
+      # Build the amd64 image locally first and run the test suite against it.
+      # The image is test-free; test_docker.sh mounts the fixtures and runs
+      # pytest with the image's own Python/tshark/hcxtools. Nothing is pushed
+      # until this passes, so a failing build never reaches a registry.
+      - name: Build image for testing
+        uses: docker/build-push-action@ac9327eae2b366085ac7f6a2d02df8aa8ead720a # v2
+        with:
+          context: .
+          file: ./Dockerfile
+          platforms: linux/amd64
+          load: true
+          push: false
+          tags: wifi_db:test
+
+      - name: Test the built image
+        run: |
+          chmod +x ./test_docker.sh
+          ./test_docker.sh wifi_db:test
+
+      # test_docker.sh leaves the Cobertura report at ./coverage.xml. Upload it
+      # so the Codacy dashboard reflects the coverage of the code just built,
+      # instead of freezing on whatever was last sent by hand. This workflow
+      # only runs on push to master / tags, so the CODACY_PROJECT_TOKEN secret
+      # is always available here.
+      - name: Upload coverage to Codacy
+        uses: codacy/codacy-coverage-reporter-action@89d6c85cfafaec52c72b6c5e8b2878d33104c699 # v1.3.0
+        with:
+          project-token: ${{ secrets.CODACY_PROJECT_TOKEN }}
+          coverage-reports: coverage.xml
 
       # manager docker
       - name: Extract metadata (tags, labels) for Docker
         id: meta
-        uses: docker/metadata-action@v5
+        uses: docker/metadata-action@c299e40c65443455700f0fdfc63efafe5b349051 # v5
         with:
-          images: ${{ github.actor }}/wifi_db
+          images: ${{ github.repository }}
 
       - name: Login to DockerHub
-        uses: docker/login-action@v1 
+        uses: docker/login-action@dd4fa0671be5250ee6f50aedf4cb05514abda2c7 # v1 
         with:
           username: ${{ github.actor }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
       - name: Build and push docker
         id: docker_build
-        uses: docker/build-push-action@v2
+        uses: docker/build-push-action@ac9327eae2b366085ac7f6a2d02df8aa8ead720a # v2
         with:
           context: .
           file: ./Dockerfile
@@ -48,11 +80,11 @@ jobs:
 
       - name: Build and push github
         id: docker_build_ghcr
-        uses: docker/build-push-action@v5
+        uses: docker/build-push-action@ca052bb54ab0790a636c9b5f226502c73d547a25 # v5
         with:
           context: .
           file: ./Dockerfile
           platforms: linux/amd64,linux/arm64
           push: true
-          tags: ghcr.io/${{ github.actor }}/wifi_db:latest, ghcr.io/${{ steps.meta.outputs.tags }}
+          tags: ghcr.io/${{ github.repository }}:latest, ghcr.io/${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}

.github/workflows/sync_readme.yml

@@ -12,10 +12,10 @@ jobs:
 
     steps:
       - name: Checkout
-        uses: actions/checkout@v2
+        uses: actions/checkout@ee0669bd1cc54295c223e0bb666b733df41de1c5 # v2
 
       - name: Docker Hub Description
-        uses: peter-evans/dockerhub-description@v2
+        uses: peter-evans/dockerhub-description@616d1b63e806b630b975af3b4fe3304307b20f40 # v2
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_PASSWORD }}

.gitignore

@@ -8,3 +8,11 @@ __pycache__/
 .flake8
 .vscode/
 *.sqlite-journal
+
+# Coverage artifacts (generated by pytest --cov / Coverage.py)
+.coverage
+.coverage.*
+coverage.xml
+.coverage_out.*
+htmlcov/
+.pytest_cache/

CHANGELOG.md

@@ -0,0 +1,21 @@
+# Changelog
+
+## v1.6.0
+
+### Added
+- Per-AP RSN/WPA security breakdown (WPA version, AKM suites, pairwise/group ciphers, enterprise flag, PMF) in a new `SecurityAP` view, with human-readable decodes of the `wps_config_methods` and `rsn_capabilities` bitmasks (e.g. `0x00c0` → `MFPR, MFPC`) stored beside the raw values; existing databases migrate automatically.
+- X.509 certificate extraction from enterprise (802.1X) EAP into the new `Certificate` table and `CertificateAP` view (based on the @x4v1l0k idea in PR #57).
+- EAP enrichment on identities: realm and method-type lookup, plus EAP-MD5 challenge/response capture for offline cracking (`EAPMD5` table, `hashcat -m 4800`).
+- Probe-request fingerprinting (`fingerprint`, `ie_order`) with randomized-MAC detection, and hidden (cloaked) SSID recovery from probe responses and (re)association requests.
+- AP management-capability detection from beacons/probe responses (802.11r/k/v fast roaming, Multiple BSSID, Channel Switch Announcement) in a new `CapabilitiesAP` view.
+
+### Fixed
+- Parsing/detection corrections: inflated WPS error count and SSID-dependent WPS 2.0 detection; MFP/PMF now read from RSN capability bits (capable vs required); `cloaked` and `firstTimeSeen` no longer clobbered when a later frame enriches an AP; EAP Success/Failure frames no longer counted as `Identity` errors.
+- asyncio child-watcher crash on Python 3.14.
+- Container fixes: in-container self-update (`git` now shipped) and non-root/podman writes to the bind-mounted `db.SQLITE`; plus Docker build, dependency CVEs and Codacy findings (including a SQL injection).
+
+### Updated
+- Merged the 1:1 Security, WPS and probe-fingerprint attributes onto the `AP`/`Probe` rows, and improved the `SummaryAP` view (grouped by SSID **and** encryption, showing WPA version, PMF, every manufacturer and client counts).
+- Slimmed the Docker image ~360 MB → ~220 MB (Alpine base, ship only `hcxpcapngtool`, drop build caches); amd64/arm64 builds, the full test suite and the built image now verified in CI.
+- Refactored internals with no behaviour change: split the oversized `utils/wifi_db_aircrack.py` into modules behind a re-export facade and cut parser/DB cyclomatic complexity below the Codacy limit (shared `safe_insert`/`cap_runner`, lookup tables); public `parse_*` API and callers unchanged.
+- Refreshed the bundled IEEE OUI / mac-vendors database (~16k new vendor prefixes).