Skip to content

Translated agent into python - #19

Open
irapatil07 wants to merge 74 commits into
swar09:linux-agent-devfrom
irapatil07:agent-implementation
Open

Translated agent into python#19
irapatil07 wants to merge 74 commits into
swar09:linux-agent-devfrom
irapatil07:agent-implementation

Conversation

@irapatil07

Copy link
Copy Markdown

Summary

Type

  • feat — new functionality
  • fix — bug fix
  • chore — dependency update, refactor, tooling
  • docs — documentation only
  • sec — security fix or hardening

Target Module / Crate

SDK

  • sdk — shared types, proto definitions

Agent Workspace

  • agent/agent-core — binary entry point, orchestrator
  • agent/ebpf-collector — eBPF programs and loader
  • agent/osquery-client — OSQuery socket IPC client
  • agent/event-buffer — local disk buffer (sled)
  • agent/fleet-client — gRPC client to Fleet Server
  • agent/isolation — IPTables-based network isolation

Backend Services

  • fleet-server — gRPC fleet server (enrollment, streaming, C2)
  • kafka-pipeline — event processor + normaliser + DB writer
  • rule-engine — YARA scanning, MITRE mapping, alert generation
  • api-backend — REST API + WebSocket for frontend

Frontend

  • frontend — React/Vite/TypeScript dashboard

Infrastructure

  • infra — Docker Compose, K8s manifests, Terraform, scripts

Checklist

  • Linked issue: closes #
  • No secrets or credentials in code
  • Tests added or updated
  • docker-compose up tested locally
  • Breaking changes documented in PR description

How to verify

swar09 and others added 30 commits May 28, 2026 18:29
…y and maintenance workflows, and implement scheduled query loading
	modified:   agent/crates/agent-core/src/orchestrator.rs
	modified:   agent/crates/fleet-client/src/lib.rs
- Created `fleet-manager` crate with a basic fleet management function.
- Added `fleet-tracing` crate for initializing tracing and logging.
- Introduced `grpc-listener` crate with build script for gRPC service.
- Implemented health tracking in `health-tracker` crate.
- Added Kafka publishing functionality in `kafka-handler` crate.
- Created `node-enrollment` crate for node enrollment functionality.
- Established Postgres interface in `postgres-interface` crate.
- Updated `fleet-server-bin` to initialize and run the fleet server.
…persistence

    This commit implements the foundational architecture for the `fleet-server` backend,
    including domain logic, robust database persistence, and local infrastructure setup.

    Major Additions:
    - `postgres-interface`: Implemented production-ready database layer using `sqlx`
      for compile-time checked SQL queries and strict state transitions.
    - `node-enrollment`: Added endpoint registration flow generating secure 24-hour JWTs.
    - `health-tracker`: Added time-series heartbeat recording.
    - Infra: Added `docker-compose.yml`, `Dockerfile`, SQL migrations, and seed data.
    - Config: Integrated `dotenvy` to parse `.env` files and added `.env.example`.
    - CI: Added `CHANGELOG.md` and `local-ci.sh` script to streamline pre-push checks.

    Security & Architectural Fixes:
    - Enforced server-side status coercion to prevent agents from self-reporting
      as "isolated" or overriding operator commands.
    - Transitioned away from using `xmax` system columns for UPSERT detection,
      opting for explicit `SELECT FOR UPDATE` patterns to guarantee query reliability.
    - Fixed a silent clock failure vulnerability in the JWT signing process.
    - Fixed `unused_import`, `dead_code`, and 32-bit truncation warnings to
      ensure the workspace compiles with zero clippy warnings.
refactor(agent): migrate fleet communication and event buffering from…
… management, and osquery configuration orchestration
swar09 and others added 26 commits June 27, 2026 19:09
…-server

- ci: upgrade actions/checkout v3 -> v4 in all four workflow jobs
- agent/agent.toml: remove hardcoded node_id (written by enrollment flow on first run)
- agent-bin/main.rs: fail fast on malformed UUID from fleet server (expect vs unwrap_or_default); remove unused imports (File, BufRead, BufReader, error); drop blanket #![allow]
- command_handler: remove blanket #![allow], drop unused tracing imports
- agent-core/lib.rs: remove blanket #![allow]
- agent-core/orchestrator.rs: remove blanket #![allow]
- fleet-client/lib.rs: remove blanket #![allow] + unused imports; propagate UUID parse error in enroll(); log payload serialization failures and skip instead of silently empty-vec; replace silent while-let stream loop with explicit match that logs errors and graceful close
- osquery/osquery.flags: fix contradictory --disable_audit=false -> true; remove --enable_syslog and --audit_allow_sockets that contradict eBPF-only architecture
- agent/tests: mark unimplemented integration test with #[ignore]
- test_query.rs: fix 'FAMED' -> 'FRAMED' typo
- kafka-handler: replace Timeout::Never with Timeout::After(30s) to prevent indefinite blocking
- kafka-pipeline/Dockerfile: replace libssl-dev with libssl3 in runtime stage; add --no-install-recommends to both apt-get calls; add non-root 'edr' user and switch to it before CMD
- kafka-pipeline/Cargo.toml: remove redundant commented-out rdkafka workspace dep line
- kafka-admin.rs: replace brittle positional arg parsing with flag-name-based parser; remove advertised-but-unimplemented verify-topics/describe-topic doc comments
- consumer.rs: disable auto-commit (enable.auto.commit=false); remove blanket #![allow] and unused tracing imports
- event_router.rs: remove blanket #![allow] and unused warn import
- guide.md: fix stale function reference determine_topic -> route_topic
- README.md: standardize spelling normalised -> normalized
find the .proto file having ServerCommand and then complete this one
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 203 files, which is 103 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4af34dfb-74b3-4713-a18c-05ec7758dbdd

📥 Commits

Reviewing files that changed from the base of the PR and between ebd63de and d965ea1.

⛔ Files ignored due to path filters (18)
  • frontend/package-lock.json is excluded by !**/package-lock.json
  • frontend/public/apple-touch-icon.png is excluded by !**/*.png
  • frontend/public/favicon-96x96.png is excluded by !**/*.png
  • frontend/public/favicon.ico is excluded by !**/*.ico
  • frontend/public/favicon.svg is excluded by !**/*.svg
  • frontend/public/icons.svg is excluded by !**/*.svg
  • frontend/public/logo.png is excluded by !**/*.png
  • frontend/public/web-app-manifest-192x192.png is excluded by !**/*.png
  • frontend/public/web-app-manifest-512x512.png is excluded by !**/*.png
  • frontend/src/apple-touch-icon.png is excluded by !**/*.png
  • frontend/src/assets/hero.png is excluded by !**/*.png
  • frontend/src/assets/react.svg is excluded by !**/*.svg
  • frontend/src/assets/vite.svg is excluded by !**/*.svg
  • frontend/src/favicon-96x96.png is excluded by !**/*.png
  • frontend/src/favicon.ico is excluded by !**/*.ico
  • frontend/src/favicon.svg is excluded by !**/*.svg
  • frontend/src/web-app-manifest-192x192.png is excluded by !**/*.png
  • frontend/src/web-app-manifest-512x512.png is excluded by !**/*.png
📒 Files selected for processing (203)
  • .dev.env.example
  • .dockerignore
  • .env.example
  • .github/CI_CACHING_STRATEGY.md
  • .github/workflows/agent-release.yml
  • .github/workflows/autofix.yml
  • .github/workflows/ci.yml
  • .github/workflows/kafka-pipeline.yml
  • .github/workflows/update-deps.yml
  • .gitignore
  • .sqlx/query-02f0784a2aefc0e48b6a78a86aa0575a0e98e87c85e72e5da49d79827a0d7997.json
  • .sqlx/query-4d3c9d60a0f85348d286e4f41ce28b0a313b6f164c48480f1ec9cf00a8d8c7bb.json
  • .sqlx/query-7a42f879c809ab1bea8dda4b06aa1fce6a105ad460025ccb4b0fa10949b6b486.json
  • .sqlx/query-7df2e8394e5f55c0217192c061668f00fd00774b2b86f414a6bfb04f900b21a8.json
  • .sqlx/query-be24aaa28047630cd45f9701f9e74fac4438460404967d1d348532cad39259ae.json
  • .sqlx/query-e5760d97933de085f8e2038f937b8924cf27b99f433d3f625e5deb6cced11ee6.json
  • .testingignore
  • CHANGELOG.md
  • CONTRIBUTING.md
  • Cargo.toml
  • EDR_IMPLEMENTATION_GUIDE.md
  • LICENSE
  • README.md
  • TIMELINE.md
  • agent/.env.example
  • agent/Dockerfile
  • agent/INSTALLATION_GUIDE.md
  • agent/agent-dev-guide.md
  • agent/agent.toml
  • agent/crates/agent-core/src/lib.rs
  • agent/crates/agent-core/timeline.md
  • agent/crates/agent_bin/Cargo.toml
  • agent/crates/agent_bin/src/main.rs
  • agent/crates/agent_core/Cargo.toml
  • agent/crates/agent_core/src/command_handler.py
  • agent/crates/agent_core/src/command_handler.rs
  • agent/crates/agent_core/src/config.py
  • agent/crates/agent_core/src/config.rs
  • agent/crates/agent_core/src/lib.py
  • agent/crates/agent_core/src/lib.rs
  • agent/crates/agent_core/src/orchestrator.py
  • agent/crates/agent_core/src/orchestrator.rs
  • agent/crates/agent_core/src/preflight.py
  • agent/crates/agent_core/src/preflight.rs
  • agent/crates/agent_tracing/Cargo.toml
  • agent/crates/agent_tracing/src/lib.py
  • agent/crates/agent_tracing/src/lib.rs
  • agent/crates/ebpf-collector/Cargo.toml
  • agent/crates/ebpf-collector/src/lib.rs
  • agent/crates/ebpf-collector/timeline.md
  • agent/crates/event-buffer/Cargo.toml
  • agent/crates/event-buffer/src/lib.rs
  • agent/crates/event-buffer/timeline.md
  • agent/crates/event_buffer/Cargo.toml
  • agent/crates/event_buffer/src/__init__.py
  • agent/crates/event_buffer/src/lib.py
  • agent/crates/event_buffer/src/lib.rs
  • agent/crates/event_buffer/tests/event_buffer_test.py
  • agent/crates/event_buffer/tests/event_buffer_test.rs
  • agent/crates/fleet-client/src/lib.rs
  • agent/crates/fleet-client/timeline.md
  • agent/crates/fleet_client/Cargo.toml
  • agent/crates/fleet_client/src/codec.py
  • agent/crates/fleet_client/src/codec.rs
  • agent/crates/fleet_client/src/connection.py
  • agent/crates/fleet_client/src/connection.rs
  • agent/crates/fleet_client/src/enrollment.py
  • agent/crates/fleet_client/src/enrollment.rs
  • agent/crates/fleet_client/src/heartbeat.py
  • agent/crates/fleet_client/src/heartbeat.rs
  • agent/crates/fleet_client/src/lib.py
  • agent/crates/fleet_client/src/lib.rs
  • agent/crates/fleet_client/src/stream.py
  • agent/crates/fleet_client/src/stream.rs
  • agent/crates/fleet_client/src/types.py
  • agent/crates/fleet_client/src/types.rs
  • agent/crates/isolation/isolation_manager.py
  • agent/crates/isolation/src/lib.rs
  • agent/crates/isolation/test_isolation_manager.py
  • agent/crates/isolation/timeline.md
  • agent/crates/osquery-client/src/lib.rs
  • agent/crates/osquery-client/timeline.md
  • agent/crates/osquery_client/Cargo.toml
  • agent/crates/osquery_client/src/bin/test_query.rs
  • agent/crates/osquery_client/src/client.rs
  • agent/crates/osquery_client/src/diff.rs
  • agent/crates/osquery_client/src/lib.rs
  • agent/crates/osquery_client/src/scheduler.rs
  • agent/crates/osquery_client/src/types.rs
  • agent/install.sh
  • agent/limits/99-aigis-zero.conf
  • agent/osquery-edr-linux-guide.md
  • agent/osquery/osquery.conf
  • agent/osquery/osquery.flags
  • agent/sysctl/60-aigis-zero.conf
  • agent/systemd/aigis-zero.service
  • agent/systemd/osqueryd.service.d/aigis-zero.conf
  • agent/tests/agent_integration.rs
  • agent/timeline.md
  • agent/uninstall.sh
  • api-backend/Cargo.toml
  • api-backend/Dockerfile
  • api-backend/timeline.md
  • fleet-server/.env.example
  • fleet-server/Cargo.toml
  • fleet-server/Dockerfile
  • fleet-server/crates/fleet-manager/Cargo.toml
  • fleet-server/crates/fleet-manager/src/lib.rs
  • fleet-server/crates/fleet-manager/src/ports.rs
  • fleet-server/crates/fleet-server-bin/Cargo.toml
  • fleet-server/crates/fleet-server-bin/src/main.rs
  • fleet-server/crates/fleet-server-bin/src/ports.rs
  • fleet-server/crates/fleet-server-bin/src/settings.rs
  • fleet-server/crates/fleet-tracing/Cargo.toml
  • fleet-server/crates/fleet-tracing/src/config.rs
  • fleet-server/crates/fleet-tracing/src/init.rs
  • fleet-server/crates/fleet-tracing/src/lib.rs
  • fleet-server/crates/grpc-listener/Cargo.toml
  • fleet-server/crates/grpc-listener/src/auth.rs
  • fleet-server/crates/grpc-listener/src/config.rs
  • fleet-server/crates/grpc-listener/src/error.rs
  • fleet-server/crates/grpc-listener/src/lib.rs
  • fleet-server/crates/grpc-listener/src/server.rs
  • fleet-server/crates/grpc-listener/src/service.rs
  • fleet-server/crates/health-tracker/Cargo.toml
  • fleet-server/crates/health-tracker/src/error.rs
  • fleet-server/crates/health-tracker/src/lib.rs
  • fleet-server/crates/health-tracker/src/store.rs
  • fleet-server/crates/health-tracker/src/tracker.rs
  • fleet-server/crates/kafka-handler/Cargo.toml
  • fleet-server/crates/kafka-handler/src/lib.rs
  • fleet-server/crates/node-enrollment/Cargo.toml
  • fleet-server/crates/node-enrollment/src/enroller.rs
  • fleet-server/crates/node-enrollment/src/error.rs
  • fleet-server/crates/node-enrollment/src/lib.rs
  • fleet-server/crates/node-enrollment/src/store.rs
  • fleet-server/crates/node-enrollment/src/token.rs
  • fleet-server/crates/postgres-interface/Cargo.toml
  • fleet-server/crates/postgres-interface/src/error.rs
  • fleet-server/crates/postgres-interface/src/health_store.rs
  • fleet-server/crates/postgres-interface/src/lib.rs
  • fleet-server/crates/postgres-interface/src/node_store.rs
  • fleet-server/crates/postgres-interface/src/pool.rs
  • fleet-server/docker-compose.yml
  • fleet-server/migrations/001_create_nodes.sql
  • fleet-server/migrations/20260601000001_create_nodes.sql
  • fleet-server/migrations/20260601000002_create_enrollment_events.sql
  • fleet-server/migrations/20260601000003_create_node_health.sql
  • fleet-server/migrations/seed.sql
  • fleet-server/src/main.rs
  • fleet-server/timeline.md
  • frontend/Dockerfile
  • frontend/README.md
  • frontend/eslint.config.js
  • frontend/index.html
  • frontend/package.json
  • frontend/public/site.webmanifest
  • frontend/src/App.css
  • frontend/src/App.tsx
  • frontend/src/AsciiBackground.css
  • frontend/src/AsciiBackground.tsx
  • frontend/src/index.css
  • frontend/src/main.tsx
  • frontend/src/site.webmanifest
  • frontend/timeline.md
  • frontend/tsconfig.app.json
  • frontend/tsconfig.json
  • frontend/tsconfig.node.json
  • frontend/vite.config.ts
  • infra/docker-compose.dev.yml
  • infra/docker-compose.kafka-cluster.yml
  • infra/k8s/keda-scaler.yml
  • infra/scripts/create-topics.sh
  • infra/timeline.md
  • kafka-pipeline/.env.example
  • kafka-pipeline/Cargo.toml
  • kafka-pipeline/Dockerfile
  • kafka-pipeline/guide.md
  • kafka-pipeline/src/bin/kafka-admin.rs
  • kafka-pipeline/src/consumer.rs
  • kafka-pipeline/src/event_router.rs
  • kafka-pipeline/src/health.rs
  • kafka-pipeline/src/lib.rs
  • kafka-pipeline/src/main.rs
  • kafka-pipeline/src/metrics.rs
  • kafka-pipeline/tests/integration_test.rs
  • kafka-pipeline/timeline.md
  • rule-engine/Cargo.toml
  • rule-engine/Dockerfile
  • rule-engine/timeline.md
  • rust-toolchain.toml
  • scheduled_queries.toml
  • sdk/Cargo.toml
  • sdk/build.rs
  • sdk/src/codec.rs
  • sdk/src/lib.rs
  • sdk/src/models/enrollment.rs
  • sdk/src/models/envelope.rs
  • sdk/src/models/event.rs
  • sdk/src/models/heartbeat.rs
  • sdk/src/models/mod.rs
  • sdk/timeline.md
  • typos.toml

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@swar09

swar09 commented Aug 12, 2026

Copy link
Copy Markdown
Owner

This pr has too much overlap with other work , it will take time for review . Thanks for PR !

@swar09

swar09 commented Aug 12, 2026

Copy link
Copy Markdown
Owner

There are several issues in pr first is you have forked from wrong branch i guess. we will discuss this later .

@irapatil07

irapatil07 commented Aug 12, 2026 via email

Copy link
Copy Markdown
Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants