Skip to content
View karthigaiselvanm's full-sized avatar
2 followers · 2 following

Achievements

Achievement: YOLOAchievement: Pull SharkAchievement: Quickdraw

Achievements

Achievement: YOLOAchievement: Pull SharkAchievement: Quickdraw

Block or report karthigaiselvanm

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
karthigaiselvanm/README.md

Hi, I'm Karthigai Selvan

Data Platform Engineer • 10+ YOE • Building Secure & Scalable Data Systems

AWS • Databricks • Spark • Trino • Apache Ranger • Terraform • Kubernetes
🌐 Blog · 💼 LinkedIn · 📧 Email

🚀 What I Do

  • Design and build scalable data platforms with strong focus on governance, security, and cost optimization
  • Enable multi-tenant analytics with fine-grained access control (Apache Ranger + Trino)
  • Build production-grade pipelines using Spark, streaming, and modern orchestration
  • Drive platform standardization (Infra as Code, data contracts, reproducibility)

🧠 Core Expertise

  • Data Platforms: Spark, Delta Lake, Trino, dbt, Airflow, Kafka
  • Cloud & Infra: AWS (S3, Glue, IAM), Terraform, Docker, Kubernetes
  • Governance & Security: Apache Ranger, OCSF mappings, audit pipelines
  • Languages: Python, SQL

⭐ Featured Projects

🔐 Trino + Apache Ranger on Kubernetes (Helm)

👉 https://github.com/karthigaiselvanm/trino-ranger-k8s

  • Production-ready Trino + Ranger integration on Kubernetes
  • Solved real-world issues:
    • Ranger plugin misconfigurations (cred.jceks, service naming)
    • JVM tuning and config injection challenges
  • Includes Helm charts + StatefulSet deployment
  • Designed for secure, repeatable platform setups

⚡ Spark Learning Lab

👉 https://github.com/karthigaiselvanm/spark-learning

  • Hands-on deep dive into:
    • DataFrame APIs
    • Structured Streaming
    • Performance optimization techniques
  • Companion to my Spark Deep Dive roadmap

🧩 PySpark Patterns & Use Cases

👉 https://github.com/karthigaiselvanm/pyspark-sample-projects

  • Real-world patterns:
    • Incremental processing
    • Window functions
    • Advanced joins & transformations
  • Designed for interview prep + production scenarios

📚 Writing & Thought Leadership

  • ✍️ Apache Ranger × Trino deep dive (Helm, pitfalls, architecture) — Coming soon
  • ✍️ Secure multi-tenant data platform patterns
  • 🌐 https://k2ddna.com

🏅 Achievements

  • Databricks Certified Data Engineer Associate (2024)
  • Built OCSF-aligned pipelines for security analytics
  • Designed ingestion frameworks using Benthos
  • 10+ years delivering enterprise-grade data solutions

📊 GitHub Stats

🤝 Let’s Collaborate

I’m open to:

  • Platform architecture discussions
  • OSS contributions (Trino, Ranger, Data Governance)
  • Building secure, scalable data systems

📩 Reach out via LinkedIn or email

Popular repositories Loading

  1. pyspark-sample-projects pyspark-sample-projects Public

    Sample projects done in Jupyter Notebooks using spark DataFrame & spark SQL

    Jupyter Notebook 3 2

  2. udacity-data-engineering-projects udacity-data-engineering-projects Public

    Few sample projects related to Udacity Data Engineer Program including Data modeling in Postgres & Apache Cassandra, Setting up a Cloud Data Warehouse, Creating a data lake using Spark & Data pipel…

    Jupyter Notebook 1

  3. trino-ranger-k8s trino-ranger-k8s Public

    Secure Trino deployments on Kubernetes with Apache Ranger — includes Helm charts, init containers, and configs for policy-driven access control.

    Dockerfile 1 1

  4. great_expectations great_expectations Public

    Forked from fivetran/great_expectations

    Always know what to expect from your data.

    Python 1

  5. sample_secrets_api sample_secrets_api Public

    Forked from GitGuardian/sample_secrets

    Jupyter Notebook

  6. sql-injection-example sql-injection-example Public

    Forked from kodtodya/sql-injection-example

    this example demonstrates the vulnerable sql injection code

    Java