Skip to content

BOSSincrypto/analytics-for-telegram-bot

BranchesTags

Repository files navigation

📊 analytics-for-telegram-bot

Lightweight, privacy-first analytics for Telegram bots (and other chatbots)

Python License: MIT Tests Dependencies mypy ruff Code style: black-compatible

Features · Installation · Quickstart · Metrics · Dashboard · Privacy

#telegram #telegram-bot #analytics #python #sqlite #flask #chatbot #privacy-first #zero-dependencies #DAU #funnel-analysis #data-retention #open-source #MIT

Track users, messages, events, funnels, and ratings in your Telegram bot. Get in-bot Markdown reports and an optional Flask web dashboard with live charts.

The core library has zero third-party runtime dependencies — pure Python stdlib. The dashboard is an optional extra requiring Flask.

Live demo & docs: GitHub Pages site

✨ Features

Feature Description
⚡ Non-blocking writes All tracking calls enqueue instantly. A background writer thread batches and commits — never block your bot's event loop.
🔒 Privacy by default Usernames off, raw JSON off, text truncated, optional HMAC-SHA256 user ID hashing with per-DB salt.
📊 22 metric functions DAU/WAU/MAU/YAU, new users, accumulation, hourly heatmap, message analysis, funnels, ratings, reviews.
🛡️ SQL injection proof Every query uses bound parameters. Read-only mode=ro connections for all reads. Custom sql_query allows only SELECT/WITH.
🎨 Flask dashboard Optional web UI with card-grid layout, client-side Chart.js graphs, hashed-password auth, read-only JSON API.
🗑️ GDPR-ready retention Automatic data purging with configurable retention_days. Cross-surface consistency guaranteed.
🤖 Multi-bot isolation Multiple Analytics instances with different bot_id write to one DB. Each metric is bot-scoped.
📝 Markdown reports Five report categories delivered directly in chat. Admin-gated allowlist access.
📦 Zero dependencies Core library uses only Python stdlib. pip install . and you're done.

📦 Installation

# Core library only (zero third-party deps)
pip install .

# With the web dashboard
pip install '.[dashboard]'

# For development
pip install -e '.[dashboard,dev]'

🚀 Quickstart

Sync (telebot / pyTelegramBotAPI)

from analytics_for_telegram_bot import Analytics, Config

config = Config(
    db_path="analytics.db",
    bot_id="my_bot",
    admin_ids=[123456789],  # Telegram user IDs allowed to see reports
)
ana = Analytics(config)

# /start handler — track a new user
ana.track_update(update)

# Track custom events
ana.event("purchase", event_type="shop", user_id=user_id)
ana.rating(5, user_id)
ana.review("Great bot!", user_id)
ana.funnel_step("checkout", "payment", user_id)

# Get a Markdown report in chat
from analytics_for_telegram_bot.reports import report, is_admin

if is_admin(user_id, config.admin_ids):
    text = report("users", ana, period_start="2024-01-01")
    bot.send_message(user_id, text)

ana.close()  # flushes all pending writes

Async (aiogram / python-telegram-bot)

from analytics_for_telegram_bot import Analytics, Config

config = Config(db_path="analytics.db", bot_id="my_bot")
ana = Analytics(config)

# In any async handler — track_update is non-blocking and thread-safe
async def on_message(update):
    ana.track_update(update.dict())  # pass a dict, not the framework object

# The writer thread handles persistence asynchronously.
# Call ana.close() on shutdown to flush.

📊 Metrics API

All metrics open a read-only connection (mode=ro), work after close(), concurrently with the writer thread, and return safe empty values on DB errors.

Auto-derived (from updates)

Method Returns
analyze_total() Total users, messages, updates, averages
analyze_new_user() New users per day
analyze_user_accumulation() Cumulative growth (running total)
analyze_dau() Daily Active Users per day
analyze_wau() Weekly Active Users
analyze_mau() Monthly Active Users
analyze_yau() Yearly Active Users
analyze_hour_activity() 24×7 matrix (hour × day of week)
analyze_messages_number(message_type=) Messages per day (filter by type)
analyze_messages(limit=) Top messages (text + count)
analyze_messages_type() Distribution by message type
analyze_update_type() Distribution by update type
analyze_chat_type() Distribution by chat type
analyze_language() Distribution by user language
analyze_bots_users() User counts across all bots in DB

Manual (from events / ratings / funnel_steps)

Method Returns
analyze_events(limit=) Event distribution (name → count)
analyze_events_number(event_type=) Events per day (filter by type)
analyze_events_type() Distribution by event type
analyze_events_funnel(steps=) Event funnel (counts per step)
analyze_assessment() Rating distribution (1–5)
analyze_review(limit=) Text reviews (latest-first)
analyze_funnel(steps=, source=, funnel=) Unified funnel: events or updates

Custom read-only SQL

rows = ana.sql_query(
    "SELECT event, COUNT(*) AS c FROM events WHERE bot_id = :bid GROUP BY event ORDER BY c DESC",
    params={"bid": "my_bot"},
    limit=100,
)
# -> [{"event": "click", "c": 42}, ...]

Only SELECT / WITH statements are allowed. Parameters are always bound, never interpolated. DML/DDL is rejected with ValueError.

🔧 Manual Instrumentation API

ana.event("click", event_type="ui", user_id=uid, value=1.0, props={"color": "red"})
ana.rating(4, user_id)
ana.review("Could be faster", user_id)
ana.funnel_step("signup", "email_verified", user_id)

🎨 Web Dashboard

# After: pip install '.[dashboard]'
python -m analytics_for_telegram_bot dashboard --db analytics.db
# Optional: --host 0.0.0.0 --port 8787 --bot-id my_bot

The dashboard provides:

  • Card-grid UI with live charts powered by vendored Chart.js (no CDN)
  • Read-only JSON API: GET /api/fetch?m=<metric>&period_start=...&period_end=...
  • HTTP Basic Auth with hashed password (constant-time verify, 401 without redirect)
  • Auto-polling every 5 seconds to keep values current
  • Multi-bot support: auto-detects bot or pins via --bot-id

Set a password hash via Config(dashboard_password_hash=...):

from analytics_for_telegram_bot.dashboard.auth import hash_password
hash_str = hash_password("your_password")

Without a password, the dashboard is open only when bound to localhost.

📝 In-Bot Reports (Markdown)

from analytics_for_telegram_bot.reports import report, is_admin

if is_admin(user_id, config.admin_ids):
    text = report("users", ana, period_start="2024-01-01", period_end="2024-01-31")
    bot.send_message(chat_id, text)

Five report categories: total, users, messages, events, ratings. Each returns a Markdown string with tables, lists, and summaries.

🗑️ Retention Purge

config = Config(
    db_path="analytics.db",
    bot_id="my_bot",
    retention_days=90,    # delete data older than 90 days
    auto_purge=True,      # purge at init and close automatically
)
ana = Analytics(config)

# Or purge manually
deleted = ana.purge()  # returns number of rows deleted
  • Purges all 5 data tables: updates, events, ratings, funnel_steps, users
  • users table purged by last_seen (active old users are preserved)
  • retention_days=None disables purge entirely (no-op)
  • Cross-surface consistency: metrics, reports, and dashboard all see the same trimmed data

🔒 Privacy & Security

Setting Default Description
collect_username False Telegram usernames are not collected
collect_raw False Full update JSON is not stored
hash_user_id False When True: HMAC-SHA256 with per-DB random salt
collect_text True Message text stored, truncated to text_max_len
text_max_len 200 Max characters of message text retained
collect_language True language_code from Telegram user object

Additional security guarantees:

  • All metric queries are parameterized SQL, scoped by bot_id
  • Read-only connections (mode=ro) for all read operations
  • sql_query only allows SELECT/WITH, runs on a read-only connection
  • DB errors translated to safe empty values (no SQL/stack leaks)
  • Writer-thread logs are sanitized (no raw exception text, no bound values)

⚙️ Full Configuration

Config(
    db_path="analytics.db",        # path to SQLite
    bot_id="my_bot",               # bot identifier
    # Privacy
    collect_text=True,
    text_max_len=200,
    collect_command_text_always=True,
    collect_username=False,
    collect_language=True,
    collect_raw=False,
    hash_user_id=False,
    # Retention
    retention_days=None,           # int or None
    auto_purge=False,
    # Access
    admin_ids=[123456],
    dashboard_password_hash=None,  # bcrypt-style hash
    dashboard_host="127.0.0.1",
    dashboard_port=8787,
    # Writer tuning
    batch_size=100,
    flush_interval=0.1,
    queue_maxsize=0,               # 0 = unbounded
    # Display
    tz_display="UTC",
)

🏗️ Architecture

  ┌──────────────────────────────────────────────────────────────┐
  │                     Your Telegram Bot                         │
  │            (telebot / aiogram / python-telegram-bot)          │
  └──────────┬───────────────────┬────────────────────┬──────────┘
             │                   │                    │
        track_update()      event()              rating()
        funnel_step()       review()
             │                   │                    │
             ▼                   ▼                    ▼
  ┌──────────────────────────────────────────────────────────────┐
  │                    In-Memory Queue                            │
  │            (non-blocking, thread-safe enqueue)                │
  └──────────────────────────┬───────────────────────────────────┘
                             │ batch + flush_interval
                             ▼
  ┌──────────────────────────────────────────────────────────────┐
  │               Background Writer Thread                        │
  │     (single daemon thread, sole write connection)             │
  │     INSERT OR IGNORE dedup · error rollback · no data loss    │
  └──────────────────────────┬───────────────────────────────────┘
                             │
                             ▼
  ┌──────────────────────────────────────────────────────────────┐
  │                     SQLite Database                           │
  │  ┌─────────┐ ┌────────┐ ┌─────────┐ ┌────────────┐          │
  │  │ updates │ │ events │ │ ratings │ │ funnel_step│  + users  │
  │  └─────────┘ └────────┘ └─────────┘ └────────────┘          │
  └──────┬───────────────────────────────────────────┬──────────┘
         │ mode=ro (read-only)                        │ mode=ro
         ▼                                            ▼
  ┌──────────────┐    ┌───────────────────┐    ┌─────────────────┐
  │  Metrics API │    │  Markdown Reports │    │  Flask Dashboard │
  │  (22 funcs)  │    │  (5 categories)   │    │  (Chart.js + API)│
  └──────────────┘    └───────────────────┘    └─────────────────┘

🤖 Multi-Bot Isolation

Multiple Analytics instances with different bot_id can write to the same SQLite database. Each metric is scoped by bot_id. The dashboard auto-detects the bot with the most updates rows, or you can pin it via --bot-id / ?bot_id= query param.

🧪 Testing

Check Result
pytest 254 tests, all passing
ruff clean
mypy clean (15 files)
E2E smoke test passed (data ingestion, all metrics, sql_query, dashboard) 
# Run the full test suite
pip install -e '.[dashboard,dev]'
pytest tests/ -v
ruff check analytics_for_telegram_bot
mypy analytics_for_telegram_bot

📁 Project Structure

analytics_for_telegram_bot/
├── analytics.py          # Public facade: track_update, event, rating, review, funnel_step
├── config.py             # Config dataclass with validation
├── db.py                 # SQLite connection + schema bootstrap
├── parser.py             # Telegram update parser (dict -> ParsedUpdate)
├── privacy.py            # User ID hashing, text resolution, salt management
├── retention.py          # Data purge logic
├── writer.py             # Background writer thread (batch + flush)
├── metrics.py            # 22 metric fetchers + REGISTRY (dashboard allow-list)
├── repository.py         # Parameterized SQL queries (read-only)
├── reports.py            # Markdown report generators
├── __main__.py           # CLI entry point (dashboard subcommand)
└── dashboard/
    ├── app.py            # Flask app factory (auth, routes, API)
    ├── auth.py           # Password hashing + constant-time verify
    └── static/           # Vendored Chart.js + card-grid frontend

🔄 CI/CD

This repo includes GitHub Actions workflows:

  • CI (.github/workflows/ci.yml) — runs ruff, mypy, and pytest on Python 3.10–3.13
  • Deploy Pages (.github/workflows/deploy-pages.yml) — auto-deploys the landing page on every push to master
  • Release (.github/workflows/release.yml) — creates a GitHub Release with built wheels when a v* tag is pushed
# Trigger a release
git tag v0.1.0
git push origin v0.1.0

📄 License

MIT © BOSSincrypto

⬆ Back to top

Made with ⚡ for the Telegram bot community

#telegram #telegram-bot #analytics #python #sqlite #flask #chatbot #privacy-first #zero-dependencies #DAU #funnel-analysis #data-retention #open-source #MIT

About

📊 Lightweight, privacy-first analytics for Telegram bots. Zero runtime dependencies. 22 metrics, Flask dashboard, funnels, DAU/WAU/MAU, GDPR-ready retention.

bossincrypto.github.io/analytics-for-telegram-bot/

Topics

python flask privacy telegram analytics telegram-bot metrics sqlite chatbot mit-license funnel dau data-retention zero-dependencies

Resources

Readme

License

MIT license
Activity

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages