Skip to content

User_Manual.md

Latest commit

 

History

History
165 lines (123 loc) · 7.53 KB

User_Manual.md

File metadata and controls

165 lines (123 loc) · 7.53 KB
Raw

Sifter — User Manual

Sifter is an automated reconnaissance pipeline designed for penetration testers, bug bounty hunters, and CTF players. It chains nmap, gobuster, and feroxbuster into a single workflow, parses the results, and highlights high-interest findings.

This manual covers everything from basic setup to advanced automation use cases.

Table of Contents

  1. Prerequisites & Dependencies
  2. Installation
  3. Execution Modes
    • Interactive Mode
    • CLI Mode (Automation)
  4. Understanding the Output
    • Terminal Output
    • Generated Files
  5. Real-World Use Cases
  6. Troubleshooting & FAQ

1. Prerequisites & Dependencies

Sifter requires root privileges to run nmap -sCV (service and version detection).

Ensure the following tools are installed on your system (Kali/Parrot OS have these by default):

Tool Purpose Install Command (Debian/Ubuntu)
nmap Port scanning & service detection sudo apt install nmap
gobuster Fast directory brute-forcing sudo apt install gobuster
feroxbuster Recursive content discovery See Feroxbuster Releases
jq Parsing feroxbuster JSON output sudo apt install jq

Default Wordlist: Sifter uses /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt by default. If you do not have it, install dirbuster (sudo apt install dirbuster) or pass a custom wordlist using the -w flag.

2. Installation

No complex installation is required. Simply clone the repository and make the script executable.

git clone https://github.com/Harshil015/sifter.git
cd sifter
chmod +x sifter.sh

3. Execution Modes

Sifter can be used in two ways: interactively (prompting you for input) or via CLI arguments (for scripting and automation).

Mode A: Interactive Mode (Default)

Best for single-target manual testing.

sudo ./sifter.sh

Workflow:

  1. Sifter checks for dependencies and root privileges.
  2. It will prompt: Enter target IP or Hostname (e.g., 10.10.10.10 or example.com):
  3. Type the target IPv4 address or hostname/FQDN (e.g., 10.10.11.45 or example.com) and press Enter.
  4. The scan runs automatically.

Mode B: CLI Mode (For Automation)

Best for batch scanning, scripts, or CI/CD pipelines.

Flags:

  • -t <target> : Specify the target IP address or hostname/FQDN.
  • -w <path> : Specify a custom wordlist.

Examples:

# Scan a specific IP using the default wordlist
sudo ./sifter.sh -t 10.10.11.45

# Scan a specific IP using a custom wordlist
sudo ./sifter.sh -t 10.10.11.45 -w /usr/share/wordlists/seclists/Discovery/Web-Content/common.txt

4. Understanding the Output

When a scan completes, Sifter creates a timestamped folder in the current directory (e.g., sifter_10.10.11.45_20231024_153000/).

A. Terminal Output

Sifter provides a color-coded, categorized summary directly in your terminal:

  • Discovered Paths: Grouped by parent directory.
  • Status Codes: Green (200), Cyan (3xx Redirects), Yellow (401/403 Forbidden), Red (5xx Errors).
  • Risk Tags & Stars (★):
    • 🔴 High-Interest: Script/Config/Sensitive files (.php, .env, .bak, .sql).
    • 🟡 Medium-Interest: Data/Config files (.json, .xml, .log).
  • Open Directory Listings: Flagged with a ⚠ icon if directory browsing is enabled.
  • Summary Box: A quick grid showing total URLs, directories, and high-risk files found.

B. Generated Files

Inside the output folder, you will find:

File Description
nmap_scan.txt Human-readable Nmap output.
nmap_scan.xml Machine-readable Nmap output (for parsing).
gobuster_portXX.txt Raw Gobuster results for the specific port.
feroxbuster_portXX_report.txt Clean, formatted plain-text report of findings.
feroxbuster_portXX_raw.json Raw Feroxbuster JSON data (kept for debugging).

5. Real-World Use Cases

Use Case 1: OSCP Exam / CTF Box Enumeration

Scenario: You just spawned an HTB or OSCP lab machine and only have an IP address. Action: Run Sifter in interactive mode.

sudo ./sifter.sh

Why it helps: While you manually enumerate the landing page, Sifter runs in the background. When it finishes, it immediately points out .bak files, configuration files, or /admin directories that you need to exploit, saving you 15-20 minutes of manual tool chaining.

Use Case 2: Bug Bounty Asset Monitoring

Scenario: You want to monitor a specific IP range for newly exposed sensitive files over time. Action: Write a bash cron job that loops through IPs using Sifter's CLI mode.

#!/bin/bash
# weekly_scan.sh
TARGET_IP="192.168.1.50"
sudo /path/to/sifter.sh -t $TARGET_IP -w /path/to/api_wordlist.txt

Why it helps: You can schedule this to run weekly. If a developer accidentally leaves a config.php.bak file exposed, Sifter's summary report will flag it with a 🔴 High-Interest tag, and you can submit it for a bug bounty.

Use Case 3: Wide Wordlist API Enumeration

Scenario: You know the target has hidden API endpoints, but the default Dirbuster wordlist is too web-focused. Action: Pass a SecLists API-specific wordlist to Sifter.

sudo ./sifter.sh -t 10.10.10.10 -w /usr/share/seclists/Discovery/Web-Content/api/api-endpoints.txt

Why it helps: Feroxbuster will recursively dig through API endpoints, and Sifter will categorize any discovered .json or .xml responses as 🟡 Medium-Interest, allowing you to quickly spot data leakage.

Use Case 4: Integration into Larger Pentest Reports

Scenario: You are writing a formal penetration test report for a client and need evidence of your directory brute-forcing. Action: Run Sifter, then navigate to the output directory. Why it helps: The feroxbuster_portXX_report.txt file is cleanly formatted with timestamps, status codes, and byte sizes. You can copy/paste this directly into an appendix in your formal deliverable.

6. Troubleshooting & FAQ

Q: I get the error This script requires root privileges for nmap -sCV. A: You forgot to run the script with sudo. Nmap's service detection (-s) requires raw socket access, which only root has. Run: sudo ./sifter.sh

Q: The script says Default wordlist not found... Exiting. A: Your system doesn't have the Dirbuster wordlists installed. You can either install them (sudo apt install dirbuster) or pass your own wordlist using the -w flag.

Q: Sifter finishes the nmap scan but says No HTTP/HTTPS services detected. A: The target only has non-web ports open (e.g., SSH, FTP). Directory brute-forcing only works on web services, so Sifter skips stages 3a and 3b automatically.

Q: The IP validation fails even though my target is correct. A: Sifter accepts IPv4 addresses (e.g., 10.10.10.10) and hostnames/FQDNs (e.g., example.com). It does not support IPv6 (e.g., fe80::1), and a purely numeric string that isn't a valid IPv4 address (e.g., a stray port number like 8080) will also be rejected.

Q: Can I change the number of threads for Gobuster/Feroxbuster to make it faster? A: Yes, but you must edit the script directly. Open sifter.sh and change --threads 40 to --threads 100 (or your desired number) in the run_gobuster() and run_feroxbuster() functions. Note: Higher threads may cause network congestion or get your IP blocked by the target.