AutoDefender

An AI-powered security tool that monitors Suricata network logs in real-time and analyzes historical log files to detect threats, provide AI-generated explanations, and recommend security actions.

Try it online: https://autodefenderhackathon.streamlit.app/

Features

• Real-time Monitoring: Watches Suricata eve.json log files and processes events as they occur
• Multi-Source Aggregation: Monitor multiple Suricata instances or log sources simultaneously
• Historical Analysis: Batch processes existing Suricata log files for threat detection
• AI-Powered Detection: Uses Ollama to analyze threats and provide plain English explanations
• GeoIP Enrichment: Automatic geographic context for external IPs (location, ISP, AS number)
• Threat Detection: Identifies port scans, unusual traffic patterns, and suspicious activity
• Action Recommendations: Suggests security actions based on threat severity
• Dual Interface: Choose between Terminal UI (Rich-based TUI) or Web UI (Streamlit)
• Web Dashboard: Modern, intuitive web interface with real-time monitoring, interactive charts, and action management
• Playbook Editor: Create and customize response workflows directly from the browser
• Database Storage: SQLite database for storing threats, actions, and statistics
• Threat Filtering: Filter threats by severity, type, IP address, or date range
• Search Functionality: Search threats by description, IP, or event type
• Export Capabilities: Export threats and statistics to CSV or JSON format
• IP Management: Whitelist trusted IPs and blacklist known malicious IPs
• Configurable AI Analysis: Choose which threat severities to analyze with AI
• Agentic Suricata Integration: AI-driven automatic Suricata rule generation with permission prompts and safety controls
• Action Playbooks: Group multiple response steps (rule + log + webhook) into single approval prompts
• Webhook Notifications: Optional Slack/Teams integration for approved threats (opt-in, privacy-first)
• Interactive Approvals: Real-time CLI prompts for reviewing and approving AI-generated rules

Installation

1. Clone the repository:

git clone <repository-url>
cd AutoDefender_Hackathon

2. Install dependencies:

pip install -r requirements.txt

3. Ensure Ollama is installed and running locally:

# Install Ollama from https://ollama.ai
ollama pull <your-chosen-model>  # e.g., llama3, mistral, phi4-mini, etc.
ollama serve  # Start Ollama server

Note: On first run, AutoDefender will automatically create:

• autodefender.db - SQLite database for threats and actions (user-specific, not in repository)
• ip_lists.json - IP whitelist/blacklist (starts empty, user-specific, not in repository)
• suricata_rules/ - Custom Suricata rules directory

The demo database (demo/demo_config.db) is pre-populated and included in the repository - no setup needed to use it.

Step-by-Step Setup (Non-Technical Friendly)

1. Install the prerequisites

   • Python 3.10+
   • Git
   • Suricata IDS (follow the installer for your platform or use docs/SURICATA_SETUP.md)
   • Ollama for local AI models
   • Optional: Slack/MS Teams (or any webhook endpoint) if you want automated notifications (Microsoft Teams incoming webhooks work the same way as Slack)
   • Optional: Slack/MS Teams (or any webhook endpoint) if you want automated notifications

2. Prepare Suricata

   • Enable the eve.json output in suricata.yaml (already enabled by default)
   • Start Suricata on the interface you want to monitor:
     suricata -c suricata.yaml -i <interface>
   • Official docs: https://docs.suricata.io/
     Windows quick start: docs/SURICATA_SETUP.md

3. Start Ollama

   ollama serve
   ollama pull phi4-mini   # Example model

4. Clone AutoDefender and install Python requirements

   git clone <repository-url>
   cd AutoDefender_Hackathon
   pip install -r requirements.txt

5. Choose how you want to run AutoDefender

   • Streamlit UI (recommended for most people):
     python -m streamlit run streamlit_app.py

     1. The browser opens at http://localhost:8501
     2. Go to the Setup page and fill in:
        • Suricata eve.json path(s): the full path to eve.json (or multiple paths, one per line for aggregation)
        • Database path: leave default autodefender.db or point somewhere else
        • Ollama endpoint: http://127.0.0.1:11434
        • Ollama model: the model you pulled (e.g., phi4-mini)
     • (Optional) Notification webhook URL: paste the Slack/Teams webhook if you want approved actions to ping that channel
     • Optional: enable Suricata rule management and pick a rules directory (default ./suricata_rules)
     3. Click Save configuration. You can now navigate to Dashboard, Threat Analysis, etc.
     4. Need a quick demo? Use the Load demo configuration button on the Setup page. It auto-fills:
        • Log path: demo/example_suricata_log.json
        • Database: demo/demo_config.db
        • Ollama endpoint: http://127.0.0.1:11434
        • Model: phi4-mini
        • Rules dir: ./suricata_rules with dry-run enabled Save the form afterwards, and switch back to your real paths when ready.

   • Command-line interface (CLI):
     python main.py --monitor <path-to-eve.json>
     The CLI dashboard appears in the terminal and shows live stats. Use python main.py --help to see all options.

Usage

AutoDefender offers two interfaces: Command-Line Interface (CLI) and Web UI (Streamlit).

Web UI (Recommended for Interactive Use)

Start the web interface:

python -m streamlit run streamlit_app.py

Hosted Streamlit App: https://autodefenderhackathon.streamlit.app/

The UI will open at http://localhost:8501 and provides:

• Real-time dashboard with live threat monitoring
• Multi-source aggregation for monitoring multiple log files
• Threat analysis with filtering, search, and export tools
• Action management for approving AI-generated rules and playbooks
• Playbook Editor for customizing response workflows
• IP whitelist and blacklist management
• Settings for Suricata, Ollama, and database options
• Built-in documentation

First steps:

1. Complete the Setup page before navigating elsewhere. Provide the Suricata log path, database path, and Ollama details.
2. Set the AUTODEFENDER_UI_PASSWORD environment variable to require a password when launching the console.
3. After setup is marked complete, open the Dashboard to begin monitoring.

Additional guides now live under the docs/ directory, including:

• docs/SURICATA_SETUP.md for Suricata installation and configuration (Windows, Linux, Mac)
• docs/AGENTIC_GUIDE.md for AI-driven agentic automation features

Command-Line Interface (CLI)

Real-time Monitoring

Monitor a Suricata log file in real-time:

python main.py --monitor /var/log/suricata/eve.json

Tip: Add --read-log-from-start to process the entire file instead of tailing only new events.

Historical Analysis

Analyze one or more existing log files:

python main.py --analyze /path/to/log1.json /path/to/log2.json

Filtering Threats

Filter threats by severity:

python main.py --analyze demo/example_suricata_log.json --filter-severity HIGH CRITICAL

Search Threats

Search for specific threats:

python main.py --analyze demo/example_suricata_log.json --search "SSH"

AI Analysis Selection

Choose which threats to analyze with AI:

# Analyze only HIGH and CRITICAL threats with AI
python main.py --analyze demo/example_suricata_log.json --ai-severities HIGH CRITICAL

# Analyze all MEDIUM threats with AI
python main.py --analyze demo/example_suricata_log.json --filter-severity MEDIUM --ai-severities MEDIUM

Export Threats

Export filtered threats to CSV or JSON:

# Export to JSON
python main.py --analyze demo/example_suricata_log.json --filter-severity HIGH CRITICAL --export threats.json

# Export to CSV
python main.py --analyze demo/example_suricata_log.json --search "SSH" --export ssh_threats.csv

Using AI Models

Specify any Ollama model for AI explanations (required for AI features):

# Use llama3
python main.py --analyze demo/example_suricata_log.json --filter-severity HIGH --model llama3

# Use mistral
python main.py --analyze demo/example_suricata_log.json --model mistral

# Use phi4-mini
python main.py --analyze demo/example_suricata_log.json --model phi4-mini

# Use any installed Ollama model
python main.py --monitor /var/log/suricata/eve.json --model gemma:7b

IP Whitelist/Blacklist Management

Manage trusted and malicious IP addresses:

# Add IP to whitelist (threats from this IP will be ignored)
python main.py --whitelist 192.168.1.100

# Add IP to blacklist (threats from this IP will be auto-blocked)
python main.py --blacklist 10.0.0.50

# Remove from whitelist
python main.py --remove-whitelist 192.168.1.100

# Remove from blacklist
python main.py --remove-blacklist 10.0.0.50

# List all whitelisted and blacklisted IPs
python main.py --list-ips

Combined Operations

Combine multiple features:

# Filter, analyze with AI, and export
python main.py --analyze demo/example_suricata_log.json \
  --filter-severity HIGH CRITICAL \
  --ai-severities HIGH CRITICAL \
  --export high_priority_threats.json

Both Modes

Run real-time monitoring and historical analysis simultaneously:

python main.py --both /var/log/suricata/eve.json /backup/logs/

Demo Script

Run the interactive demo to see all features in action:

python demo/demo.py

Demo features:

• Threat detection and analysis
• AI-powered explanations
• Threat filtering and search
• Export functionality
• IP whitelist/blacklist management

Demo Configuration

The project ships with a built-in demo dataset that works on both localhost and Streamlit Cloud:

• demo/example_suricata_log.json - sample Suricata log file
• demo/demo_config.db - pre-populated demo SQLite database (included in repository)
• demo/log_replayer.py - optional tool to replay demo events

Note: The demo database (demo/demo_config.db) is included in the repository and contains example/test IP addresses only (no personal information). It's ready to use immediately.

In the Streamlit Setup page, click Load demo configuration to pre-fill:

• Suricata log path: demo/example_suricata_log.json
• Database path: demo/demo_config.db
• Ollama endpoint: http://127.0.0.1:11434
• Ollama model: phi4-mini
• Suricata rules directory: ./suricata_rules
• Suricata rule management enabled with dry-run mode

Review the fields and click Save configuration to apply. The demo works immediately on both localhost and the hosted Streamlit app. Switch back to your real paths afterwards to monitor live data.

Need more help with Suricata itself? Check the following resources:

• Official docs: https://docs.suricata.io/
• Windows quick start and troubleshooting: docs/SURICATA_SETUP.md
• General testing instructions with real Suricata logs: see docs/SURICATA_SETUP.md and the upstream Suricata documentation portal

Demo Database Features

The demo database (demo/demo_config.db) is pre-populated and included in the repository. It contains realistic sample threats showcasing all of AutoDefender's capabilities:

• GeoIP-enriched threats: Examples from Russia, Germany, USA with ISP and location data
• Diverse attack types: SSH brute force, Tor exit node scans, data exfiltration attempts, MITM attacks
• Playbook actions: Multi-step response workflows (drop rule + log + webhook) demonstrating action bundling
• Various action states: RECOMMENDED, EXECUTED, REJECTED, and FAILED actions for UI testing
• Rich AI explanations: Detailed, context-aware threat descriptions explaining what happened, why it matters, and what to do
• Safe test data: All IP addresses are example/test IPs (private ranges, TEST-NET, etc.) - no personal information

The demo database is ready to use immediately on both localhost and Streamlit Cloud. It includes 6 sample threats and 9 associated actions.

To refresh the demo database with new sample threats at any time (optional):

python tools/populate_demo_db.py

This command regenerates demo/demo_config.db with fresh sample data for testing and demonstrations.

Slack / Teams Webhook Example (Optional)

1. Create an Incoming Webhook:
   • Slack: open your workspace settings -> Integrations -> Incoming Webhook -> add new webhook and copy the URL (https://hooks.slack.com/services/...).
   • Microsoft Teams:
     1. In the Teams channel where you want alerts, click the ... menu next to the channel name.
     2. Choose Connectors -> search for Incoming Webhook -> click Configure.
     3. Give the webhook a friendly name (e.g., "AutoDefender Alerts") and optionally upload an icon.
     4. Click Create, then copy the URL (https://<region>.webhook.office.com/webhookb2/...).
2. Paste the webhook URL into the "Notification webhook URL" field on the Streamlit Setup page and click Save configuration.
3. Open Action Management. When you approve a playbook step that says WEBHOOK_NOTIFY, AutoDefender sends a simple JSON payload (same format for Slack and Teams) with threat details such as severity, source IP, and description. Example payload:

   {
     "text": "[Playbook] Critical SSH brute force response executed",
     "threat": {
       "id": 42,
       "severity": "CRITICAL",
       "source_ip": "192.168.1.101",
       "event_type": "alert",
       "description": "Suricata Alert: SSH brute force attempts"
     }
   }

   • Slack will display the text field automatically.
   • Microsoft Teams shows the same text content inside the channel message; you can optionally wrap it in an Adaptive Card by pointing the webhook to your own middleware.
4. To disable notifications, clear the webhook field and save the Setup form. No data leaves your machine unless you explicitly configure the webhook.

Agentic Enhancements

• Playbook Actions: AutoDefender bundles common responses (drop rule + log + webhook) into a single approval prompt. Approve once and all steps execute in order, keeping humans in the loop but reducing clicks.
• Playbook Editor: Customize response workflows from the Streamlit UI. Define conditions (severity, keywords) and action sequences without editing JSON files manually.
• Webhook Notifications: When you approve a playbook step with WEBHOOK_NOTIFY, the console sends a JSON payload to your configured webhook (e.g., Slack/Teams). Leave the webhook URL blank if you prefer to stay offline - no data leaves your machine by default.
• GeoIP Context: Public IPs are automatically enriched with location, ISP, and network data. This context appears in AI explanations and webhook notifications.
• Multi-Source Monitoring: Monitor multiple Suricata instances, archived logs, or distributed sensors by entering multiple file paths (one per line) in the Setup page.

Configuration

Edit config.py or create a config.ini file to customize:

• Suricata log file paths
• Ollama endpoint (default: http://localhost:11434)
• Database path
• Detection thresholds (port scan threshold, suspicious ports)
• Action policies (auto-approval settings)

Note: Use the --model flag to specify which Ollama model to use for AI features.

Environment Variables

# Set Ollama model (or use --model flag) - user must choose their model
export OLLAMA_MODEL=your-model-name

# Set Ollama endpoint
export OLLAMA_ENDPOINT=http://localhost:11434

# Suricata integration (optional)
export SURICATA_ENABLED=true
export SURICATA_RULES_DIR=./suricata_rules
export AUTO_APPROVE_SURICATA=false
export SURICATA_DRY_RUN=false
export WEBHOOK_URL=https://your-webhook-url

Quick Reference (Non-Technical)

• Try online: https://autodefenderhackathon.streamlit.app/ - Demo database is pre-loaded and ready to use
• Start Suricata: open PowerShell -> cd "C:\Program Files\Suricata" -> .\suricata.exe -c suricata.yaml -i "Wi-Fi"
• Run AutoDefender UI: in the project folder -> python -m streamlit run streamlit_app.py
• Run CLI monitor: python main.py --monitor "C:\Program Files\Suricata\log\eve.json" --model phi4-mini
• Load demo data: Setup page -> "Load demo configuration" -> Save (works on both localhost and Streamlit Cloud)
• Replay demo log (optional): python demo/log_replayer.py demo/example_suricata_log.json --interval 0.5 --loop
• Refresh demo database (optional): python tools/populate_demo_db.py - Note: demo database is pre-populated in the repository
• Enable Slack/Teams alerts: paste your webhook URL into the Setup page, then approve a WEBHOOK_NOTIFY action in Action Management.

Suricata Integration (Agentic Features)

AutoDefender can automatically generate and manage Suricata rules based on detected threats using AI.

Features

• AI-Driven Rule Generation: Uses Ollama to generate context-aware Suricata drop rules
• Permission Prompts: Requires manual approval by default (can be configured for auto-approval)
• Interactive CLI Workflow: Prompts appear in the terminal to approve or reject each AI-generated rule
• Dry-Run Mode: Test rule generation without making changes
• Automatic Backups: Creates timestamped backups before modifying rule files
• Path Validation: Only modifies files in safe, app-controlled directories
• Rule Validation: Validates rule syntax before writing
• Real-Time Integration: Processes HIGH/CRITICAL threats immediately

Configuration

Via Environment Variables:

export SURICATA_ENABLED=true                    # Enable Suricata integration
export SURICATA_RULES_DIR=./suricata_rules      # Path to rules directory
export AUTO_APPROVE_SURICATA=false              # Require manual approval (recommended)
export SURICATA_DRY_RUN=false                   # Dry-run mode (test without executing)

Via config.ini:

[suricata]
enabled = true
rules_dir = ./suricata_rules
auto_approve = false
dry_run = false

Usage

1. Enable Suricata Integration:

export SURICATA_ENABLED=true
python main.py --monitor /var/log/suricata/eve.json

With AUTO_APPROVE_SURICATA=false, AutoDefender will pause to show an interactive approval prompt for each AI-generated rule.

2. Dry-Run Mode (Test Without Executing):

export SURICATA_ENABLED=true
export SURICATA_DRY_RUN=true
python main.py --monitor /var/log/suricata/eve.json

3. Auto-Approve Mode (Advanced):

export SURICATA_ENABLED=true
export AUTO_APPROVE_SURICATA=true  # Use with caution
python main.py --monitor /var/log/suricata/eve.json

Manual Approval Workflow

When AUTO_APPROVE_SURICATA is disabled (default), threats that qualify for Suricata remediation trigger an interactive prompt:

1. AutoDefender displays the full AI-generated rule along with context about the triggering threat.
2. Press y to approve (the rule is written to the Suricata rules file) or n to reject.
3. Batch Approval: If 3+ actions are pending, you'll be offered the option to approve/reject all at once or review individually.
4. The dashboard's Pending Agentic Actions panel updates in real time to reflect approvals/rejections.
5. Backups are created automatically before each approved rule is written.

This workflow keeps humans in the loop while still benefiting from real-time AI triage.

How It Works

1. Threat Detection: AutoDefender detects HIGH or CRITICAL threat
2. AI Analysis: AI generates a Suricata drop rule based on threat context
3. Permission Prompt: System asks for approval (unless auto-approve is enabled)
4. Rule Execution: Upon approval, rule is added to custom rules file with automatic backup
5. Dashboard Display: Pending actions shown in real-time dashboard

Safety Features

• Default to Manual Approval: Requires user confirmation before executing rules
• Batch Approval: Efficiently approve/reject multiple pending actions at once (3+ actions)
• Dry-Run Mode: Test rule generation without making changes
• Automatic Backups: Timestamped backups created before each modification
• Path Validation: Only modifies files within app-controlled directories
• Rule Validation: Checks rule syntax before writing
• Master Switch: SURICATA_ENABLED flag to disable all operations
• Audit Logging: All actions logged to database for review
• Health Monitoring: Automatic checks of rules directory, disk space, and file permissions
• Restart Notifications: Dashboard alerts when Suricata needs restart to apply new rules

Windows Support

Suricata integration works on Windows using file-based rule management:

• Rules are written to custom rules file immediately
• Dashboard displays a restart banner when new rules have been added
• Suricata must be restarted or reloaded to pick up new rules
• No control socket (suricatasc) required
• Health monitoring tracks rules file status and disk space

Project Structure

AutoDefender_Hackathon/
|-- main.py                 # CLI entry point
|-- config.py               # Configuration management
|-- analyzer.py             # Historical analysis
|-- monitor.py              # Real-time monitoring
|-- parser.py               # Suricata JSON log parsing
|-- detector.py             # Threat detection engine
|-- ai_explainer.py         # AI integration (Ollama)
|-- action_engine.py        # Action recommendations
|-- suricata_manager.py     # Suricata rule file management
|-- approval_handler.py     # Permission prompt system
|-- database.py             # SQLite database operations
|-- filter.py               # Threat filtering and search
|-- exporter.py             # Export functionality (CSV/JSON)
|-- ip_manager.py           # IP whitelist/blacklist management
|-- ui/                     # Terminal dashboard components
|   |-- dashboard.py        # CLI dashboard with threat/stats panels
|-- streamlit_app.py        # Streamlit entry point
|-- streamlit_pages/        # Streamlit page implementations
|-- demo/
|   |-- demo.py             # Interactive demo script
|   |-- demo_config.db      # Pre-populated demo database (included in repository)
|   |-- example_suricata_log.json
|   |-- log_replayer.py     # Utility to replay events into eve.json
|   |-- generated/          # Temporary files created during demo
|   |-- outputs/            # Demo export artifacts
|-- docs/                   # Additional guides and references
|   |-- SURICATA_SETUP.md   # Suricata installation and configuration
|   |-- AGENTIC_GUIDE.md    # AI-driven agentic automation guide
|-- requirements.txt        # Python dependencies
|-- README.md               # This file

Security & Privacy

• Local Processing: All analysis runs locally - no data sent to external services
• SQL Injection Protection: All database queries use parameterized statements
• Input Validation: All user inputs are validated and sanitized
• File Permissions: Database and IP list files use appropriate permissions
• No Data Collection: No telemetry or usage data is collected
• Secure Storage: Sensitive data stored in local SQLite database with proper access controls

Requirements

• Python 3.10+
• Ollama (local installation)
• Suricata log files (JSON format - eve.json)

Troubleshooting

Ollama Connection Issues

If you see "model not found" errors:

# Check if Ollama is running
ollama list

# Start Ollama if not running
ollama serve

# Pull the model you want to use (choose any model you prefer)
ollama pull <your-model-name>

Database Errors

If you encounter database errors:

• Check file permissions on the database file
• Ensure sufficient disk space
• Verify the database file isn't locked by another process

Log File Issues

If log files aren't being processed:

• Verify the file path is correct
• Check file permissions (read access required)
• Ensure the file is valid JSON format
• Check that Suricata is writing to the file

Examples

Example 1: Quick Threat Analysis

python main.py --analyze demo/example_suricata_log.json

Example 2: Focus on High-Priority Threats

python main.py --analyze demo/example_suricata_log.json \
  --filter-severity HIGH CRITICAL \
  --ai-severities HIGH CRITICAL \
  --export critical_threats.json

Example 3: Search and Export

python main.py --analyze demo/example_suricata_log.json --search "SSH" --export ssh_threats.json