TheHiddenOwl/NightWatch

# NIGHTWATCH — Behavioral IDS NIGHTWATCH is a modular, behavioral-based Intrusion Detection System (IDS) designed for low-and-slow threat detection. It monitors network traffic, audit logs, and DNS queries to build a baseline of normal behavior and flag anomalies using machine learning. ## Features - **Multi-Source Collection:** - **Packet Capture:** Real-time network sniffing via Scapy. - **Audit Ingestion:** Regex-based parsing of `audit.log` for process-to-network correlation. - **DNS Monitoring:** Ingestion of `dnsmasq` query logs to detect tunneling and rare domain lookups. - **Behavioral Baselining:** - Unsupervised machine learning using `IsolationForest`. - Periodic retraining of models per-host and per-hour to handle time-of-day variations. - **Event Correlation:** - Link packet spikes, high-entropy DNS queries, and off-hours process activity into unified alert clusters. - Cross-source escalation for high-confidence detections. - **Flexible Alerting:** - Built-in support for Slack webhooks, PagerDuty Events API, and Email notifications. - Rate-limiting and deduplication to prevent alert fatigue. - **Operational Ready:** - **Learning Mode:** Collect and score events without sending external alerts to tune sensitivity. - **Mock Mode:** Replay PCAP and log files for testing and staging. - **Structured Logging:** JSON-formatted logs for SIEM integration. ## Technology Stack - **Language:** Python 3.11+ - **Machine Learning:** `scikit-learn` (Isolation Forest), `pandas` - **Networking:** `scapy` - **Storage:** `SQLAlchemy` + `SQLite` - **Alerting:** `httpx` (Async), `smtplib` - **Configuration:** `PyYAML` ## Installation 1. **Clone the repository:** git clone https://github.com/your-repo/nightwatch.git cd nightwatch 2. **Install dependencies:** pip install -r requirements.txt 3. **(Optional) Install dev tools:** pip install -r requirements-dev.txt ## Setup 1. **Initialize the Database:** python nightwatch.py --init-db 2. **Configure the system:** Edit `config.yaml` to set your network interfaces, log paths, and alerting credentials. 3. **Generate Sample Data (for testing):** python tools/generate_synthetic_data.py ## Usage ### Running the Daemon **Foreground Mode (Development):** python nightwatch.py --foreground --mode mock **Production (Live):** python nightwatch.py --mode live ### Management Commands - **Check Database Status:** python nightwatch.py --db-status - **Review Learning Mode Alerts:** python nightwatch.py --review-learning-alerts - **Train Models Manually:** python -c "from nightwatch.store.db import DBManager; from nightwatch.analysis.baseline import BaselineEngine; db = DBManager('nightwatch.db'); be = BaselineEngine(db); be.train_models()" ### Deployment A systemd unit file is provided in `deploy/nightwatch.service`. Copy it to `/etc/systemd/system/` and grant `CAP_NET_RAW` to the python binary if running as a non-root user. ## License MIT