Silwalkg/cyberwatch-siem

# CyberWatch Pro - SIEM Dashboard A comprehensive Security Information and Event Management (SIEM) platform for real-time threat monitoring, event correlation, and incident response. ## 🎯 Features ### Core SIEM Capabilities - **Log Ingestion**: Syslog receiver (UDP/TCP) for collecting logs from network devices, firewalls, IDS/IPS systems - **Multi-Format Parsing**: Support for syslog, CEF, JSON, firewall logs, IDS alerts, and application logs - **Event Normalization**: Standardized event schema across all log sources - **Event Correlation**: Rule-based threat detection engine identifying security incidents - **Real-Time Alerting**: Immediate notifications for critical security events - **Incident Management**: Grouping related events into security incidents ### Dashboard Features - **Real-Time Overview**: KPI cards showing total events, critical alerts, threats blocked, system uptime - **Live Event Stream**: Filterable log stream with severity-based color coding - **Threat Intelligence**: Global attack origin map with animated threat indicators - **Network Monitoring**: Traffic analysis, protocol distribution, port scan detection - **Alert Management**: Alert acknowledgment workflow and incident tracking - **Forensic Investigation**: Pivot capabilities to explore related events by IP, user, or hostname ### Security Features - **Role-Based Access Control**: Admin, analyst, and viewer roles - **Audit Logging**: Complete audit trail for compliance - **Authentication**: JWT-based API authentication - **Data Encryption**: TLS/SSL support for data in transit - **Compliance Ready**: SOC2, HIPAA, PCI-DSS compliance framework ## 🏗️ Architecture ┌─────────────────────────────────────────────────────────────┐ │ Frontend (Web UI) │ │ - Dashboard with real-time charts │ │ - Event filtering and search │ │ - Alert management interface │ └────────────────────┬────────────────────────────────────────┘ │ REST API / WebSocket ┌────────────────────▼────────────────────────────────────────┐ │ FastAPI Backend (Python) │ │ - REST API endpoints │ │ - Event correlation engine │ │ - Alert generation and management │ │ - User authentication & RBAC │ └────────────────────┬────────────────────────────────────────┘ │ ┌────────────┼────────────┐ │ │ │ ┌───────▼──┐ ┌──────▼──┐ ┌─────▼──────┐ │ Syslog │ │PostgreSQL│ │ Redis │ │ Receiver │ │ Database │ │ Cache │ └──────────┘ └──────────┘ └────────────┘ ## 🚀 Quick Start ### Prerequisites - Docker & Docker Compose (recommended) - Python 3.11+ (for local development) - PostgreSQL 15+ (for production) ### Option 1: Docker Compose (Recommended) # Clone the repository git clone https://github.com/yourusername/siem-dashboard.git cd siem-dashboard # Start all services docker-compose up -d # Access the dashboard # Frontend: http://localhost # API Docs: http://localhost:8000/docs # Syslog: localhost:514 (UDP/TCP) ### Option 2: Local Development # Install backend dependencies cd backend pip install -r requirements.txt # Set up environment cp .env.example .env # Edit .env with your configuration # Initialize database python -c "from database import init_db; init_db()" # Start the backend uvicorn main:app --reload # In another terminal, start syslog receiver python syslog_receiver.py # Open frontend in browser # http://localhost:8000 (served by backend) ## 📊 API Endpoints ### Events - `GET /api/events` - List events with filtering - `GET /api/events/{id}` - Get event details - `GET /api/events/source/{ip}` - Get events from source IP - `GET /api/events/stats` - Get event statistics - `POST /api/events` - Ingest new event ### Alerts - `GET /api/alerts` - List alerts - `GET /api/alerts/{id}` - Get alert details - `PATCH /api/alerts/{id}` - Update alert status - `GET /api/alerts/stats` - Get alert statistics ### Incidents - `GET /api/incidents` - List incidents - `GET /api/incidents/{id}` - Get incident details - `PATCH /api/incidents/{id}` - Update incident status ### Rules - `GET /api/rules` - List correlation rules - `PATCH /api/rules/{id}` - Enable/disable rule ### Health - `GET /health` - Health check ## 🔧 Configuration ### Environment Variables # Database DATABASE_URL=postgresql://user:password@localhost:5432/siem_db # API API_HOST=0.0.0.0 API_PORT=8000 # Syslog SYSLOG_HOST=0.0.0.0 SYSLOG_PORT=514 # Security SECRET_KEY=your-secret-key ALGORITHM=HS256 # Logging LOG_LEVEL=INFO ### Correlation Rules Rules are stored in the database and can be managed via API. Default rules include: 1. **Brute Force SSH Attack** - 5+ failed SSH attempts in 5 minutes 2. **SQL Injection Attempt** - SQL injection pattern detected 3. **Port Scan Detection** - 10+ port scan attempts in 2 minutes 4. **Privilege Escalation** - Unauthorized privilege escalation 5. **Data Exfiltration** - Unusual outbound data transfer Create custom rules by adding entries to the `correlation_rules` table. ## 📝 Log Ingestion ### Syslog Format Send logs to `localhost:514` (UDP or TCP): # Example: Send syslog from Linux logger -n localhost -P 514 "SSH Brute Force: 5 failed attempts from 192.168.1.100" # Example: Send from firewall echo "<34>Jan 15 10:30:45 firewall ALLOW TCP 192.168.1.100:54321 -> 8.8.8.8:53 (DNS)" | nc -u localhost 514 ### JSON Format POST to `/api/events`: curl -X POST http://localhost:8000/api/events \ -H "Content-Type: application/json" \ -d '{ "timestamp": "2024-01-15T10:30:45Z", "source_ip": "192.168.1.100", "destination_ip": "8.8.8.8", "event_type": "dns_query", "severity": "low", "message": "DNS query to 8.8.8.8", "source_system": "dns_server" }' ## 🔍 Event Correlation The correlation engine detects security incidents by: 1. **Threshold-Based Detection**: Triggers when event count exceeds threshold in time window 2. **Sequence Detection**: Identifies event sequences (e.g., failed login → successful login) 3. **Anomaly Detection**: Detects deviations from baseline behavior Example: Brute force detection Event 1: Failed SSH login from 192.168.1.100 (10:00:00) Event 2: Failed SSH login from 192.168.1.100 (10:00:15) Event 3: Failed SSH login from 192.168.1.100 (10:00:30) Event 4: Failed SSH login from 192.168.1.100 (10:00:45) Event 5: Failed SSH login from 192.168.1.100 (10:01:00) → Alert: SSH Brute Force Attack (5 attempts in 5 minutes) → Incident: Created and linked to all events ## 📈 Dashboards ### Overview - KPI cards with 24-hour trends - Event timeline chart - Threat severity distribution gauge - Top attack sources table ### Live Events - Real-time event stream - Severity-based filtering - Search by IP, type, protocol - Auto-refresh toggle ### Threat Intelligence - Global attack origin map - CVE feed - Threat category distribution - Attack pattern analysis ### Network Monitor - Network traffic trends - Protocol distribution - Port scan detections - Network health status ### Alerts & Incidents - Active alert cards - Incident timeline - Alert acknowledgment workflow - Incident status tracking ## 🔐 Security Best Practices 1. **Change default credentials** in production 2. **Use strong SECRET_KEY** for JWT signing 3. **Enable TLS/SSL** for all communications 4. **Implement rate limiting** on API endpoints 5. **Regular database backups** for disaster recovery 6. **Monitor SIEM system itself** for tampering 7. **Rotate API keys** regularly 8. **Implement network segmentation** for SIEM infrastructure ## 📚 SIEM Concepts ### Log Aggregation Collecting logs from multiple sources (servers, firewalls, IDS, applications) into a central repository for analysis. ### Event Normalization Converting logs from different sources into a standardized format with consistent field names and values. ### Event Correlation Analyzing multiple events to identify patterns indicating security incidents. Example: Multiple failed logins followed by successful login = suspicious activity. ### Threat Intelligence Enriching events with external threat data (GeoIP, reputation feeds, CVE databases) to assess risk. ### Incident Response Coordinating response to detected security incidents through alerts, escalation, and remediation workflows. ## 🔄 Comparison to Commercial SIEM | Feature | CyberWatch Pro | Splunk | Elastic Stack | ArcSight | |---------|---|---|---|---| | Log Ingestion | ✓ | ✓ | ✓ | ✓ | | Event Correlation | ✓ | ✓ | ✓ | ✓ | | Real-Time Alerting | ✓ | ✓ | ✓ | ✓ | | Dashboarding | ✓ | ✓ | ✓ | ✓ | | Cost | Free | $$$$ | Free/$$$ | $$$$ | | Scalability | Medium | Enterprise | Enterprise | Enterprise | | Learning Curve | Low | High | Medium | High | ## 🚨 Limitations 1. **Log Quality**: SIEM effectiveness depends on quality and completeness of ingested logs 2. **Rule Quality**: Detection accuracy depends on correlation rule quality 3. **False Positives**: Poorly tuned rules generate alert fatigue 4. **Storage**: Large-scale deployments require significant storage capacity 5. **Performance**: Real-time correlation on millions of events requires optimization 6. **Expertise**: Effective SIEM operation requires skilled security analysts ## 📖 Documentation - [Architecture Guide](./docs/ARCHITECTURE.md) - [API Reference](./docs/API.md) - [Deployment Guide](./docs/DEPLOYMENT.md) - [Rule Development](./docs/RULES.md) - [Troubleshooting](./docs/TROUBLESHOOTING.md) ## 📄 License MIT License - See LICENSE file for details ## 🎓 Learning Resources - [SIEM Fundamentals](https://www.sans.org/white-papers/) - [Log Analysis Best Practices](https://www.nist.gov/) - [Incident Response Guide](https://www.cisa.gov/) - [Threat Intelligence](https://www.mitre.org/attack/) **CyberWatch Pro** - Enterprise-Grade SIEM for Security Teams