Vai-bhv/network-monitoring-dashboard

# 📡 网络监控仪表板 一个使用 **Python、Elasticsearch 和 Kibana** 构建的端到端网络可观测性管道。 该项目通过收集、处理、分析和可视化网络性能指标，模拟真实的网络监控系统。 ## 🚀 概述 该项目展示了现代监控系统的工作原理： 它提供以下洞察： - 网络延迟 - 吞吐量性能 - 丢包率 - 流量体积 - 系统健康与异常 ## ⚙️ 功能 - 📊 网络指标的时间序列分析 - 🚨 使用统计阈值 + 孤立森林进行异常检测 - 📈 可交互的 Kibana 仪表板 - 🧠 KPI 监控（延迟、吞吐量、丢包、流量、异常） - 🔍 设备级与区域级洞察 - ⚡ 使用 `bootstrap.sh` 的自动化管道 ## 🏗️ 技术栈 - **Python**（Pandas、NumPy、Scikit-learn） - **Elasticsearch**（数据索引与查询） - **Kibana**（可视化与仪表板） - **Docker Compose**（基础设施） - **Shell 脚本**（自动化） ## 📁 项目结构 networkMonitoringDashboard/ │ ├── data/ │ ├── network_metrics.csv │ └── network_metrics_processed.csv │ ├── scripts/ │ ├── generate_sample_data.py # 数据生成（提取） │ ├── process_metrics.py # 处理 + 异常检测（转换） │ └── ingest_to_elasticsearch.py # 索引到 Elasticsearch（加载） │ ├── docker-compose.yml ├── requirements.txt ├── bootstrap.sh # 一键设置 + 管道 └── README.md ## ⚡ 快速启动（推荐） 使用 **单个命令** 运行所有内容： ``` chmod +x bootstrap.sh ./bootstrap.sh This will: Create virtual environment Install dependencies Start Elasticsearch & Kibana Generate dataset Process metrics Ingest data into Elasticsearch 🌐 Access Dashboard After setup: 👉 Open Kibana: http://localhost:5601 Then: Go to Dashboards Open your saved dashboard 🔁 Manual Execution (Optional) If you want to run step-by-step: # 启动服务 docker compose up -d # 激活环境（可选） source venv/bin/activate # 运行管道 python scripts/generate_sample_data.py python scripts/process_metrics.py python scripts/ingest_to_elasticsearch.py 📊 Dashboard Overview The dashboard is structured like a real monitoring system: 🔝 KPI Panels (Top Row) High-level system health indicators: Average Latency → overall network delay Average Throughput → data transfer efficiency Average Packet Loss → reliability metric Total Traffic Volume → network usage Total Anomalies → detected issues in system 👉 These provide an instant snapshot of system health 📈 Time-Series Analysis Average Latency Over Time Shows latency trends across time Helps identify spikes or instability Average Throughput Over Time Tracks bandwidth performance Detects performance degradation Traffic Volume Over Time Displays traffic spikes and usage patterns Useful for capacity planning 📉 Comparative Analysis Average Packet Loss by Region Compares network reliability across regions Helps identify problematic geographic zones 🚨 Anomaly Analysis Anomalies by Device Shows which devices generate the most anomalies Helps identify unstable infrastructure 🟢 System Health Healthy vs Degraded Status Displays ratio of healthy vs degraded states Quick system-wide health overview 📋 Incident Analysis Top Severity Events Table Lists highest severity network incidents Includes: Device Region Latency spikes Packet loss Traffic surges Sorted by severity for prioritization 🧠 Anomaly Detection Two approaches used: Rule-based detection High latency High packet loss Machine Learning (Isolation Forest) Detects abnormal traffic patterns 🔄 Data Pipeline The pipeline follows standard data engineering stages: Extract: scripts/generate_sample_data.py Generate synthetic network data Transform: scripts/ingest_to_elasticsearch.py Feature engineering Anomaly detection Load : scripts/process_metrics.py Index data into Elasticsearch ```

标签：Cutter, Docker Compose, Elasticsearch数据索引, Kibana可视化, KPI监控, NumPy, Python数据分析, Scikit-learn, SEO: 实时网络分析, SEO: 端到端网络监控, SEO: 网络监控仪表盘, Shell脚本, 丢包率监控, 交互式仪表盘, 区域级监控, 吞吐量监控, 孤立森林, 延迟监控, 异常检测, 时间序列分析, 版权保护, 端到端监控, 系统健康监控, 网络观测, 自动化数据流水线, 设备级监控, 越狱测试, 逆向工具