fivetran-jasonchletsos/LayoffOutage-ODI-Demo

GitHub: fivetran-jasonchletsos/LayoffOutage-ODI-Demo

Stars: 0 | Forks: 0

# LayoffOutage-ODI-Demo A reproducible cross-reference between corporate layoff events and public status-page incident histories. For each layoff at a tracked tech company, the pipeline counts incidents in the 90-day windows before and after, then renders the result as a scatter chart, sortable table, and per-company timeline overlays. ## Background This was built in response to a LinkedIn prompt from Connor Dickson (Analytics Engineer): Jason Chletsos (Fivetran Sales Engineering) replied "I'll try to build it." This is the build. ## Data sources ## Repo layout LayoffOutage-ODI-Demo/ ├── etl/ │ └── build.py # one-shot Python ETL: fetches sources, writes JSON ├── app/ # Vite + React + TypeScript viewer │ ├── src/ │ │ ├── App.tsx │ │ ├── pages/ │ │ │ ├── HomePage.tsx │ │ │ ├── CompanyPage.tsx │ │ │ ├── MethodologyPage.tsx │ │ │ ├── ArchitecturePage.tsx │ │ │ └── SourcesPage.tsx │ │ └── types.ts │ └── public/data/ # generated JSON files (committed) ├── docs/ # reserved for future └── README.md ## ETL outputs `etl/build.py` writes five JSON files into `app/public/data/`: - `layoffs.json` — every layoff event at a tracked company - `incidents.json` — per-company list of incident dates - `companies.json` — per-company rollup with monthly incident counts and layoff events - `correlations.json` — per-event before/after counts and delta - `meta.json` — provenance, source URLs, caveats The JSON files are committed so the static viewer works without re-running the ETL. ## How to run git clone https://github.com/fivetran-jasonchletsos/LayoffOutage-ODI-Demo cd LayoffOutage-ODI-Demo # Refresh the dataset python3 etl/build.py # Run the viewer cd app && npm install && npm run dev ## Architecture This is one of 38 demos in the Fivetran ODI (Open Data Infrastructure) portfolio. Each demo in the portfolio follows the same medallion shape: Sources, then Fivetran connectors, then Bronze (Iceberg on S3), then dbt, then Silver/Gold, then Snowflake / Athena / DuckDB. Sister repos in the portfolio: - Hub catalog: `Fivetran-Demo-Repository` - Concepts and intro: `00-Intro-ODI-Demo` - Observability: `ODI-Mission-Control` ## Methodology and caveats Read these before drawing conclusions from the charts. - **Correlation, not causation.** Incident counts before vs after a layoff are descriptive. A change in either direction does not imply the layoff caused (or prevented) outages. - **Status-page severity is not normalized.** A "minor" Stripe incident counts equally with a "major" Discord incident. Incident counts are a coarse proxy for reliability. - **Atom feeds cap at ~25 most-recent incidents.** For high-incident companies that is roughly six months of history. Many layoff events from 2022 and 2023 sit outside the data window and therefore have incomplete or empty "before" buckets. - **layoffs.fyi is community-sourced** with selection bias toward layoffs that get press coverage. Smaller or unreported events are missing. - **Tracked-company coverage is limited** to companies that publish an Atlassian Statuspage-compatible Atom feed. Companies on other status-page platforms, or with no public status page, are not included. The methodology page on the live site has the full discussion. ## License ## Credit - **Data:** Roger Lee (layoffs.fyi), and the open status-page feeds published by Atlassian, Slack, Stripe, Shopify, Twilio, Cloudflare, Heroku, Notion, Discord, DigitalOcean, Datadog, Snowflake, and the 30-plus other companies whose public history feeds make this dataset possible. - **Prompt:** Connor Dickson on LinkedIn (the original "I want someone to make..." post). - **Build:** Jason Chletsos, Sales Engineering at Fivetran. Built with Claude Code.
标签:自动化攻击