adithya-s-k/omniparse

# OmniParse  [](https://github.com/adithya-s-k/omniparse/stargazers) [](https://github.com/adithya-s-k/omniparse/network/members) [](https://github.com/adithya-s-k/omniparse/issues) [](https://github.com/adithya-s-k/omniparse/pulls) [](https://github.com/adithya-s-k/omniparse/blob/main/LICENSE) ## Try it out [](https://colab.research.google.com/github/adithya-s-k/omniparse/blob/main/examples/OmniParse_GoogleColab.ipynb) ## Intro https://github.com/adithya-s-k/omniparse/assets/27956426/457d8b5b-9573-44da-8bcf-616000651a13 ## Features ✅ Completely local, no external APIs \ ✅ Fits in a T4 GPU \ ✅ Supports ~20 file types \ ✅ Convert documents, multimedia, and web pages to high-quality structured markdown \ ✅ Table extraction, image extraction/captioning, audio/video transcription, web page crawling \ ✅ Easily deployable using Docker and Skypilot \ ✅ Colab friendly \ ✅ Interative UI powered by Gradio ### Why OmniParse ? It's challenging to process data as it comes in different shapes and sizes. OmniParse aims to be an ingestion/parsing platform where you can ingest any type of data, such as documents, images, audio, video, and web content, and get the most structured and actionable output that is GenAI (LLM) friendly. ## Installation git clone https://github.com/adithya-s-k/omniparse cd omniparse Create a Virtual Environment: conda create -n omniparse-venv python=3.10 conda activate omniparse-venv Install Dependencies: poetry install # or pip install -e . # or pip install -r pyproject.toml ### 🛳️ Docker To use OmniParse with Docker, execute the following commands: 1. Pull the OmniParse API Docker image from Docker Hub: 2. Run the Docker container, exposing port 8000: 👉🏼[Docker Image](https://hub.docker.com/r/savatar101/omniparse) docker pull savatar101/omniparse:0.1 # if you are running on a gpu docker run --gpus all -p 8000:8000 savatar101/omniparse:0.1 # else docker run -p 8000:8000 savatar101/omniparse:0.1 Alternatively, if you prefer to build the Docker image locally: Then, run the Docker container as follows: docker build -t omniparse . # if you are running on a gpu docker run --gpus all -p 8000:8000 omniparse # else docker run -p 8000:8000 omniparse ## Usage Run the Server: python server.py --host 0.0.0.0 --port 8000 --documents --media --web - `--documents`: Load in all the models that help you parse and ingest documents (Surya OCR series of models and Florence-2). - `--media`: Load in Whisper model to transcribe audio and video files. - `--web`: Set up selenium crawler. Download Models: If you want to download the models before starting the server python download.py --documents --media --web - `--documents`: Load in all the models that help you parse and ingest documents (Surya OCR series of models and Florence-2). - `--media`: Load in Whisper model to transcribe audio and video files. - `--web`: Set up selenium crawler. ## Supported Data Types | Type | Supported Extensions | |-----------|-----------------------------------------------------| | Documents | .doc, .docx, .pdf, .ppt, .pptx | | Images | .png, .jpg, .jpeg, .tiff, .bmp, .heic | | Video | .mp4, .mkv, .avi, .mov | | Audio | .mp3, .wav, .aac | | Web | dynamic webpages, http://.com |

API Endpoints

- [API Endpoints](#api-endpoints) - [Document Parsing](#document-parsing) - [Parse Any Document](#parse-any-document) - [Parse PDF](#parse-pdf) - [Parse PowerPoint](#parse-powerpoint) - [Parse Word Document](#parse-word-document) - [Media Parsing](#media-parsing) - [Parse Any Media](#parse-any-media) - [Parse Image](#parse-image) - [Process Image](#process-image) - [Parse Video](#parse-video) - [Parse Audio](#parse-audio) - [Website Parsing](#website-parsing) - [Parse Website](#parse-website) ### Document Parsing #### Parse Any Document Endpoint: `/parse_document` Method: POST Parses PDF, PowerPoint, or Word documents. Curl command: curl -X POST -F "file=@/path/to/document" http://localhost:8000/parse_document #### Parse PDF Endpoint: `/parse_document/pdf` Method: POST Parses PDF documents. Curl command: curl -X POST -F "file=@/path/to/document.pdf" http://localhost:8000/parse_document/pdf #### Parse PowerPoint Endpoint: `/parse_document/ppt` Method: POST Parses PowerPoint presentations. Curl command: curl -X POST -F "file=@/path/to/presentation.ppt" http://localhost:8000/parse_document/ppt #### Parse Word Document Endpoint: `/parse_document/docs` Method: POST Parses Word documents. Curl command: curl -X POST -F "file=@/path/to/document.docx" http://localhost:8000/parse_document/docs ### Media Parsing