eCrimeLabs/cratos-fastapi
GitHub: eCrimeLabs/cratos-fastapi
Stars: 13 | Forks: 3
[
.yaml | This contains the configuration files related to each MISP instance |
### config/config.yaml
Here is a oneliner that can be used to to create an encryption key
openssl rand -base64 4096 | sha256sum | tr '[:lower:]' '[:upper:]'
Copy the SHA256 string into the "encryption_key" field.
for the "salt" it will be similar:
< /dev/urandom tr -dc 'A-Za-z0-9!#?' | head -c 32; echo
Now update the "config/config.yaml and save.
---
debug: False
encryption_key: ""
salt: ""
memcached_user: ""
memcached_pass: ""
memcached_host: "127.0.0.1"
memcached_port: "11211"
allways_allowed_ips:
- "100.64.3.0/24"
- "10.0.0.0/8"
- "172.16.0.0/12"
- "192.168.0.0/16"
***NOTICE: The "allways_allowed_ips" is globally set for all sites, these will typically be your monitoring setup to ensure that the service is running, the site specific if defined further down.***
## Configuring your first MISP connection config
In the folder sites you generate a file with the name "\.yaml" this can also be "\.yaml" but it has to map the MISP instance, as it is used as one of the validators if Cratos FastAPI is allowed to connect to this instance and how.
The configuraiton files is located in the folder "sites"
### misp.example.net
So in this scenario our MISP instance is "misp.example.net" so we create the file in the sites folder:
$ touch sites/misp.example.net.yaml
Now edit the file:
---
enabled: true
debug: false
company: Example ApS
tag: example
mispVerifyCert: true
mispTimeoutSeconds: 100
mispDebug: true
memcached_all_timeout: 300
falsepositive_timeout: "1w"
list_stats: "1w"
allowed_ips:
- "10.0.0.0/8"
- "127.0.0.1/32"
- "192.168.1.0/24"
custom_feeds:
cust1: ":incident-classification=cust1"
cust2: ":incident-classification=cust2"
cust3: ":incident-classification=cust3"
cust4: ":incident-classification=cust4"
cust5: ":incident-classification=cust5"
The field from the "tag" combined with som build-in feeds and 5 custom feeds will be mapped towards the tagging system, so with this setup you will have to following tags that map to these overall feed groups.
| feed name/group | MISP tags you must create (Remember to lock the tags to your organization) |
| --------------- | ------------------------------------------------------------------- |
| incident | example:incident-classification=incident |
| alert | example:incident-classification=alert |
| block | example:incident-classification=block |
| hunt | example:incident-classification=hunt |
| cust1 | example:incident-classification=sinkhole |
| cust2 | example:incident-classification=permanent-block |
| cust3 | example:incident-classification=tor-exitnode |
| cust4 | example:incident-classification=cust4 |
| cust5 | example:incident-classification=cust5 |
## Installing system dependencies
$ sudo apt install nginx libnginx-mod-http-headers-more-filter supervisor gcc openssl libssl-dev python3-dev python3-venv
## Configure reverse proxy settings if used
In the 'config.yaml' file there are the following two options
- reverse_proxy (Boolean)
- reverse_proxy_header (String)
in the event that no reverse proxy is used in front of the API set the "reverse_proxy: False", else if "True" ensure that the
correct reverse proxy header key that contains the real source IP.
Known headers that reverse proxies are seen using:
- X-Forwarded-For
- X-Real-IP
## Memcached (Optional)
This is "Optional" in the event that you are using an allready existing memcached sever or cluster
Ensure that memcached is running and enabled at reboot
sudo apt install memcached libmemcached-tools
sudo systemctl status memcached
sudo systemctl enable memcached
Be sure to add enough memory in the memcached config to store the data
File: "/etc/memcached.conf" and look for below option "-m"
# memory (Allocated in MB)
-m 4096
If modifying this file remember to restart the service
sudo systemctl restart memcached
## Creating and installing Cratos python dependencies
$ python3 -m venv .venv
$ .venv/bin/pip install -r requirements.txt
## Add the fastapi user
sudo adduser fastapiuser --system --no-create-home --shell /usr/sbin/nologin
## Configuraiton and setup of Nginx (If needed)
In the [nginx.conf](/INSTALLATION/nginx.conf) be sure to modify the setup to match your environment, and also install a SSL certificate, either through your own or services like Let's Encrypt.
Replace the current "nginx.conf" located at "/etc/nginx/nginx.conf"
## Configuration and setup of supervisord (Boot type 1)
[Supervisor](http://supervisord.org/introduction.html) is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.
Modify [uvicorn_start](/INSTALLATION/uvicorn_start) to fit your needs, and copy it to the root directory of the project
$ cp INSTALLATION/uvicorn_start uvicorn_start.sh
# We want to test that it is working
$ chmod +x uvicorn_start.sh
$ ./uvicorn_start.sh
It should start Cratos FastAPI application using Uvicorn with the predefined settings.
$ cp INSTALLATION/cratos.conf_example /etc/supervisor/conf.d/cratos.conf
Remember to modify the configuraiton in "cratos.conf" to be adapted to your environment.
Testing and installing the configuration (Also use this if you make changes to the cratos.conf)
$ sudo supervisorctl reread
$ sudo supervisorctl update
And finally test if it starts the Cratos application managed by Supervisor
sudo supervisorctl start cratos
Good command to know with supervisor
sudo supervisorctl start cratos
sudo supervisorctl stop cratos
sudo supervisorctl restart cratos
sudo supervisorctl status cratos
sudo supervisorctl
## Configuration and setup of Systemd (Boot type 2) - Recommended
systemd can automatically start Gunicorn when the system boots and restart it if it crashes, ensuring high availability. Additionally systemd provides advanced resource management features, such as CPU and memory limits, which can help prevent Gunicorn from consuming too many resources.
Utilizing systemd also offers security features like sandboxing and process isolation, which can help improve the security of your Gunicorn service.
**Ensure to update INSTALLATION/gunicorn.service_example to fit your installation path**
cp INSTALLATION/gunicorn.service_example /etc/systemd/system/gunicorn.service
sudo systemctl daemon-reload
sudo systemctl start gunicorn
sudo systemctl enable gunicorn
# Everything is working
If the Cratos FastAPI is running you should be able to connect to is and we recommend starting on the help page "https://cratos.yourdomain.com/v1/help"
"https://cratos.yourdomain.com/v1/generate_token_form" to generate your auth token
# Unit testing
If commiting code to this project please ensure to run a unit test prior, to identify potential errors
Pre-requisit
copy the file "test.token.example" to "test.token" and insert a valid API token.
*Standard Test*
pytest tests/unit/test_api.py
*Standard Test with HTML report*
pytest tests/unit/test_api.py --html=report.html
# License
This software is licensed under [MIT](https://github.com/eCrimeLabs/cratos-fastapi/blob/main/LICENSE)
# Todo
# Video presentation from Hack.lu 2023
"Cratos - Use your bloody indicators"
https://www.youtube.com/watch?v=yFvvFIq7TKk