FuzzAnything/PromptFuzz

# Prompt Fuzzing for Fuzz Driver Generation PromptFuzz is an automated tool that generates high-quality fuzz drivers for libraries via a fuzz loop constructed on mutating LLMs' prompts. The fuzz loop of PromptFuzz aims to guide the mutation of LLMs' prompts to generate programs that cover more reachable code and explore complex API interrelationships, which are effective for fuzzing.  PromptFuzz is currently regarded as the leading approach for generating fuzz drivers both in academia and industry. The fuzz drivers generated by **PromptFuzz achieved a branch coverage of 40.12\% on the tested libraries**, which is 1.61x greater than *[OSS-Fuzz](https://github.com/google/oss-fuzz)* and 1.67x greater than *[Hopper](https://arxiv.org/pdf/2309.03496)*. Besides, PromptFuzz detected 33 valid security bugs from 49 unique crashes.  ## Relase Notes: ## ✨Features ## 🏆Trophy The fuzz drivers generated by PromptFuzz can detect a wide range of bugs, most of which are security bugs. For instances, [CVE-2023-6277](https://nvd.nist.gov/vuln/detail/CVE-2023-6277), [CVE-2023-52355](https://nvd.nist.gov/vuln/detail/CVE-2023-52355) and [CVE-2023-52356](https://nvd.nist.gov/vuln/detail/CVE-2023-52356). PromptFuzz detects uniquely interesting bugs: | ID | **Library** | **Buggy Function** | **Bug Type** | **Status** | **Track Link** | |-------------|------------------|-----------------------------------------|-------------------|-----------------|-------------------| | 1. | libaom | highbd_8_variance_sse2 | SEGV | Confirmed | [3489](https://bugs.chromium.org/p/aomedia/issues/detail?id=3489) | | 2. | libaom | av1_rc_update_framerate | Uninitialized Stack | Confirmed | [3509](https://bugs.chromium.org/p/aomedia/issues/detail?id=3509) | | 3. | libaom | timebase_units_to_ticks | Integer Overflow | Confirmed | [3510](https://bugs.chromium.org/p/aomedia/issues/detail?id=3510) | | 4. | libaom | encode_without_recode | SEGV | Confirmed | [3534](https://bugs.chromium.org/p/aomedia/issues/detail?id=3534) | | 5. | libvpx | vp8_peek_si_internal | SEGV | Confirmed | [1817](https://bugs.chromium.org/p/webm/issues/detail?id=1817) | | 6. | libvpx | update_fragments | Buffer Overflow | Confirmed | [1827](https://bugs.chromium.org/p/webm/issues/detail?id=1827) | | 7. | libvpx | vp8e_encode | Integer Overflow | Confirmed | [1828](https://bugs.chromium.org/p/webm/issues/detail?id=1828) | | 8. | libvpx | encode_mb_row | Integer Overflow | Confirmed | [1831](https://bugs.chromium.org/p/webm/issues/detail?id=1831) | | 9. | libvpx | vpx_free_tpl_gop_stats | SEGV | Confirmed | [1837](https://bugs.chromium.org/p/webm/issues/detail?id=1837) | | | 10. | libmagic | mkdbname | Buffer Overflow | Confirmed | [481](https://bugs.astron.com/view.php?id=481) | | 11. | libmagic | magic_setparam | Buffer Overflow | Waiting | [482](https://bugs.astron.com/view.php?id=482) | | 12. | libmagic | check_buffer | Buffer Overflow | Confirmed | [483](https://bugs.astron.com/view.php?id=483) | | 13. | libmagic | mget | Integer Overflow | Waiting | [486](https://bugs.astron.com/view.php?id=486) | | 14. | libTIFF | TIFFOpen | OOM | Confirmed | [614](https://gitlab.com/libtiff/libtiff/-/issues/614) | | 15. | libTIFF | PixarLogSetupDecode | OOM | Confirmed | [619](https://gitlab.com/libtiff/libtiff/-/issues/619) | | 16. | libTIFF | TIFFReadEncodedStrip | OOM | Confirmed | [620](https://gitlab.com/libtiff/libtiff/-/issues/620) | | 17. | libTIFF | TIFFReadRGBAImageOriented | OOM | Confirmed | [620](https://gitlab.com/libtiff/libtiff/-/issues/620) | | 18. | libTIFF | TIFFRasterScanlineSize64 | OOM | Confirmed | [621](https://gitlab.com/libtiff/libtiff/-/issues/621) | | 19. | libTIFF | TIFFReadRGBATileExt | SEGV | Confirmed | [622](https://gitlab.com/libtiff/libtiff/-/issues/622) | | 20. | sqlite3 | sqlite3_unlock_notify | Null Pointer crash | Confirmed | [e77a5](https://www.sqlite.org/forum/forumpost/e77a5c3445) | | 21. | sqlite3 | sqlite3_enable_load_extension | Null Pointer crash | Confirmed | [9ce83](https://www.sqlite.org/forum/forumpost/9ce835fe96) | | 22. | sqlite3 | sqlite3_db_config | Null Pointer crash | Confirmed | [5e3fc](https://www.sqlite.org/forum/forumpost/5e3fc453a6) | | 23. | c-ares | config_sortlist | Memory Leak | Confirmed | [d62627](https://github.com/c-ares/c-ares/commit/d62627e8b39ef793c3b1c7b054724b0d581eb4fb) | | 24. | c-ares | config_sortlist | Memory Leak | Confirmed | [d62627](https://github.com/c-ares/c-ares/commit/d62627e8b39ef793c3b1c7b054724b0d581eb4fb) | | 25. | libjpeg-turbo | tj3DecodeYUV8 | Integer Overflow | Confirmed | [78eaf0](https://github.com/libjpeg-turbo/libjpeg-turbo/security/advisories/GHSA-x7cp-qgf3-9896) | | 26. | libjpeg-turbo | tj3LoadImage16 | OOM | Confirmed | [735](https://github.com/libjpeg-turbo/libjpeg-turbo/issues/735) | | 27. | libpcap | pcap_create | File Leak | Confirmed | [1233](https://github.com/the-tcpdump-group/libpcap/issues/1233) | | 28. | libpcap | pcapint_create_interface | Null Pointer crash | Confirmed | [1239](https://github.com/the-tcpdump-group/libpcap/issues/1239) | | 29. | libpcap | pcapint_fixup_pcap_pkthdr | Misaligned Address | Confirmed | - | | | 30. | cJSON | cJSON_SetNumberHelper | Error Cast | Confirmed | [805](https://github.com/DaveGamble/cJSON/issues/805) | | 31. | cJSON | cJSON_CreateNumber | Error Cast | Confirmed | [806](https://github.com/DaveGamble/cJSON/issues/806) | | 32. | cJSON | cJSON_DeleteItemFromObjectCaseSensitive | TimeOut | Confirmed | [807](https://github.com/DaveGamble/cJSON/issues/807) | | 33. | curl | parseurl | Assertion Failure | Confirmed | [12775](https://github.com/curl/curl/pull/12775) | # Usage See in [Usage](USAGE.md) ## Trouble Shooting PromptFuzz customized both static and dynamic analysis on the generated harnesses to ensure the harness effectiveness and bug detection accuracy. As LLMs can generate a diverse both C and C++ syntax, our predefined analysis could be panic when it encouters some corner cases. If you encountered that cases, please reproduce that issue with `RUST_BACKTRACE=1` and report it. If possible, let your coding pilot (using a powerful LLM) try to identifiy the issues and fix them. Welcome PRs. ## 🎈Future Works - **Close-source libraries:** Apply PromptFuzz to close-source libraries by fine tuning LLMs on private code corpus. - **Generalization**: Generalize PromptFuzz to binary programs.