Skip to content

V
visual-fuzz-llvm
Commit c25c4505 by phoon
Options

update README.md

parent aa1ad63c
Showing with 42 additions and 185 deletions
README.md
# The LLVM Compiler Infrastructure
This directory and its subdirectories contain source code for LLVM,
a toolkit for the construction of highly optimized compilers,
optimizers, and runtime environments.
The README briefly describes how to get started with building LLVM.
For more information on how to contribute to the LLVM project, please
take a look at the
[Contributing to LLVM](https://llvm.org/docs/Contributing.html) guide.
## Getting Started with the LLVM System
Taken from https://llvm.org/docs/GettingStarted.html.
### Overview
Welcome to the LLVM project!
The LLVM project has multiple components. The core of the project is
itself called "LLVM". This contains all of the tools, libraries, and header
files needed to process intermediate representations and converts it into
object files. Tools include an assembler, disassembler, bitcode analyzer, and
bitcode optimizer. It also contains basic regression tests.
C-like languages use the [Clang](http://clang.llvm.org/) front end. This
component compiles C, C++, Objective C, and Objective C++ code into LLVM bitcode
-- and from there into object files, using LLVM.
Other components include:
the [libc++ C++ standard library](https://libcxx.llvm.org),
the [LLD linker](https://lld.llvm.org), and more.
### Getting the Source Code and Building LLVM
The LLVM Getting Started documentation may be out of date. The [Clang
Getting Started](http://clang.llvm.org/get_started.html) page might have more
accurate information.
This is an example workflow and configuration to get and build the LLVM source:
1. Checkout LLVM (including related subprojects like Clang):
* ``git clone https://github.com/llvm/llvm-project.git``
* Or, on windows, ``git clone --config core.autocrlf=false
https://github.com/llvm/llvm-project.git``
2. Configure and build LLVM and Clang:
* ``cd llvm-project``
* ``mkdir build``
* ``cd build``
* ``cmake -G <generator> [options] ../llvm``
Some common generators are:
* ``Ninja`` --- for generating [Ninja](https://ninja-build.org)
build files. Most llvm developers use Ninja.
* ``Unix Makefiles`` --- for generating make-compatible parallel makefiles.
* ``Visual Studio`` --- for generating Visual Studio projects and
solutions.
* ``Xcode`` --- for generating Xcode projects.
Some Common options:
* ``-DLLVM_ENABLE_PROJECTS='...'`` --- semicolon-separated list of the LLVM
subprojects you'd like to additionally build. Can include any of: clang,
clang-tools-extra, libcxx, libcxxabi, libunwind, lldb, compiler-rt, lld,
polly, or debuginfo-tests.
For example, to build LLVM, Clang, libcxx, and libcxxabi, use
``-DLLVM_ENABLE_PROJECTS="clang;libcxx;libcxxabi"``.
* ``-DCMAKE_INSTALL_PREFIX=directory`` --- Specify for *directory* the full
pathname of where you want the LLVM tools and libraries to be installed
(default ``/usr/local``).
* ``-DCMAKE_BUILD_TYPE=type`` --- Valid options for *type* are Debug,
Release, RelWithDebInfo, and MinSizeRel. Default is Debug.
* ``-DLLVM_ENABLE_ASSERTIONS=On`` --- Compile with assertion checks enabled
(default is Yes for Debug builds, No for all other build types).
* Run your build tool of choice!
* The default target (i.e. ``ninja`` or ``make``) will build all of LLVM.
* The ``check-all`` target (i.e. ``ninja check-all``) will run the
regression tests to ensure everything is in working order.
* CMake will generate build targets for each tool and library, and most
LLVM sub-projects generate their own ``check-<project>`` target.
* Running a serial build will be *slow*. To improve speed, try running a
parallel build. That's done by default in Ninja; for ``make``, use
``make -j NNN`` (NNN is the number of parallel jobs, use e.g. number of
CPUs you have.)
* For more information see [CMake](https://llvm.org/docs/CMake.html)
Consult the
[Getting Started with LLVM](https://llvm.org/docs/GettingStarted.html#getting-started-with-llvm)
page for detailed information on configuring and compiling LLVM. You can visit
[Directory Layout](https://llvm.org/docs/GettingStarted.html#directory-layout)
to learn about the layout of the source code tree.
## 容器创建
docker run -it unifuzz:2.0 bash
## 编译源码工程
**首先在进行以下编译之前，要保证正常情况下是能够编译通过的，也就是工程相关的依赖都已经安装，我们假设工程依赖都已经安装好。**
我们以vim工程为例，说明下编译流程
- 用AFL编译工程生成用于fuzz的可执行文件（我们称之为fuzz_binary），该编译过程可以参照AFL编译流程，对应的vim编译流程为:
- $cd /path/to/vim-afl 进入到vim工程源码目录
- $CC=/home/AFL/afl-gcc ./configure
- $make -j20
- 完成之后我们可以得到AFL插桩后的可执行文件vim，也就是fuzz_binary
- 用llvm编译工程生成用于产生精确覆盖率的可执行文件（我们称之为visual_binary）和用于可视化展示的静态分析json文件（我们称之为visual_json），对应的vim编译流程为:
- $cd /path/to/vim-llvm 进入到另外一个vim工程源码目录（一般需要将源码压缩文件解压到两个不同的目录中，一个用于AFL的编译，一个用于llvm的编译）
- $CC="/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang" CFLAGS="-w -flto -g /home/afl-llvm-rt.o.c.o" CXX="/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang++" CXXFLAGS="-w -flto -g /home/afl-llvm-rt.o.c.o" LDFLAGS="-L/home/llvm-10-visual-mode-1.0/llvm-build-release-install/lib/clang/10.0.0/lib/linux -lclang_rt.builtins-x86_64" RANLIB=/bin/true AR=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-ar NM=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-nm ./configure（该构建命令对应的是Makefile构建系统，如果构建系统是cmake，相应的命令为cmake -DCMAKE_C_COMPILER=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang -DCMAKE_CXX_COMPILER=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang++ -DCMAKE_C_FLAGS="-w -flto -g /home/afl-llvm-rt.o.c.o" -DCMAKE_CXX_FLAGS="-w -flto -g /home/afl-llvm-rt.o.c.o" -DCMAKE_RANLIB=/bin/true -DCMAKE_AR=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-ar -DCMAKE_NM=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-nm .）
- Makefile构建系统的工程中，有时候上述configure构建命令会在编译时出现xxx.o: file not recognized: file format not recognized导致编译失败，这时可以用CC="/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang -w -flto -g /home/afl-llvm-rt.o.c.o" CXX="/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang++ -w -flto -g /home/afl-llvm-rt.o.c.o" LDFLAGS="-L/home/llvm-10-visual-mode-1.0/llvm-build-release-install/lib/clang/10.0.0/lib/linux -lclang_rt.builtins-x86_64" RANLIB=/bin/true AR=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-ar NM=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-nm ./configure构建命令来替代原来的构建命令
- $make -j20
- 完成之后我们可以得到llvm插桩后的可执行文件vim，也就是visual_binary，还有用于可视化展示的静态分析json文件vim-172825.json，其中172825表示该可执行文件中包含的基本块个数
# 镜像制作
- 用docker cp命令从容器中拷贝fuzz_binary、visual_binary、visual_json三个文件到本地
- docker cp container-id:/path/to/fuzz_binary /local/path/dir
- docker cp container-id:/path/to/visual_binary /local/path/dir
- docker cp container-id:/path/to/visual_json /local/path/dir
- 从容器中删除llvm目录和AFL目录（为了使生成的docker镜像尽可能小）
- docker exec -it container-id bash
- cd /home && rm -rf llvm-10-visual-mode-1.0 && rm -rf AFL
- export容器到镜像文件
- docker export -o image_name.tar container-id
- 用save命令生成镜像文件（原因是目前unifuzz网页系统不支持export命令生成的镜像文件，只支持save命令生成的镜像文件）
- docker import image_name.tar REPOSITORY:TAG
- docker run -it REPOSITORY:TAG bash（假设生成的容器为container-id2）
- docker commit container-id2 REPOSITORY1:TAG1
- docker save -o image_name2.tar REPOSITORY1:TAG1
**最终我们生成一个名字为image_name2.tar的镜像文件**
# 测试
- 正常的unifuzz测试参照unifuzz使用文档
- 精确覆盖率测试
- 首先在程序管理中找到要被测程序，点击修改
- 分别上传分析程序（对应于生成的visual_binary）、源代码（对应于源代码的压缩包）、JSON文件（对应于生成的visual_json）
- 在任务管理中找到对应的任务，点击启动按钮进行正常fuzz测试，运行一段时间后，点击“暂停”按钮，再点击“分析”按钮开始做精确覆盖率分析，等待分析完成，点击“详情”按钮，在详情页面点击“精确覆盖率”
how_to_compile.txt deleted 100755 → 0
sudo apt install gcc-9-plugin-dev
if [ $1 == 'debug' ]
then
cd $2
if [ -d "llvm-build-debug" ]
then
rm -rf llvm-build-debug
fi
if [ -d "llvm-build-debug-install" ]
then
rm -rf llvm-build-debug-install
fi
mkdir llvm-build-debug
mkdir llvm-build-debug-install
cd llvm-build-debug
echo "cmake -DLLVM_ENABLE_RTTI=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Debug -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_USE_SPLIT_DWARF=ON -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_TARGETS_TO_BUILD=host -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_DOCS=OFF -DCMAKE_CXX_FLAGS=\"-std=c++14 -g\" -DLLVM_ENABLE_PROJECTS=\"clang;polly;compiler-rt;lld\" -DCMAKE_INSTALL_PREFIX=$2/llvm-build-debug-install -DLLVM_BINUTILS_INCDIR=/usr/lib/gcc/x86_64-linux-gnu/9/plugin/include $2/llvm"
cmake -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_RTTI=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Debug -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_USE_SPLIT_DWARF=ON -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_TARGETS_TO_BUILD=host -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_DOCS=OFF -DCMAKE_CXX_FLAGS="-std=c++11 -g" -DLLVM_ENABLE_PROJECTS="clang;polly;compiler-rt;lld" -DCMAKE_INSTALL_PREFIX=$2/llvm-build-debug-install -DLLVM_BINUTILS_INCDIR=/usr/lib/gcc/x86_64-linux-gnu/9/plugin/include $2/llvm
elif [ $1 == 'release' ]
then
cd $2
if [ -d "llvm-build-release" ]
then
rm -rf llvm-build-release
fi
if [ -d "llvm-build-release-install" ]
then
rm -rf llvm-build-release-install
fi
mkdir llvm-build-release
mkdir llvm-build-release-install
cd llvm-build-release
echo "cmake -DLLVM_ENABLE_RTTI=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_DUMP=ON -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_USE_SPLIT_DWARF=ON -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_TARGETS_TO_BUILD=host -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_DOCS=OFF -DCMAKE_CXX_FLAGS=\"-std=c++14 -O3\" -DLLVM_ENABLE_PROJECTS=\"clang;polly;compiler-rt;lld\" -DCMAKE_INSTALL_PREFIX=$2/llvm-build-release-install -DLLVM_BINUTILS_INCDIR=/usr/lib/gcc/x86_64-linux-gnu/9/plugin/include $2/llvm"
cmake -DLLVM_INSTALL_UTILS=ON -DLLVM_ENABLE_RTTI=ON -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_DUMP=ON -DLLVM_ENABLE_ASSERTIONS=OFF -DLLVM_USE_SPLIT_DWARF=ON -DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_TARGETS_TO_BUILD=host -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF -DLLVM_INCLUDE_DOCS=OFF -DCMAKE_CXX_FLAGS="-std=c++11 -O3" -DLLVM_ENABLE_PROJECTS="clang;polly;compiler-rt;lld" -DCMAKE_INSTALL_PREFIX=$2/llvm-build-release-install -DLLVM_BINUTILS_INCDIR=/usr/lib/gcc/x86_64-linux-gnu/9/plugin/include $2/llvm
fi
how_to_use.txt deleted 100644 → 0
## 容器创建
docker run -it unifuzz:2.0 bash
## 编译源码工程
**首先在进行以下编译之前，要保证正常情况下是能够编译通过的，也就是工程相关的依赖都已经安装，我们假设工程依赖都已经安装好。**
我们以vim工程为例，说明下编译流程
- 用AFL编译工程生成用于fuzz的可执行文件（我们称之为fuzz_binary），该编译过程可以参照AFL编译流程，对应的vim编译流程为:
- $cd /path/to/vim-afl 进入到vim工程源码目录
- $CC=/home/AFL/afl-gcc ./configure
- $make -j20
- 完成之后我们可以得到AFL插桩后的可执行文件vim，也就是fuzz_binary
- 用llvm编译工程生成用于产生精确覆盖率的可执行文件（我们称之为visual_binary）和用于可视化展示的静态分析json文件（我们称之为visual_json），对应的vim编译流程为:
- $cd /path/to/vim-llvm 进入到另外一个vim工程源码目录（一般需要将源码压缩文件解压到两个不同的目录中，一个用于AFL的编译，一个用于llvm的编译）
- $CC="/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang" CFLAGS="-w -flto -g /home/afl-llvm-rt.o.c.o" CXX="/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang++" CXXFLAGS="-w -flto -g /home/afl-llvm-rt.o.c.o" LDFLAGS="-L/home/llvm-10-visual-mode-1.0/llvm-build-release-install/lib/clang/10.0.0/lib/linux -lclang_rt.builtins-x86_64" RANLIB=/bin/true AR=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-ar NM=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-nm ./configure（该构建命令对应的是Makefile构建系统，如果构建系统是cmake，相应的命令为cmake -DCMAKE_C_COMPILER=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang -DCMAKE_CXX_COMPILER=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang++ -DCMAKE_C_FLAGS="-w -flto -g /home/afl-llvm-rt.o.c.o" -DCMAKE_CXX_FLAGS="-w -flto -g /home/afl-llvm-rt.o.c.o" -DCMAKE_RANLIB=/bin/true -DCMAKE_AR=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-ar -DCMAKE_NM=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-nm .）
- Makefile构建系统的工程中，有时候上述configure构建命令会在编译时出现xxx.o: file not recognized: file format not recognized导致编译失败，这时可以用CC="/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang -w -flto -g /home/afl-llvm-rt.o.c.o" CXX="/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/clang++ -w -flto -g /home/afl-llvm-rt.o.c.o" LDFLAGS="-L/home/llvm-10-visual-mode-1.0/llvm-build-release-install/lib/clang/10.0.0/lib/linux -lclang_rt.builtins-x86_64" RANLIB=/bin/true AR=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-ar NM=/home/llvm-10-visual-mode-1.0/llvm-build-release-install/bin/llvm-nm ./configure构建命令来替代原来的构建命令
- $make -j20
- 完成之后我们可以得到llvm插桩后的可执行文件vim，也就是visual_binary，还有用于可视化展示的静态分析json文件vim-172825.json，其中172825表示该可执行文件中包含的基本块个数
# 镜像制作
- 用docker cp命令从容器中拷贝fuzz_binary、visual_binary、visual_json三个文件到本地
- docker cp container-id:/path/to/fuzz_binary /local/path/dir
- docker cp container-id:/path/to/visual_binary /local/path/dir
- docker cp container-id:/path/to/visual_json /local/path/dir
- 从容器中删除llvm目录和AFL目录（为了使生成的docker镜像尽可能小）
- docker exec -it container-id bash
- cd /home && rm -rf llvm-10-visual-mode-1.0 && rm -rf AFL
- export容器到镜像文件
- docker export -o image_name.tar container-id
- 用save命令生成镜像文件（原因是目前unifuzz网页系统不支持export命令生成的镜像文件，只支持save命令生成的镜像文件）
- docker import image_name.tar REPOSITORY:TAG
- docker run -it REPOSITORY:TAG bash（假设生成的容器为container-id2）
- docker commit container-id2 REPOSITORY1:TAG1
- docker save -o image_name2.tar REPOSITORY1:TAG1
**最终我们生成一个名字为image_name2.tar的镜像文件**
# 测试
- 正常的unifuzz测试参照unifuzz使用文档
- 精确覆盖率测试
- 首先在程序管理中找到要被测程序，点击修改
- 分别上传分析程序（对应于生成的visual_binary）、源代码（对应于源代码的压缩包）、JSON文件（对应于生成的visual_json）
- 在任务管理中找到对应的任务，点击启动按钮进行正常fuzz测试，运行一段时间后，点击“暂停”按钮，再点击“分析”按钮开始做精确覆盖率分析，等待分析完成，点击“详情”按钮，在详情页面点击“精确覆盖率”
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment