试用clusterfuzzlite

发表于 |

clusterfuzzlite是是一种持续的模糊测试解决方案，作为持续集成 (CI) 工作流的一部分运行，比如我们一旦push代码，便可以自动build，之后自动fuzz。

比如它支持GitHub Actions，GitLab，Google Cloud Build和Prow，我们最常见到的应该是GitHub Actions

如何让项目支持clusterfuzzlite呢

ClusterFuzzLite 重用了 OSS-Fuzz 工具链来简化构建。这意味着 ClusterFuzzLite 将在 docker 容器中构建您的项目。所以加入你熟悉OSS-Fuzz，这就看着很像了。只不过多了一些参数，比如–external

有一个问题就是他这个只支持libfuzzer！！！，所以局限性还是有的。

环境配置及文件编辑

首先环境的配置好：

首先需要在项目根目录新建.clusterfuzzlite文件夹，包含下面三个文件

  • .clusterfuzzlite/project.yaml
  • .clusterfuzzlite/Dockerfile
  • .clusterfuzzlite/build.sh

上面的文件不用我们新建，可以使用命令新建模板

1
2
3
$ cd /path/to/oss-fuzz
$ export PATH_TO_PROJECT=<path_to_your_project>
$ python infra/helper.py generate --external --language=c++ $PATH_TO_PROJECT

以https://github.com/libexpat/libexpat.git为例

1
2
3
cd / && git clone https://github.com/libexpat/libexpat.git expat
export PATH_TO_PROJECT=/expat
python infra/helper.py generate --external --language=c $PATH_TO_PROJECT

主要编辑build.sh文件即可，主要最后编译出来的fuzzer需要复制到$OUT即可

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
cd $SRC/expat/

: ${LD:="${CXX}"}
: ${LDFLAGS:="${CXXFLAGS}"}  # to make sure we link with sanitizer runtime

cmake_args=(
    # Specific to Expat
    -DEXPAT_BUILD_FUZZERS=ON
    -DEXPAT_OSSFUZZ_BUILD=ON
    -DEXPAT_SHARED_LIBS=OFF

    # C compiler
    -DCMAKE_C_COMPILER="${CC}"
    -DCMAKE_C_FLAGS="${CFLAGS}"

    # C++ compiler
    -DCMAKE_CXX_COMPILER="${CXX}"
    -DCMAKE_CXX_FLAGS="${CXXFLAGS}"

    # Linker
    -DCMAKE_LINKER="${LD}"
    -DCMAKE_EXE_LINKER_FLAGS="${LDFLAGS}"
    -DCMAKE_MODULE_LINKER_FLAGS="${LDFLAGS}"
    -DCMAKE_SHARED_LINKER_FLAGS="${LDFLAGS}"
)

mkdir -p build
cd build
cmake ../expat "${cmake_args[@]}"
make -j$(nproc)

for fuzzer in fuzz/*;
do
  cp $fuzzer $OUT
done

本地测试

上搞完就可以本地测试了

  1. 构建您的 docker 镜像和 fuzz 目标

模板命令

1
2
$ python infra/helper.py build_image --external $PATH_TO_PROJECT
$ python infra/helper.py build_fuzzers --external $PATH_TO_PROJECT --sanitizer <address/undefined/memory>

实际命令

1
2
python infra/helper.py build_image --external $PATH_TO_PROJECT
python infra/helper.py build_fuzzers --external $PATH_TO_PROJECT --sanitizer address

假如没问题说明build.sh写得没问题，环境库依赖也没问题

  1. 通过运行 check_build 命令查找要修复的常见构建问题

这将检查您的模糊测试目标是否使用正确的sanitizer编译，并且在模糊测试几秒钟后不会崩溃。

1
$ python infra/helper.py check_build --external $PATH_TO_PROJECT --sanitizer <address/undefined/memory>
  1. 运行特定的模糊测试目标，使用 run_fuzzer
1
$ python infra/helper.py run_fuzzer --external --corpus-dir=<path-to-temp-corpus-dir> $PATH_TO_PROJECT <fuzz_target>

<path-to-temp-corpus-dir>就是宿主机你自己准备好的种子文件路径
<fuzz_target>是编译出来的fuzzer的名字

  1. 如果您打算使用 ClusterFuzzLite 的代码覆盖率报告功能，最好测试覆盖率报告生成是否有效。这将使用本地语料库目录中上一个 run_fuzzer 步骤生成的语料库。
1
2
$ python infra/helper.py build_fuzzers --external --sanitizer coverage $PATH_TO_PROJECT
$ python infra/helper.py coverage --external $PATH_TO_PROJECT --fuzz-target=<fuzz_target> --corpus-dir=<path-to-temp-corpus-dir>

步骤4是可选的，run_fuzzer没问题即可就行了

Running ClusterFuzzLite

ClusterFuzzLite可以以pull request为触发，也可以cron定时触发

下面就以GitHub Actions为例了，需要在.github/workflows目录新建文件

1
2
3
4
.github/workflows/cflite_pr.yml (for PR fuzzing, pull request为触发)
.github/workflows/cflite_build.yml (for continuous builds, 用于持续构建，push代码就执行build)
.github/workflows/cflite_batch.yml (for batch fuzzing，用于批量模糊测试)
.github/workflows/cflite_cron.yml(for tasks done on a cron schedule: pruning and coverage)

之后展示了一些默认的配置设置，默认配置已经适用于大多数项目

.github/workflows/cflite_pr.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
name: ClusterFuzzLite PR fuzzing
on:
  pull_request:
    paths:
      - '**'
permissions: read-all
jobs:
  PR:
    runs-on: ubuntu-latest
    concurrency:
      group: ${{ github.workflow }}-${{ matrix.sanitizer }}-${{ github.ref }}
      cancel-in-progress: true
    strategy:
      fail-fast: false
      matrix:
        sanitizer:
        - address
        # Override this with the sanitizers you want.
        # - undefined
        # - memory
    steps:
    - name: Build Fuzzers (${{ matrix.sanitizer }})
      id: build
      uses: google/clusterfuzzlite/actions/build_fuzzers@v1
      with:
        language: c++ # Change this to the language you are fuzzing.
        github-token: ${{ secrets.GITHUB_TOKEN }}
        sanitizer: ${{ matrix.sanitizer }}
        # Optional but recommended: used to only run fuzzers that are affected
        # by the PR.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".
    - name: Run Fuzzers (${{ matrix.sanitizer }})
      id: run
      uses: google/clusterfuzzlite/actions/run_fuzzers@v1
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        fuzz-seconds: 600
        mode: 'code-change'
        sanitizer: ${{ matrix.sanitizer }}
        # Optional but recommended: used to download the corpus produced by
        # batch fuzzing.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".

字段解析：

language： 更改为目标代码的语言
sanitizers： 更改或启用更多消毒剂。
fuzz-seconds： 更改模糊测试的时间。
parallel-fuzzing：使用并行模糊测试。

接下是.github/workflows/cflite_batch.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
name: ClusterFuzzLite batch fuzzing
on:
  schedule:
    - cron: '0 0/6 * * *'  # Every 6th hour. Change this to whatever is suitable.
permissions: read-all
jobs:
  BatchFuzzing:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        sanitizer:
        - address
        # Override this with the sanitizers you want.
        # - undefined
        # - memory
    steps:
    - name: Build Fuzzers (${{ matrix.sanitizer }})
      id: build
      uses: google/clusterfuzzlite/actions/build_fuzzers@v1
      with:
        language: c++ # Change this to the language you are fuzzing.
        sanitizer: ${{ matrix.sanitizer }}
    - name: Run Fuzzers (${{ matrix.sanitizer }})
      id: run
      uses: google/clusterfuzzlite/actions/run_fuzzers@v1
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        fuzz-seconds: 3600
        mode: 'batch'
        sanitizer: ${{ matrix.sanitizer }}
        # Optional but recommended: For storing certain artifacts from fuzzing.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".

.github/workflows/cflite_build.yml 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
name: ClusterFuzzLite continuous builds
on:
  push:
    branches:
      - main  # Use your actual default branch here.
permissions: read-all
jobs:
  Build:
   runs-on: ubuntu-latest
   concurrency:
     group: ${{ github.workflow }}-${{ matrix.sanitizer }}-${{ github.ref }}
     cancel-in-progress: true
   strategy:
     fail-fast: false
     matrix:
        sanitizer:
        - address
        # Override this with the sanitizers you want.
        # - undefined
        # - memory
   steps:
   - name: Build Fuzzers (${{ matrix.sanitizer }})
     id: build
     uses: google/clusterfuzzlite/actions/build_fuzzers@v1
     with:
        language: c++ # Change this to the language you are fuzzing.
        sanitizer: ${{ matrix.sanitizer }}
        upload-build: true

.github/workflows/cflite_cron.yml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
name: ClusterFuzzLite cron tasks
on:
  schedule:
    - cron: '0 0 * * *'  # Once a day at midnight.
permissions: read-all
jobs:
  Pruning:
    runs-on: ubuntu-latest
    steps:
    - name: Build Fuzzers
      id: build
      uses: google/clusterfuzzlite/actions/build_fuzzers@v1
      with:
        language: c++ # Change this to the language you are fuzzing
    - name: Run Fuzzers
      id: run
      uses: google/clusterfuzzlite/actions/run_fuzzers@v1
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        fuzz-seconds: 600
        mode: 'prune'
        # Optional but recommended.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".
  Coverage:
    runs-on: ubuntu-latest
    steps:
    - name: Build Fuzzers
      id: build
      uses: google/clusterfuzzlite/actions/build_fuzzers@v1
      with:
        language: c++ # Change this to the language you are fuzzing.
        sanitizer: coverage
    - name: Run Fuzzers
      id: run
      uses: google/clusterfuzzlite/actions/run_fuzzers@v1
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}
        fuzz-seconds: 600
        mode: 'coverage'
        sanitizer: 'coverage'
        # Optional but recommended.
        # See later section on "Git repo for storage".
        # storage-repo: https://${{ secrets.PERSONAL_ACCESS_TOKEN }}@github.com/OWNER/STORAGE-REPO-NAME.git
        # storage-repo-branch: main   # Optional. Defaults to "main"
        # storage-repo-branch-coverage: gh-pages  # Optional. Defaults to "gh-pages".

之后将代码都提交github，之后在Actions那里可以看到Workflow

此外谷歌官方人员也有个curl示例

1
https://github.com/oliverchang/curl/

参考

https://google.github.io/clusterfuzzlite/

打赏专区
paypal: https://www.paypal.me/giantbranch
假如你看不到评论，可能是你访问Disqus被墙了，请使用代理访问