bazel-execlog-cmp

CLI 工具，帮助比较 Bazel 执行日志。

这是什么

当尝试 调试 Bazel 远程缓存 时，获取 Bazel 的 执行日志 以进行比较（--execution_log_json_file）是很有帮助的。

实际上，这些日志可能非常大，手动比较它们可能很困难；这个工具试图帮助解决这个问题。

安装

cargo install bazel-execlog-cmp

如何使用此工具

首先，使用执行日志的路径运行工具

bazel-execlog-cmp <paths to a bunch of JSON execution logs>

然后，要求它比较您感兴趣的艺术品的操作

> cmp bazel-out/k8-opt/bin/foo.out
Input Mismatches:
  `bazel-out/k8-opt/bin/foo.o`
        ../execlog1.json: {Bytes:       9809, SHA-256: 9316644c2e21e3f5e238ae4b503b13935d997364b711731f1955af819e983e22}
        ../execlog2.json: {Bytes:       9809, SHA-256: a34d2d7c69bdda43de87d392439232649dfe0d787c0aced1245b8ff5b342d97a}

Output Mismatches:
  `bazel-out/k8-opt/bin/foo.out`
        ../execlog1.json: {Bytes:      16783, SHA-256: 8bc8118a9c5114910965057759b32c581d02963d2d3118f849b91ee92526d5b4}
        ../execlog2.json: {Bytes:      16782, SHA-256: 7482bd31539cb3fee803d4f0fac191d1fd96d549f8aa0808cc43df3b140b6b36}

通常您会从顶部开始（您的构建图的一个叶节点或您感兴趣的艺术品），然后递归地追踪不匹配的输入。例如，对于上面的例子，我们接下来会要求关于 foo.o 的信息

> cmp bazel-out/k8-opt/bin/foo.o
Environment Variable Mismatches:
  $SOME_ENV_VAR_THATS_DIFFERENT_FOR_SOME_REASON
        ../execlog1.json: hello
        ../execlog2.json: 👋

Output Mismatches:
  `bazel-out/k8-opt/bin/foo.o`
        ../execlog1.json: {Bytes:       9809, SHA-256: 9316644c2e21e3f5e238ae4b503b13935d997364b711731f1955af819e983e22}
        ../execlog2.json: {Bytes:       9809, SHA-256: a34d2d7c69bdda43de87d392439232649dfe0d787c0aced1245b8ff5b342d97a}

或者，如果您想查看所有操作中跨所有构建艺术品的不同环境变量/输入/输出的完整列表，可以使用 tcmp

> tcmp bazel-out/k8-opt/bin/foo.out
Environment Variable Mismatches:
  $SOME_ENV_VAR_THATS_DIFFERENT_FOR_SOME_REASON
        ../execlog1.json: hello
        ../execlog2.json: 👋

Input Mismatches:
  `bazel-out/k8-opt/bin/foo.o`
        ../execlog1.json: {Bytes:       9809, SHA-256: 9316644c2e21e3f5e238ae4b503b13935d997364b711731f1955af819e983e22}
        ../execlog2.json: {Bytes:       9809, SHA-256: a34d2d7c69bdda43de87d392439232649dfe0d787c0aced1245b8ff5b342d97a}

Output Mismatches:
  `bazel-out/k8-opt/bin/foo.o`
        ../execlog1.json: {Bytes:       9809, SHA-256: 9316644c2e21e3f5e238ae4b503b13935d997364b711731f1955af819e983e22}
        ../execlog2.json: {Bytes:       9809, SHA-256: a34d2d7c69bdda43de87d392439232649dfe0d787c0aced1245b8ff5b342d97a}
  `bazel-out/k8-opt/bin/foo.out`
        ../execlog1.json: {Bytes:      16783, SHA-256: 8bc8118a9c5114910965057759b32c581d02963d2d3118f849b91ee92526d5b4}
        ../execlog2.json: {Bytes:      16782, SHA-256: 7482bd31539cb3fee803d4f0fac191d1fd96d549f8aa0808cc43df3b140b6b36}

要排除来自其他不匹配艺术品的下游艺术品（因此可能是差异的 来源）以及顶级输出，请使用 edges（⚠️ 警告：此命令的输出可能不准确）

> edges bazel-out/k8-opt/bin/foo.out
Environment Variable Mismatches:
  $SOME_ENV_VAR_THATS_DIFFERENT_FOR_SOME_REASON
        ../execlog1.json: hello
        ../execlog2.json: 👋

Output Mismatches:
  `bazel-out/k8-opt/bin/foo.out`
        ../execlog1.json: {Bytes:      16783, SHA-256: 8bc8118a9c5114910965057759b32c581d02963d2d3118f849b91ee92526d5b4}
        ../execlog2.json: {Bytes:      16782, SHA-256: 7482bd31539cb3fee803d4f0fac191d1fd96d549f8aa0808cc43df3b140b6b36}

还有其他一些命令

> help
usage:
  - `quit` or `q` to quit
  - `cmp <output path>` to compare items of interest within the action for an output path
  - `transitive-cmp <output path>` or `tcmp` to compare all transitive dependencies of an output path
  `edges <output path>` *attempts* to determine the inputs that caused the executions of the output path to diverge; may not be accurate
  - `diff <output path>` to print a textual diff of the fields from `view <output path>`
  - `view <output path>` to print selected fields of interest from the action for an output path

最后，还有带有模糊搜索的自动补全；这对于输出路径特别有用，因为输出路径通常很长且难以手动输入。

还有其他什么吗？

此包有一个功能：json-dump-command。启用此功能可解锁 json 命令。

> json bazel-out/k8-opt/bin/foo.out
`../execlog1.json`:
{
  "commandArgs": ["..."],
  "environmentVariables": [{
    "name": "PATH",
    "value": "/bin:/usr/bin:/usr/local/bin"
  }, {
    "name": "PWD",
    "value": "/proc/self/cwd"
  }],
  "platform": {
    "properties": []
  },
  "inputs": [{
    "path": "bazel-out/k8-opt/bin/foo.o",
    "digest": {
      "hash": "9316644c2e21e3f5e238ae4b503b13935d997364b711731f1955af819e983e22",
      "sizeBytes": "9809",
      "hashFunctionName": "SHA-256"
    }
  }],
  "listedOutputs": ["bazel-out/k8-opt/bin/foo.out"],
  "remotable": true,
  "cacheable": true,
  "timeoutMillis": "0",
  "progressMessage": "...",
  "mnemonic": "CppCompile",
  "actualOutputs": [{
    "path": "bazel-out/k8-opt/bin/foo.out",
    "digest": {
      "hash": "8bc8118a9c5114910965057759b32c581d02963d2d3118f849b91ee92526d5b4",
      "sizeBytes": "16783",
      "hashFunctionName": "SHA-256"
    }
  }],
  "runner": "remote cache hit",
  "remoteCacheHit": true,
  "status": "",
  "exitCode": 0
}

`../execlog2.json`:
{
  "commandArgs": ["..."],
  "environmentVariables": [{
    "name": "PATH",
    "value": "/bin:/usr/bin:/usr/local/bin"
  }, {
    "name": "PWD",
    "value": "/proc/self/cwd"
  }],
  "platform": {
    "properties": []
  },
  "inputs": [{
    "path": "bazel-out/k8-opt/bin/foo.o",
    "digest": {
      "hash": "a34d2d7c69bdda43de87d392439232649dfe0d787c0aced1245b8ff5b342d97a",
      "sizeBytes": "9809",
      "hashFunctionName": "SHA-256"
    }
  }],
  "listedOutputs": ["bazel-out/k8-opt/bin/foo.out"],
  "remotable": true,
  "cacheable": true,
  "timeoutMillis": "0",
  "progressMessage": "...",
  "mnemonic": "CppCompile",
  "actualOutputs": [{
    "path": "bazel-out/k8-opt/bin/foo.out",
    "digest": {
      "hash": "7482bd31539cb3fee803d4f0fac191d1fd96d549f8aa0808cc43df3b140b6b36",
      "sizeBytes": "16782",
      "hashFunctionName": "SHA-256"
    }
  }],
  "runner": "processwrapper-sandbox",
  "remoteCacheHit": false,
  "status": "",
  "exitCode": 0
}

与 view 命令相比，json 会打印出有关生成特定艺术品的执行的所有详细信息。如果您希望查看一些 view 省略的详细信息，例如运行的完整命令，这很有用。

此功能默认是禁用的。请注意，启用它大约会将工具的加载时间加倍，并且大大增加内存使用。

我应该使用这个吗？

我不确定。

直到写完这篇文章后我才意识到，实际上有一个第一方工具也试图使比较执行日志变得更简单。

它操作于二进制日志格式（protobuf而不是JSON；体积小得多），但它也更简单：它给你排序后的文本文件，然后你可以用其他工具比较这些文件。