Lib.rs

› 命令行工具
#docker #kubernetes #cgroup #low-overhead #cli #resource-monitor

bin+lib radvisor

Docker 容器和 Kubernetes Pod 的资源利用率监控工具，具有细粒度和低开销

由 Joseph Azevedo、Bhanu Garg 和 2 位贡献者 开发

1 个稳定版本

1.3.1 2020 年 11 月 19 日

#2306 在 命令行工具

MIT 许可证

165KB
3K SLoC

rAdvisor

build/test security License: MIT Latest release FOSSA Status

以细粒度和低开销监控 Linux 上的 Docker 容器和 Kubernetes Pod 的系统资源利用率，以 CSVY（csv + yaml）格式输出资源利用率日志。最初，rAdvisor 是作为自定义工具在 Rust 中开发的，用于帮助检测和分析容器化在线系统中的微瓶颈，rAdvisor 通过每秒轮询目标提供者（本地 Docker 守护进程或 Kubernetes API 服务器）来运行，以获取活动、正在运行的容器/Pod 列表。从这个列表中，rAdvisor 每隔 50ms 运行一个收集线程，使用 Linux cgroups 获取每个活动目标的资源利用率数据，将结果日志输出到 /var/log/radvisor/stats。

🖨️ 示例输出

注意：文件名对应于容器/Pod 的 ID/UID，并在末尾附加收集器初始化时间戳。

🐋 Docker

/var/log/radvisor/stats/c0cd2077ec95e1b340e85c2...b_1585108344.log
---
Version: 1.3.1
Provider: docker
Metadata:
  Created: "2020-10-11T04:22:18Z"
  Command: "bash -c 'sleep 2s; apt-get update; sleep 2s; DEBIAN_FRONTEND=noninteractive apt-get install -y stress wget; sleep 2s; dd if=/dev/zero of=/tmp/file1 bs=512M count=1 oflag=direct; sleep 2s; stress --cpu 8 --io 4 --vm 4 --vm-bytes 1024M --timeout 10s; sleep 2s; wget \"http://ipv4.download.thinkbroadband.com/10MB.zip\"; sleep 2s'"
  Id: 7762ff15c99a2d238f4d26c22b5eda5b97ebc03bd0a711693104dcb6f71fe411
  Image: ubuntu
  Labels: {}
  Names:
    - /silly_elion
  Ports: []
  Status: Up Less than a second
  SizeRw: ~
  SizeRootFs: ~
PerfTable:
  Delimiter: ","
  Columns:
    cpu.usage.percpu:
      Type: int
      Count: 32
    read:
      Type: epoch19
System:
  OsType: Linux
  OsRelease: 4.15.0
  Distribution:
    Id: ubuntu
    IdLike: debian
    Name: Ubuntu
    PrettyName: Ubuntu 18.04.1 LTS
    Version: 18.04.1 LTS (Bionic Beaver)
    VersionId: "18.04"
    VersionCodename: bionic
    CpeName: ~
    BuildId: ~
    Variant: ~
    VariantId: ~
  MemoryTotal: 65870408
  SwapTotal: 3145724
  Hostname: node-0.sandbox.infosphere.emulab.net
  CpuCount: 32
  CpuOnlineCount: 4
  CpuSpeed: 1279
Cgroup: system.slice/docker-7762ff15c99a2d238f4d26c22b5eda5b97ebc03bd0a711693104dcb6f71fe411.scope
CgroupDriver: systemd
PolledAt: 1602390140142271945
InitializedAt: 1602390140157676566
---
read,pids.current,pids.max,cpu.usage.total,cpu.usage.system,cpu.usage.user,cpu.usage.percpu,cpu.stat.user,cpu.stat.system,cpu.throttling.periods,cpu.throttling.throttled.count,cpu.throttling.throttled.time,memory.usage.current,memory.usage.max,memory.limit.hard,memory.limit.soft,memory.failcnt,memory.hierarchical_limit.memory,memory.hierarchical_limit.memoryswap,memory.cache,memory.rss.all,memory.rss.huge,memory.mapped,memory.swap,memory.paged.in,memory.paged.out,memory.fault.total,memory.fault.major,memory.anon.inactive,memory.anon.active,memory.file.inactive,memory.file.active,memory.unevictable,blkio.time,blkio.sectors,blkio.service.bytes.read,blkio.service.bytes.write,blkio.service.bytes.sync,blkio.service.bytes.async,blkio.service.ios.read,blkio.service.ios.write,blkio.service.ios.sync,blkio.service.ios.async,blkio.service.time.read,blkio.service.time.write,blkio.service.time.sync,blkio.service.time.async,blkio.queued.read,blkio.queued.write,blkio.queued.sync,blkio.queued.async,blkio.wait.read,blkio.wait.write,blkio.wait.sync,blkio.wait.async,blkio.merged.read,blkio.merged.write,blkio.merged.sync,blkio.merged.async,blkio.throttle.service.bytes.read,blkio.throttle.service.bytes.write,blkio.throttle.service.bytes.sync,blkio.throttle.service.bytes.async,blkio.throttle.service.ios.read,blkio.throttle.service.ios.write,blkio.throttle.service.ios.sync,blkio.throttle.service.ios.async,blkio.bfq.service.bytes.read,blkio.bfq.service.bytes.write,blkio.bfq.service.bytes.sync,blkio.bfq.service.bytes.async,blkio.bfq.service.ios.read,blkio.bfq.service.ios.write,blkio.bfq.service.ios.sync,blkio.bfq.service.ios.async
1602390175053135973,18,4915,45675783181,0,45675783181,9719044209 12310201631 11027849186 12618688155 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0,2925,1668,0,0,0,2802823168,3667771392,9223372036854771712,9223372036854771712,0,9223372036854771712,,35323904,2754781184,0,28672,,6837817,6156636,6854722,0,0,2754711552,7380992,27942912,0,2273087370,1306336,0,668844032,662777856,6066176,0,331937,331753,184,0,68057100860,68011971780,45129080,0,0,0,0,0,222907407415,222860666999,46740416,0,32,0,32,0,668844032,662777856,6066176,0,331937,331753,184,,,,,,,,
1602390175103189646,18,4915,45876609757,0,45876610443,9767491855 12362201213 11076227542 12670689833 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0,2938,1676,0,0,0,2968367104,3667771392,9223372036854771712,9223372036854771712,0,9223372036854771712,,35323904,2920067072,0,28672,,6878171,6156636,6895076,0,0,2919976960,7380992,27942912,0,2273087370,1306336,0,668844032,662777856,6066176,0,333749,333565,184,0,68057100860,68011971780,45129080,0,0,0,0,0,222907407415,222860666999,46740416,0,32,0,32,0,668844032,662777856,6066176,0,333750,333566,184,,,,,,,,
...

有关每个列表示什么的信息，可以在 文档页面 中找到。

⚓ Kubernetes

/var/log/radvisor/stats/9f0b1893-15e7-4...c_1585470948.log.log
---
Version: 1.3.1
Provider: kubernetes
Metadata:
  Uid: 9f0b1893-15e7-442a-966a-b0d19a35fc1c
  Name: kube-proxy-hsplg
  CreatedAt: "2020-03-29T04:32:35Z"
  Labels:
    controller-revision-hash: c8bb659c5
    k8s-app: kube-proxy
    pod-template-generation: "1"
  Namespace: kube-system
  NodeName: node-0.sandbox.infosphere.emulab.net
  HostIp: 130.127.133.26
  Phase: Running
  QosClass: BestEffort
  StartedAt: "2020-03-29T04:32:36Z"
PerfTable:
  Delimiter: ","
  Columns:
    cpu.usage.percpu:
      Type: int
      Count: 32
    read:
      Type: epoch19
System:
  OsType: Linux
  OsRelease: 4.15.0
  Distribution:
    Id: ubuntu
    IdLike: debian
    Name: Ubuntu
    PrettyName: Ubuntu 18.04.1 LTS
    Version: 18.04.1 LTS (Bionic Beaver)
    VersionId: "18.04"
    VersionCodename: bionic
    CpeName: ~
    BuildId: ~
    Variant: ~
    VariantId: ~
  MemoryTotal: 65870408
  SwapTotal: 3145724
  Hostname: node-0.sandbox.infosphere.emulab.net
  CpuCount: 32
  CpuOnlineCount: 32
  CpuSpeed: 1198
PolledAt: 1585470948008442929
Cgroup: /kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f0b1893_15e7_442a_966a_b0d19a35fc1c.slice
CgroupDriver: systemd
InitializedAt: 1585470948030565581
---
read,pids.current,pids.max,cpu.usage.total,cpu.usage.system,cpu.usage.user,cpu.usage.percpu,cpu.stat.user,cpu.stat.system,cpu.throttling.periods,cpu.throttling.throttled.count,cpu.throttling.throttled.time,memory.usage.current,memory.usage.max,memory.limit.hard,memory.limit.soft,memory.failcnt,memory.hierarchical_limit.memory,memory.hierarchical_limit.memoryswap,memory.cache,memory.rss.all,memory.rss.huge,memory.mapped,memory.swap,memory.paged.in,memory.paged.out,memory.fault.total,memory.fault.major,memory.anon.inactive,memory.anon.active,memory.file.inactive,memory.file.active,memory.unevictable,blkio.time,blkio.sectors,blkio.service.bytes.read,blkio.service.bytes.write,blkio.service.bytes.sync,blkio.service.bytes.async,blkio.service.ios.read,blkio.service.ios.write,blkio.service.ios.sync,blkio.service.ios.async,blkio.service.time.read,blkio.service.time.write,blkio.service.time.sync,blkio.service.time.async,blkio.queued.read,blkio.queued.write,blkio.queued.sync,blkio.queued.async,blkio.wait.read,blkio.wait.write,blkio.wait.sync,blkio.wait.async,blkio.merged.read,blkio.merged.write,blkio.merged.sync,blkio.merged.async,blkio.throttle.service.bytes.read,blkio.throttle.service.bytes.write,blkio.throttle.service.bytes.sync,blkio.throttle.service.bytes.async,blkio.throttle.service.ios.read,blkio.throttle.service.ios.write,blkio.throttle.service.ios.sync,blkio.throttle.service.ios.async,blkio.bfq.service.bytes.read,blkio.bfq.service.bytes.write,blkio.bfq.service.bytes.sync,blkio.bfq.service.bytes.async,blkio.bfq.service.ios.read,blkio.bfq.service.ios.write,blkio.bfq.service.ios.sync,blkio.bfq.service.ios.async
1602390175053135973,18,4915,45675783181,0,45675783181,9719044209 12310201631 11027849186 12618688155 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0,2925,1668,0,0,0,2802823168,3667771392,9223372036854771712,9223372036854771712,0,9223372036854771712,,35323904,2754781184,0,28672,,6837817,6156636,6854722,0,0,2754711552,7380992,27942912,0,2273087370,1306336,0,668844032,662777856,6066176,0,331937,331753,184,0,68057100860,68011971780,45129080,0,0,0,0,0,222907407415,222860666999,46740416,0,32,0,32,0,668844032,662777856,6066176,0,331937,331753,184,,,,,,,,
1602390175103189646,18,4915,45876609757,0,45876610443,9767491855 12362201213 11076227542 12670689833 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0,2938,1676,0,0,0,2968367104,3667771392,9223372036854771712,9223372036854771712,0,9223372036854771712,,35323904,2920067072,0,28672,,6878171,6156636,6895076,0,0,2919976960,7380992,27942912,0,2273087370,1306336,0,668844032,662777856,6066176,0,333749,333565,184,0,68057100860,68011971780,45129080,0,0,0,0,0,222907407415,222860666999,46740416,0,32,0,32,0,668844032,662777856,6066176,0,333750,333566,184,,,,,,,,
...

📜 运行时选项

可以通过命令行界面控制收集的许多具体细节。目前，这包括收集/轮询间隔和输出目录。要查看有关可用 CLI 选项的信息，请运行 radvisor help

$ radvisor help
radvisor 1.3.1
Joseph Azevedo <[email protected]>, Bhanu Garg <[email protected]>
Monitors container resource utilization with high granularity and low overhead

USAGE:
    radvisor [FLAGS] [OPTIONS] <SUBCOMMAND>

FLAGS:
    -h, --help       Prints help information
    -q, --quiet      Whether to run in quiet mode (minimal output)
    -v, --verbose    Whether to run in verbose mode (maximum output)
    -V, --version    Prints version information

OPTIONS:
    -c, --color <color-mode>    Color display mode for stdout/stderr output [default: auto]

SUBCOMMANDS:
    help    Prints this message or the help of the given subcommand(s)
    run     Runs a collection thread that writes resource statistics to output CSV files

📇 子命令

radvisor run

$ radvisor run <provider>

radvisor 的主要子命令是 run，它还需要目标提供者（Docker 或 Kubernetes），以用于发现收集目标。例如，要运行 rAdvisor 并每 40ms 收集 Docker 容器的资源利用率统计数据，可以使用以下命令

$ radvisor run docker -i 40ms
Initializing Docker API provider
Beginning statistics collection
Identified cgroupfs as cgroup driver

☑️ 支持的操作系统

目前，rAdvisor仅支持Linux（由于其高度依赖cgroups），尽管有一个跟踪问题，用于扩展其功能以与Windows自带的容器化API HCS协同工作：radvisor/issues/#3。

🏗️ 构建

🐋 使用Docker

要使用Docker构建rAdvisor，请运行以下命令（需要已安装并运行docker，可能需要以root用户运行）

$ sudo make

对于Docker构建方法，使用Rust稳定镜像（rust）来运行一个预先安装了必要工具链的Docker容器。

💽 直接从源码

要从源码构建rAdvisor，使用Rust 稳定版。我们建议使用rustup来安装Rust工具链。

现在，在已克隆的存储库根目录下，运行make compile以在./target/release/radvisor生成一个发布级别的二进制文件。此构建过程可能需要十分钟。

$ make compile
cargo build --release --bins \
--target x86_64-unknown-linux-gnu
   Compiling libc v0.2.68
   Compiling autocfg v1.0.0
   Compiling cfg-if v0.1.10
   ...
   Compiling shiplift v0.6.0
   Compiling radvisor v1.3.1 (/home/jazev/dev/radvisor)
    Finished release [optimized] target(s) in 4m 52s
$ ./radvisor --version
radvisor 1.3.1

⚖️ 许可证

本项目采用MIT许可证。

🔍 FOSSA状态

FOSSA Status

依赖项

~13–39MB
~631K SLoC