消息队列工作负载生成器

简介

Apache RocketMQ和Apache Kafka性能测试工具。

快速入门

安装

通过cargo安装

cargo install mq-workload-generator

手动安装

从发布页面下载二进制文件。目前仅支持Linux和macOS。对于其他平台，您需要自行从源代码构建。

基本用法

每秒向主题test发送和接收100条消息

# To set a parameter, choose either the CLI or ENV mode.
export ACCESS_POINT=localhost:9092
mq-workload-generator --topic test --qps 100
# example output: 
# this tool will print the current send and receive tps every second.
Jul 25 10:38:44.203 INFO[src/main.rs:32:5] Begin generating workload and wait for the server to come online...
Jul 25 10:38:44.204 INFO[src/main.rs:82:17] current send tps: 100, receive tps: 100
Jul 25 10:38:45.206 INFO[src/main.rs:82:17] current send tps: 100, receive tps: 100
Jul 25 10:38:46.205 INFO[src/main.rs:82:17] current send tps: 100, receive tps: 100
Jul 25 10:38:47.205 INFO[src/main.rs:82:17] current send tps: 100, receive tps: 100

所有可用选项

A tool for testing the performance of Apache RocketMQ and Apache Kafka

Usage: mq-workload-generator [OPTIONS] --topic <TOPIC>

Options:
  -d, --driver <DRIVER>
          Work load driver, available option: rocketmq, kafka [env: DRIVER=] [default: kafka]
  -a, --access-point <ACCESS_POINT>
          Access point of the mq cluster, default is localhost:8081 for RocketMQ and localhost:9092 for Kafka [env: ACCESS_POINT=]
  -t, --topic <TOPIC>
          Target topic [env: TOPIC=]
  -g, --group <GROUP>
          Group used by consumer [env: GROUP=] [default: mq_workload_generator]
  -p, --parallelism <PARALLELISM>
          Number of the client [env: PARALLELISM=] [default: 1]
  -q, --qps <QPS>
          Send tps of the sum of all producers [env: QPS=] [default: 100]
      --key-size <KEY_SIZE>
          Random key with specified size for each record sent by producer [env: KEY_SIZE=]
      --min-payload-size <MIN_PAYLOAD_SIZE>
          Minimum message payload size, measured in bytes [env: MIN_PAYLOAD_SIZE=]
      --max-payload-size <MAX_PAYLOAD_SIZE>
          Maximum message payload size, measured in bytes [env: MAX_PAYLOAD_SIZE=]
      --payload-size <PAYLOAD_SIZE>
          Fixed message payload size, measured in bytes. The priority is given to the dynamic payload size, which ranges from **min_payload_size** to **max_payload_size** [env: PAYLOAD_SIZE=] [default: 4096]
      --mode <MODE>
          Mode of the workload test, available values: producer, consumer, producer_and_consumer [env: MODE=] [default: producer_and_consumer]
      --access-key <ACCESS_KEY>
          Access Key to the topic [env: ACCESS_KEY=] [default: ]
      --secret-key <SECRET_KEY>
          Secret Key to the topic [env: SECRET_KEY=] [default: ]
  -v, --verbose
          Print detail error [env: VERBOSE=]
  -h, --help
          Print help
  -V, --version
          Print version

在Kubernetes中使用

有一个现成的Kubernetes清单文件，可用于在Kubernetes中部署工作负载生成器。

kubectl create namespace benchmark
kubectl apply -f deployment-consumer.yaml
kubectl apply -f deployment-producer.yaml

待办事项

• 添加更多选项：用户特定的消费者组、负载大小、消费时间、延迟消息计数等。
• 支持更多平台：Apache Kafka。
• 添加更多指标：发送/接收延迟等。
• 添加更多测试用例：发送/接收大消息或延迟/事务消息。