kafka学习 -- 服务端命令学习

写在前面

这里介绍一下 kafka_2.11-1.0.2 版本服务端命令

一、服务端所有脚本

connect-distributed.sh
kafka-consumer-perf-test.sh
kafka-replica-verification.sh
kafka-verifiable-producer.sh

connect-standalone.sh
kafka-delete-records.sh
kafka-run-class.sh
trogdor.sh

kafka-acls.sh
kafka-log-dirs.sh
kafka-server-start.sh
windows
kafka-broker-api-versions.sh
kafka-mirror-maker.sh
kafka-server-stop.sh
zookeeper-security-migration.sh

kafka-configs.sh
kafka-preferred-replica-election.sh
kafka-simple-consumer-shell.sh
zookeeper-server-start.sh

kafka-console-consumer.sh
kafka-producer-perf-test.sh
kafka-streams-application-reset.sh
zookeeper-server-stop.sh

kafka-console-producer.sh
kafka-reassign-partitions.sh
kafka-topics.sh
zookeeper-shell.sh

kafka-consumer-groups.sh
kafka-replay-log-producer.sh
kafka-verifiable-consumer.sh

二、常用的脚本

2.1、服务端启动配置

bin/zookeeper-server-start.sh config/zookeeper.properties

bin/kafka-server-start.sh config/server.properties
bin/kafka-server-start.sh config/server1.properties
bin/kafka-server-start.sh config/server2.properties

2.2、broker 节点

在这里插入代码片

2.3、Topic

bin/kafka-topics.sh --create --zookeeper 192.168.1.74:2181 --replication-factor 3 --partitions 1 --topic topic2
bin/kafka-topics.sh  --delete --zookeeper 192.168.1.74:2181  --topic topic1

bin/kafka-topics.sh --list --zookeeper 192.168.1.74:2181

bin/kafka-topics.sh --describe --zookeeper 192.168.1.74:2181 --topic topic1

2.4、producer

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic topic1

2.5、consumer 和 group

consumer start

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic topic1
bin/kafka-console-consumer.sh --zookeeper 192.168.1.74:2181 --from-beginning --topic topic1

group命令，这里可看到消费者组的Topic 管理（分区）

bin/kafka-consumer-groups.sh --zookeeper 192.168.1.74:2181 --list
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list


bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group dltGroup --describe
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group program --describe
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group fooGroup --describe
bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group multiGroup --describe

2.6、zookeeper

启动 zookeeper 客户端，管理 kafka

bin/zookeeper-shell.sh 192.168.1.74:2181
ls /
ls /brokers/topics

查看节点

ls /brokers/ids

查看 topic

ls /brokers/topics

查看删除的消息

ls /admin/delete_topics

其他

ls /log_dir_event_notification
ls /consumers
ls /consumers/console-consumer-45795
ls /consumers/console-consumer-45795/offsets
ls /consumers/console-consumer-45795/offsets/topic2

ls /consumers/console-consumer-97962/offsets/topic1
ls /consumers/console-consumer-97962/owners/topic1
ls /consumers/console-consumer-97962/ids

ls /vod_clip_dev_new
ls /latest_producer_id_block
ls /controller_epoch
ls /isr_change_notification
ls /admin
ls /admin/delete_topics
ls /zookeeper
ls /zookeeper/quota
ls /config
ls /cluster
ls /cluster/id
ls /controller
ls /brokers
ls /dubbo-dev

2.7、性能测试

bin/kafka-consumer-perf-test.sh --broker-list localhost:9092 --group fooGroup --topic topic1 --messages 1024

2.8、日志查询

bin/kafka-log-dirs.sh --describe --bootstrap-server 127.0.0.1:9092 --topic-list foos

结果

{
   
  "version": 1,
  "brokers": [
    {
   
      "broker": 0,
      "logDirs": [
        {
   
          "logDir": "/kafka/kafka-logs-3d0b0150932f",
          "error": null,
          "partitions": [
            {
   
              "partition": "foos-0",
              "size": 384,
              "offsetLag": 0,
              "isFuture": false
            }
          ]
        }
      ]
    }
  ]
}

然后，查看日志

ls  /kafka/kafka-logs-3d0b0150932f/foos-0

tail -f /kafka/kafka-logs-3d0b0150932f/foos-0/00000000000000000000.log

kafka学习 -- 服务端命令学习