Home / server / Questions / 914122
In Process
Sachin PK
Asked: 2018-05-29 05:15:07 +0800 CST

How to perform bulk topic deletion in kafka

  • 772

We have 3 node Kafka cluster with replication factor 3 and default log retention period of 4 days. We have enabled the auto topic creation and It will create topics for each of our tasks. Once the task is finished we don't need the topic created for it. Now we have 2000 unused topics there in Kafka, can we perform bulk deletion of this 2000 topics without shutting down our cluster ?. Do bulk topic deletion affects our cluster's performance?. If yes what is the safe way to delete unwanted topics without shutting down cluster

kafka

1 Answers

  1. Look into the kafka-topics script from your install folder:

    Put the list command into a list(so you can later use it in your favorite language)

    list topics

    bin/kafka-topics.sh --zookeeper localhost:2181 --list

    For each topic from the above list, run the delete command:

    delete topic

    bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic mytopic

    Yes, performance will be affected depending on:

    • How many messages you have in a topic and overall

    • How big is the message size

    Please be carefull and only delete a couple of them at a time at most, then verify if the cluster has finished the delete operations afterwards :)

    • 0