How do I use consumer groups in Kafka using Go?

Using consumer groups in Kafka with Go allows multiple consumers to share the processing load. Each consumer in a group reads data from different partitions, enabling horizontal scalability and failover capabilities. In this guide, we will demonstrate how to implement Kafka consumer groups in Go.

Keywords: Kafka, Go, consumer groups, data processing, horizontal scalability, failover

Description: Learn how to implement consumer groups in Kafka using Go to scale your data processing efficiently.


        package main

        import (
            "context"
            "fmt"
            "github.com/confluentinc/confluent-kafka-go/kafka"
            "log"
        )

        func main() {
            // Create a new consumer
            c, err := kafka.NewConsumer(&kafka.ConfigMap{
                "bootstrap.servers": "localhost:9092",
                "group.id":          "my-consumer-group",
                "auto.offset.reset": "earliest",
            })
            if err != nil {
                log.Fatalf("Failed to create consumer: %s\n", err)
            }
            defer c.Close()

            // Subscribe to topic
            c.SubscribeTopics([]string{"my-topic"}, nil)

            // Start consuming messages
            for {
                msg, err := c.ReadMessage(context.Background())
                if err == nil {
                    fmt.Printf("Message on %s: %s\n", msg.TopicPartition, string(msg.Value))
                } else {
                    // Handle errors
                    fmt.Printf("Consumer error: %v\n", err)
                }
            }
        }

How do I use consumer groups in Kafka using Go?

Popular Topics

Recent Languages

How do I use consumer groups in Kafka using Go?

Related Questions

Popular Topics

Recent Languages