How do I fan-in and fan-out patterns in Go?

In Go, the fan-in and fan-out patterns are common concurrency patterns used to manage goroutines effectively. The fan-out pattern allows multiple goroutines to process tasks simultaneously, while the fan-in pattern consolidates results from multiple goroutines into a single channel. Here’s how you can implement these patterns in Go.

fan-in, fan-out, concurrency, goroutines, Go language, channels

This example demonstrates how to use fan-in and fan-out patterns in Go to handle concurrent processing efficiently.


        package main

        import (
            "fmt"
            "sync"
            "time"
        )

        // worker simulates a task that processes data.
        func worker(id int, jobs <-chan int, wg *sync.WaitGroup) {
            defer wg.Done()
            for job := range jobs {
                fmt.Printf("Worker %d processing job %d\n", id, job)
                time.Sleep(time.Second) // Simulate work
            }
        }

        // fanOut creates multiple workers that process jobs from the channel.
        func fanOut(numWorkers int, jobs <-chan int) {
            var wg sync.WaitGroup
            for i := 0; i < numWorkers; i++ {
                wg.Add(1)
                go worker(i, jobs, &wg)
            }
            wg.Wait()
        }

        // fanIn merges results from multiple channels into a single channel.
        func fanIn(channels ...<-chan int) <-chan int {
            out := make(chan int)
            var wg sync.WaitGroup
            
            // Helper function to merge a single channel.
            output := func(c <-chan int) {
                for job := range c {
                    out <- job
                }
                wg.Done()
            }

            wg.Add(len(channels))
            for _, c := range channels {
                go output(c)
            }

            // Close the output channel once all input channels are processed.
            go func() {
                wg.Wait()
                close(out)
            }()
            return out
        }

        func main() {
            jobs1 := make(chan int, 5)
            jobs2 := make(chan int, 5)

            // Populate jobs
            for i := 1; i <= 5; i++ {
                jobs1 <- i
                jobs2 <- i + 5
            }
            close(jobs1)
            close(jobs2)

            // Fan out using multiple workers
            go fanOut(3, fanIn(jobs1, jobs2))

            // Wait for a while to let workers finish
            time.Sleep(6 * time.Second)
        }

fan-in fan-out concurrency goroutines Go language channels

How do I fan-in and fan-out patterns in Go?

Popular Topics

Recent Languages

How do I fan-in and fan-out patterns in Go?

Related Questions

Popular Topics

Recent Languages