In Python DevOps, how do I parallelize workloads?

In Python DevOps, you can parallelize workloads using various libraries and techniques. One of the most popular methods is to use the `concurrent.futures` module, which provides a high-level interface for asynchronously executing callables.

Below is an example of how to use the `ThreadPoolExecutor` to run tasks in parallel.

import concurrent.futures
import time

# Function to simulate a workload
def workload(task_id):
    print(f"Starting task {task_id}")
    time.sleep(2)  # Simulate a long-running task
    print(f"Finished task {task_id}")
    return f"Result of task {task_id}"

# Main execution block
if __name__ == "__main__":
    tasks = [1, 2, 3, 4, 5]  # List of tasks to execute
    results = []

    # Using ThreadPoolExecutor to parallelize the tasks
    with concurrent.futures.ThreadPoolExecutor() as executor:
        futures = [executor.submit(workload, task) for task in tasks]
        for future in concurrent.futures.as_completed(futures):
            results.append(future.result())

    print("All tasks completed.")
    print("Results:", results)
        

Python DevOps parallelize workloads concurrent.futures ThreadPoolExecutor