Jason Brownlee's Blog, page 17

The GIL Was Removed From Python!?

The GIL has been removed from Python! Has it though? Okay, let me clear some things up: Okay, allow me to elaborate. Let’s get started. Python Global Interpreter Lock (GIL) The internals of the default Python interpreter (called CPython) are not thread-safe. This means that there can be race conditions between multiple threads within a […]

How to use ThreadPoolExecutor submit()

You can issue one-off tasks to the ThreadPoolExecutor using the submit() method. This returns a Future object that gives control over the asynchronous task executed in the thread pool. In this tutorial, you will discover how to use the ThreadPoolExecutor submit() method. Let’s get started. What is the ThreadPoolExecutor The ThreadPoolExecutor provides a pool of […]

Concurrent File I/O Programming Patterns

You can use reusable concurrent programming patterns when speeding up file IO using concurrency in Python. In this tutorial, you will discover patterns for concurrent file IO in Python. Let’s get started. Types of File I/O Operations We may need to perform file IO in our Python programs. There are many types of file IO […]

ThreadPoolExecutor Tasks Submit New Tasks

You can submit new tasks from tasks running in the ThreadPoolExecutor by passing the thread pool as an argument to the task or having the task access the thread pool via a global variable. In this tutorial, you will discover how to submit new tasks from tasks running in the ThreadPoolExecutor. Let’s get started. Need […]

Faster File I/O With Concurrency

File I/O operations are inherently slower compared to working with data in main memory. The performance of file I/O is constrained by the underlying hardware of the hard drive, resulting in significantly slower execution times when reading from or writing to files. This performance gap creates an opportunity for optimizing file I/O tasks with concurrency, […]

ThreadPoolExecutor Share Queue With Worker Threads

You can share a thread-safe queue with workers in the ThreadPoolExecutor using a function argument, global variable, and variable defined in the worker thread initialization function. In this tutorial, you will discover how to share a queue with tasks executed in the ThreadPoolExecutor. Let’s get started. Need to Share a Queue with All Workers in […]

ThreadPoolExecutor Share Data With Workers

You can share data with workers in the ThreadPoolExecutor using a function argument, global variable, and variable defined in the worker thread initialization function. In this tutorial, you will discover how to share data from the main thread with tasks executed by workers in the ThreadPoolExecutor. Let’s get started. Need to Share Data From Main […]

ThreadPoolExecutor Workers Stop Main Thread From Exiting

The ThreadPoolExecutor will block the Python interpreter from exiting if there are running tasks. This can cause a program to hang before exiting, if exited normally, exiting with an exception, or exiting explicitly via a call to sys.exit(). In this tutorial, you will discover what happens if we forget or are unable to shut down […]

ThreadPoolExecutor When Are Workers Started

The ThreadPoolExecutor will create worker threads on demand, rather than creating all worker threads when the thread pool is created. In this tutorial, you will discover when worker threads in the ThreadPoolExecutor are created. Let’s get started. Need to Know When Workers Are Started in the ThreadPoolExecutor The ThreadPoolExecutor provides a pool of reusable worker […]

ThreadPoolExecutor map() Configure chunksize

You can set the chunksize argument to the map() method of the ThreadPoolExecutor to any value as it has no effect. Batches of tasks issued to the ThreadPoolExecutor via the map() method are not chunked, meaning that the chunksize argument is ignored. In this tutorial, you will discover how to configure the chunksize argument to […]