Python Multithreading and Synchronization

Python Multithreading and Synchronization

Python programming language is a multi-threading language. It means this language is capable of executing multiple program threads at a time or concurrently. A Single Thread is a lightweight process that performs a particular task during its lifecycle until it is terminated after that task completion. Multithreading approach of programming has the following benefits.

Threads allow Python programs to handle multiple functions at once as opposed to running a sequence of commands individually.

A process may have multiple threads which share the same data space within the main thread. Thus, they can interact with each other and can share required information which is easier with less performance overhead as compared to separate processes.

As threads are light-weight processed; therefore, they do not require much memory overhead. In terms of memory and performance, the threads are cheaper than processes.

Each thread has a life cycle as the start, the execution and the termination. Each thread has an instruction pointer that keeps track of its context where it is currently running.

During the life cycle of a thread, the following events can also occur.

A Thread can be pre-empted or interrupted.

A Thread can be put on hold temporarily or sleep while other threads are executing or running. This is also known as yielding.

Starting a New Thread using “thread” module

Python’s “thread” module has the method available that starts a new thread. Following is the syntax to start a new Thread in Python programming language.

thread.start_new_thread ( function, args[, kwargs] )

Above method is used to create a new thread in both Linux and Windows operating systems. This method call returns instantly, and the child thread starts to call the function that is passed in the list of arguments (args). When the called function returns, the thread will be terminated. In the above syntax, the args is a tuple of arguments. If we want to call the function without passing any arguments, then we may pass an empty tuple as args. The parameter kwargs is an optional dictionary of keyword arguments.

The Threading Module

This is a new module that is included with Python 2.4. The “threading” module exposes all the methods that are present in the “thread” module and provides some additional methods as follows.

Method threading.activeCount (): This method returns the number of thread objects that are active.
Method threading.currentThread (): This method returns the number of thread objects in the caller’s thread control.
Method threading.enumerate (): This method restores a list of all thread objects that are currently active.

In addition to these methods, the threading module has the Thread class that implements threading. Following are the methods provided by the Thread class.

Method run (): The run () method of the Thread class is the entry point for a thread.
Method start (): The start () method of the Thread class starts a thread by calling the run method.
Method join ([time]): The join () method of the Thread class waits for threads to terminate.
Method isAlive (): The isAlive () method of the Thread class checks whether a thread is still executing.
Method getName (): The getName () method of the Thread class returns the name of a thread.
Method setName (): The setName () method of the Thread class sets the name of a thread.

Creating Thread Using Threading Module

Following are the steps to implement a new thread using the threading module.

Firstly, define a new subclass of the Thread class.

After inheritance, Override the __init__ (self [, args]) method to add additional arguments.

Next, override the run (self [, args]) method to implement what the thread should do when started.

After doing above steps, we can now create the instance of subclass and then start a new thread by invoking the start () method, which in turn will call the run () method.

Following is the Thread example by using “threading” threading module in Python language.

Output

When we execute the above Python program, we will observe the following output.

Communicating between threads

There are multiple threads in your code, and you need to safely communicate between them.

You can use a Queue from the queue library.

from queue import Queue

from threading import Thread

# create a data producer

def producer(output_queue):

while True:

data = data_computation()

output_queue.put(data)

# create a consumer

def consumer(input_queue):

while True:

# retrieve data (blocking)

data = input_queue.get()

# do something with the data

# indicate data has been consumed

input_queue.task_done()

Creating producer and consumer threads with a shared queue

q = Queue()

t1 = Thread(target=consumer, args=(q,))

t2 = Thread(target=producer, args=(q,))

t1.start()

t2.start()

Create a Custom Thread Class

Using threading.Thread class we can subclass new custom Thread class. Therefore, we must override run method in a subclass.

Example

from threading import Thread

import time

class Sleepy(Thread):

def run(self):

time.sleep(5)

print(“Hello form Thread”)

if __name__ == “__main__”:

t = Sleepy()

t.start() # start method automatic call Thread class run method.

# print ‘The main program continues to run in foreground.’

t.join()

print(“The main program continues to run in the foreground.”)

Global Interpreter Lock

Python multithreading execution can provide suffer due to the Global Interpreter Lock. In short, even though we can have several threads in a Python program, only one bytecode instruction can implement in parallel at any one time, regardless of the number of CPUs. As such, multithreading in cases where operations are blocked by external events – like network access – can be quite effective:

import threading

import time

def process():

time.sleep(2)

start = time.time()

process()

print(“One run took %.2fs” % (time.time() – start))

start = time.time()

threads = [threading.Thread(target=process) for _ in range(4)]

for t in threads:

t.start()

for t in threads:

t.join()

print(“Four runs took %.2fs” % (time.time() – start))

Output:

When we execute the following command, it results in the following output:

One run took 2.00s

Four runs took 2.00s

Running in Multiple Processes

Use multiprocessing. The process to run a function in another process. The interface is similar to threading. Thread:

Example

import multiprocessing

import os

def process():

print(“Pid is %s” % (os.getpid(),))

processes = [multiprocessing.Process(target=process) for _ in range(4)]

for p in processes:

p.start()

for p in processes:

p.join()

Output:

When we execute the following command, it results in the following output:

Pid is 11206

Pid is 11207

Pid is 11208

Pid is 11209

Sharing State Between Processes

Code running in different processes do not, by default, share the same data. However, the multiprocessing module contains primitives to help share values across multiple processes.

Example

import multiprocessing

plain_num = 0

shared_num = multiprocessing.Value(‘d’, 0)

lock = multiprocessing.Lock()

def increment():

global plain_num

with lock:

# ordinary variable modifications are not visible across processes

plain_num += 1

# multiprocessing.Value modifications are

shared_num.value += 1

ps = [multiprocessing.Process(target=increment) for n in range(4)]

for p in ps:

p.start()

for p in ps:

p.join()

print(“plain_num is %d, shared_num is %d” % (plain_num, shared_num.value))

Output:

When we execute the following command, it result the following output:

plain_num is 0, shared_num is 4

Synchronizing Threads in Python

The simple-to-implement locking mechanism is provided in the “threading” module of Python that permits us to synchronize threads. It has following methods to achieve Thread synchronization.

Method Lock (): When this method is called, it returns the new lock.
Method acquires (blocking):

This method of the new lock object is used to force threads to run synchronously. It accepts an optional blocking parameter that enables us to control whether the thread waits to acquire the lock. If the value of blocking is set to 0, then the thread returns immediately with a 0 value if the lock cannot be acquired and with a 1 if the lock was acquired. Suppose the value of blocking is set to 1 then the thread blocks and wait for the lock to be released.

Method release (): This method of the new lock object is used to release the lock when it is no longer required.

Let’s understand thread synchronization with the help of the following example.

Output

When we execute the above Python program, we will observe the following output.

Apply now for Advanced Python Training Course