Parallelization
Parallelization
Embedded Files
Large Scale Algorithms
Large Scale Algorithms
Large-scale algorithms are designed to solve gigantic complex problems. The characterizing feature of large-scale algorithms is their need to have more than one execution engine due to the scale of their data and processing requirements.
Human beings like to be challenged. For centuries, various human innovations have allowed us to solve really complex problems in different ways. From predicting the next target area of a locust attack to calculating the largest prime number, the methods to provide answers for complex problems around us kept on evolving. With the advent of the computer, we found a powerful new way to solve complex algorithms.
A well-designed, large-scale algorithm has the following two characteristics:
It is designed to handle a huge amount of data and processing requirements using an available pool of resources optimally.
It is scalable. As the problem becomes more complex, it can handle the complexity simply by provisioning more resources.
Threading and Multiprocessing
Threading and Multiprocessing
Threading is a sequence of instructions in a program that can be executed independently of the remaining process. You can see them as different units of your process that do jobs independently when scheduled. If they need to wait for a slow external operation to finish (such as a network request, or disk access), they sleep for a while and enable the scheduler to spend time executing another thread.
What is the Process?
A process is an executable instance of a computer program. Usually, a process is executed in a single sequence of control flow.
See the Key differences between thread and process in Python from the following table:
Threading is a way of achieving multitasking in Python. It allows a program to have multiple threads of execution simultaneously. Each thread runs independently and can perform different tasks concurrently. This means that if one thread is blocked or waiting for input/output, other threads can continue to run and keep the program responsive.
Python provides a threading module that makes it easy to create and manage threads in a program. With this module, you can create multiple threads, start them, and synchronize their execution.
Example of Multiprocessing:
import time
import multiprocessing
def count(num):
a = 0
for i in range(num):
a += i
time.sleep(1)
print(a)
if __name__ == '__main__':
p1=multiprocessing.Process(target=count,args=(50000,))
p1.start()
p2=multiprocessing.Process(target=count,args=(7000,))
p2.start()
print("the processes have started")
print("this message is show before the completion of a non joined process")
print("Starting newer processes")
p3=multiprocessing.Process(target=count,args=(100000,))
p3.start()
p4=multiprocessing.Process(target=count,args=(8000,))
p4.start()
# Join allows the program to wait till the processes have completed
p3.join()
p4.join()
print("This message is shown after the processes have terminated successfully")
"""
Output:
the processes have started
this message is show before the completion of a non joined process
Starting newer processes
24496500
1249975000
31996000
4999950000
This message is shown after the processes have terminated successfully
"""
Example of Threading:
import threading
import time
def count(num,id):
print(id,"Running")
a = 0
for i in range(num):
a += i
#One second delay
time.sleep(1)
print(id,a)
t1 = threading.Thread(target=count,args=(1000000,"Thread 1:"))
t1.start()
t2 = threading.Thread(target=count,args=(5000,"Thread 2:"))
t2.start()
t3 = threading.Thread(target=count,daemon=True,args=(2000000,"Thread 3:"))
t3.start()
print("Threads are running")
"""
Output:
Thread 1: Running
Thread 2: Running
Thread 3: Running
Threads are running
Thread 2: 12497500
Thread 1: 499999500000
"""
Why did thread 3 not complete its execution?
Beacuse it is a daemon thread. A daemon thread is a thread that runs in the background, and is not expected to complete its execution before the program exits. In order to solve this problem, we can add the following line after t3.start(): t3.join()
Page updated
Google Sites
Report abuse