Commit f29f13b0 authored by PARWORK Lambert's avatar PARWORK Lambert
Browse files
parents 4001b0ee bbe79613
......@@ -13,7 +13,7 @@ cat 2.txt | parallel -k
code.py 1
./code.py 1
seq 1 8 |parallel ./code.py {}
time seq 1 8 |parallel -j1 ./code.py {}
time seq 1 8 |parallel ./code.py {}
......
Study speed-up for "example2" with different N ( -size parameters ).
Study speed-up for "example2" with different N ( -size=10000 for example ).
Make sure that for big N speed-up is bigger.
......@@ -33,12 +33,3 @@ do
done
rm -f times_5000.txt
for N in `seq 1 12`
do
export OMP_NUM_THREADS=$N
python ../code3.py 5000 >> times_5000.txt
done
Parallelize code3.py with Pool+map.
1. rewrite code3.py with build in map function.
2. Parallelize it with Pool+map (see code2.py as an example).
3. Make sure that matrix multipication use only one thread per
process (use "mkl.set_num_threads(1)" in the code and "export
OMP_NUM_THREADS=1" in the batch file).
4. Compare perforamance of code3.py with your new code.
5. We've made sure that we use only one thread per process. Now, try to
set 12 threads per process (set "mkl.set_num_threads(12)" in the code
and OMP_NUM_THREADS=12 in the batch file) and see how performance will
degrade.
1. Read documentation about apply_async method of multiprocessing.Pool
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.apply_async
This might be useful for server application where you are
continuously receiving data and want to treat them in parallel.
2. See code5.py which is a parallel version code1.py with apply_async
3. Compare performance of code2.py (version with Pool+map) and
code5.py. code5.py is slower because it is equivalent to code2.py
with chunksize=1.
#!/usr/bin/env python
import numpy as np
import argparse
from multiprocessing import Pool
import mkl
# To be sure that we use 1 thread in MKL
mkl.set_num_threads(1)
# Parse command line arguments
parser = argparse.ArgumentParser()
parser.add_argument("Nm", help="Number of matrices", type=int);
parser.add_argument("-ncores", help="Number of cores to use", type=int);
args = parser.parse_args()
Nm = args.Nm
......@@ -19,19 +13,10 @@ def fun(i):
np.random.seed(i)
A = np.random.rand (100, 100)
B = np.random.rand (100, 100)
return np.sum(np.dot(A, B))
p = Pool(args.ncores);
return np.min(np.dot(A, B))
async_rez = []
rez = 0
for i in range(Nm):
async_rez.append( p.apply_async(fun, (i,) ) )
rez = 0;
for i in range(Nm):
rez += async_rez[i].get()
rez += fun(i) * np.sqrt(i)
print (rez)
p.close()
p.join()
#!/usr/bin/env python
import numpy as np
import argparse
from multiprocessing import Pool
import mkl
# To be sure that we use 1 thread in MKL
mkl.set_num_threads(1)
# Parse command line arguments
parser = argparse.ArgumentParser()
parser.add_argument("Nm", help="Number of matrices", type=int);
parser.add_argument("-ncores", help="Number of cores to use", type=int);
args = parser.parse_args()
Nm = args.Nm
def fun(i):
np.random.seed(i)
A = np.random.rand (100, 100)
B = np.random.rand (100, 100)
return np.sum(np.dot(A, B))
p = Pool(args.ncores);
async_rez = []
for i in range(Nm):
async_rez.append( p.apply_async(fun, (i,) ) )
rez = 0;
for i in range(Nm):
rez += async_rez[i].get()
print (rez)
p.close()
p.join()
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment