Commit deb1940e authored by Médéric Boquien's avatar Médéric Boquien

Expand the SharedArray class so that from the point of view of the rest of the...

Expand the SharedArray class so that from the point of view of the rest of the code it behaves like a Numpy array. This is a very minimal implementation that works with the operations we do on RawArrays. Internally we use a RawArray or a Numpy array representation depending on whether the operation done is scalar or vector.
parent aeb0e6d7
......@@ -25,24 +25,59 @@ def get_info(cls):
class SharedArray(object):
"""Class to Create a shared array that can be read/written by parallel
"""Class to create a shared array that can be read/written by parallel
processes, were data related to the models is going to be stored. For
memory efficiency reasons, we use RawArrays that will be passed in argument
to the pool. Each worker will fill a part of the RawArrays. It is
important that there is no conflict and that two different workers do
not write on the same section.
To simplify the interface, from the point of view of the rest of the code,
this will behave like a regular Numpy array. This is a minimal
implementation and if new operations are done on these arrays, it may be
necessary to define them here.
"""
def __init__(self, size):
self.array = RawArray(ctypes.c_double, size)
"""The RawArray is stored in raw, which is protected by a setter and
a getter. The array property returns raw as a regular Numpy array. It
is important to access both the RawArray and the Numpy array forms. The
conversion from a RawArray to a Numpy array can be costly, in
particular if it is to just set or get an element. Conversely a
RawArray is dramatically slower when using slices. To address this
issue we selectively work with array or raw depending on whether the
operation is done with a slice or not.
"""
self.raw = RawArray(ctypes.c_double, size)
self.size = size
def __setitem__(self, idx, data):
if isinstance(idx, slice):
self.array[idx] = data
else:
self.raw[idx] = data
def __getitem__(self, idx):
if isinstance(idx, slice):
return self.array[idx]
return self.raw[idx]
def __len__(self):
return self.size
def __rmul__(self, other):
return other * self.array
@property
def array(self):
return np.ctypeslib.as_array(self._array)
return np.ctypeslib.as_array(self.raw)
@property
def raw(self):
return self._raw
@array.setter
def array(self, array):
if isinstance(array, ctypes.Array):
self._array = array
@raw.setter
def raw(self, raw):
if isinstance(raw, ctypes.Array):
self._raw = raw
else:
raise TypeError("Type must be RawArray.")
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment