Remove the SED parts dedicated to the lines as lines are perfectly generic now. No need to have this still hanging around.

If the code is to be run only on 1 core, then there is no need to fork a new process for that. As a side effect, it should at least allow people who have a multiprocessing code to run the code, even if only on one core.

Improve memory management. Several techniques are used. First, we use arrays in shared memory when computing the SEDs (this should also be done for the analysis later). This avoids using temporary objects that have to be sent back into a list from pool.starmap. Second, we do not save the sed.info data until the best sed has been computed. Only then we recompute the SED to get the info. Third, the list of changed parameters for cache cleaning was too big. It contained the key and the value of the changed parameter to identify models to be discarded. Now we simply return an array of integers giving the index of the module to be discarded. Much simpler, much smaller. There are also some small improvements here and there that I probably forget. The main culprit now is by far the list of parameters. It gets horribly big. 3.5 GB for 30M models. Anyway, now pcigale runs with 30M models.

Remove the variables. The file does not need to be initialised upon load anyway. The module description has been also clarified.

Change how the IMF is handled. Now rather than a string with require an integer. It is slightly less user friendly but it will make possible to store all the sed.info in a numpy array, which reduces the amount of memory required per model.

Make Denis happy. Now we display the number of models/objects already computed/analysed along with the time spent and the speed.

Prevent the model computation worker from returning a view of sed.info.values(). This meant the entire dictionary was actually copied, wasting a lot of memory. Also adopt a usual list rather than a numpy array to store the info in pdf_analysis.

When creating the SED of the best SED, select only parameters at a given redshift first as the index of the best χ² is computed on this subset.

Make model computations much faster. On a test run we spent 20% of the time just computing the luminosity distance. Now the information is just provided by the redshifting module. If the information is not present we just assume a distance of 10 pc.

Treat the total galaxy mass as a normal parameter. The special handling of galaxy_mass already got removed in the previous commit.

The initial implementation of the parallel analysis ended up grinding pcigale to a halt. This was due to the numerous array transfers between the main process and subprocesses. To solve this problem, we now share arrays through a module. This has the advantage
that thanks to copy-on-write, we almost never need to actually copy arrays. Now all the subprocessing run at full speed. Quite a few changes for a patch that only starting with the activation of cache clearing.

Implement partial cache clearing in order not to clog the memory with models that are not necessary anymore. This requires the proper computation of models that will not be used anymore. This is why we used ordered dictionaries to ensures that parameters are always considered in the same order.

If there is no redshift information, we assumed that the redshift is 0. Also correct a bogus test in the redshifting module.

Make pdf_analysis parallel. This required important changes. Functions saving data have been moved to the utils.py file. Workers for the parallel processes have been put in workers.py. To make sure we eliminate models incompatible with the age of the universe now the latter is indicated in sed.info for easy access. Ditto for the redshift.

Move the plotting functions out of the analysis modules. Plots are now generated using an external script named pcigale-plots. This script is ready for multiprocessing but it is not yet activated. Finally, we can now save the reduced χ² values (which is required to plot them).

Tiny optimisation. Create arrays are empty rather than being zero-filled as we are going to assign all elements later on anyway.

Change the way redshifts are handled. Now all redshifts are specified in pcigale.ini. To compute the χ², we simply take the models corresponding to the closest χ².