Move read_table() and Counter() to a top-level utils directory as they are used both by pcigale and pcigale-plots.

Do display the model counter twice when the numbers of models is a multiple of the printing frequency.

The counter can become a bottleneck when many parallel processes are running concurrently because of the lock to update it. To address this we implement a Counter class that only updates the counter periodically rather than at each iteration. The number of models actually computed at any moment is kept in a local counter.

Switch savefluxes to using a models manager and make all the necessary adjustments with the workers.

Rename ParametersHandler to ParametersManager to be consistent with new managers that will come in the near future.

Move the backup_dir() function to the base class of the analysis modules as it makes sense for analysers to prepare the output directories.

Remove the output_file and output_format parameters from the savefluxes module. Now results are systematically saved as ASCII and FITS tables.

The optionally saved spectra in the pdf_analysis and savefluxes modules were saved in the VO-table format. The most important downside is that it is very slow to write to, which proves to be a major bottleneck in the computing speed. To overcome this issue, we rather save the spectra using the FITS formation. Instead of having one file containing the spectra (including the various components) and the SFH in a single file, now we have one file for the spectra and one file for the SFH.

Change the vague column_list keyword in pcigale.ini to bands. This will require the user to rebuild their pcigale.ini but that is necessary for the validation anyway so that is the right moment to do so.

Pratical implementation of the validation of the parameters. The patch is quite long as it has a direct effect on the structure of the configuration dictionary. The validation has the advantage of automatically convert the parameters to the right type. Therefore rather than building a dictionary ourselves, we rather use the ready-made dictionary from ConfigObj. Because the names of the sections are not the same, quite a bit of code had to be adapted. Finally, note that the validation file containing the specification of each variable, pcigale.ini.spec, is created while building the pcigale.ini file. Also because it is needed to convert the data to the right type, one cannot run cigale without a correct validation file.

Specify the types and the constraints of the input parameters for the SED creation modules and the analysis modules. Once validation is activated, the validation module will be able to infer the constraints from the specified information.

Abstract the call to the parameters handler to allow for the creation of different handlers depending on how the parameters are provided, for instance either a systematic grid by giving then in pcigale.ini or through a file with each line providing a the parameters for a different model. The change is transparent to the rest of pcigale so it does not matter which handler is actually used.

If the list of analysed/saved physical properties is left empty in pcigale.ini, savefluxes saved all physical properties. This has been moved to the Configuration class and easily extended to the pdf_analysis module.

When the redshifts are not indicated in pcigale.ini, compute them from the input flux file when the configuration dictionary is returned rather than in the analysis module, which should not have to change the configuration.

Simplify the interface of the analysis modules as it is needlessly restrictive by forcing a precise list of parameters. Different modules do not necessarily need the same information and the need for different input parameters may emerge in the future. To make the interface future-proof, we simply pass the file containing the configuration file. Then each analysis module picks the data it needs from it.

Introduce the possibility of saving only specific output parameters with savefluxes. This should accelerate the computation while reducing memory usage.

We need to sort the sed.info before saving the data. The reason is that the dictionary orders the keys with a hash so it appears random. In addition the order also changes from one process to another (I suspect the hash also depends on a seed). Thanks to Yannick for pointing out this issue.

We rely on OrderedDict to keep the order of the parameters. However given the way the code is structured currently, there is really no reason for this. So let's just switch to the standard dictionary. This has two advantages. First the standard dictionary is much faster than OrderedDict (implemented in C rather than in pure python). Then, standard dictionaries are marshalable, which makes them usable as keys for other dictionaries.

Implement a cache for the filters to compute the fluxes. Rather than passing the transmission table as an argument to compute_fnu(), which makes caching difficult and/or slow to compute the hash, we rather pass the filter name. Then the SED object fetches the filter from the database and resamples it on the optimal wavelength grid. The result is stored in cache to avoid carrying out this operation repeatedly.

We use only a memory-based cache, so we get rid of the option. For the time being we keep the architecture in place.

Add back the possibility to read the redshifts from the input file. To do so, do not give any redshift to the redshifting module. To avoid having too many redshifts, it is rounded to 2 decimals. This number can be changed in the analysis modules.

Rewrite the savefluxes module to make use of the new infrastructure. Though this implementation is not terribly scalable in case of some really large arrays. It should be enough for now anyway. Famous last words.