GitLab

The optionally saved spectra in the pdf_analysis and savefluxes modules were saved in the VO-table format. The most important downside is that it is very slow to write to, which proves to be a major bottleneck in the computing speed. To overcome this issue, we rather save the spectra using the FITS formation. Instead of having one file containing the spectra (including the various components) and the SFH in a single file, now we have one file for the spectra and one file for the SFH.

Change the vague column_list keyword in pcigale.ini to bands. This will require the user to rebuild their pcigale.ini but that is necessary for the validation anyway so that is the right moment to do so.

Pratical implementation of the validation of the parameters. The patch is quite long as it has a direct effect on the structure of the configuration dictionary. The validation has the advantage of automatically convert the parameters to the right type. Therefore rather than building a dictionary ourselves, we rather use the ready-made dictionary from ConfigObj. Because the names of the sections are not the same, quite a bit of code had to be adapted. Finally, note that the validation file containing the specification of each variable, pcigale.ini.spec, is created while building the pcigale.ini file. Also because it is needed to convert the data to the right type, one cannot run cigale without a correct validation file.

Specify the types and the constraints of the input parameters for the SED creation modules and the analysis modules. Once validation is activated, the validation module will be able to infer the constraints from the specified information.

Abstract the call to the parameters handler to allow for the creation of different handlers depending on how the parameters are provided, for instance either a systematic grid by giving then in pcigale.ini or through a file with each line providing a the parameters for a different model. The change is transparent to the rest of pcigale so it does not matter which handler is actually used.

If the list of analysed/saved physical properties is left empty in pcigale.ini, savefluxes saved all physical properties. This has been moved to the Configuration class and easily extended to the pdf_analysis module.

When the redshifts are not indicated in pcigale.ini, compute them from the input flux file when the configuration dictionary is returned rather than in the analysis module, which should not have to change the configuration.

Simplify the interface of the analysis modules as it is needlessly restrictive by forcing a precise list of parameters. Different modules do not necessarily need the same information and the need for different input parameters may emerge in the future. To make the interface future-proof, we simply pass the file containing the configuration file. Then each analysis module picks the data it needs from it.

Assign the fluxes to the correct array. For some reason I was sure I corrected that before committing the changes last Friday.

Introduce the possibility of saving only specific output parameters with savefluxes. This should accelerate the computation while reducing memory usage.

Magic values are a bit of a pain to handle and are not safe. This patch removes the use of magic values and rather replaces them with NaN whenever appropriate. The logic of the input file stays the same but the magic values are converted internally after reading it so that the rest of the code does not have to deal with that.

As the code is much faster now, let's update the number of computed models a bit less often as printing something on screen causes a context switch

We need to sort the sed.info before saving the data. The reason is that the dictionary orders the keys with a hash so it appears random. In addition the order also changes from one process to another (I suspect the hash also depends on a seed). Thanks to Yannick for pointing out this issue.

We rely on OrderedDict to keep the order of the parameters. However given the way the code is structured currently, there is really no reason for this. So let's just switch to the standard dictionary. This has two advantages. First the standard dictionary is much faster than OrderedDict (implemented in C rather than in pure python). Then, standard dictionaries are marshalable, which makes them usable as keys for other dictionaries.

Implement a cache for the filters to compute the fluxes. Rather than passing the transmission table as an argument to compute_fnu(), which makes caching difficult and/or slow to compute the hash, we rather pass the filter name. Then the SED object fetches the filter from the database and resamples it on the optimal wavelength grid. The result is stored in cache to avoid carrying out this operation repeatedly.

Add a carriage return after we have computed the last model or analysed the last object so that the information does not get erased.