Commit 9440c959 authored by Médéric Boquien's avatar Médéric Boquien

Implement the analysis of the observations by blocks of models.

parent 86300034
......@@ -6,9 +6,12 @@
- An option `redshift\_decimals` is now provided in `pdf\_analysis` to indicate the number of decimals to round the observed redshifts to compute the grid of models. By default the model redshifts are rounded to two decimals but this can be insufficient at low z and/or when using narrow-band filters for instance. This only applies to the grid. The physical properties are still computed for the redshift at full precision. (Médéric Boquien)
- Bands with negative fluxes are now considered valid and are fitted as any other band. (Médéric Boquien)
- Allow the models to be computed by blocks in `savefluxes`. This can be useful when computing a very large grid and/or to split the results file into various smaller files as large files can be difficult to handle. The number of blocks is set with the `blocks` parameters in the pcigale.ini. (Médéric Boquien)
- Allow the observations to be analysed by blocks of models in `pdf\_analysis`. This is useful when computing a very large grid of models that would not fit in memory. The number of blocks is set with the `blocks` parameters in the pcigale.ini. (Médéric Boquien)
### Changed
- Make the timestamp more readable when moving the out/ directory. (Médéric Boquien)
- To accommodate the analysis of the observations by blocks, all models are now included in the estimation of the physical properties and no cut in chi² is done anymore. (Médéric Boquien)
- To accommodate the analysis of the observations by blocks, the `save_pdf` option has been eliminated. To plot PDF one needs to set `save_chi2` to True and then run `pcigale-plots pdf`. (Médéric Boquien)
### Fixed
- Corrected a typo that prevented `restframe\_parameters` from being listed among the available modules. (Médéric Boquien)
......
......@@ -7,44 +7,6 @@
Various utility functions for pcigale analysis modules
"""
import numpy as np
from astropy import log
from astropy.table import Table, Column
log.setLevel('ERROR')
def save_fluxes(model_fluxes, model_parameters, filters, names):
"""Save fluxes and associated parameters into a table.
Parameters
----------
model_fluxes: RawArray
Contains the fluxes of each model.
model_parameters: RawArray
Contains the parameters associated to each model.
filters: list
Contains the filter names.
names: List
Contains the parameters names.
"""
out_fluxes = np.ctypeslib.as_array(model_fluxes[0])
out_fluxes = out_fluxes.reshape(model_fluxes[1])
out_params = np.ctypeslib.as_array(model_parameters[0])
out_params = out_params.reshape(model_parameters[1])
out_table = Table(np.hstack((out_fluxes, out_params)),
names=filters + list(names))
out_table.add_column(Column(np.arange(model_fluxes[1][0]), name='id'),
index=0)
out_table.write("out/computed_fluxes.fits")
out_table.write("out/computed_fluxes.txt", format='ascii.fixed_width',
delimiter=None)
def nothread():
"""Some libraries such as Intel's MKL have automatic threading. This is
good when having only one process. However we already do our own
......
......@@ -49,11 +49,13 @@ def _chi2_worker(obj_name, var_name):
Name of the analysed variable..
"""
if os.path.isfile("out/{}_{}_chi2.fits".format(obj_name, var_name)):
chi2 = Table.read("out/{}_{}_chi2.fits".format(obj_name, var_name))
fname = "out/{}_{}_chi2.npy".format(obj_name, var_name)
if os.path.isfile(fname):
data = np.memmap(fname, dtype=np.float64)
data = np.memmap(fname, dtype=np.float64, shape=(2, data.size // 2))
figure = plt.figure()
ax = figure.add_subplot(111)
ax.scatter(chi2[var_name], chi2['chi2'], color='k', s=.1)
ax.scatter(data[1, :], data[0, :], color='k', s=.1)
ax.set_xlabel(var_name)
ax.set_ylabel("Reduced $\chi^2$")
ax.set_ylim(0., )
......@@ -77,11 +79,34 @@ def _pdf_worker(obj_name, var_name):
Name of the analysed variable..
"""
if os.path.isfile("out/{}_{}_pdf.fits".format(obj_name, var_name)):
pdf = Table.read("out/{}_{}_pdf.fits".format(obj_name, var_name))
fname = "out/{}_{}_chi2.npy".format(obj_name, var_name)
if os.path.isfile(fname):
data = np.memmap(fname, dtype=np.float64)
data = np.memmap(fname, dtype=np.float64, shape=(2, data.size // 2))
likelihood = np.exp(-data[0, :] / 2.)
model_variable = data[1, :]
Npdf = 100
min_hist = np.min(model_variable)
max_hist = np.max(model_variable)
Nhist = min(Npdf, len(np.unique(model_variable)))
if min_hist == max_hist:
pdf_grid = np.array([min_hist, max_hist])
pdf_prob = np.array([1., 1.])
else:
pdf_prob, pdf_grid = np.histogram(model_variable, Nhist,
(min_hist, max_hist),
weights=likelihood, density=True)
pdf_x = (pdf_grid[1:]+pdf_grid[:-1]) / 2.
pdf_grid = np.linspace(min_hist, max_hist, Npdf)
pdf_prob = np.interp(pdf_grid, pdf_x, pdf_prob)
figure = plt.figure()
ax = figure.add_subplot(111)
ax.plot(pdf[var_name], pdf['probability density'], color='k')
ax.plot(pdf_grid, pdf_prob, color='k')
ax.set_xlabel(var_name)
ax.set_ylabel("Probability density")
ax.minorticks_on()
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment