Commit d0759666 authored by Yannick Roehlly's avatar Yannick Roehlly

[WIP] Correct function adjusting data

Correct the function adjusting the data and implements a new way to indicate
upper limits.

FIXME: This breaks other parts of the code that must be corrected because now
upper limits are indicated with a negative flux.
parent b78e682a
......@@ -3,6 +3,18 @@
# Unreleased
### Added
### Changed
- The way to indicate the fluxes in the input table has changed because there
was bugs in the way upper limits were handled. Here are the new rules:
- When the flux is around 0 (i.e. it's absolute value is lower than the
tolerance threshold) or when it's lower than -9990 (typically -9999) it
is considered as invalid and the source is not analysed.
- When the flux is negative (between -threshold and -9990) this indicates
a upper-limit: the value of the upper limit is given in the flux column
and the value of the error on the upper limit is given in the error
column.
- When the error is negative or below the tolerance threshold, it is
replace by the default error.
- The tolerance threshold is 1e-12.
- Various descriptions have been improved and clarified. (Médéric Boquien)
- The `output\_file` and `output\_format` parameters have been removed from the `savefluxes` module. They served little purpose and made the code more complex. The same strategy as for the `pdf\_analysis` modules is now adopted, saving the output both as FITS and ASCII tables. Old configuration file still work, with these two parameters simply ignored. (Médéric Boquien)
......
......@@ -127,9 +127,21 @@ def get_module(module_name):
def adjust_data(fluxes, errors, tolerance, lim_flag, default_error=0.1,
systematic_deviation=0.1):
"""Adjust the fluxes and errors replacing the invalid values by NaN, and
adding the systematic deviation. The systematic deviation changes the
errors to: sqrt(errors² + (fluxes*deviation)²)
"""Adjust fluxes and errors
Adjust fluxes and errors provided by the user to fit CIGALE prescriptions:
- When the flux is around 0 (i.e. it's absolute value is lower than the
tolerance threshold) or when it's lower than -9990 (typically -9999) it
is considered as invalid and both flux and error are set to NaN.
- When the flux is negative (between -threshold and -9990) this indicates
a upper-limit. If the lim_flag is set to false, the flux is marked as
invalid.
- When the error is negative or below the tolerance threshold, it is
replace by the default error.
For valid fluxes, a systematic deviation is added that change the error to
sqrt(errors² + (fluxes*deviation)²).
Parameters
----------
......@@ -149,6 +161,8 @@ def adjust_data(fluxes, errors, tolerance, lim_flag, default_error=0.1,
Returns
-------
fluxes: array of floats
The corrected fluxes.
error: array of floats
The corrected errors.
......@@ -162,25 +176,24 @@ def adjust_data(fluxes, errors, tolerance, lim_flag, default_error=0.1,
fluxes = fluxes.copy()
errors = errors.copy()
# Valid fluxes mask
valid_mask = np.isfinite(fluxes)
if lim_flag:
valid_mask &= ~(np.abs(fluxes) <= tolerance) | (fluxes < -9990.)
else:
valid_mask &= ~(fluxes <= tolerance)
# We set invalid data to NaN
mask_invalid = np.where((fluxes <= tolerance) | (errors < -9990.))
fluxes[mask_invalid] = np.nan
errors[mask_invalid] = np.nan
fluxes[~valid_mask] = np.nan
errors[~valid_mask] = np.nan
# Replace missing errors by the default ones.
mask_noerror = np.where((fluxes > tolerance) & ~np.isfinite(errors))
errors[mask_noerror] = (default_error * fluxes[mask_noerror])
# Replace upper limits by no data if lim_flag==False
if not lim_flag:
mask_limflag = np.where((fluxes > tolerance) & (errors < tolerance))
fluxes[mask_limflag] = np.nan
errors[mask_limflag] = np.nan
no_error_mask = valid_mask & (errors <= tolerance)
errors[no_error_mask] = default_error * np.abs(fluxes[no_error_mask])
# Add the systematic error.
mask_ok = np.where((fluxes > tolerance) & (errors > tolerance))
errors[mask_ok] = np.sqrt(errors[mask_ok]**2 +
(fluxes[mask_ok]*systematic_deviation)**2)
errors[valid_mask] = np.sqrt(
errors[valid_mask]**2 + (fluxes[valid_mask] * systematic_deviation)**2)
return fluxes, errors
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment