Commit 262eefc2 authored by Médéric Boquien's avatar Médéric Boquien

Check against the duplication of id in the input file and emit an Exception when relevant.

parent f33d19cf
Pipeline #2105 skipped with stage
......@@ -22,6 +22,7 @@
- The computation of all the cosmology-dependent quantities has been consolidated in pcigale/utils/cosmology.py and optimised. This leads to a slightly faster startup, in particular when there are many objects to fit, and it makes it easier to change the cosmology. (Médéric Boquien)
- The time spent computing is now displayed in hours, minutes, and seconds rather than just seconds to improve legibility. (Médéric Boquien)
- Some expected warnings have been silenced in `pcigale-plots` to avoid flooding the terminal. (Médéric Boquien)
- The id given in the input files are now checked against duplications. In case there are duplications, an exception is emitted. This prevents possible crashes later in the run. (Médéric Boquien)
### Fixed
- Make sure we can plot the PDF of equivalent widths. (Médéric Boquien)
- Fix a crash when generating a mock catalogue containing intensive properties. (Médéric Boquien)
......
......@@ -62,6 +62,7 @@ class ObservationsManagerPassbands(object):
self.tofit_err = self.bands_err + self.intprops_err + self.extprops_err
# Sanitise the input
self._check_id()
self._check_filters()
self._check_errors(defaulterror)
self._check_invalid(config['analysis_params']['lim_flag'],
......@@ -91,6 +92,19 @@ class ObservationsManagerPassbands(object):
return obs
raise StopIteration
def _check_id(self):
  • If you just want to know if there are duplicated id, why not just a len(table) - len(set(table['id']))?

    Edited by Yannick Roehlly
  • Because I want to give the list of duplicated id.

  • Oops, I didn't see the ", ".join(dup_set).

    Then using numpy should be faster:

    value, counts = np.unique(self.table['id'], return_counts=True)
    duplicates = values[counts > 1]
Please register or sign in to reply
"""Check that the id of each object is unique. If not trigger an
exception as it may cause issues down the road.
"""
seen_set = set()
dup_set = set(i for i in self.table['id']
if i in seen_set or seen_set.add(i))
if len(dup_set) > 0:
raise Exception("The input file has the following duplicated id: " +
", ".join(dup_set) + ". The id must be unique.")
def _check_filters(self):
"""Check whether the list of filters and poperties makes sense.
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment