data_management.md

Data management

Introduction

ANIS provides scientific projects and for each project datasets extracted from databases.

ANIS also allows to associate files such as fits, png, xml, etc, to illustrate projects and datasets.

This page describes how to associate those files with projects and datasets.

Server data path

First of all you have to specify the directory containing the future files and data. This directory becomes the root directory for the ANIS server. Go to the docker-compose.yml file to configure the DATA_PATH environment variable.

Here you can see that the data directory is configured to /data.

While downloading and installing ANIS, remember you downloaded examples file to start the app. You then symlinked them to the directory data at the root of the project. The latter files and directory find their use now. They are mounted in the server container at the root of the ANIS server for it to serve them properly.

Admin files explorer

An administrator can list files or folders contained in the root directory. Please note that if authentication is enabled you will need to send a token to perform this action.

If you want to explore the ANIS root directory type:

$ curl "http://localhost:8080/admin-file-explorer"

If you want to explore the DEFAULT directory type:

$ curl "http://localhost:8080/admin-file-explorer/DEFAULT"

Instance data path

You can associate a data directory for each of your scientific projects. The directory path will start from the data root directory of anis-server. To configure the data path of a project you have to fill in the property data_path in the metamodel.

Example:

$ curl -d '{
  "name": "default",
  "label": "Default instance",
  "description": "Instance for the test",
  "scientific_manager": "M. Durand",
  "instrument": "Multiple",
  "wavelength_domain": "Visible imaging / Spectroscopy",
  "display": 10,
  "data_path": "/DEFAULT",
  "files_path": "/INSTANCE_FILES",
  "public": true,
  "portal_logo": "",
  "design_color": "#7AC29A",
  "design_background_color": "#ffffff",
  "design_logo": "/logo.png",
  "design_favicon": "/favicon.ico",
  "samp_enabled": true,
  "back_to_portal": true,
  "search_by_criteria_allowed": true,
  "search_by_criteria_label": "Search",
  "search_multiple_allowed": false,
  "search_multiple_label": "Search multiple",
  "search_multiple_all_datasets_selected": false,
  "documentation_allowed": false,
  "documentation_label": "Documentation"
}' \
--header 'Content-Type: application/json' \
-X POST "http://localhost/instance"

You can also configure a directory for storing files related to the project: files_path. The directory path will start from the data path directory of the project. Files related to the project such as the logo, images will be stored in this directory.

Instance files explorer

A user can list files or folders contained in the instance files path directory. Please note that if the instace is private you will need to send a token to perform this action.

If you want to explore the files for default project directory type:

$ curl "http://localhost:8080/instance/default/file-explorer"

Dataset data path

You can associate a data directory for each of your datasets. The directory path will start from the scientific project root directory (instance data_path). To configure the data_path of a dataset you have to fill in the property in the metamodel.

Example:

$ curl -d '{
  "name": "vipers_dr2_w1",
  "table_ref": "aspic_vipers_dr2_w1",
  "label": "VIPERS-W1 (DR2)",
  "description": "VIPERS W1 dataset",
  "display": 10,
  "data_path": "/ASPIC/VIPERS_DR2",
  "public": true,
  "download_json": true,
  "download_csv": true,
  "download_ascii": true,
  "download_vo": false,
  "server_link_enabled": false,
  "datatable_enabled": true,
  "datatable_selectable_rows": false,
  "id_database": 1
}' \
--header 'Content-Type: application/json' \
-X POST "http://localhost/dataset-family/1/dataset"

Here you can see that the data directory for this dataset is set to /ASPIC/VIPERS_DR2

Dataset files explorer

A user can list files or folders contained in the dataset directory. Please note that if the dataset is private you will need to send a token to perform this action.

Example:

$ curl "http://localhost:8080/dataset/vipers_dr2_w1/file-explorer"

If you want explore the spec1d directory you can execute:

$ curl "http://localhost:8080/dataset/vipers_dr2_w1/file-explorer/spec1D"

If a user want to download the file VIPERS_101121877_bis.fits you can execute:

$ curl \
"http://localhost:8080/dataset/vipers_dr2_w1/file-explorer/spec1D/VIPERS_101121877_bis.fits"