Astropy Proposal for Enhancement 6: Enhanced Character Separated Values table format (APE 6)

Main Authors: Aldcroft, Tom, Taylor, Mark
Format: Document publication-technicalnote Journal
Terbitan: , 2021
Online Access: https://zenodo.org/record/4792325
Daftar Isi:
  • Data tables in astronomical analysis frequently contain additional metadata beyond just the column names and data values. Common attributes include the numerical data type, the physical unit, and a longer textual description of the column content. These attributes can be reprented in binary formats such as FITS, but the available options for a text-only format are inadequate. APE6 proposes to fill this void by specification of a standard for the interchange of tabular data in a text-only format. The format handles the key issue of serializing column specifications and table metadata by using a YAML-encoded data structure. The actual tabular data are stored in a standard character separated values (CSV) format, giving compatibility with a wide variety of non-specialized CSV table readers. Using YAML makes it extremely easy for applications *and humans* to read both the standardized data format elements (e.g. column name, type, description) as well as complex metadata structures. YAML also lends itself to simple table modifications by humans in a plain text editor. The reference Python implementation in ``astropy.io.ascii`` is relatively straightforward and will provide a significant benefit of allowing text serialization of most astropy Table objects, persistent storage, and subsequent interchange with other users. Although developed in the context of Astropy, there is nothing astronomy-specific in this format and it could be useful for other domains or languages. However, promoting that is beyond the scope of this APE.