Venus Climate Orbiter, AKATSUKI Data sets Users' Guide

Shin-ya Murakami

2018-10-10

Revision History
Revision 0.1 2018-10-10
First draft.

Table of Contents

This document introduces Venus Climate Orbiter (VCO, also known as AKATSUKI and PLANET-C) data sets.

1.  Overview of the mission

Venus Climate Orbiter, also known as AKATSUKI and PLANET-C mission is to explore Venus atmosphere, especially the super-rotation of the atmosphere. It carries six instruments to observe Venus; Ultraviolet Imager (UVI), 1‐μm Camera (IR1), 2‐μm Camera (IR2), Longwave Infrared Camera (LIR), Lightning and Airglow Camera (LAC), and Ultra Stable Oscillator (USO) used by Radio Science experiment (RS).

The spacecraft was launched on 2010-05-20, and failed its first Venus orbit insertion (VOI-1) on 2010-12-07 due to malfunction of the main propulsion system. After the failure of VOI-1, it orbited around the Sun five years. On 2015-12-07, second trial of Venus orbit insertion (VOI-R1) had been done successfully using the reaction control system (RCS) thrusters.

The spacecraft started nominal observation of Venus after VOI-R1. Data acquired by the observation are archived in data sets. The following sections briefly introduce these data sets.

For more information on the mission, see mission.cat in PDS3 data sets (e.g., mission.cat.

For more information on the instruments, see *inst.cat in PDS3 data sets, uviinst.cat for UVI, ir1inst.cat for IR1, ir2inst.cat for IR2, lirinst.cat for LIR, rsinst.cat for RS (USO), SPICE FK, vco_spacecraft_v26.tf, and SPICE IKs, vco_uvi_v12.ti, vco_ir1_v11.ti, vco_ir2_v10.ti, vco_lir_v08.ti, vco_lac_v13.ti.

2.  How to get data?

2.1.  List of data sets

Before introducing the data distribution site, we'll briefly describe list of data sets. There are 26 data sets (List of Venus Climate Orbiter, Akatsuki Data sets.).

Table 1. List of Venus Climate Orbiter, Akatsuki Data sets.
instrument target product description DATA_SET_ID coverage
UVI Venus Level 1b raw data VCO-V-UVI-2-EDR-V1.0 2010-05-21 -- 2017-06-06
Level 2b calibrated data VCO-V-UVI-3-CDR-V1.0 2010-05-21 -- 2017-06-06
geometry geometry data VCO-V-UVI-3-SEDR-V1.0 2010-05-21 -- 2017-06-06
Level 3b gridded data -- (non-PDS) 2010-12-09 -- 2017-06-06
CMV cloud motion vector -- (non-PDS) (not released yet)
IR1 Venus Level 1b raw data VCO-V-IR1-2-EDR-V1.0 2010-05-21 -- 2016-12-09
Level 2b, 2c calibrated data VCO-V-IR1-3-CDR-V1.0 2010-05-21 -- 2016-12-09
geometry geometry data VCO-V-IR1-3-SEDR-V1.0 2010-05-21 -- 2016-12-09
Level 3b, 3c gridded data -- (non-PDS) 2010-12-09 -- 2016-12-09
CMV cloud motion vector -- (non-PDS) (not released yet)
IR2 Venus Level 1b raw data VCO-V-IR2-2-EDR-V1.0 2010-05-21 -- 2016-12-09
Level 2b calibrated data VCO-V-IR2-3-CDR-V1.0 2010-05-21 -- 2016-12-09
geometry geometry data VCO-V-IR2-3-SEDR-V1.0 2010-05-21 -- 2016-12-09
Level 3b gridded data -- (non-PDS) 2010-12-09 -- 2016-12-09
CMV cloud motion vector -- (non-PDS) (not released yet)
LIR Venus Level 1b raw data VCO-V-LIR-2-EDR-V1.0 2010-05-21 -- 2017-06-06
Level 2b, 2c calibrated data VCO-V-LIR-3-CDR-V1.0 2010-05-21 -- 2017-06-06
geometry geometry data VCO-V-LIR-3-SEDR-V1.0 2010-05-21 -- 2017-06-06
Level 3b, 3c gridded data -- (non-PDS) 2010-12-09 -- 2017-06-06
CMV cloud motion vector -- (non-PDS) (not released yet)
LAC Venus Level 1 raw data VCO-V-LAC-2-EDR-V1.0 (not released yet)
RS Sun Level 1 raw data VCO-S-RS-2-OCC-V1.0 (not released yet)
Venus Level 1 raw data VCO-V-RS-2-OCC-V1.0 (not released yet)
Level 2 processed data VCO-V-RS-3-OCC-V1.0 2016-03-03 -- 2016-07-23
Level 3,4 processed data VCO-V-RS-5-OCC-V1.0 2016-03-03 -- 2016-07-23
SPICE -- -- SPICE kernels VCO-V-SPICE-6-V1.0 2010-05-21 -- 2017-12-31


There are two kinds of data sets, one is PDS3-compliant data set and the other is non-PDS3-compliant data set.

For PDS3-compliant data set, data set can be identified by DATA_SET_ID. DATA_SET_ID is keyword which is terminology in PDS (Planetary Data System; https://pds.nasa.gov).

For non-PDS3-compliant data set, we can identify data set by the combination of instrument and product, e.g., UVI Level 3b data set.

2.2.  Distribution sites

The data are currently available from DARTS/JAXA and PDS/NASA:

2.3.  How to retrieve data sets?

To retrieve data sets, please get .tar.xz or .tar.gz files and uncompress it.

  • For Linux and other modern Unix users, you can uncompress .tar.xz and .tar.gz files by tar(1) command, e.g., execute

    tar xf foo.tar.xz

  • For Windows users, you can uncompress .tar.xz and .tar.gz files by using 7-Zip (https://www.7-zip.org/).

3.  PDS3-compliant data set

In this section, we will introduce PDS33-compliant data sets.

3.1.  How is the directory structure?

3.1.1.  Naming of the top directory

You can find "vco-v-uvi-3-cdr-v1.0" directory that is lower case of DATA_SET_ID, "VCO-V-UVI-3-CDR-V1.0" ("DATA_SET_ID directory"). Under this directory, all of the calibrated UVI data are available.

3.1.2.  Naming of the directory under top directory

You can find "vcouvi_1001" directory that is lower case of VOLUME_ID, "VCOUVI_1001" ("VOLUME_ID directory") under "vco-v-uvi-3-cdr-v1.0" directory. The "VOLUME_ID has naming convention as the following format: VCOINST_XTVV, where

INST

abbreviation of instruments or experiments, UVI, IR1, IR2, LIR, LAC, RS, and SP.

X

product type of the data sets (0: raw data, 1: calibrated data, 2: geometry information data)

T

main target (0: Venus, 1: Sun)

VV

version of the release in two digits in decimal.

For the SPICE kernels data set, XTVV is set to 1000 regardless of the convention described above.

For example, vcouvi_1001 is for the 1st release of the calibrated data, and vcouvi_1002 is for the 2nd release.

3.1.3.  Directory structure under the "VOLUME_ID" directory

The directory structure under the "VOLUME_ID" directory is described in aareadme.txt, e.g., aareadme.txt in "VCO-V-UVI-3-CDR-V1.0" like the following:

   [top-level-directory]
   |
   |-- aareadme.txt           This file.
   |
   |-- errata.txt             Comments and errata on this volume.
   |
   |-- voldesc.cat            Description of the contents of this volume.
   |
   |--[browse]                Directory containing the JPEG files created from
   |  |                       files in the ``/data'' directory for browsing.
   |  |
   |  |-- browinfo.txt        Description of files in the browse directory.
   |  |
   |  +--[l2b]                Subdirectory for level 2b (l2b) data.
   |     |
   |     |-- list.html        HTML file including hyperlinks to the HTML
   |     |                    files in each orbit number string subdirectory.
   |     |
   |     |-- list.lbl         The corresponding detached PDS label file.
   |     |
   |     +--[PVVVV]           A series of subdirectories organized by the
   |        |                 orbit number string.
   |        |
   |        |-- list_*.html   HTML file that enumerates JPEG files in this
   |        |                 directory.
   |        |
   |        |-- *.jpg         A JPEG file created from corresponding
   |        |                 calibrated image in FITS format.
   |        |
   |        +-- *.lbl         The corresponding detached PDS label file.
   |
   |--[calib]                 Directory containing the calibration files.
   |  |
   |  |-- calinfo.txt         Description of files in the calib directory.
   |  |
   |  |-- *.fit               A flat-field image file used in the original
   |  |                       processing of the data, stored in FITS format.
   |  |                       A flat conversion factor file that can be used
   |  |                       to generate image suitable for photometry
   |  |                       analysis.
   |  |
   |  +-- *.lbl               The corresponding detached PDS label file.
   |
   |--[catalog]               Directory containing PDS catalog objects.
   |  |
   |  |-- catinfo.txt         Description of files in the catalog directory.
   |  |
   |  |-- dataset.cat         Description of the data set.
   |  |
   |  |-- uviinst.cat         Description of the instrument.
   |  |
   |  |-- uviref.cat          List of instrument-specific publications
   |  |                       mentioned in the uviinst.cat file.
   |  |
   |  |-- mission.cat         Description of the mission.
   |  |
   |  |-- person.cat          References for personnel who created this
   |  |                       volume.
   |  |
   |  |-- ref.cat             List of mission-wide publications mentioned in
   |  |                       *.cat files.
   |  |
   |  |-- target_dust.cat     Description of the target, dust.
   |  |
   |  |-- target_earth.cat    Description of the target, the earth.
   |  |
   |  |-- target_sun.cat      Description of the target, the sun.
   |  |
   |  |-- target_venus.cat    Description of the target, Venus.
   |  |
   |  +-- vcohost.cat         Description of the spacecraft.
   |
   |--[data]                  Directory containing the data files and PDS
   |  |                       labels describing the contents of those files.
   |  |
   |  +--[l2b]                Subdirectory for level 2b (l2b) data.
   |     |
   |     +--[PVVVV]           A series of subdirectories organized by the
   |        |                 orbit number string.
   |        |
   |        |-- *.fit         A VCO calibrated data product file in FITS
   |        |                 format.
   |        |
   |        +-- *.lbl         The corresponding detached PDS label file.
   |
   |--[document]              Directory containing volume related documents.
   |  |
   |  |-- docinfo.txt         Description of files in the document directory.
   |  |
   |  |-- uvical.txt          Description of calibration procedure.
   |  |
   |  |-- imagecoord.txt      Description of image coordinate used in this
   |  |                       dataset and corresponding indexes of the data
   |  |                       stored in FITS files.
   |  |
   |  |-- onlabels.txt        Description of PDS labels for this data set.
   |  |
   |  |-- vco_fits_dic_vVV.*  Header keyword dictionary file for Venus
   |  |                       Climate Orbiter FITS file in several formats
   |  |                       and corresponding detached PDS label file.
   |  |
   |  |-- vco_obsprg_vVV.*    Description of observation programs (sequence
   |  |                       of image acquisition) in several formats and
   |  |                       corresponding detached PDS label file.
   |  |
   |  +--[ft]                 Directory containing filter transmission data
   |     |                    files.
   |     |
   |     |-- ftinfo.txt       Description of filter transmission data files.
   |     |
   |     |-- ft_uvi_*.tab     Filter transmission data files.
   |     |
   |     +-- ft_uvi_*.lbl     PDS label for each filter transmission data
   |                          file.
   |
   +--[index]                 Directory containing index files.
      |
      |-- indxinfo.txt        Description of files in the index directory.
      |
      |-- checksum.tab        Table of MD5 checksums for files in this data
      |                       set.
      |
      |-- checksum.lbl        PDS label for checksum.tab file.
      |
      |-- index.tab           Index table of the data on this volume.
      |
      +-- index.lbl           PDS label for index.tab file.

The data are placed under data/ directory. There are subdirectories with "orbit number string" that is described in aareadme.txt.

For geometry information data sets, the main data are placed under geometry/ directory.

For quick-look purpose, see browse/ directory. Please find list.html file.

For calibrated data sets, there are files used to calibrated data in the data/ directory in calib/ directory. Note that data files are already calibrated, so you don't need to use files under calib/ directory in general.

The catalog files that are descriptive information about a data set, are placed under catalog/ directory. There are following files:

list of major catalog files
dataset.cat

detailed descriptions on the data set, e.g, dataset.cat in VCO-V-UVI-3-CDR-V1.0

mission.cat

mission overview on the data set, e.g, mission.cat in VCO-V-UVI-3-CDR-V1.0

ref.cat

references that are appeared in mission.cat, e.g, ref.cat in VCO-V-UVI-3-CDR-V1.0

vcohost.cat

description of the spacecraft for the mission on the data set, e.g, vcohost.cat in VCO-V-UVI-3-CDR-V1.0

INSTinst.cat

description for the instrument, INST, on the data set, e.g, uviinst.cat in VCO-V-UVI-3-CDR-V1.0

INSTref.cat

references that are appeared in the instrument, INST, on the data set, e.g, uviref.cat in VCO-V-UVI-3-CDR-V1.0

target_foo

general information of the target of the mission on the data set, e.g, target_venus.cat in VCO-V-UVI-3-CDR-V1.0

The documents except for catalog files are placed under document/ directory. There are following files:

list of major documents
onlabels.txt

an example of PDS33 labels for the data and brief description of PDS33 labels used in .lbl files in the data set, e.g, onlabels.txt in VCO-V-UVI-3-CDR-V1.0

imagecoord.txt

image coordinates used in the data set, e.g, imagecoord.txt in VCO-V-UVI-3-CDR-V1.0

INSTcal.txt

calibration procedures applied in the data set, e.g, uvical.txt in VCO-V-UVI-3-CDR-V1.0

vco_fits_dic_vVV.{html,txt,pdf}

FITS header keyword dictionary used in the data set, e.g, vco_fits_dic_v04.html in VCO-V-IR1-2-EDR-V1.0

vco_obsprg_vVV.{html,txt,pdf}

List of observation programs and descriptions, e.g, vco_obsprg_v03.html in VCO-V-IR1-2-EDR-V1.0

The summary of the data files is placed under index/ directory. The table of MD5 checksum also placed under this directory.

The extra data files are placed under extras/ directory. Currently, the list of executed observation programs are stored as ASCII text table in this directory, e.g, vco_obsprg_executed_v02.tab in VCO-V-UVI-3-CDR-V1.0

Except for data/ directory, there are fooinfo.txt files. These files briefly describe files in the directory.

3.2.  How to open data file?

UVI, IR1, IR2, and LIR data are provided in FITS format (see FITS Standard Page). The data are mostly compliant with FITS Standard Version 4.0, so you can open the FITS files of Akatsuki data by applications that support FITS Standard.

For example, the following applications can read Akatsuki's FITS files.

There are some libraries or packages to handle FITS format file. For example:

For LAC and RS data, the data are just an ASCII text table file (space-separated file). You can use Python, IDL, R, Gnuplot, etc.

3.3.  How to understand data?

To understand data, there are two ways: one is reading PDS3 label file (.lbl), and the other is reading header of the FITS file (.fit). The latter way can be used for UVI, IR1, IR2, and LIR data that are stored to FITS file.

3.3.1.  Getting meta-data from PDS3 label

In PDS3 label file (e.g., uvi_20151207_051953_283_l2b_v10.lbl), you can find the following lines:

/* TARGET INFORMATION */

TARGET_NAME                  = VENUS
VCO:SPHERICAL_RADIUS         = 6051.8 <km>
VCO:ASSUMED_CLOUD_ALTITUDE   = 70.0 <km>

Meta data are described in "keyword = value" format. The "TARGET_NAME" describes name of target, VENUS.

You can also find the following keywords: INSTRUMENT_NAME, FILTER_NAME, EXPOSURE_DURATION, etc.

There is a way to describe meta data associated with not a whole part of the file but a part of the file. The OBJECT block defines a named group of the meta data and points to a part of the file by a pointer. An example of a pointer that points to a part of the file by the OBJECT block named "UVI_LEVEL2B_IMAGE". A pointer is expressed as a leading caret, "^", and name of the block, "UVI_LEVEL2B_IMAGE". An assignment of value of the pointer is expressed as follows:

^UVI_LEVEL2B_IMAGE           = ("uvi_20151207_051953_283_l2b_v10.fit", 8)

and an example of OBJECT block is as follows:

OBJECT                       = UVI_LEVEL2B_IMAGE
  LINE_SAMPLES               = 1024
  LINES                      = 1024
  SAMPLE_BITS                = 32
  SAMPLE_TYPE                = IEEE_REAL
  UNIT                       = "W m**-2 sr**-1 m**-1"
  MAXIMUM                    = 2.56932E+07
  MINIMUM                    = -759021    
  MEAN                       = 6.64511E+06
  STANDARD_DEVIATION         = 7.81745E+06
  UNKNOWN_CONSTANT           = NULL  /* dead pixel flag value */
  INVALID_CONSTANT           = UNK  /* saturated pixel flag value */
  MISSING_CONSTANT           = -3.4E+38
  LINE_DISPLAY_DIRECTION     = UP
  DESCRIPTION                = "Data of secondary HDU of UVI L2B FITS
                                file that contains the image data."
END_OBJECT                   = UVI_LEVEL2B_IMAGE

You can find descriptions of meta-data from the following:

Data dictionary for mission-specific entries is considered to be merged to the official PDS Data Dictionary. Until the they will be merged, the dictionary will put somewhere (TBD).

3.3.2.  Getting meta-data from FITS header

FITS file has an unit called as Header Data Unit ("HDU"). HDU has two parts: the header and the data. For usual FITS data file in the Akatsuki's data sets, the primary HDU has general information of the file in the header and no data in the data part, and the secondary HDU has main image data in the data part and its header in the header part.

In the header of FITS file, there are several records with each record of 80 characters. One record usually contains keyword and its value, separated by "=". You can see records by pager like less(1) in the terminal with 80 characters-width. There are various keywords, e.g., OBJECT (target of the image), INSTRUME (instrument used to acquire image), FILTER (filter used to acquire image), EXPOSURE (exposure time for acquiring image). See VCO FITS header keyword dictionary for the details.

You can extract a value of keyword by your script. For example, to extract value of P_ID that describes a combination of the instrument and the filter/channel used to acquire image in string from uvi_20151207_051953_283_l2b_v10.fit, use the following Python 3 script with the Astropy package:

from astropy.io import fits

hdulist = fits.open('uvi_20151207_051953_283_l2b_v10.fit')
p_id = hdulist[1].header['P_ID']
print(p_id)

and the output of this script will be:

VCO_UVI_283

This output means the image is acquired by "VCO_UVI_283" as NAIF name that represents UVI with the 283-nm filter onboard VCO (Akatsuki).

On special flag values, see catalog/dataset.cat for each data set.

4.  uncategorized contents under preparation

4.1.  How to find the specific data?

Use index/index.tab. Each column are described in index/index.lbl.

4.2.  Fundamental things on image

image coordinates, ROI function, locate subimage in the full-size image coordinate, upward of the image corresponds to +Y-axis of the spacecraft, +Y-axis direction of the spacecraft, geometry information.

4.3.  Knowing quality of the data

Events related to S/C are found in mission.cat. Events on each instrument are found in dataset.cat.

UVI, IR1, IR2, LIR: salvaged area

UVI: filter wheel position.

IR1, IR2: four quadrants, smear (characteristics of detector using CSD).

IR1: temperature?

IR2: temperature

LIR: temperature, deep space pixels

LIR, L2c: interpolated temperature in time used to calibrate data.

4.4.  Q and A (Trouble shooting)

Q and A (Trouble shooting)

4.4.1.

Q1...

A1...

4.5.  Data products and stored data type

Let us introduce product type or data type (integer, float).

There are three kinds of products:

  • EDR: uncalibrated, count value data

  • CDR: calibrated data with physical dimension

  • SEDR: geometry information data

  • EDR, CDR, SEDR is shorthanded name (defined by CODMAC ?).

  • EDR is count value data, so data type is 16 bit signed integer.

  • CDR is data with physical dimension, so data type is 32 bit floating-point number.

  • SEDR will be described later (in section 6).

a. Check by header

For the value of BITPIX, positive sign means integer type, and number means bits representing one data.

Summary:

BITPIX  data type
------  ----------------------------
16      16 bit singed integer
-32     32 bit floating point number

You can also check it by filename, however, this is categorized by what the value means.

level    quantity           data type
-------  -----------------  ----------------------------
l1b      count value        16 bit singed integer 
l2b,l2c  physical quantity  32 bit floating point number

Reading BITPIX header keyword in FITS file by Python script is as follows:

...

4.6.  Check the size of image

Introduce structure of FITS file, FITS keyword dictionary

Size of image is NAXIS1 and NAXIS2 in HDU 1 (1st IMAGE extention)

Size of image can vary for UVI, IR1, and IR2 because of reducing data to downlink.

All of the keywords used in the FITS file header are described in VCO FITS header keyword dictionary. (URL)

4.7.  Handling special values

introduce special values and how to handle it.

before reading data values, introduce missing flag value, saturated pixel flag value, and dead pixel flag value.

missing flag value means .... saturated ... dead pixel flag value means ...

introduce P_DPIXV, P_DPIXN, ....

4.8.  Direction of image and coordinate system

Direction of image is always +Y-axis of spacecraft upward. What is the +Y-axis of spacecraft? Which direction is "upward" for a coordinate that is used for image? introduce imagecoord.txt and many frames in FK read the value, ...

4.9.  Useful meta data

introduce various meta-data and dictionary orbit number, subspacecraft point, S_SSCLON, S_SSCLAT, S_SSCPX, S_SSCPY

4.10.  geometry information file

what is geometry information?

How about the structure of the geo file?

4.11.  Calibrate by yourself

introduce INSTcal.txt

note for UVI data with photometric analysis (introduce "conversion" files)

4.12.  Combined analysis with image taken by other cameras

introduce observation programs, list of observation programs, list of executed observation programs. (stationary feature in LIR, UVI, and/or IR2 for the example?)

4.13.  statistics

seasons, LAC imaging or RS observation chances, full-disk Venus near periapsis, full-disk Venus near apoapsis,

Table 2. List of number of Venus images for each filter/channel.
instrument product day or night filter/channel # of images # of images larger than 4 degrees
UVI L2b day 283 2037 404
365 2095 437
IR1 L2b day 09D 800 228
night 09N 226 61
097 225 61
101 228 62
L2c day 09D 1069 228
night 09N 227 61
097 225 61
101 228 62
IR2 L2b day 202 1613 380
night 174 560 183
226 559 183
232 552 175
LIR L2b day and night PIC 8871 5571
L2c 8873 5558


5.  non-PDS3-compliant data set

In this section, we will introduce non-PDS3-compliant data sets.

5.1.  How is the directory structure?

5.1.1.  Naming of the top directory

You can find "vco_uvi_l3_v1.0" directory. Under this directory, all of the UVI Level 3 data are available.

5.1.2.  Naming of the subdirectory under the top directory

The subdirectory under the top directory ("release" directory) is named as rel_VVVV, where

VV

version of the release in four digits in decimal.

For example, rel_0001 is for the 1st release of the data, and rel_0002 is for the 2nd release.

5.1.3.  Directory structure under the "release" directory

00readme.txt, browse, data, doc

5.2.  How to open data file?

NetCDF 4

5.3.  What is the l3bx or l3cx?

to be written

6.  References