Usage of HSC pipeline¶
HSC pipeline will provide you coadd data and catalog data. You can obtain these data by performing reduction in order of following process:
- Data Downloading (You need to setup computer environment for the use of HSC pipeline.)
- Making Detrend
- Applying Detrend Data to CCDs
- Mosaicing
- Warp and Stack
Figure 1 represents the flow chart of data reduction with HSC pipeline. Then please go to the following process;
The link under the Next Step in the side-bar is guidepost for the next process.
The filter information in each pipeline version is summarized in HSC pipeline filter list.
If you want to get a jump on data reduction, please move to Data Downloading. For others, please read The structure of Repository in this page, and then move to Data Downloading.
Fig 1: The flow chart of data reduction with HSC pipeline.
HSC pipeline filter list¶
Here is the filter information for HSC pipeline ver 4.x and 3.x. Detailed filter information for HSC is here .
| Filter | hscPipe 4.x | hscPipe 3.x |
|---|---|---|
| g | Yes | Yes |
| r | Yes | Yes |
| i | Yes | Yes |
| z | Yes | Yes |
| y | Yes | Yes |
| i2 | Yes (Version 4.0.2 or later) | No |
| r2 | Yes (Version 4.0.5 or later) | No |
| Filter | Open from | hscPipe 4.x | hscPipe 3.x |
|---|---|---|---|
| NB387 | S16B | No (1) | No (1) |
| NB468 | S16A | No | No |
| NB515 | S15A | No (1) | No (1) |
| NB527 | S16A | No | No |
| NB656 | S16A | No | No |
| NB718 | S16B | No | No |
| NB816 | S15B | Yes | Yes |
| NB921 | S15A | Yes | Yes |
| NB926 | S17A | No | No |
| IB945 | S17A | No | No |
| NB973 | S17B | No | No |
(1): For these filters, color terms are not defined but you can analyze the data.
Please refer to How to perform processing for NB387 and NB515 for detailed information.
The structure of Repository¶
We introduce the structure of repository. In default, HSC pipeline generates data base (repository) under the directory where the _mapper file is present. You will find detrends, coadd data and catalogs in the repository after reducing your HSC data with HSC pipeline. Registories named registry.sqlite3 and calibRegistry.sqlite3 tell you information of data and repository.
For example, here we show the outline of repository under ~/hsc . Details of products with HSC pipeline are given in each reduction process.
~/hsc # a directory for data reduction by HSC pipeline
|-- _mapper
|-- registry.sqlite3 # a regstory for all data
|-- CALIB # a directory for detrend data
| `--- calibRegistry.sqlite3 # a regstory for detrend data
| |-- FLAT # a directory for Flat data
| | `--- [dateObs]
| | `--- [filter]
| | `--- [calibVersion]
| | `--- FLAT-[ccd].fits # Flat data
| |
| |-- BIAS* # a directory for Bias data (the structure is the same as "Flat data")
| |-- DARK* # a directory for Dark data (the structure is the same as "Flat data")
|
|-- rerun
`--- [rerun]
`--- _parent
|-- config # a directory for parameter of reduction
| `--- eups.versions # a version of HSC pipeline
| |-- mosaic.py
| |-- processExposure.py
| |-- stacker.py
| |-- forcedCcd.py
|
|-- [pointing]
| `--- [filter]
| `--- corr # a directory for detrend applied data and mosaic data of 1 visit 1 CCD
| | `--- [tract]
| | | `--- fcr-[visit]-[ccd].fits
| | | |-- wcs-[visit]-[ccd].fits
| | | |-- CALEXP-[visit]-[ccd].fits
| | |
| | |-- BKGD-[visit]-[ccd].fits
| | |-- CORR-[visit]-[ccd].fits
| |
| |-- output # a directory for catalogs produced detrend applied data of 1 visit 1 CCD
| | `--- [tract]
| | | `--- CALSRC-[visit]-[ccd].fits
| | |
| | |-- CAMERAGEOM-[visit].paf
| | |-- ICSRC-[visit]-[ccd].fits
| | |-- MATCH-[visit]-[ccd].fits
| | |-- ML-[visit]-[ccd].fits
| | |-- SRC-[visit]-[ccd].fits
| | |-- SRCMATCH-[visit]-[ccd].fits
| | |-- SRCML-[visit]-[ccd].fits
| |
| |-- processExposure_metadata # output of the middle of reduction (unnecessary data for the science use)
| | `--- [visit].boost
| |
| |-- qa # a directory for evaluation results of 1 visit 1 CCD data
| | `--- ellPaGrid-[visit]-[ccd].fits
| | |-- ellipseGrid-[visit]-[ccd].png
| | |-- ellipseMap-[visit]-[ccd].png
| | |-- ellipticityGrid-[visit]-[ccd].fits / .png
| | |-- ellipticityMap-[visit]-[ccd].png
| | |-- fwhmGrid-[visit]-[ccd].fits / .png
| | |-- magHist-[visit]-[ccd].png
| | |-- psfModelGrid-[visit]-[ccd].fits / .png
| | |-- psfSrcGrid-[visit]-[ccd].txt / .png
| | |-- seeingMap-[visit]-[ccd].txt /.png
| | |-- seeingRobust-[visit]-[ccd].png
| | |-- seeingRough-[visit]-[ccd].png
| |
| |-- thumbs # thumbnails for detrend applied data of 1 visit 1 CCD
| | `--- flattened-[visit]-[ccd].png
| | |-- oss-[visit]-[ccd].png
| |
| |-- tract[tract] # a directory for catalogs produced after stack.py of 1 vist 1 CCD
| `--- FORCEDSRC-[visit]-[ccd].fits
| |-- forcedPhotCcd_metadata
| `--- [visit]-[ccd].boost
|
|-- deepCoadd
| `--- skyMap.pickle # the information about tract definition
| |-- [filter]
| `--- [tract] # a directory for stacked image data of esch tract
| `--- [patch]
| | `--- warp-[filter]-[tract]-[patch]-[visit].fits
| |
| |-- [patch].fits
|
|-- deepCoadd-results # a directory for catalogs produced after stack.py of each patch
| `--- [filter]
| `--- [tract]
| `--- [patch]
| `--- bkgd-[filter]-[tract]-[patch].fits
| |-- det-[filter]-[tract]-[patch].fits
| |-- src-[filter]-[tract]-[patch].fits
|
|-- metadata # the meta information about a tract
| `--- deep_makeDiscreteSkyMap.boost
|
|-- schema # a directory for defining the structure of catalogs
`--- deepCoadd_det.fits
|-- icSrc.fits
|-- src.fits
|-- forced_src_schema.fits