Pipeline tools

Some useful pipeline tools are included in hscPipe. (More tools will be updated)

butler

The hscPipe tool butler can search and load catalogs and images created in hscPipe processing. The way to specify data is different from its type. For example, visit and ccd should be specified to analyze CORR image (it is called ‘calexp’ in hscPipe process), but tract, patch, and filter is needed for forced_src catalog (‘deepCoadd_forced_src’). If you want to check the correspondence of file name and one in hscPipe process, please refer to hscpipe/7.9.1/stack/miniconda3-4.5.12-1172c30/Linux64/obs_base/7.0-hsc+1/policy/exposures.yaml, datasets.yaml or hscpipe/7.9.1/stack/miniconda3-4.5.12-1172c30/Linux64/obs_subaru//7.7.2-hsc+4/policy/HscMapper.paf. The following case shows Loading a catalog by PyFITS with butler.

# Setup hscPipe5
setup-hscpipe

# Launch python
python

# Load python module for butler
>>> import lsst.daf.persistence as dafPersist
# Load ds9 module
>>> import lsst.afw.display.ds9 as ds9

# Call butler
# Specify rerun name
>>> butler = dafPersist.Butler("~/HSC/rerun/[rerun]")

# Load CORR image (calexp) and display with ds9
>>> dataId = {'visit':22222, 'ccd':43}
>>> corr = butler.get('calexp', dataId)
>>> ds9.mtv(corr)

# Load SRC catalog
>>> cat = butler.get('src', dataId)
>>> cat
# Output
#       <lsst.afw.table.tableLib.SourceCatalog; proxy of <Swig Object of type 'lsst::afw::table::SortedCatalogT< lsst::afw::table::SourceRecord > *' at 0x7f66e0d924b0> >
>>> cat['base_SdssCentroid_x']
# Output
#       array([  718.        ,  1912.86014728,  1530.75661309,   573.97121016,
        1207.51468474,   484.28527248,    53.33994211,  1079.15382039,
        652.99781412,  1008.24738213,  1183.51441946,  1865.93879445,
        ...
        1241.45153334,   733.9278414 ,   723.57418556,   863.39894481,
        863.75880726,   751.31009308,   755.85760008,  1659.56569431,
        1632.        ])

# Plot on ds9 window
>>> xs = cat['base_SdssCentroid_x']
>>> ys = cat['base_SdssCentroid_y']
>>> for x,y in zip(xs,ys):
...     ds9.dot('o', x, y, size=30)

The following table shows target name and data type.

butler target example

Target

Output file name

note

Type

dataId

bias

BIAS-[dateObs]-[ccd].fits

BIAS data

ExposureF

visit, ccd

dark

DARK-[dateObs]-[ccd].fits

DARK data

ExposureF

visit, ccd

flat

FLAT-[dateObs]-[ccd].fits

FLAT data

ExposureF

visit, ccd

fringe

FRINGE-[dateObs]-[ccd].fits

FRINGE data

ExposureF

visit, ccd

sky

SKY-[dateObs]-[ccd].fits

SKY data

ExposureF

visit, ccd

calexp

CORR-[visit]-[ccd].fits

Corrected CCD image

ExposureF

visit, ccd

src

SRC-[visit]-[ccd].fits

Catalog of detected sources

SourceCatalog

visit, ccd

wcs

wcs-[visit]-[ccd].fits

WCS file

ExposureI

tract, visit, ccd

fcr

fcr-[visit]-[ccd].fits

Flux scaling file

ExposureI

tract, visit, ccd

deppCoadd_calexp

calexp-[filter]-[tract]-[patch].fits

Coadd image

ExposureF

tract, visit, ccd

deepCoadd_forced_src

forced_src-[filter]-[tract]-[patch].fits

Source catalogs generated by a forced measurement

SourceCatalog

tract, visit, ccd