Pipeline tools

Some useful pipeline tools are included in hscPipe. (More tools will be updated)

butler

The hscPipe tool butler can search and load catalogs and images created in hscPipe processing. The way to specify data is different from its type. For example, visit and ccd should be specified to analyze CORR image (it is called ‘calexp’ in hscPipe process), but tract, patch, and filter is needed for forced_src catalog (‘deepCoadd_forced_src’). If you want to check the correspondence of file name and one in hscPipe process, please refer to hscpipe/8.4/stack/miniconda3-4.7.10-4d7b902/Linux64/obs_base/8.1-hsc/policy/exposures.yaml, datasets.yaml, or hscpipe/8.4/stack/miniconda3-4.7.10-4d7b902/Linux64/obs_subaru/8.4-hsc/policy. The following case shows Loading a catalog by PyFITS with butler.

# Setup hscPipe8
setup-hscpipe

# Launch python
python

# Load python module for butler
>>> import lsst.daf.persistence as dafPersist
# Load ds9 module
>>> import lsst.afw.display.ds9 as ds9

# Call butler
# Specify rerun name
>>> butler = dafPersist.Butler("~/HSC/rerun/[rerun]")

# Load CORR image (calexp) and display with ds9
>>> dataId = {'visit':22222, 'ccd':43}
>>> corr = butler.get('calexp', dataId)
>>> ds9.mtv(corr)

# Load SRC catalog
>>> cat = butler.get('src', dataId)
>>> cat
# Output
#       <lsst.afw.table.tableLib.SourceCatalog; proxy of <Swig Object of type 'lsst::afw::table::SortedCatalogT< lsst::afw::table::SourceRecord > *' at 0x7f66e0d924b0> >
>>> cat['base_SdssCentroid_x']
# Output
#       array([  718.        ,  1912.86014728,  1530.75661309,   573.97121016,
        1207.51468474,   484.28527248,    53.33994211,  1079.15382039,
        652.99781412,  1008.24738213,  1183.51441946,  1865.93879445,
        ...
        1241.45153334,   733.9278414 ,   723.57418556,   863.39894481,
        863.75880726,   751.31009308,   755.85760008,  1659.56569431,
        1632.        ])

# Plot on ds9 window
>>> xs = cat['base_SdssCentroid_x']
>>> ys = cat['base_SdssCentroid_y']
>>> for x,y in zip(xs,ys):
...     ds9.dot('o', x, y, size=30)

The following table shows target name and data type.

butler target example
Target Output file name note Type dataId
bias BIAS-[dateObs]-[ccd].fits BIAS data ExposureF visit, ccd
dark DARK-[dateObs]-[ccd].fits DARK data ExposureF visit, ccd
flat FLAT-[dateObs]-[ccd].fits FLAT data ExposureF visit, ccd
fringe FRINGE-[dateObs]-[ccd].fits FRINGE data ExposureF visit, ccd
sky SKY-[dateObs]-[ccd].fits SKY data ExposureF visit, ccd
calexp CORR-[visit]-[ccd].fits Corrected CCD image ExposureF visit, ccd
src SRC-[visit]-[ccd].fits Catalog of detected sources SourceCatalog visit, ccd
wcs wcs-[visit]-[ccd].fits WCS file ExposureI tract, visit, ccd
fcr fcr-[visit]-[ccd].fits Flux scaling file ExposureI tract, visit, ccd
deppCoadd_calexp calexp-[filter]-[tract]-[patch].fits Coadd image ExposureF tract, visit, ccd
deepCoadd_forced_src forced_src-[filter]-[tract]-[patch].fits Source catalogs generated by a forced measurement SourceCatalog tract, visit, ccd