Tips for Sky subtraction

Not use default sky subtraction in HSC pipeline

Sky subtraction is performed in reduceFrames.py and stack.py in HSC pipeline. The process in reduceFrames.py is for object detection and stack.py for object detection for science use. Sky background subtracted in reduceFrames.py can be recovered by stack.py.

# Restore sky background subtracted in reduceFrames.py process, then run stack.py
stack.py $home/hsc --calib=$home/hsc/CALIB --rerun=dith_16h_test --id filter=HSC-I tract=0 --selectId visit=902798..902808:2 ccd=0..103 --config makeCoaddTempExp.bgSubtracted=False

# Parameter:
#     --config makeCoaddTempExp.bgSubtracted=False：It is set to True as default, taking over the subtraction result of reduceFrames.py

The data restored sky background are stored in $home/hsc/rerun/[rerun]/deepCoadd/[filter]/[tract]/[patch]/ as warp-[filter]-[tract]-[patch]-[visit].fits.

Warning

In case that you have already run stack.py and deepCoadd directory has been created under rerun, the new warp data is not created when you run the second stack.py because the existing warp data in the deepCoadd directory is loaded. Workaround is to change [filter] directory name under deepCoadd. After this action, the data recovered sky background is made.


Change size of region for sky estimate in reduceFrames.py

Please add the following option when you perform reduceFrames.py.

# Change the size to 256 × 256 pixel
reduceFrames.py $home/hsc --calib=$home/hsc/CALIB --rerun=dith_16h_test --id filter=HSC-I visit=902798..902808:2 --config processCcd.isr.doFringe=False processCcd.detection.background.binSize=256 processCcd.detection.background.binSize=256

# Parameter
#     --config processCcd.detection.background.binSize=256 processCcd.calibrate.background.binSize=256

Usage of global sky subtraction

Since the sky subtraction in current hscpipe is executed on each, it is not adequate for extended objects. Then we have developed global sky subtraction package. You can download it from here .

There are two steps in the process.

  • Sky pattern subtraction: Smooth the gain between CCDs. Sky pattern is created by median stack of some sky field images and subtracted from postISR images. We recommend that different field data are used to make sky pattern. You have to run this process for bands which have large gain variation in each CCD (e.g., g, r, and i), while you can skip it for r2 or i2 band. Please check here if you don’t know the way to output postISR images.
  • Global sky subtraction: Make sky pattern through the entire field and subtract it. In this process, large objects extended in multiple CCDs are masked.

Please refer to this material for detailed information.

After downloading the package, please follow the procedure for setup.

# Extract the package
tar xvzf hscPipe-globalskysub.tgz

# Setup environment valiable
setup hscpipe
setup -jr hscPipe-globalskysub

Then run the commands. If you try sky pattarn subtraction, you have to execute it before reduceFrames.py.

# Make sky pattern (Skip here if you don't need sky pattern subtraction.)
makeSkyPattern.py $home/hsc --calib=$home/hsc/CALIB --rerun=dith_16h_test --id visit=91028..91050:2 --sky-pattern-dir skypattern --config isr.doDark=False isr.doBias=False

# You can perform makeSkyPattern with a batch process.
# Please specify some sky fields for sky pattern in visit.
# The sky pattern will be output in the directory which you specify as --sky-pattern-dir.


# After sky pattern subtraction, execute reduceFrames.py.
reduceFrames.py $home/hsc --calib=$home/hsc/CALIB --rerun=dith_16h_test --id filter=HSC-I visit=90278..90288:2 --config doGlobalSkySubtraction=True processCcd.doIsr=False skyPatternDir=skypattern hscIsrWithGlobalSkySubtraction.isr.doBias=False hscIsrWithGlobalSkySubtraction.isr.doDark=False

# If you do not run sky pattern subtraction, please add the following option.
#       -config hscIsrWithGlobalSkySubtraction.doSkyPatternSubtraction=False


# When you run stack.py, you have to add
--config detectCoaddSources.detection.reEstimateBackground=False makeCoaddTempExp.bgSubtracted=False