QA for errors in command execution¶
- Tract is not defined after makeDiscreteSkyMap.py
- FATAL: Failed in task initialization error occurs after command execution
- reduceFrames.py stops with the error message “OpenBLAS: pthread_create error”
- reduceFrames.py for narrow-band filter analysis stops with the error message “ColortermNotFoundError”
- Failed to solve linear equation during mosaic.py
- During making detrend data in hanaco, the process stops with the error “qsub:command not found”
Tract is not defined after makeDiscreteSkyMap.py¶
If you find the following error messages during makeDiscreteSkyMap.py,
Note
makeDiscreteSkyMap FATAL: Failed: No data found from which to compute convex hull
the observed region is outside of SDSS field. So astrometry does not succeeded, and the tract is not generated. Please prepare for catalog files for astrometry by yourself (here).
FATAL: Failed in task initialization error occurs after command execution¶
If you find the following error after reduceFrames.py or stack.py,
Note
FATAL: Failed in task initialization: Config does match existing config on disk for this task; tasks configurations must be consistent within the same output repo (override with –clobber-config)
please add
--clobber-config
at the end of the command. The pipeline refuses changing configuration in the same rerun. You can run the command with –clobber-config which overrides the configuration. Please refer to General info about HSC pipeline commands.
There is a bug in reduceFrames, so the same error is sometimes found despite no configuration change. This can avoid adding –clobber-config same as above.
reduceFrames.py stops with the error message “OpenBLAS: pthread_create error”¶
When reduceFrame.py stops with the following message,
Note
OpenBLAS: pthread_create error in blas_thread_init function. Error code:11
please set the environment variable like below:
$ OMP_NUM_THREADS=1 reduceFrames.py ~/HSC --rerun= ....
This is the environment variable which defines the number of thread in OpenBLAS solving linear equation. Basically, pipeline creates 1 thread per 1 process. However BLAS library executes many threads. OMP_NUM_THREADS defines the number of thread per 1 process in Open BLAS.
Warning
Do NOT set OMP_NUM_THREADS to 1 in mosaic.py because it executes many threads in 1 process.
reduceFrames.py for narrow-band filter analysis stops with the error message “ColortermNotFoundError”¶
When you execute reduceFrames.py for the narrow-band filters annotated in HSC pipeline filter list , you will get an error message (in case of NB515);
Note
ColertermNotFound: No colorterm set for filter N515 with astrometry_net_data version xxxxxxx
In that case, the config.py file which describes color term setting should be loaded and hsc is prefixed to the catalog name (see Installing and setup for your own astrometry catalog directory).
# config.py
root.processCcd.calibrate.astrometry.solver.filterMap["N515"]="NB0515"
from lsst.meas.photocal.colorterms import ColortermConfig
root.processCcd.calibrate.photocal.colorterms.library['hsc*'].group['N515']=ColortermConfig()
root.processCcd.calibrate.photocal.colorterms.library['hsc*'].group['N515'].c2= 0.0
root.processCcd.calibrate.photocal.colorterms.library['hsc*'].group['N515'].c1= 0.0
root.processCcd.calibrate.photocal.colorterms.library['hsc*'].group['N515'].c0= 0.0
root.processCcd.calibrate.photocal.colorterms.library['hsc*'].group['N515'].primary= 'N515'
root.processCcd.calibrate.photocal.colorterms.library['hsc*'].group['N515'].secondary= 'N515'
After config.py preparation, you can execute reduceFrames.py adding -C config.py.
reduceFrames.py $home/hsc --calib=$home/hsc/CALIB --rerun=dith_16h_test --id filter=HSC-I visit=902798..902808:2 --config processCcd.isr.doFringe=False -C config.py
Then you need to make a similar configuration file for mosaic.
# config_mosaic.py
from lsst.meas.photocal.colorterms import ColortermConfig
root.colorterms.library['hsc*'].group['N515']=ColortermConfig()
root.colorterms.library['hsc*'].group['N515'].c2= 0.0
root.colorterms.library['hsc*'].group['N515'].c1= 0.0
root.colorterms.library['hsc*'].group['N515'].c0= 0.0
root.colorterms.library['hsc*'].group['N515'].primary= 'N515'
root.colorterms.library['hsc*'].group['N515'].secondary= 'N515'
The command for mosaic becomes as follows;
mosaic.py $home/hsc --calib=$home/hsc/CALIB --rerun=dith_16h_test --id visit=902798..902808:2 ccd=0..103 tract=0 -C config_mosaic.py
If you run mosaic.py again, you need to add –clobber-config (detail of –clobber-config is here ).
Failed to solve linear equation during mosaic.py¶
If you find the following error during mosaic.py,
Note
Exception: solving linear equation failed: dgesv returned 2620
it may fail because of the lack of objects used for solving linear equation. Then, the number of objects in MATCH-/ML-xxxx.fits should be checked. The contents of MATCH and ML lists are basically same. They are different in terms of source ID; the one of ML- can be used with other software. For example, you can check the number using the following commands.
for i in output/xxxxx/HSC-I/output/ML-*.fits ; do echo $i $(head -c$((80*36*4)) $i | fold | grep NAXIS2) ; done
After checking the number of object, you can execute mosaic without some CCDs which have less matched objects. If you want to remove CCD = 9 and 13, you use;
mosaic.py <a directory for data reduction> --calib=<a directory for detrend data> --rerun=xxxxxxxx --id visit=xxxxx ccd=0..8^10..12^14..103 tract=0
During making detrend data in hanaco, the process stops with the error “qsub:command not found”¶
Hanaco does not have a job scheduler, then please add the option processing on a local machine to the command. Detailed information is here . After making detrend data, if there is a comment “You can perform xxxxxx.py with a batch process”, you need to add this option.
Failed to create FLAT images due to a specific CCD¶
After 21st Nov. 2016, there has been hardware fault on CCD 33. Because of the fault, reduceFlat.py often fails with following errors;
flat FATAL: Failed: Bad exposure scales: [[9.502126862373885 9.499400187124834 9.498856535788773 9.496713880612939
...
[9.755296139139855 9.752648112376978 9.752002573201265 9.749887192083907
9.748172542114258 9.752851785173549 9.753830962276854 9.752436002528357
9.752959752363898 9.751412547931334]] --> [ nan nan nan nan nan nan nan nan nan nan]
First of all, please run reduceFlat.py without CCD 33.
reduceFlat.py $home/hsc --calib=$home/hsc/CALIB --rerun=<rerun> --id visit=902690..902704:2 ccd=0..32^34..111 --detrendId calibVersion=all
If FLAT images for all CCDs except 33 are created, the issue is caused by it. You can avoid it by defining and registering defects.
For pipeline in your local machine¶
Move to defects directory. The following example is used hscpipe 4.0.5.
$ cd /<pipeline DIR>/hscpipe/4.0.5/Linux64/obs_subaru/HSC-4.0.3/hsc/defects
$ ls
2013-01-31 2014-04-03 2014-06-01 2014-09-01 defectRegistry.sqlite3
# Please take backup if you need.
# If you use different version, the directory name ../HSC-4.0.3/ will be changed.
Make a new directory to store a new defects.dat. Its name should be the date when the defect is applied.
$ mkdir 2016-11-21
$ vi defects.dat
#
# Defects file
#
# Convert to a fits table using getDefectFits.py; this is done for you by running scons
#
# CCD x0 y0 width height
# Dead amps
33 0 0 1024 4176
Generate the defects.fits.
$ setup-hscpipe
$ genDefectFits.py /<pipeline DIR>/hscpipe/4.0.5/Linux64/obs_subaru/HSC-4.0.3/hsc/hsc_geom.paf defects.dat .
$ ls
defects.dat defects_33.fits
# defects_33.fits should be created.
Register new data to defectRegistry.sqlite3.
$ genDefectRegistry.py --create --root . -v
$ sqlite3 defectRegistry.sqlite3
sqlite> .header on
sqlite> select * from defect;
id|path|version|ccdSerial|validStart|validEnd
1|2013-01-31/defects_14.fits|./2013-01-31|14|2013-01-31|2037-12-31
2|2013-01-31/defects_17.fits|./2013-01-31|17|2013-01-31|2037-12-31
3|2013-01-31/defects_23.fits|./2013-01-31|23|2013-01-31|2037-12-31
4|2013-01-31/defects_33.fits|./2013-01-31|33|2013-01-31|2016-11-20T23:59:59
5|2016-11-21/defects_33.fits|./2016-11-21|33|2016-11-21|2037-12-31
6|2013-01-31/defects_45.fits|./2013-01-31|45|2013-01-31|2037-12-31
7|2014-09-01/defects_0.fits|./2014-09-01|0|2014-09-01|2037-12-31
8|2014-04-03/defects_9.fits|./2014-04-03|9|2014-04-03|2014-05-31T23:59:59
9|2014-06-01/defects_9.fits|./2014-06-01|9|2014-06-01|2014-08-31T23:59:59
10|2014-09-01/defects_9.fits|./2014-09-01|9|2014-09-01|2037-12-31
11|2013-01-31/defects_90.fits|./2013-01-31|90|2013-01-31|2037-12-31
12|2013-01-31/defects_100.fits|./2013-01-31|100|2013-01-31|2037-12-31
13|2013-01-31/defects_78.fits|./2013-01-31|78|2013-01-31|2037-12-31
# defects_33.fits should be added.