hscPipe7 installation

Note

This page is based on HSC Pipeline Binary Distribution . There are some documents about pipeline installation and quick tutorial, and test data.

Environment

hscPipe requires the following machine environment to reduce a set of raw data consisting from 300 shots.

		Remarks
CPU	x86_64	Only x86_64 is tested.
Cores	12	Not a requirement but a recommendation.
Memory	64 GB	Stacking too wide a region might require more.
HDD	10 TB	for 300 shots

Download

Binary package

The binary package is available for CentOS 7 users. If you use other OS, please follow the instruction Installing the LSST Science Pipelines (please refer to Build from source).

The latest version is hscPipe 7.9.1

OS	Link
CentOS 7	DL
CentOS 7 with OpenBLAS	DL
build script	DL

Reference catalog

You can download the reference catalog which used for photometric and astrometric calibration from Pan-STARRS reference catalog in LSST format . The total size is over 400GB, so we recommend that you get a subset of the catalog. If you need all files, please download reference-ps1_pv3_3pi_20170110.tar.xz .

Installation

1. For root users

Become root user, then expand the program archive into /opt.

# Move to /opt, then expand the downloaded package in ~/.
cd /opt
tar xvf ~/hscPipe-7.9.1-openblas-centos-7-x86_64.tar.xz

When you can install the program on the above directory, there is nothing else you have to do as to program installation.

2. For general users

If you cannot become root user, please put the downloaded package on any directory. In the following example, it is put on ~/ana. After extracting it, run the included script to install hscPipe.

# Create /ana directory, then expand the package in ~/.
mkdir ~/ana
cd ~/ana
tar xvaf ~/hscPipe-7.9.1-openblas-centos-7-x86_64.tar.xz

# Execute the included script in newly created hscpipe directory.
cd hscpipe/7.9.1
./install.sh

Reference catalog

The total amount of the catalog is over 400GB, so please check the disk space first.

# Expand into ~/astrometry_data as an example
cd ~/astrometry_data
tar xvaf ~/reference-ps1_pv3_3pi_20170110.tar.xz

It is not necessary to install the catalog in the latest version.

Register a setup command

Finally, add setup command in ~/.bashrc because hscPipe runs in the bash environment.

# Register setup command in ~/.bashrc.
vi ~/.bashrc

: (Original contents)
:
source ~/opt/hscpipe/7.9.1/bashrc # Add this line and save the file

# Reload .basgrc
source ~./bashrc

Then setup-hscpipe command can be used. You have to execute this command when you log-in to the machine or open a new terminal.

# hscPipe setup
setup-hscpipe

# Reset environment variable
reset-env

Build from source

If you do not use Cent OS or want to install to macOS, you can build the pipeline by yourself. The build process becomes more stable than previous ones.

You can install hscPipe with Installing the LSST Science Pipelines . Please check Prerequisites before installation.

The following steps are required for installation.

# Create a directory for hscPipe 7.
mkdir ~/hscpipe7
cd ~/hscpipe7

export EUPS_PKGROOT=http://tigress-web.princeton.edu/~HSC/hscPipe7/
wget $EUPS_PKGROOT/newinstall.sh
bash newinstall.sh # Answer "yes".

source ~/hscpipe7/loadLSST.bash
eups distrib install hscPipe 7.9.1

Then install the package to use ds9. Unfortunately, hscPipe7 does not include the package for ds9 by default. The installation steps are as follows.

# Install ds9 package.
source /path/to/lsst/stack/loadLSST.bash
export EUPS_PKGROOT=http://master.ipmu.jp/~price/packages-hscPipe7/
eups distrib install -j xpa 2.1.15.lsst3

# Before install display_ds9, need to setup the environment.
# The following setup commands and option "-j" reproduce the environment when the package is installed.
setup afw
setup xpa
setup python_future

eups distrib install -j display_ds9 13.0+5

# Setup command
setup display_ds9

# The setup command should be executed when you log-in to the machine same as setup-hscpipe command.
# If you use the binary package, setup-hscpipe command has already included display_ds9 command.