hscPipe5 installation

Note

This page is editted based on HSC Pipeline Binary Distribution . There are some documents about pipeline installation and quick tutorial, and test data.

Environment

The following machine environment is required for 300 shots raw data.

		Remarks
CPU	x86_64
Cores	12
Memory	64 GB	Stackling too wide area might need more.
HDD	10 TB	for 300 shots

Download

Binary package

The binary package is available for CentOS 6 or 7 users. If you use other OS, please follow the instruction Installing the LSST Science Pipelines (please refer to Build from source).

The latest version is hscPipe 5.4.

OS	Link
CentOS 6	DL
CentOS 7	DL
build script	DL

Reference catalog

You can dowload the reference catalog which used for photometric and astorometric calibration from Pan-STARRS reference catalog in LSST format . The total size is over 400GB, so we recommend that you get a subset of the catalog. If you need all files, please download reference-ps1_pv3_3pi_20170110.tar.xz .

Installation

1. For root users

Become root user, then expand the program archive into /opt.

# Move to /opt, then expand the downloaded package in ~/.
cd /opt
tar xvf ~/hscPipe-5.4-centos-6-x86_64.tar.xz

When you can install the program on the above directory, there is nothing else you have to do as to program installation.

2. For general users

If you cannot become root user, please put the downloaded package on any directory. In the following example, it is put on ~/ana. After extracting it, run the included script to install hscPipe.

# Create /ana directory, then expand the package in ~/.
mkdir ~/ana
cd ~/ana
tar xvaf ~/hscPipe-5.4-centos-6-x86_64.tar.xz

# Execute the included script in newly created hscpipe directory.
cd hscpipe/5.4
./install.sh

Reference catalog

The total amount of the catalog is over 400GB, so please check the disk space first.

# Expand into ~/astrometry_data as an example
cd ~/astrometry_data
tar xvaf ~/reference-ps1_pv3_3pi_20170110.tar.xz

It is not necessary to install the catalog in the latest version.

Register a setup command

Finally, add setup command in ~/.bashrc because hscPipe runs in the bash environment.

# Register setup command in ~/.bashrc.
vi ~/.bashrc

: (Original contents)
:
source ~/opt/hscpipe/5.4/bashrc # Add this line and save the file

# Reload .basgrc
source ~./bashrc

Then setup-hscpipe command can be used. You have to execute this command when you log-in to the machine or open a new terminal.

# hscPipe setup
setup-hscpipe

# Reset environment variable
reset-env

Build from source

If you do not use Cent OS or want to install to macOS, you can build the pipeline by yourself. The build process becomes more stable than previous ones.

If you have gcc 4.8 or later, you can install hscPipe with Installing the LSST Science Pipelines . Please check Prerequisites before installation.

The following steps are required for installation.

# Create a directory for hscPipe5.
mkdir ~/hscpipe5
cd ~/hscpipe5

export EUPS_PKGROOT=http://master.ipmu.jp/~price/packages-hscPipe5/
wget $EUPS_PKGROOT/newinstall.sh
bash newinstall.sh # Answer "yes".

source ~/hscpipe5/loadLSST.bash
eups distrib install hscPipe 5.4

If you do not have gcc 4.8 or later, please use “build script”.

Then install the package to use ds9. Unfortunately, hscPipe5 does not include the package for ds9 by default. The installation steps are as follows.

# Install ds9 package.
source /path/to/lsst/stack/loadLSST.bash
export EUPS_PKGROOT=http://master.ipmu.jp/~price/packages-hscPipe5/
eups distrib install -j xpa 2.1.15.lsst3

# Before install display_ds9, need to setup the environment.
# The following setup commands and option "-j" reproduce the environment when the package is installed.
setup afw
setup xpa
setup python_future

eups distrib install -j display_ds9 13.0+5

# Setup command
setup display_ds9

# The setup command should be executed when you log-in to the machine same as setup-hscpipe command.
# If you use the binary package, setup-hscpipe command has already included display_ds9 command.