HSC pipeline installation

Warning

This page is editted based on HSC pipeline Binary Distribution . The pipeline distributed here is for engineering use only. We are possible to modify the page heavily depending on the pipeline version.

Environment

We present a typical envoroment to reduce 300 shots of HSC data.

	Spec	Comments
CPU	x86_64
Core	12
Memory	64 GB	Data analysis for HSC data observed in large area requries additional memory.
HDD Storage	10 TB	In case of 300 shots for objects.

Downloading

Please download the binary package of HSC pipeline to your computer. Current version of HSC pipeline is 4.0.5.

Binary

OS	Link
CentOS 6	DL
CentOS 7	DL
Ubuntu 16	DL
build script	DL

Catalog for astrometory installation

SDSS DR9 (about 20 GB)

Pan-STARRS1 (about 400 GB, The new install script is also included. Plese refer to 2. Install the astrometry catalog.)

Installation

1. Install the program

First of all, you expand the binary package under a directory which has an amount of space.

# Make "pipe" directory under home directory, and expand the binary package in there.
mkdir ~/pipe
cd pipe
tar xvaf ~/Download/hscPipe-4.0.5-centos-6-x86_64.tar.xz

Once the binary package is expanded, hscpipe directory is generated. Then, move to hscpipe directory, and run the install script. Installation will finish within ~ 10 minutes.

# Run the install script
cd hscpipe/4.0.5
./post-install.sh

2. Install the astrometry catalog

You need to install the astrometry catalog which is used to decide both astrometry and zero-point magnitude.

# Download the astrometry catalog tar file under the same directory of HSC pipeline in this example (you can place this catalog other than ~/pipe/hscpipe), and expand it
cd ~/pipe/hscpipe
# SDSS
tar xvaf ~/Download/astrometry_net_data-sdss-dr9-fink-v5b.tar.xz
# Pan-STARRS
tar xvaf ~/Download/astrometry_net_data-ps1_pv3_3pi_20170110-and.tar.xz

# Register the astrometry catalog in HSC pipeline
# SDSS
cd ~/pipe/hscpipe/4.0.5
./install-astrometry_net_data.sh ~/pipe/hscpipe/astrometry_net_data/sdss-dr9-fink-v5b

# Pan-STARRS
# To install Pan-STARRS catalog, you need to replace the install script included in pipeline package.
cd ~/pipe/hscpipe/4.0.5
mv install-astrometry_net_data.sh install-astrometry_net_data_old.sh
mv ~/pipe/hscpipe/astrometry_net_data/install-astrometry_net_data.sh .
chmod +x install-astrometry_net_data.sh
./install-astrometry_net_data.sh ~/pipe/hscpipe/astrometry_net_data/ps1_pv3_3pi_20170110-and

3. Register setup commands

Final step for HSC pipeline installation is to register setup commands in .bashrc and .bash_profile, since HSC pipeline runs in bash environment.

# Register setup commands in ~/.bashrc
vi ~/.bashrc

: (contents of .bashrc)
:
source ~/pipe/hscpipe/4.0.5/bashrc


# Register setup commands in ~/.bash_profile
vi ~/.bash_profile

: (contents of .bash_profile)
:
. ~/.bashrc

Now you finish all installation. You can use HSC pipeline with setup commands by following way. Note that every time you log in, or you launch a new terminal, setup-hscpipe is necessary.

# Setup commands for HSC pipeline
setup-hscpipe

# Setup commands for the astrometry catalog
# SDSS
setup astrometry_net_data sdss-dr9-fink-v5b
# Pan-STARRS
setup astrometry_net_data ps1_pv3_3pi_20170110-and

4. Install LaTex and dvipng for data evaluation

Note

If you have already installed LaTex and dvipng, you can skip this section.

If you evaluate your data with image output, you need LaTex and dvipng so please install them on your PC. For Linux system, the following command is useful.

# RedHat system
sudo yum install dvipng texlive-latex

# Debian system
sudo apt-get install dvipng texlive-latex