How to make a catalog for


Here is explained how to make an astrometry_net_data by yourself. Proceed with this page in one hand, and the legitimate manual in another.

Get the latest

If you are using hscPipe 3.x or later, you should skip this section. But if you are using hscPipe 2.12.4d_hsc, there are some required tools that are not offered by the in that version of hscPipe because the version of the is old. You must build the latest by yourself.

Required libraries

yum install zlib-devel bzip-devel \
    libpng-devel libjpeg-turbo-devel \

Download the source

Download the latest source from the originator's download page. Version 0.49 is the latest as of 2014-06-18.

Build & install

[user@example ~]$ cd /dev/shm
[user@example shm]$ tar xvaf ~/
[user@example]$ cd
[user@example]$ make \
    INSTALL_DIR=$HOME/usr/astrometry               $(: as you like) \
    CFITS_INC="-I $HOME/usr/cfitsio/include"       $(: where cfitsio resides) \
    CFITS_LIB="$HOME/usr/cfitsio/lib/libcfitsio.a" $(: where cfitsio resides) \
[user@example]$ cd ..
[user@example shm]$ rm -rf
[user@example shm]$ cd ~
[user@example ~]$ export PATH="$HOME/usr/astrometry/bin:$PATH"

Prepare a catalog

Prepare a CSV file that looks like the following: (If you already have a FITS BinTable, you can skip this section.)

(Values are from SDSS DR8)
Description of columns
idobject's ID
rara (degree)
decdec (degree)
ra_errerror of ra
dec_errerror of dec
starnotgalstar (1) / otherwise (0)
r,u,g,i,zmagnitude measured in each filter
r_err,u_err,g_err,i_err,z_errerror of magnitude

The names of filters and the number of filters are arbitrary within, but in hscPipe, you have to obey the following rule to prevent trouble.

Note that the magnitudes will be used not only in astrometry but also in photometry calibration.

Convert it to a FITS BinTable

Next, convert the CSV to a FITS BinTable. (Instead of converting a CSV, you may prepare a FITS BinTable with your favorite tools.)

Download the conversion script and do as follows:

[user@example ~]$ setup-hscpipe
[user@example ~]$ python src-catalog.csv src-catalog.fits

Separate the catalog into fragments

Separate into fragments the FITS BinTable you have prepared above↑, so that each of the fragments will correspond to a Healpix pixel. Thus:

[user@example ~]$ mkdir workdir
[user@example ~]$ hpsplit -o workdir/cat-%i.fits -r ra -d dec -m 1 src-catalog.fits -n 2

Create index files

Create files that actually uses:

[user@example ~]$ filter=r # The default filter
[user@example ~]$ mkdir output
[user@example ~]$ # For all the catalog files that've been created in workdir:
[user@example ~]$ find workdir -name "cat-*.fits" -print0 | while read -r -d '' catalog
    # From catalog="workdir/cat-###.fits", generate index="output/index-###.fits"
    # From index="output/index-###.fits", trim the extension ".fits"

    # Build preset 0
    build-astrometry-index -v -i $catalog -o $index-0.fits -P 0 -S $filter -n 100 -r 1 -j 0.4 -E
    # preset 1, 2, 3
    for p in {1..3}
        build-astrometry-index -v -1 $index-0.fits -o $index-$p.fits -P $p -S $filter -n 100 -r 1 -j 0.4 -E

Create a config file

Describe in a config file what the index files are.

[user@example ~]$ vi output/
# Write as follows
import os.path
import os
import glob

root.starGalaxyColumn = "starnotgal"
filters = ("r", "u", "g", "i", "z") # Enumerate the filters in the catalog you prepared
root.defaultMagColumn = "r"
root.magColumnMap = dict([(f,f) for f in filters])
root.magErrorColumnMap = dict([(f, f + "_err") for f in filters])
root.indexFiles = [os.path.basename(path) for path in glob.glob(os.path.join(os.environ["ASTROMETRY_NET_DATA_DIR"], "index-*.fits"))]

Create a UPS directory

Eups requires a UPS directory so that it can use the output directory you have created above.

[user@example ~]$ mkdir -p output/ups
[user@example ~]$ touch output/ups/astrometry_net_data.table # empty file


Rename the output directory an appropriate name, and move it into an appropriate directory.

[user@example ~]$ mv output MYCATALOG
[user@example ~]$ mv MYCATALOG /opt/hscpipe/astrometry_net_data/

Note that MYCATALOG must match one of the following wildcards:

If the catalog is from SDSS, prefix "sdss" to the catalog name. The pipeline will take account of the difference in filter property between SDSS and HSC.
If the catalog is from Pan-STARRS, prefix "ps1" to the catalog name. The pipeline will take account of the difference in filter property between Pan-STARRS and HSC.
This is used when the catalog is from HSC itself, but it can also be used when you do not want color correction.

Register it in eups

[user@example ~]$ eups declare \
    -m /opt/hscpipe/astrometry_net_data/MYCATALOG/ups/astrometry_net_data.table \
    -r /opt/hscpipe/astrometry_net_data/MYCATALOG/ \
    astrometry_net_data  MYCATALOG

To make use the data you have created, set it up thus:

[user@example ~]$ setup-hscpipe
[user@example ~]$ setup astrometry_net_data MYCATALOG