obs_subaru patch for Y background subtraction

Install

curl -L https://github.com/lsst/obs_subaru/archive/master.tar.gz | tar -xzf -
curl -LO https://hscdata.mtk.nao.ac.jp/hsc_bin_dist/4fc0f523/obs_subaru.py3.patch
curl -LO https://hscdata.mtk.nao.ac.jp/hsc_bin_dist/4fc0f523/yb-dictionary-6-0.tar.xz
cd obs_subaru-master
patch -p1 < ../obs_subaru.patch
tar --strip-components=1 -xvJf ../yb-dictionary-6-0.tar.xz
...
singleFrameDriver.py ... --config processCcd.isr.doSubtractStraylight=True processCcd.isr.doKawanomotoCorrection=True

~~The patch is actually for hscPipe-5.4, which is on python2. The master branch patched the way above won't work. We have yet to rewrite the patch for python3.~~ I have rebased the patch onto the master branch

Files

The background file has been updated on 2017-12-06 JST. This latest version includes dark exposures with INR=92 and 93, which were accidentally omitted from the old version. If you are reluctant to alter this file, the previous version probably work well because these angles were linearly interpolated.

The background file (A) is the recommended version. We tried strengthening the compression level by one to make (B) and (C). In (B), compared to (A), the data was more compressed along the axis of the instrument rotator angle. In (C), compared to (A), the data was more compressed along the spatial axes. In both (B) and (C) we observed some cases where subtraction didn't go well.

The background files were generated by this program

Test

Quick Look: Focal Plane Quick Look: We subtracted computed backgrounds from the images which the compressed background data had been made from. We show the residuals in the linked page. There are 6 options at the top of the linked page: "ql##-#-#". The first number (100, 30 or 300) indicates the value range: when it is 100, the value range is ±100. The second number indicates the compression level along the spatial axes. The third number indicates the compression level along the instrument-rotator-angle axis.
no subtraction / subtracted without kawanomoto-correction / subtracted with kawanomoto-correction: We have decided to compute instrument rotator angle for ourselves, not relying on what the FITS header says. In the FITS header there is only a single value that indicates the instrument rotator angle, though in fact the instrument rotator rotates during an exposure. Kawanomoto-san provided a program to compute the instrument rotator angle from the observation date etc.
with/without subtraction: Here are some more comparisons.
HDFN field: We processed HDFN field upto coaddDriver with the master branch and with the modified version:
Processed with the master branch
Processed with Y background subtraction enabled