###Last update 19.4.2005#### ###Last update 1.2.2006 (WCS CDELT1 -> CD1_1, calibration farmes process=no)#### In these reduction notes I haven't used OBJECT-keyword in CCDPROC and all the data of different filters have their own input file (@file). The converted (from single image to multi extensions) MEF-files have the same names as the "raw" data (e.g. mc090012.fits), this is useful later when combining bad pixel masks. These notes are more detailed for the inital reductions, bias, dark and flat field correction and for adjusting the WCS. For the final combination see e.g. The NOAO Deep Wide-Field Survey MOSAIC Data Reductions http://www.noao.edu/noao/noaodeep/ReductionOpt/frames.html and Reducing Palomar 200-inch LFC Data Part 1 - Image Processing http://bubba.ucdavis.edu/~gal/science/lfcred/lfc_red.html And you can find more MOSCAIC related articles from: http://iraf.noao.edu/projects/ccdmosaic/ ################################################### The data is stored in MultiExtension FITS format file (MEF file). This follows the FITS standards and is understood by e.g. IRAF MSCRED. The lack of single FILTER- and OBSTYPE-keywords are the main "non-IRAF" features and one needs to use "@file" instead of picking up the e.g. bias/dark/science-frames by the OBSTYPE-keyword. So in zeroc/dark/flatcombine check that the switch "ccdtype = " "". ################################################## Data: Bias & dark frames, twilight flats and science exposures Cross talk has not been detected with MOSCA: no correction. Calibration files available from NOT: "static" bad pixel masks (initial version) WCS databases for (XXX filters) In order to use IRAF (V.2.12.2)/MSCRED, MOSCA MEF headers are needed to be slightly modified. ################PREPARATIONS: ############################### First of all make sure that your data is multi extension FITS (MEF). These fits header changes are most handy to do for the whole night's data in one go. # add keyword "SATURATE" and value of e.g. 59000 ADUs hedit mm18*fits[1] saturate 59000 add+ ver- sho+ do this for every extension This keyword will be used to create bad pixel mask # Add keyword BPM which points to the existing bad pixel mask hedit mm18*fits[1] bpm mask.im1 add+ ver- sho+ hedit mm18*fits[2] bpm mask.im2 add+ ver- sho+ .... again all four extensions # convert RA from degrees (FITS-standard) to hours hedit mm18*fits[0] RA '((ra)/15)' up+ sho+ ver- ####IF you have binned data#### Check wether your images have cdelt1 or CD1_1-keywords # Scale CD1_1 & CD2_2 (or Cdelt1 & 2) hedit mm18*fits[im1] CD1_1 '((CD1_1)/2.)' up+ sho+ ver- hedit mm18*fits[im1] CD2_2 '((CD2_2)/2.)' up+ sho+ ver- hedit mm18*fits[im2] CD1_1 '((CD1_1)/2.)' up+ sho+ ver- hedit mm18*fits[im2] CD2_2 '((CD2_2)/2.)' up+ sho+ ver- ..... (all extensions) # Scale crpix1 & 2 hedit mm18*fits[im1] crpix1 '((crpix1)*2.)' up+ sho+ ver- hedit mm18*fits[im1] crpix2 '((crpix2)*2.)' up+ sho+ ver- hedit mm18*fits[im2] crpix1 '((crpix1)*2.)' up+ sho+ ver- hedit mm18*fits[im2] crpix2 '((crpix2)*2.)' up+ sho+ ver- ..... (all extensions) MOSCA MEF-files are set in such a way that when the raw data is displayed e.g. using stand alone ds9, the WCS-coordinates (RA and DEC) are correctly displayed, however it seems that IRAF wants the unbinned pixel scale and reference pixel ############## REDUCTIONS OF THE CALIBRATION FRAMES ############## (bias, dark, twilight flat) (see http://www.noao.edu/noao/noaodeep/ReductionOpt/frames.html ) This might not be the most compact way to reduce the data, but for clarity I think it is better to show the different steps rather than do everything in one go. MASTER BIAS FRAME create a list of bias frames e.g. ms>hsel mc09*fits[0] $I 'exptime == 0.00' > bias.lst use your favorite editor and remove '.fits[0]' from "bias.lst" ### For the MOSCA data 1/2006 or later all the extensions have keyword EXPTIME. Next comment can be valid for the data before 2006: Note at this point only extension [0] has keyword 'EXPTIME' this won't work for other extensions. #### trim bias images: ms>ccdpr @bias.lst trim+ fixpix- oversc- zeroc- darkc- flatc- sflatc- merge- master bias frame: ms>zeroc @bias.lst out=mc09bias rej=minmax comb=median nh=2 nl=2 proc- MASTER DARK FRAME create a list of dark frames You can use e.g. iraf/hselect to create the list of dark frames mscred> hsel MO*[0] $I,tcstgt,exptime,shstat 'shstat == "CLOSED"' trim and bias subtract dark-frames ms>ccdpr @dark.lst trim+ zeroc+ zero=mc09bias oversc- combine dark frames check that the option CCDTYPE= "" ms>darkc @dark.lst out=mc09dark combine=average reject=ccdclip check that you have set the keywords "rdnoise" & "gain" right TWILIGHT FLAT create a list of twilight flats (all filters) e.g. ms> hsel mc09*s[0] $I,tcstgt yes > flat.tmp remove non-blank fields from the "flat.tmp"-file using your favorite editor ms> hsel @flat.tmp $I 'exptime != 0.00' > flat.lst removes bias-frames finally remove '.fits[0]' process twilight flats: ms>ccdpr @flat.lst trim+ darkc+ zeroc+ zero=mc09bias dark=mg24dark oversc- If you get error message like: mscred> ccdpr @flat.tmp trim+ darkc+ zeroc+ zero=pa28bias dark=pa28dark ERROR: Bias section not specified or given as full image "low_reject=low_reject, high_reject=high_reject, grow=grow)" line 286: mscsrc$ccdproc.cl called as: `ccdproc (images=@flat.tmp, trim=yes, zerocor=yes, darkcor=yes, zero=pa28bias, dark=pa28dark)' Check that the in CCDPR option "overscan = no" If you get error message like: ERROR on line 491: Image header parameter not found (exptime) You have to run ccdproc twice: First pass: ms> ccdpr @flat.lst trim+ darkc- zeroc- fixpix- i.e. trim the image first, because at this point many of the fits header extension[0] are copied to other extensions. Second pass: ms>ccdpr @flat.lst darkc+ zeroc+ zero=mc09bias dark=mg24dark Combine each filter twilight flats: you can use hsel to find the right frames first add '[1]' after each and every filenames in "flat.lst" e.g mc09022[1] mc09023[1] ... then use the number of the filters: filter number filter name 104 U Bessell 105 B Bessell 106 V Bessell 107 R Bessell 108 I interference 109 u SDSS 110 r SDSS 111 i SDSS 112 z SDSS 120 g SDSS ms> hsel @flat.lst $I 'afiltid == 109' > u109.lst etc..... or 'bfiltid == 109" Remember to remove '[1]' from the list Check that the flat images are OK Combine master flat for each filter: ms> flatc @r110.lst out=mc09r110flat proc- subs- etc... ############## REDUCTION OF THE SCIENCE FRAMES ################## (see http://www.noao.edu/noao/noaodeep/ReductionOpt/BasicObjproc.html) Create a list of all science exposures. Check if EXPTIME keyword is in the science exposure extensions e.g. using hselect or imhead (see above) ms>hsel sci_exp[1] exptime yes if not then you have to run ccdproc two times CCDPR first pass (trimming images): ms> ccdpr @sci.exp trim+ fixpix- zeroc- darkc- bpma="" second pass (bias & dark correction, flagging bad pixels) remember to set the saturation = "!saturate" and bleed = " your_favorite_value" (see ccdpr help page) ms>ccdpr @sci.exp fixp+ zeroc+ darkc+ zero=mc09bias dark=mg24dark bpma=@sci.bpm files sci.exp and sci.bpm look like this mc090013 bpmmc090013 mc090014 bpmmc090014 mc090015 bpmmc090015 ... ... check that the masks are OK: ms> mscdi image 1 fi+ bpdi=overlay bpma=BPM third pass, flat field correction: @list for each filter; first add [1] in the sci.lst file second ms>hsel @sci.lst $I 'afiltid == 106' > v106.sci etc.... third remove [1] from v106.sci etc. ms>ccdpr @b105.sci flatc+ flat=mc09b105flat ##### REMOVING fringes and extra light### (see http://www.noao.edu/noao/noaodeep/ReductionOpt/Skyflat.html) At the moment there is an unidentified red light somewhere in the adapter, which is visible only in the I, iSDSS and z SDSS filter images. ################ Removing fringes & other patterns ############### (I, i SDSS and z SDSS-filters) http://www.noao.edu/noao/noaodeep/ReductionOpt/Skyflat.html ###I have tested this only for one set of data, which wasn't very good quality & quantity#### At this point all the science frames are trimmed, dark- & bias corrected and (twilight)flat corrected. First you have to create objmaks for images to be corrected: ms>objmask @iIzframes @obj.mask omty=numbers skys=@obj.sky obj.mask obj.sky mc090123om mc090123sky mc090124om mc090124sky mc090125om mc090125sky ... ... NOTE, remember to check the hidden parameters objmask1 Combine preliminary fringe image: ms>combine @images ifringev1 masktype=!objmask scale=mode Median image from the ifringev1 ms>mscmed ifringev1 ifringev1med 189 189 Remove the round excess light: first create a mask ms>!cp ifringev1med.fits mc09imask.fits Then blink an i-band image and mc09imask and locate the ADU level where the excess light begins in the smoothed image mc09imask (say, it would be 4760 ADU). Then replace all the pixel values either with 0s or 1s. np> imrepla mc09imask[1] 1. up=4760 np> imrepla mc09imask[1] 0. lo=4760 np> imrepla mc09imask[2] 1. up=4760 ..... (go through all extensions) Check that you have chosen the right level by blinking images: ms>mc090123 1 fi+ ms> mc090123 1 fi+ bpdi=overlay bpma=mc09imask After this subtract the background from the median image ms>mscarith ifringev1med - 3456 mc09iexcess Finally remove the excess light: np> rmpup @images @pimages mc09iexcess @iobj.mask pupilm=mc09imask input output excess light masks mask ( I hope this works.... maybe you have to change the scaling of the mc09iexcess, if needed, I haven't tested this very much....) So now the images should have more or less smooth background + fringes Now combine excess light subtracted images: ms> combine @images ifringev2 masktype=!objmask scale=mode Median image from the ifringev2 ms>mscmed ifringev2 ifringev2med 189 189 Final fringe image: ms> mscarith ifringev2 - ifringev2med mc09ifringe Subtracting the fringes: ms> rmfringe @pimages @fpimages mc09ifringe @iobj.mask Hopefully this works.... ################################################ ######## Improving the World Coordinate system ########## ( see http://www.noao.edu/noao/noaodeep/ReductionOpt/wcsfix.html ) WCS: If you have filter.db available use ms>mscsetwcs @b105.sci B105lj18.db First you have to get a catalogue with appropiate magnitude limits ms>mscgetcat mc090144 144.cat Try: ms>msccmatch mc090024 24.cat and if it doesn't work, run msczero. However even if it works fine check that the coordinates are fixed in a ciorrect way: e.g. ms>mscdi mc090144 1 fi+; msctv 144.cat 1 mark=circ rad=30 Setting WCS using msczero goes like this: ms>msczero mc090144 this is interactive task and on top of the ds9-display press 'm', give the name of the coordinate file etc. List of coordinates (abc.cat):144.cat Radii of concentric circles (40): Gray level of marks to be drawn (205): Label the marked coordinates (no): if you are happy with the size of the circle, colour and no-label, press 'enter'. Now you should have circles overlaid on the image, then choose one circle & star pair and press 's' on top of the circle and then 'z' on top of the star. After this something similar to this should appear: Selected: 12:31:17.792 64:11:28.13 13.900 12:31:21.09 64:11:34.7 13.900 RA (hours) (12:31:17.792): DEC (degrees) (64:11:28.13): Again 'enter' for RA and DEC If you want to check that you picked up the correct star press 'm' again and use the same catalogue, but for clarity change the colour to something else e.g. ... Gray level of marks to be drawn (205):204 .... If msczero gives an error message like "cursor position not in an image" try: cl> set disable_wcs_maps="" cl> flpr Then run msccmatch ms>msccmatch mc090024 24.cat For MOSCA images the offset in RA and DEC can be up to +-20 arcsec. This is due to bad pointing of the telescope. ################################################### ******MOSCA extra***** Display every image and see if the background level of im4 needs to be adjusted. Correct the variable bias level in im4, if needed: >mscsplit myimage del+ verb+ >imarith myimage_4 + 10. myimage_4 > mscjoin myimage del+ ver+ *************************** ############### Projecting Your Individual Images ################ (see http://www.noao.edu/noao/noaodeep/ReductionOpt/Project.html) Cosmic ray removal: use task craverage to find cosmicrays from each image Create file which will look something like this (crave.mc09): craverage mc090014[im1] output="" crmask="crmaskmc090014_1" craverage mc090014[im2] output="" crmask="crmaskmc090014_2" craverage mc090014[im3] output="" crmask="crmaskmc090014_3" craverage mc090014[im4] output="" crmask="crmaskmc090014_4" craverage mc090015[im1] output="" crmask="crmaskmc090015_1" craverage mc090015[im2] output="" crmask="crmaskmc090015_2" ...... To create this file use e.g. i) ms>mscstat @images.sci field=image > cr.tmp the file images.sci looks like this: mc090014 mc090015 mc090016 ... cr.tmp looks like: mc090013[im1] mc090013[im2] mc090013[im3] mc090013[im4] mc090014[im1] ... ii) awk '{print "craverage", $1, "output=\"\" crmask=\"crmask"substr($1,0,8)"_"substr($1,12,1)"\""}' cr.tmp > crave.mc09 ms>cl < crave.mc09 To run this task takes quite a while: about 50 images in an hour (Pentium 4, 1.6GHz) #############Combine bad pixel masks################ Combine bad pixel masks using modified "crplusbpmask.cl": Modified crplusbpmask.cl works i) If the reduced frames are named as mc090123.fits ii) bad pixel masks: - BPM: bpmmc090203/bpm_im2.pl etc. - CR :crmaskmc090203_2.fits etc. ms>crplusbpmask mc090203 Fix bad pixels using task proto/fixpix ms>fixpix @objects.lst BPM -objects.lst: mc090013[im1] mc090013[im2] mc090013[im3] mc090013[im4] mc090014[im1] .... Generating the tangent-plane projected image im> mscimag @1136.in @1136.out e.g. 1136.in: 1136.out mc090144 mc090144pro mc090150 mc090150pro mc090151 mc090151pro mc090152 mc090152pro For the final steps see: http://www.noao.edu/noao/noaodeep/ReductionOpt/prepStack.html and http://www.noao.edu/noao/noaodeep/ReductionOpt/makeStack.html im> mscskysub @1136.out @1136.outs im> mscimat @1136.outs 154s.cat If you have any comments / suggestions / improvemnts I'll be happy to include them in these notes....