I happened to know a code called High Order Transform of Psf ANd Template Subtraction, which has a sexy short name called HOTPANTS, when reading a paper. The code, developed by Andy Becker, is said to be a supersession of another optimal subtraction package ISIS, which unfortunately has a name in common with a terrorist organisation in Middle East. I clearly remember the pain when tweaking parameters so as for ISIS to work and output satisfactory results for me. So I was tempted to try HOTPANTS and see if I can benefit from its name. So I did.
I downloaded the zipped file from Github to my local directory and untarred it. I edited the paths CFITSIOINCDIR in Makefile.macosx and linked them to my CFITSIO that came with Ureka. Also deleted was -ansi in the line of COPTS =.... Then simply type
Only two warnings popped out. The rest looked good. Then I decided to try with two images taken by NEXT, Xingming Observatory C42, about 2018 C2:
Meanings about the parameters (copied from the output without any inputs):
[-inim fitsfile]: comparison image to be differenced
[-tmplim fitsfile]: template image
[-outim fitsfile] : output difference image
[-tu tuthresh]: upper valid data count, template (25000)
[-tg tgain] : gain in template (1)
[-tr trdnoise]: e- readnoise in template (0)
[-iu iuthresh]: upper valid data count, image (25000)
[-ig igain] : gain in image (1)
[-ir irdnoise]: e- readnoise in image (0)
[-gd xmin xmax ymin ymax]
: only use subsection of full image (full image)
[-ctoconvolve]: force convolution on (t)emplate or (i)mage (undef)
[-nnormalize] : normalize to (t)emplate, (i)mage, or (u)nconvolved (t)
[-hki]: print extensive kernel info to output image header (0)
[-ngngauss degree0 sigma0 .. degreeN sigmaN]
: ngauss = number of gaussians which compose kernel (3)
: degree = degree of polynomial associated with gaussian #
(6 4 2)
: sigma= width of gaussian #
(0.70 1.50 3.00)
: N = 0 .. ngauss - 1
: (3 6 0.70 4 1.50 2 3.00
But then I immediately had the following error:
ERROR: Mismatch in the CFITSIO_SONAME value in the fitsio.h include file
that was used to build the CFITSIO library, and the value in the include file
that was used when compiling the application program:
Version used to build the CFITSIO library = 2
Version included by the application program = 1393625872
Fix this by recompiling and then relinking this application program
with the CFITSIO library.
Segmentation fault: 11
So I reinstalled the latest version of CFITSIO, and changed the two paths accordingly. Reinstalling went smoothly but not quite for reinstalling HOTPANTS:
Undefined symbols for architecture x86_64:
"_curl_easy_cleanup", referenced from:
_https_open_network in libcfitsio.a(drvrnet.o)
"_curl_easy_init", referenced from:
_https_open_network in libcfitsio.a(drvrnet.o)
"_curl_easy_perform", referenced from:
_https_open_network in libcfitsio.a(drvrnet.o)
"_curl_easy_setopt", referenced from:
_https_open_network in libcfitsio.a(drvrnet.o)
"_curl_global_cleanup", referenced from:
_ffchtps in libcfitsio.a(cfileio.o)
"_curl_global_init", referenced from:
_ffihtps in libcfitsio.a(cfileio.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [hotpants] Error 1
I suspected this was due to incompatibility between CFITSIO and HOTPANTS, which was written five years ago. As a test, I downloaded an older version of CFITSIO, and repeated all the steps. Annoyingly, now I instead had these at the end when executing make:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: for architecture: i386 file: libcfitsio.a(drvrgsiftp.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: for architecture: x86_64 file: libcfitsio.a(drvrgsiftp.o) has no symbols
But the above warnings do not seem to interrupt with HOTPANTS. My guess is that this has something to do with the structure in which drvrgsiftp.c is written. Anyway, I could successfully reinstall it and obtain a difference image. So it seems working anyway, at least giving me a result rather satisfying, and saving me plenty of time. I will definitely have more tests in the future. Now I start favouring HOTPANTS.
I managed to dig out C/2017 K2 in CFHT images taken in 2013 May. The search was thought to be hopeless at the beginning, because the SSOIS of the CADC did not show any useful serendipitous images within the past three years before its discovery. The comet was discovered already at a heliocentric distance of AU after all. But after noticing a long exposure was used for those CFHT images, I decided to look into the data anyway.
The 3-sigma uncertainty region computed by the JPL HORIZONS based on then-available astrometry was pretty big, ~10′. My search was focusing on images taken from May 12, since that night there were more images. But soon I found out that the last two images were kind of useless, as the limiting magnitude was worsening and the brightness of the sky background rose up, which made my visual search more difficult.
When I was about to finish my last scan of the images from a common chip, I suddenly noticed something comet like moving at the expected direction, and also at a right speed. I measured the positions from three best images, added the astrometry to all observations about C/2017 K2 from this year, and then loaded the file into Find_Orb. Excitingly, the software successfully produced an excellent orbital solution with pretty small astrometric residuals.
With the new orbital solution, looking for C/2017 K2 in images from the second night was way easier. Unfortunately there were only two images, and one of them suffered from bad quality. I managed to find a faint blob of light at the expected pixel coordinates based on the new orbital solution.
The orbital solution by MPEC 2017-N26 puts the comet onto a heliocentric hyperbolic orbit. These CFHT serendipitous observations of C/2017 K2 turned out to be a record of the most distant comet ever observed during the preperihelion stage ( AU). The equilibrium temperature at such a distance would be as low as ~59 K, which means that the activity must be driven by supervolatiles rather than water ice. Hmm, interesting…
I feel like the discussion of the maximum of SNR (signal-to-noise ratio) will benefit my thesis, because my boring mathematical derivation can easily occupy some space to lengthen the number of pages.
To make things easy, I assume that the signal of the observed target can be described by a Gaussian distribution, namely,
where is some referenced intensity, is the radius to the centre of the source, and describes its FWHM. The integrated intensity of the source within an aperture is then calculated from
After doing some simple algebra the result is simply
The SNR of the source is given by the following equation:
For simplicity, we assume the sky background remains constant across the targeted signal source, so can be regarded as a combination constant value related to the sky background intensity and the readout noise of the CCD. To obtain the maximum SNR, we ought to find out when , which can be transformed to
With some algebra, this equation can be changed to
where , and . Unfortunately, this is a transcendental equation so I cannot find out its solution analytically. Instead, let me use the numerical method.
Assuming the targeted source is very faint, which means that , I obtain . Conversely, if the source is extremely bright, i.e., , then we have .
Anyway, the best-SNR radius as a function of is plotted in the following figure. Note that I have converted the radius in FWHM by using .
In conclusion, using FWHM looks to be a good choice.
My colleague Ariel and I went to KPNO near Tucson for our observation run just right before the spring break.
Day 1 (Mar 21)
The flight from LAX to Tucson was ~ 8 am, so I had to wake up at ~ 5:15 am, and then quickly packed up my backpack. After eating some leftover bánh trôi in the fridge, I set off to catch the 6am Flyaway At the Westwood. Within half an hour, I arrived at LAX, where I met Ariel an hour later. The flight was smooth. On board I recognizes your destination – KPNO in distance, thanks to the super obvious signature just next to KPNO, Baboquivari Peak. Of course all the domes appeared as small what dots merely.
We took a Uber to the KPNO in downtown Tucson, on campus of University of Arizona. Since the shuttle wouldn’t leave until 11 am, Ariel and I walked around the campus very briefly, but soon we decided not to continue because the temperature was scorching. Actually I’ve visited the campus three years ago during the DPS. However, it was at night so I didn’t see things clearly. It turned out to be that Ariel and I were the only two persons going to KP, and Ariel had to drive the shuttle.
The drive was ~1.5 hrs. The weather atop KP was visually perfect, although the wind was pretty strong. Starving we were, we devoured quite a lot at the cafeteria. We then went to the 36″ dome, waiting for Flynn, who showed up ~2:30 pm and gave us some very brief instructions on operating the dome and the telescope. Basically we still remember every procedure because our last visit was three months ago only. Flynn mentioned that recently there have been several reports of bear and mountain lion witness around KP and asked us to be cautious.
Ariel and I were both exhausted, so we went back to the dorm and took a nap. After dinner, we met at the dome and started working. Ariel arrived earlier. When I got there, she was taking flats already. Having adjusted the focus of the telescope by taking images of θ Gem, we started observations. Our targets were all comets, SPCs and LPCs. We did plan to observe some unusual asteroids with TJ < 2, including several Damocloids, but eventually didn’t have enough time. During the early run, we unconsciously stuck the telescope because the pointing exceeded the minimum altitude limit twice. As a result, the telescope had to be manually slewed by me through a control panel upstairs. We have now been experts on this!
We observed a PCCP object YF982D7 through BVRI filters. I quickly calibrated all the images, and measured astrometry. No cometary feature can be seen from our coadded images. It finally was designated as 2017 EF12 by the MPC on Day 3, so the cometary feature wasn’t confirmed at all. I’m wondering if some observer has been fooled by bad seeing. Otherwise it’d be quite interesting and exciting, were it a comet; it has a low-eccentricity orbit, and the semimajor axis is small too.
Before daybreak, we attempted to observe several newly discovered comets, such as C/2017 E4 (Lovejoy), only to realise that the influence from the Moon was awfully serious, so had to give up. The observing run ended at ~5:40 am, when the twilight became too strong.
After taking a shower, I went to bed and felt asleep soon.
Day 2 (Mar 22)
I woke up several times around noon, and forced me to sleep. Finally I got up at 4 pm and went to have dinner (breakfast in reality). The weather had been overcast, but the cloud dispersed and revealed a crystal clear sky before sunset. I went to the dome and met Ariel there. Again she was taking flats.
As the twilight faded, we began the observing run with the synoptic observation about a variable star in the region of M42, which happened to be the only observed target tonight. As soon as we finished taking BVRI images of it, we saw the instruction of shutting down the dome, due to the strong wind and rising humidity. So we did. The humidity went up quickly and reached ~100% later. Ariel left the dome around 2 am, and I left around 4 am.
On the way back I felt the ambience a bit horrible. The smog was pretty strong; I could hardly see things beyond ~5 m even with a torchlight. The gust kept producing weird noise. Anyway, I successfully made my way to the cafeteria, and ate some hot soups there.
Before I went to bed, it started raining. What a night!
Day 3 (Mar 23)
Again I woke up several times before the time I planned to get up. This is equivalent to suffering from jet-lag. Before I went to have my dinner, there were considerable amounts of clouds in the sky. But fortunate enough, they dispersed towards dusk. The temperature was also much colder.
We still had plenty of time after having finished taking flats. To kill the time I was busy calibrating images from Day 1 and performing astrometry. Of course I couldn’t finish the all before the sky got dark enough. So we adjusted the focus. I should point out that seeing was extremely awful. However we changed the focus, images of field stars looked fat. Anyway we began the observing run with 128P, which somehow appeared fainter than expected, perhaps due to the fact that the twilight was still there a bit.
Around 9:30 pm, a staff (I forgot his name) brought some visitors into our dome, showing them “what astronomers are doing”. It didn’t last for long, and they left within ~15 min. We then concentrate back on imaging comets. During the run we did attempt to observe Damocloid 2015 RK245, but we couldn’t spot it in individual images.
Before the end, we observed comet C/2017 E4 (Lovejoy), which appeared super bright. Judging from the morphology, it’s a gassy one. It moved pretty fast too. We obtained 10-ish images, which did give us enough SNR of it. We finally challenged the telescope a bit by observing another bright comet C/2017 E1 (Borisov) — then the twilight had been quite strong already. We did get it, but way less impressive than C/2017 E4, maybe because of the bright sky background. The session ended at ~5:45 am. The slender crescent moon looked amazing in the eastern sky.
Day 4 (Mar 24)
Maybe I’m really aging; I again woke up first at 12pm, and then woke up every hour. I dreamed a lot, probably due to which I didn’t sleep well. Anyway, the weather for this observing run appeared to be the best since we arrived at KP; the humidity was low enough, <~ 40%, basically windless, and the temperature was pretty steady, ~9 ℃. As a result, seeing was way better than on Day 3.
We began the observing run from comet C/2010 U3 (Boattini), in Perseus, followed by the synoptic observation of the M42 field. Comet C/2017 C1 (NEOWISE) was observed successfully. Originally we expected it to be faint, since no one had observed it since Feb, but it turned out to be a good one. It was readily visible in individual images, moving fast. The synoptic observation also required us to observe a field containing ρ Oph, so we wasted some time on this.
The session was ended with C/2017 E1 (Borisov). Compared to Day 3, we were able to get two more images before the twilight was too bright.
Day 5 (Mar 25)
I slept a bit better today, at least my first wakeup was close to 1pm. I could clearly remember what I dreamed during the sleep — my family, and my calligraphy teacher 張老師, and his master pieces. He suffered from a stroke last Sept, resulting in weak disability of his right arm. Since then I had had quite a lot of dreams, in which he had convalesced and could write things as before. But a good thing is that I’ve heard that he’s getting better and better.
The late afternoon witnessed a considerable amount of clouds. But thanks to them, I enjoyed a spectacular sunset. The clouds were like burnt, against a violet background of the sky. As time evolved they changed their shapes. I was simply standing outside and looking at the sky, doing nothing else. As one of my favourite things in my childhood which I did a lot was looking at the sky during sunsets, the scene certainly brought me back, just like yesterday once more. I thought I were only a little kid at a moment.
I went back into the dome after the show faded. The clouds were great decorations during the sunset, but form a bad thing for our observing run. The dome was shut down because of them. We had to wait till ~11 pm until the sky turned clear. Our first target was 315P, which displayed a fan-shaped broad tail. A super bright comet we observed was C/2015 ER61 (PANSTARRS), which was initially discovered as an asteroid by Pan-STARRS, but later a cometary feature was found.
This day’s observing run ended again by observing C/2017 E1. Earlier, our advisor Dave emailed us asking whether we’d observed it; it might be a potentially disintegrating candidate due to the rotational instability. Frankly I didn’t think about this point, but just thought that we shall observe it to improve the orbit; the current solution was still pretty nasty, because we saw a significant discrepancy between the observed and predicted positions. Anyway, this time we put more efforts to this comet, and we were able to obtain seven R-band images on it.
Before going back to the dorm, we saw the very elegantly thin crescent Moon rising in the east.
Day 6 (Mar 26)
I finally managed to adapt to the timezone for astronomers, which is good. But unfortunately it was the last day of our observing trip, which is bad.
We planned to end the whole KP observing run by observing C/2017 E1 again. However, for some reason an attempt image showed that the image quality couldn’t be even more nasty, and thus we decided to observe C/2013 C2 (Tenagra), which was predicted to be bright enough, provided that the prediction didn’t go wrong. It hadn’t been observed for almost a year, based on the MPC. So we did. But we couldn’t see anything obvious, so at least it wasn’t that bright. I haven’t yet reduced the data. Maybe something will show up, hopefully.
It’d been pretty bright at the time we shut down the dome. I knew that Venus should be visible, so on the way back we searched for it for awhile. Indeed we managed to spot it by naked eye! It just passed the inferior conjunction with the Sun a day ago! This definitely becomes my record of seeing Venus closest to the Sun during non-transits by naked eye. Alas my camera was left in my office, a bit pity.
I got up at noon the next day. Before we left the mountain I grabbed something for lunch. After checking out, Ariel and I drove back to downtown Tucson at 2 pm, and then took a Uber to the airport. Our flight was ~5:15 pm and landed in LAX an hour later. So the whole trip came to an end, but I still have to reduce and measure all the images from Day 6.
Today saw my first success of establishing an orbital linkage for a SOHO comet, SOHO-3165 , with observations from three consecutive apparitions. I have never done this work on my own — all the previous similar works I’ve done were either verifying Rainer’s solutions, or simply lucky because of lacking influential nongravs. SOHO-3165 doesn’t seem to be an easy one, as neither Bill nor Karl managed to get a solution. Bill managed to get a linkage with the first two apparitions, and third one was left unsuccessful with huge residuals. Therefore I was curious if I could solve this problem.
I haven’t done this job for long, perhaps over three years. As a result it took me a while before I could get onto the right track. A few tweaks of starting conditions enabled Aldo’s Exorb to get a decent orbit for the first apparition. In this step, a parabolic orbit had to be assumed. I then forced the program to solve an orbit for semimajor axis . With observations from the second apparitions the orbital solution was quickly refined. The trick was to fit for the first observation from the second apparition initially. Once the RMS decreased, more subsequent data could be fed in.
If there were no nongravs, the third apparition could be linked without much difficulty. However, this is not the case this time, which exactly complicated the computation work. In fact there is a huge offset in positions in the first observation of the third apparition (which was the only observation from the third apparition at this step). So there must be a nonzero transverse nongrav parameter. I tried , no success, and even worsened the solution to the previous two apparitions. Yet the program turned the optimised into a negative number, from which I immediately realised that the third apparition was postponed. Then . Success! The program quickly shrunk the RMS of the fit. My last step was adding all the remaining observations and let the program find the best-fit solution.
Yet seems like there is something wrong with the code about completing iterations. The code somehow continuously looked for but never succeeds in yielding a final convergence. I therefore manually paused the program. Luckily because the code was generally looping around small RMS. Although the above solution may well not have the smallest RMS, it has one close to already.
The motivation of this writing is simply to provide me as a reminder in the future about how I can obtain an orbital linkage when nonzero nongravs are presented.
I got in touch with Dr. Nalin Samarasinha at PSI, who wrote the codes for all of the special cometary processing filters at PSI website. He has beeing patiently answering every of my questions so I now can understand what his RVSF code is basically doing, though I know very little of FORTRAN. The discrepancy I spot between the source code and the explanation file is indeed a typo.
I added a new keyword to my IDL routine so that the user could have the option whether they like to switch on or off the function of sub-pixel sampling of the image prior to the function of the filter or not. Through some simple tests I realized that the differences between sub-pixelization and non-sub-pixelization of the image would not be very obvious in terms of visual inspection. However, the speed matters considerably, especially when the input image size or the part of regions selected is quite large — the processing time, in this case, could be quite consuming. So it’s a good idea to preview the enhanced image without sub-pixel sampling, not only would you save a great amount of time, but also this would give you a basic knowledge about if your kernel parameters are appropriate, and finally, to process the image by sub-pixelization with reasonable kernel parameters.
I couldn’t discern any difference in the enhanced images with or without sub-pixelization, however, a subtraction between the two clearly reveals what is previously hidden behind. See the following image.
Additionally, I’m very delighted to find that the resulting enhanced image processed by means of sub-pixelization shares great similarity with the one processed through Nalin’s code or PSI’s online tool. So I think my routine is quite successful.
At last, before the end of this update, I need to confess that I had mistakenly regarded the author of the source codes to be Padma Yanamandra-Fisher. I had been confused by their names, all look rather long for me… ><
I think I need to update my blog; it has been so long since my last update… Also should I prove that I’m still alive here.
I saw a message in the comets-ml posted by Martino Nicolini that the PSI website has released a web tool for processing cometary images, including azimuthal median/average/renormalization filters, 1/rho coma model division, and, quite unfamiliar to me, radially variable spatial filter (http://www.psi.edu/research/cometimen). I attempted to process an image taken by HST regarding comet C/2012 S1 (ISON) in May 2013 with the online tool, however, I had difficulty in retrieving the enhanced or processed data. Weirdly the size of the to-be-downloaded file was always 0 KB, obviously problematic. With consideration that the network is not permanently available for me, it sounds absurd and waste-of-time for me to wait for accessible network before performing specially processed cometary images, and therefore I decided to write codes for my own purpose.
So I did by taking reference to the explanation file of the enhancement techniques, understanding the global idea of how the radially variable spatial filter works. Realizing the algorithm of the filter was proved easy, and it didn’t take me long to accomplish the IDL routine.
I did some tests with the image CometCIEF_test.fits provided in this page. The following was generated with kernel A = 4.0, B = 4.0, N = 0.4. It looks correct anyway, quite similar to the appearance in the tutorial file.
However, I found my result would look somewhat different from those presented in the tutorial file if the kernel size was smaller — probably the scaling plays a role there, yet anyway my outcome would look less detailed. The corresponding FORTRAN source code of the filter in the PSI page seems to has a typo, which I have already reported to Padma Yanamandra-Fisher. I’m still comparing my codes against the PSI’s…