Search
Text Size:
Smaller Text Normal Text Larger Text

Facebook

Follow us on Twitter

FIES Trouble shooting

Problem: New Mask or Slide does not move (Bundle D)
Problem: FIES fiberhead needs to be initialised (Bundle C)
Problem: No light from Top Halogen/Thar on FIES images
Problem: No starlight on FIES images but FIESTA indicates that all is OK
Problem: Commands to FIES BIAS hang
Problem: STANCAM direct image seems vignetted
Problem: Guide star obscured for STANCAM direct image (camera-probe-stancam)
Problem: FIES-calibs results in message : FIESscript wavecal found few ThAr lines
Problem: steady 'noise' pattern in flats -- condensation
Problem: STANCAM CCD is warm: can i still observe?
Problem: fiber guider: the green cross is far away from where I think the fibre really is

Problem: The new rotating Mask or XY slide does not move

The new top unit rotary mask and slider arm are still under development and the arm is proving particularly unreliable, so be patient!

If the sequencer commands 'arm init' or 'mask init' fail to move the respective mechanism, try the following steps

  1. On florence, connect to the Windows 10 fiberhead server via the remote desktop client program 'remmina', (type remmina in a terminal on florence). To open the desktop click anywhere on the line 'FIES fiberhead win10' and select OK from the popup window.
      In the unlikely case you can not connect, the Windows pc may need a reboot:
      • Locate to Windows pc on the mount plate on the side of the huge ALFOSC lamp control box on the telescope adapter.
      • If no led light is seen on pc: Press the small on/off button on the top. If nothing happens, disconnect the network cable, momentarily press the on/off button, wait a minute and connect the network cable again.
      • If led lights are seen: again disconnect and connect the network cable. If still no connection, press the on/off button to reboot.
  2. Identify the mask & slide TCP server programs running in their terminal windows. Exit the server program for the mechanism giving trouble by pressing the 'X' on the corner of the corresponding terminal window. Usually the lower terminal is the slide (arm) server and the upper the rotary (mask) server.
  3. Start the server again by clicking on the corresponding desktop icon ('maskserver' or 'slideserverV4'). Verify communication by issuing the respective init command from a Fies Sequencer window (e.g. for the slider (arm) 'arm init') - check that output is seen on the server terminal windows. For the mask it should return in a few seconds but the slider can take some minutes. The slide should reply with something like "INFO: Initialised successfully" if successful.
  4. SLIDE:
    • If the slide does not initialise (reference) then try issuing the 'arm init' command again from the Fies Seqterm and wait the few minutes.
    • If still problems try running the 'factory' program from the Windows remote desktop. Kill the slide server and start the Servo3AxUSB program, by clicking on the icon. In the upper right click Axis 1 Goto Ref and when you see 'yes' in the Reference column for Axis 1 repeat for Axis 2. Note during the Goto Reference the window becomes unresponsive and the loop count stops. It can take a couple of minutes to reference (initialise) each axis. If successful then quit the Servo3AxUSB program by clicking the 'X' and start the slide server from the icon, and move the terminal window down to the bottom right of the desktop.
    • If still stuck, exit the server program and power cycle the controller. First disconnect the USB cable going into the windows pc, then unplug the power cable. Wait for a moment, then connect the power cable. Wait for a moment, then connect the USB cable. Finally, start the server program and initialise.


Problem: FIES fiberhead needs to be initialised
Problem: No light from Top Halogen/Thar on FIES images
Problem: No starlight on FIES images but FIESTA indicates that all is OK

Possibly the calibration arm lost its internal positioning reference. Re-initialize the mechanism by typing 'init' in the FIES SEQUENCER window; this takes about 3 minutes or so. Then proceed with e.g. 'setup-fibN-halo' or 'setup-fibN-thar', and take an image.

If 'init' takes too long, then restart the obssystem and subsequently try 'init' again.

Note that this typically happens when the power to the FIES calibration box has been lost/ toggled.
Also note that the system does not initialise by itself upon startobssys: this has to be done explicitly by the user.


Problem: Commands to FIES BIAS hang

If the SEQUENCER commands to the BIAS program do not respond and shutting down and starting up as described at the top of this page does not cure the problem, look for this process (using 'ps aux')

obs 21121 0.0 0.1 1604 840 pts/2 S Aug22 0:05 /home/software/fies/src.v1/bias_ds9/SOAP-serv/biasservserver

and kill it (in this example 'kill -9 21121'). Then the obssystem can be (re-)started.


Problem: STANCAM direct image seems vignetted

If the STANCAM direct image looks like below (or flipped in X direction),
then please check that the ALFOSC/FASU calibration lamps are switched off.


Problem: Guide star obscured for STANCAM direct image (camera-probe-stancam)

For a minor fraction of the autoguider field, the pick-off mirror can vignet the guide stars when going to 'camera-probe-stancam'. The guide star should reappear when moving to 'camera-probe-fies'.

If this is a problem, then choose a different guide star.


Problem: FIES-calibs results in message : FIESscript wavecal found few ThAr lines

This is the dying ThAr-lamp phenomenon. From experience we know that from one day to another a well-functioning ThAr lamp can go into a state in which the ionisation balance is gradually changing, and subsequently the strength of lines will change accordingly. Over the course of several days-weeks, the lamp will gradually die.
Lamp life time for the top-calib unit is typically between 0.5 and 1 year.

The phenomeneon is picked up by the QC routine triggered by fies-calibs. The routine logs the number of ThAr lines found in the ThAr frames that are exposure-level scaled to have 15000ADU in a particular blue line (4765 Angstrom, see above). The log is in fies-pipe@elena:~/datared/thar-solutions.log
If an anomalous number of lines is found (50-90% of nominal), then a warning is sent to the talker, and an email to 'duty'. If less than 50% of the lines are found, then ERROR-level warnings are sent.

If the warnings persist, or an ERROR is given, then it is likely that the lamp needs to be changed.

Example: A 'grep' for 'fib3' in file fies-pipe@elena:datared/thar-solutions.log results in the following:
Date
Image Found Fit RMS
2012-12-24 15:48 fib3.cfg waveref 554/568 523/554 0.00201
2012-12-25 17:18 fib3.cfg waveref 555/568 527/555 0.00216
2012-12-26 17:54 fib3.cfg waveref 564/568 538/564 0.00191
2012-12-27 17:23 fib3.cfg waveref 568/568 545/568 0.00182
2012-12-28 08:24 fib3.cfg waveref 526/568 497/526 0.00246
2012-12-28 17:41 fib3.cfg waveref 531/568 509/531 0.00265
2012-12-29 08:29 fib3.cfg waveref 501/568 471/501 0.00236
2012-12-29 17:30 fib3.cfg waveref 522/568 498/522 0.00248
2012-12-30 18:14 fib3.cfg waveref 525/568 498/525 0.00244
2012-12-31 16:42 fib3.cfg waveref 527/568 488/527 0.00194
2013-01-01 17:46 fib3.cfg waveref 441/568 423/441 0.0027
2013-01-02 17:08 fib3.cfg waveref 531/568 507/531 0.00234
2013-01-03 09:16 fib3.cfg waveref 505/568 477/505 0.0024
2013-01-03 18:10 fib3.cfg waveref 492/568 464/492 0.00234
2013-01-04 09:29 fib3.cfg waveref 560/568 544/560 0.00227
2013-01-04 18:03 fib3.cfg waveref 565/568 546/565 0.00241
2013-01-05 08:28 fib3.cfg waveref 568/568 547/568 0.0019
2013-01-05 17:23 fib3.cfg waveref 563/568 544/563 0.00227

In the 'Found' column you can see that between 2012-12-28 and 2013-01-03 (when the lamp got replaced) the number of lines found was lower than the nominal 568 lines. Replacing the lamp helped getting back to normality (from 2013-01-04 onwards).

Another way to find out if the lamp is still OK is by blinking the image with an image from a month before. However, proper scaling has to be chosen and even then the effect can be hard to spot to the untrained eye.

Here is an up-to-date plot of the last 800 entries in the thar-solutions.log:


Problem: steady 'noise' pattern in flats -- condensation

In Oct 2013 and in May 2014 we had a problem caused by pumping, which resulted in condensation on the CCD. As flat 2D flats are difficult to obtain with FIES, the interpretation of the symptoms may not be 100% correct. But in both cases, the symptoms disappeared after the following warm pump of the cryostat.

Left: no condensation. Right: with condensation. Seven central orders are drawn, blue to red.
Bottom: 2D division of the two, where the condensation is apparent using the scattered light between the orders as flatfield source.



Problem: the STANCAM CCD is warm: can i still observe?

When the StanCam CryoTiger does not work properly, and the CCD is above the normal operating temperature of -95C, StanCam can still be used to do observations with FIES.

The observations are then limited by the increased dark current, inhibiting target acquisition onto the fibers for faint stars. However, at a CCD temperature around -20C one can still acquire bright stars of V<10 onto the fiber.

As the CCD heater will further warmup the CCD during read outs, it is advised to stop the StanCam movie once the target is properly centered on the fiber, and to switch off fiber guiding. This to minimise the number of StanCam CCD readouts.

This problem should be reported in the fault database as soon as it happens.


Problem: fiber guider: the green cross is far away from where i think the fiber really is

In case the green cross has moved off the fiber and you are not sure where the fiber is in the movie image (e.g. all looks dark), you can return to normality using fibpos.snafu .
This will reset the green-cross = fiber position to approximate but sound X,Y coordinates on the StanCam movie image.
Proper determination of the fiber positions is required afterwards (use fibpos-man).



Back to top Last modified: 19-Sep-2017