
                       APACHE POINT OBSERVATORY SDSS 2.5M

                                 OBSERVING LOG
                              Monday May 31, 2010

                                  (MJD 55348)

---=== OBSERVING TEAM ===---

Viktor Malanushenko
Dan Oravetz
Howard Brewington (support)
French Leger (support)

---=== OBSERVING PLAN ===---

BOSS & MARVELS Science

---=== OBSERVING SUMMARY ===---

Most of the night was lost to clouds.

We observed cart. 6, 3359B.

We experienced some confusion with definitively knowing the exact 
cartridge/plate configurations in some cases as a result of the platedb 
displaying plates as being observable when the cartridges were actually 
having their fibers relaxed and also because of some miscommunication due to 
network issues in the morning. Please see the log for details.

We also experienced half a dozen rotator aborts. French is aware and will 
investigate further today.

Special thanks to Howard and French for your support tonight.

---=== OBSERVING LOG ===---

Afternoon
=========
00:54Z - Arrived to partly cloudy skies and started the checkout.
01:03Z - Verified Watcher server and STUI actors were alive and responsive.
01:05Z - Verified gcam and spectro temperatures were in spec.
01:07Z - Checked disk space on host2 and hub25m:

sdsshost2.apo.nmsu.edu % df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             142G  131G  4.2G  97% /
/dev/sda1              99M   30M   64M  33% /boot
/dev/sdb1             917G  842G   29G  97% /data
sdssfiles1p:/data1/data
                      1.8T  133G  1.7T   8% /files

[observer@hub25m ~]$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md0               48G   33G   13G  73% /
/dev/md1              145M   18M  120M  13% /boot
/dev/sdc1              92G   85G  7.6G  92% /home
/dev/sdc2             2.6T  1.6T  1.1T  61% /data
tmpfs                 2.0G     0  2.0G   0% /dev/shm

01:08Z - Checked out telescope and upper & lower enclosures.
01:12Z - Started ventilation.
01:26Z - ThAr lamp turned on.
01:40Z - Hartmanns, files 115921, 922: moved sp1 -4528 and sp2 -6873
01:57Z - Hartmanns, files 115923, 924: moved b2 -4 degrees. 
02:07Z - Hartmanns, files 115925, 926: all cameras now reporting focused.

Certain cartridges in the plate database were actually NOT plugged with the 
plates that were being reported. This affected the plug plan the night before 
and could have been problematic (e.g. lost about 15 min in the evening, but we 
didn't because we were closed for weather). WE REALLY NEED MORE TRANSPARENCY 
WITHIN THE PLATE DATABASE!

We delayed opening because of approaching radar returns and clouds.

Night
=====
05:20Z - The weather had improved enough to open.
05:25Z - Mounted cart. 9 and prepared for opening.
05:30Z - Opened. Forced a fill from the 180L dewar.
05:38Z - Cals.

Cart. 9 Confusion
-----------------
We ran into confusion regarding which plate was actually plugged in cartridge 
9. Viktor's information was inconsistent with the plate database information 
which was also inconsistent with the plugging note we received from the 
pluggers this afternoon. There were a few issues that contributed to this 
confusion.

To begin with, we were informed that we had network problems on site this 
morning. As such, the pluggers did not send an email detailing which plates 
were plugged and instead placed a hand-written note in the control room. The 
note somehow found its way behind the monitors in the control room and we 
didn't have knowledge of this plugging info until Howard helpfully reported to 
us later on that such a note existed when we approached him for input with the 
inconsistencies we'd discovered.

Before we knew of this note, Viktor used the plugging request information to 
supplement the MARVELS plan, but the plugging request email was not consistent 
with the plugging that was actually done because the request for carts. 9 & 4 
had been swapped; all the correct PLATES ended up being plugged, but they were 
simply not plugged in the exact cartridge configuration requested (normally 
not a major issue, but since we didn't know of the note at this point, it was 
important in this case). After I had returned to the control room when we were 
about to start the exposure, Viktor and I discovered the inconsistency between 
our information and proceeded to understand the differences. When we asked 
Howard for any information about these inconsistencies he told us of the 
network problems he worked on in the morning with Jon and also mentioned that 
he thought the pluggers left a note (thanks, Howard, otherwise we really would 
have been screwed). We eventually found the note and aren't sure as to why it 
migrated out of our sight.

At this point we had three pieces of information: the plate database (which 
had already given us confusion during the formulation of the plugging request 
the night before), the plugging REQUEST, and the plugging note from the 
pluggers. There was a mistake in the note, however, because it was reporting 
that cart. 9 had been plugged with plate 3495 (3490 was reported by platedb). 
So, we decided to physically inspect the cartridges and do a by-eye check of 
the cartridges and their respective plates. It turns out that the plate 
database was correctly reporting the cartridge/plate configurations, but 
during the physical inspection of the cartridge/plate configurations, we ran 
into the next problem. Again, this kind of issue does not normally arise, but 
we called it a software problem since the lack of network in the morning 
ultimately seemed to lead to the confusion.

Rotator Aborts
--------------
We sent the telescope to zenith in order to remove cart. 9 so that we could 
see which plate was on and after we reached 121,90 and pinned the altitude, 
the rotator still had a ways to go (of order 150 degrees or so). The rotator 
aborted and so we re-issued the command to bring the rotator to 0 so we could 
remove the cartridge. The rotator aborted again after having moved a few 10s 
of degrees. The rotator aborted about four times when Howard, having observed 
this behavior in prior months, indicated that the rotator wrap was the culprit 
previously. We inspected the rotator wrap and upon first inspection it felt 
tight and was unable to move freely. Another attempt at moving and after a few 
degrees another abort. After further inspection of the rotator wrap we felt as 
though the wrap had loosened, but again the rotator aborted. We woke up French 
and he had a look, but the rotator never aborted thereafter. In total, we 
experienced about half a dozen aborts and while we think the wrap is the 
issue, French saw nothing obviously wrong, but will investigate further today.

We mounted cart. 6 and proceeded.

06:59Z - Cals.

Cart. 6, 3359B, K15
*******************
07:03Z - GSOGTF.
07:14Z - Science.

07:54Z - Clouds and radar were approaching from the SSE, but we were confident 
we could finish the exposure.

08:10Z - Enclosure on after finishing the previous MARVELS exposure
10:24Z - Mounted cart. 12.
10:25Z - Shut everything down at the telescope.
10:28Z - ThAr lamp turned off.
10:36Z - endNight rejoiced.

---=== IMAGING RUN SUMMARY ===---

 Run     Time    Stripe    Lambda        Last Flavor      Comments
      Start End         Begin    End     Frame
 -------------------------------------------------------------------------

---=== IMAGING RUN DETAILS ===---

---=== SKIPPY RESULTS ===---

Run  Frame nFrames stars muErr   muRms   nuErr   nuRms rot      az    el
---------------------------------------------------------------------------

---=== LTMATCH RESULTS ===---

Run  Field nFields alt az  nGood rowMean rowSig  colMean colSig rot
------------------------------------------------------------------------

---=== SPECTROSCOPY DATA SUMMARY ===---

Summary Checked (y/n): y
QA Procedures Done (y/n): y

  n   time filename    cart  plt flav hrt exp  colA   colB   colC   Hrt
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
  1 01:34 b1-00115918 12 4025 bia         0.0  45772  46493  47144     
    01:34 b2-00115918 12 4025 bia         0.0  25761  25238  26585     
    01:34 r1-00115918 12 4025 bia         0.0  45772  46493  47144     
    01:34 r2-00115918 12 4025 bia         0.0  25761  25238  26585     
  2 01:35 b1-00115919 12 4025 fla Out    30.8  45772  46493  47144     
    01:35 b2-00115919 12 4025 fla Out    30.8  25761  25238  26585     
    01:35 r1-00115919 12 4025 fla Out    30.8  45772  46493  47144     
    01:35 r2-00115919 12 4025 fla Out    30.8  25761  25238  26585     
  3 01:39 b1-00115920 12 4025 arc Out     4.8  45772  46493  47144     
    01:39 b2-00115920 12 4025 arc Out     4.8  25761  25238  26585     
    01:39 r1-00115920 12 4025 arc Out     4.8  45772  46493  47144     
    01:39 r2-00115920 12 4025 arc Out     4.8  25761  25238  26585     
  4 01:41 b1-00115921 12 4025 arc Lef     4.8  45772  46493  47144  HART
    01:41 b2-00115921 12 4025 arc Lef     4.8  25761  25238  26585  HART
    01:41 r1-00115921 12 4025 arc Lef     4.8  45772  46493  47144  HART
    01:41 r2-00115921 12 4025 arc Lef     4.8  25761  25238  26585  HART
  5 01:42 b1-00115922 12 4025 arc Rig     4.8  45772  46493  47144  HART
    01:42 b2-00115922 12 4025 arc Rig     4.8  25761  25238  26585  HART
    01:42 r1-00115922 12 4025 arc Rig     4.8  45772  46493  47144  HART
    01:42 r2-00115922 12 4025 arc Rig     4.8  25761  25238  26585  HART
  6 01:57 b1-00115923 12 4025 arc Lef     4.8  41244  41966  42613  HART
    01:57 b2-00115923 12 4025 arc Lef     4.8  18890  18365  19712  HART
    01:57 r1-00115923 12 4025 arc Lef     4.8  41244  41966  42613  HART
    01:57 r2-00115923 12 4025 arc Lef     4.8  18890  18365  19712  HART
  7 01:59 b1-00115924 12 4025 arc Rig     4.8  41244  41966  42613  HART
    01:59 b2-00115924 12 4025 arc Rig     4.8  18890  18365  19712  HART
    01:59 r1-00115924 12 4025 arc Rig     4.8  41244  41966  42613  HART
    01:59 r2-00115924 12 4025 arc Rig     4.8  18890  18365  19712  HART
  8 02:09 b1-00115925 12 4025 arc Lef     4.8  41244  41966  42613  HART
    02:09 b2-00115925 12 4025 arc Lef     4.8  18890  18365  19712  HART
    02:09 r1-00115925 12 4025 arc Lef     4.8  41244  41966  42613  HART
    02:09 r2-00115925 12 4025 arc Lef     4.8  18890  18365  19712  HART
  9 02:10 b1-00115926 12 4025 arc Rig     4.8  41244  41966  42613  HART
    02:10 b2-00115926 12 4025 arc Rig     4.8  18890  18365  19712  HART
    02:10 r1-00115926 12 4025 arc Rig     4.8  41244  41966  42613  HART
    02:10 r2-00115926 12 4025 arc Rig     4.8  18890  18365  19712  HART
 10 02:28 b1-00115927 12 4025 arc Out     4.8  41244  41966  42613     
    02:28 b2-00115927 12 4025 arc Out     4.8  18890  18365  19712     
    02:28 r1-00115927 12 4025 arc Out     4.8  41244  41966  42613     
    02:28 r2-00115927 12 4025 arc Out     4.8  18890  18365  19712     
 11 02:37 b1-00115928 12 4025 fla Out    30.8  41244  41966  42613     
    02:37 b2-00115928 12 4025 fla Out    30.8  18890  18365  19712     
    02:37 r1-00115928 12 4025 fla Out    30.8  41244  41966  42613     
    02:37 r2-00115928 12 4025 fla Out    30.8  18890  18365  19712
 12 03:58 b1-00115929 12 4025 arc Out     4.8  41244  41966  42613     
    03:58 b2-00115929 12 4025 arc Out     4.8  18651  18606  19712     
    03:58 r1-00115929 12 4025 arc Out     4.8  41244  41966  42613     
    03:58 r2-00115929 12 4025 arc Out     4.8  18651  18606  19712     
 13 04:08 b1-00115930 12 4025 arc Out     4.8  41244  41966  42613     
    04:08 b2-00115930 12 4025 arc Out     4.8  18891  18364  19712     
    04:08 r1-00115930 12 4025 arc Out     4.8  41244  41966  42613     
    04:08 r2-00115930 12 4025 arc Out     4.8  18891  18364  19712

Image       Cart Plate Texp     <PMT>  <Image>    Exptype  Object
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -             
T05:38:01.750  0         11.   40347.   4431.      TIO    NULL             
T05:40:08.250  0        160.     212.    643.     THAR    NULL             
T05:44:43.710  0         11.   40618.   4427.      TIO    NULL             
T06:59:10.400  0         11.   43261.   4915.      TIO    NULL             
T07:01:16.790  0        160.     236.    664.     THAR    NULL             
T07:05:52.280  0         11.   43683.   4905.      TIO    NULL             
T07:14:22.020  6 3359B 3000.      39.   1123.     STAR     K15             
T08:06:09.350  0         11.   45010.   4929.      TIO    NULL             
T08:08:14.660  0        160.     234.    665.     THAR    NULL             
T08:12:51.230  0         11.   41746.   4939.      TIO    NULL

---=== TELESCOPE OFFSETS AND SCALE I ===---

Cart. 6, 3359B
--------------
ObjArcOff = -0.002067, 0.000000, 4782094666.64000, 0.000128, 0.000000, 
4782094666.64000

---=== TELESCOPE OFFSETS AND SCALE II ===---

---=== FOCUS LOG ===---

                      setmir piston                  Temp   Wind
Time     Inst    scale    M1    M2   Foc   Az   Alt   (C) MPH  Dir  filt  fwhm
------------------------------------------------------------------------------
07:35Z  6 3359B  1.00015 -1480  206   275  121  58.3  14.0  17  176  gdr.  1.5"

---=== WEATHER LOG ===---

                             Wind
      Time   Temp F Dewp F  MPH   Direction    Dust  DIMM   Sky

     00:59Z    29    29      8   262  (W)      890     -    
     01:31Z    29    29     18   262  (W)      906     -    
     02:03Z    30    30     14   273  (W)     1070     -    
     02:27Z    29    29     16   267  (W)     1264     -    
     02:59Z    29    29     11   265  (W)     1232     -    
     03:31Z    33    33      9   212  (SSW)    936     -    
     04:03Z    38    38     19   132  (SE)     785     -    
     04:36Z    38    38     16   140  (SE)     892     -    
     05:08Z    38    38     19   146  (SE)     620     -    
     05:40Z    38    38     15   155  (SSE)    710     -    
     06:12Z    37    37     20   158  (SSE)    699     -    
     06:44Z    37    37     18   155  (SSE)    706     -    
     07:16Z    36    36     18   170  (S)      802     -    
     07:48Z    37    37     16   180  (S)      655     -    
     08:21Z    37    37     17   156  (SSE)    699     -    
     08:53Z    37    37     16   178  (S)      886     -    
     09:25Z    37    37     12   177  (S)      890     -    
     09:57Z    38    38     15   173  (S)     1737     -    
     10:29Z    37    37     11   178  (S)     1866     -    

---=== TELESCOPE STATUS ===---

01:12Z - Started ventilation.
05:30Z - Opened.
08:10Z - Closed.
10:25Z - Shut everything down.

No interlocks bypassed.

Status at 10:25Z:

Telescope stowed at 121, 30, 00 mount.
Cart. 12 mounted.
Counterweights at 245.
LN2 autofill system connected and on.
180L LN2 dewar scale: 96.60 lbs. 18.57 psi

---=== SOFTWARE USED ===---

        IOP/SOP: v4_40_0
        Watcher: v2_36_0
        MCP: v6_6_5
        TPM: tpm_v3_5_0
        AstroDa: v15_13_1
        TCC: TCC 2.11.0 2009-06-24
        sdssProcedures: v3_04
        SoS: v1_1_3
        plate-mapper3: NOCVS:v5_0_7
        idlspec2d (sos3): v5_4_12
        STUI: 0.6.2 2010-05-28
        alerts: v1_1_1
        apo: trunk
        boss: v2_3_1
        boss.daq: 1.2.14 
        boss.specMech-- 3.0.1 
        gcamera: v1_1_2 
        guider: v1_0_20h 
        hub: Tag: sdss3_v1_1_3 
        mcp.plc: 24 
        platedb: v1_0_5 
        sop: v1_1_4

---=== MIRROR NUMBERS ===---

PRIMARY:
--------
Scale: 1.000000

MIGS        TONIGHT   NOMINAL
Axial A      -0.6960  -0.6940
Axial B      -0.0270  -0.0410
Axial C      -0.3720  -0.3710
Trans D      11.6080  11.6170
Lateral E    10.3759  10.3505
Lateral F    11.9507  11.9253

GALILS
Commanded: 9900. 400. 5850. 1200. 2950. 2750.
Actual:    9917. 401. 5832. 1205. 2954. 2757.

SETMIR VALUES
PriDesOrient: 0.00 -3.30 5.10 278.08 178.06
PriOrient:    0.00 -3.04 5.26 277.70 178.07

SECONDARY:
----------
Focus: 0.00
Air Temp.: 17.7
Alt.: 29.999985

MIGS        TONIGHT   NOMINAL
Axial A      18.6080  18.6150
Axial B      15.6460  15.6120
Axial C      13.8240  13.8170
Trans D     -11.7830 -11.7780

GALILS
Commanded: 1597646. 1566238. 1588084. -7600. -12400.
Actual:    1597721. 1566188. 1588095. -7597. -12380.

SETMIR VALUES
SecDesOrient: 1257.00 -4.00 -9.00 10.00 234.42
SecOrient:    1256.99 -3.99 -8.96 10.21 234.69

---=== PROBLEMS IN DETAIL ===---

==> Collimate-55348-r1-00115921.log <==
COLLIMATE:  
COLLIMATE: Min offset = -0.80 pix
COLLIMATE: Max offset = -0.60 pix
COLLIMATE: RMS across CCD =  0.06 pix
COLLIMATE:  
COLLIMATE: Mean offset = -0.74 pix
COLLIMATE:  
COLLIMATE: Camera r1 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix)
COLLIMATE: Predict (red) piston movement of  -4528 steps for spectro-1
COLLIMATE: Issue command: "boss moveColl spec=sp1 a=-4528 b=-4528 c=-4528"
COLLIMATE: current   motorPosition  45772 46493 47144
COLLIMATE: new  expected    motorPosition 41244 41965 42616
==> Collimate-55348-r2-00115921.log <==
COLLIMATE:  
COLLIMATE: Min offset = -1.30 pix
COLLIMATE: Max offset = -0.70 pix
COLLIMATE: RMS across CCD =  0.18 pix
COLLIMATE:  
COLLIMATE: Mean offset = -0.99 pix
COLLIMATE:  
COLLIMATE: Camera r2 appears to be OUT-OF-FOCUS (|mean| > 0.15 pix)
COLLIMATE: Predict (red) piston movement of  -6873 steps for spectro-2
COLLIMATE: Issue command: "boss moveColl spec=sp2 a=-6873 b=-6873 c=-6873"
COLLIMATE: current   motorPosition  25761 25238 26585
COLLIMATE: new  expected    motorPosition 18888 18365 19712
----------
