You are here

Start here!

Content owned by achene

Observing

So, you are thinking about using Gemini Observatory for your Science? Or you were just granted a Gemini Observing program? Or you downloaded Gemini data and you want to know what support you have access to for Data Reduction?

This page summarizes the basic facts you need to know before deciding what to do, as it guides you through the whole life cycle of a Gemini observing program.

Gemini observing programs life cycle consists of 3 Phases:

  • Phase I:   Proposal for telescope time and time allocation
  • Phase II:  Program preparation, observations, quality control and archiving
  • Phase III: Data reduction and analysis

Phase I : Proposal for telescope time and time allocation

If you are browsing Gemini’s instruments capabilities, or if you are already working on your proposal, you are in the first Phase of a Gemini observing program.

This section covers:

Browsing capabilities (a.k.a. Phase 0)

At the base of the Phase I is when a researcher decides if Gemini Observatory offers the right set of capabilities for their scientific goals. If you know you want to use Gemini, and are settled with our required instrument set-up, you can move to the next section.

Gemini offers a wide range of observing modes in both Optical (~4000-9000 A) and Infrared (~0.9-5 um), and Adaptive Optics (AO) correction is available with many of the instruments. You may select the best match for your required spatial and/or spectral resolution, central wavelength, field-of-view, sensitivity, etc.

For an exhaustive list of instruments available at Gemini, consult the Instrumentation page.

Selecting your proposal class

Gemini offers five classes to submit proposals for observing time. The following are brief descriptions. Where the header is a link, more detailed pages are to be found there.

Standard Semester 

For programs of normal length, carried out in the queue or in a visiting mode.

Proposal deadline: Two proposal deadlines per year (end of September for the "A" semester, end of March for the "B" semester) announced with a Call for Proposals.

Time Allocation: Proposals are evaluated by a peer review process by the National Time Allocation Committees. 

Program lifetime: 6 months (or potentially more for highly ranked program, a.k.a Band 1).

Observing modes: Queue (+eavesdropping), Target of Opportunity, Classical, Priority Visiting mode

Large and Long Programs (LLP)

For programs requiring a lot of observing time, or a limited amount spread over multiple semesters. 

Proposal deadline: Once a year, corresponding to the standard "B" semester deadline (announced in the Call for Proposals).

Time Allocation: Proposals are evaluated by a Time Allocation Committees focusing only on Large and Long Programs.

Program lifetime: From 2 to 6 semesters

Observing modes: Priority Visiting mode, Target of Opportunity.

Fast Turnaround Programs (FT)

Allowing for quick conversion of ideas into short programs which get executed relatively quickly in a dedicated FT queue. 

Proposal deadline: End of each month!

Time Allocation: Peer review is by other proposers in the same month.

Program lifetime: 3 months

Observing mode: Queue (+eavesdropping)

Director’s Discretionary Time (DDT)

Open to astronomers worldwide, a fixed amount of time per semester is set aside for discretionary allocation. Typically used for programs with potential for high-impact scientific return.

Proposal deadline: Proposals can come in at any time.

Time Allocation: Decided by the Gemini Chief Scientist.

Program lifetime: Variable (usually around 6 months)

Observing mode: Queue (+eavesdropping), Target of Opportunity

Poor Weather (PW)

For programs which can use the worst observing conditions. They help fill gaps in the observing schedule during poor weather periods. Poor weather programs are in Band 4.

Proposal deadline: Proposals can come in at any time.

Time Allocation: Decided by the Head of Science Operations at the site of the requested program.

Program lifetime: From approval to the end of the current semester.

Observing mode: Queue

Still unsure what to pick?

Regular proposal Large & Long programs Fast turnaround Director Discretionary Time Poor Weather
When Once per semester Once per year Once per month Anytime Anytime
Refereeing National Allocation Committees LLP Program Time Allocation Committee Other proposers! Gemini Chief Scientist Head of Science Operation
What for? Routine programs Large allocation and/or multi-semester Immediate, short and/or follow-up Special opportunities Bright targets
Observing mode Queue, Claccical, Priority Visitor Priority Visitor (Queue occasionally) Queue Queue Queue

If you are thinking:

  • I have a regular science program and normal (six month) timescales are fine.
    Ask for a Standard Semester Program.
  • I have a particularly compelling science idea that needs Band 1 treatment asap.
    Ask for DDT.
  • I want data soon (or sooner than the next available semester).
    Ask for FT.
  • I need a lot of data and/or expect it to take a while.
    Ask for LLP.
  • I have special conditions (e.g. IQ20) or timing constraints.
    Ask for either a Standard Semester program, or FT or LLP, depending on the length of the project and the urgency.
  • I have bright targets and can work with really bad conditions.
    Ask for PW.

Selecting the Observing Mode

If you are not familiar with the various observing modes offered at Gemini, here is a brief description. Where the header is a link, more detailed pages are to be found there.

Queue mode

The majority of Gemini proposals are handled in queue-scheduled mode. Observations are carried out by Gemini staff members. 

Target of Opportunity (ToO)

For (i) transients events needing follow-up (either immediate or within 24h) and (ii) programs which will have numerous targets which aren't known at the time of the proposal (e.g. survey follow-up). Approved observations placed in the queue and are executed by observatory staff.

Classical mode

Similar to classical observing at many observatories, where observations are scheduled during fixed time slots. PIs come to the telescope to execute the approved program. Target changes are possible given advance request. Programs are still generated well in advance of the observations, as part of the normal phase II preparation period.

Priority Visitor (PV) mode

This is the default for LLPs and can be requested for Standard Semester programs (approval pending proposal’s ranking). PIs visit the telescope for more time than they have allocated. While there, they can choose to run their own approved program when the conditions are good enough (whether better or worse than requested). Otherwise, they run the regular Gemini Queue of observing programs. Unlike Classical mode, any remaining time on the program after the PIs visit is given back to the Queue. 

Eavesdropping

All Band 1 and 2 PIs are invited to eavesdropping if they wish (once the observing programs are allocated). On a given night, after some preliminary coordination, the PI or designated contact can assist the observer remotely through Skype (or Zoom). The remote program science team member has access to a connection to the Observatory and a FITS server, and takes part of the observations,  including watching and commenting on acquisitions, inspecting data frames and commenting on whether to continue or terminate an observation. The intention to using the Eavesdropping mode can be mention in a proposal, but it needs to be requested once the approved program is created.

Still unsure what to pick?

If you are thinking:

  • Just send me my data.
    Ask for Queue mode.
  • I can demonstrate that my program needs my presence during the observations and I can accept the risk of being weathered off completely.
    Ask for Classical mode.
  • I'd like to be physically present but I would prefer to mitigate the risk of weather loss.
    Ask for PV mode. (If not granted, consider requesting Eavesdropping.)
  • I'd like the chance to interact with the observers when data are taken.
    Ask for Queue mode, and sign up for Eavesdropping.
  • I want my student to get observing experience.
    Ask for Classical or PV mode.

Creating the proposal

Investigators should use the Gemini Phase I Tool (PIT) for creating proposals. It provides an interface via which the Investigator can enter all the necessary information (e.g., Title, Abstract, Scientific Justification, Experimental design).

The Gemini PIT will highlight errors in the proposal. 

​A new Phase I Tool is released for each semester. Information on how to download and install the Tool is given on the semester's installation page. Differences from previous versions are described on the hot news page.

Help on the Tool is available via the PIT help page including video tutorials.

Proposal submission

The Gemini PIT is also used to submit proposals. Once you are ready to submit a single click on the "Submit this Proposal" button will send the proposal and PDF attachment to the servers of your selected Participant(s).

Submission to a single partner

If you are applying for time for a Standard Semester Program (Queue or Classical) or a Program Exchange (Subaru, Keck, CHFT) through one of the Gemini Participants, simply enter the total requested time in the line of the corresponding Participant (US, Canada, Argentina, Brazil, Chile, South Korea, University of Hawai’i).

Investigators applying for NOAO time using multiple NOAO telescopes for a single program may instead use the NOAO submission form. However, Large and Long Program proposals and any proposal that requests time from multiple Participants, known as a joint proposals, must use the Gemini Phase I Tool.

Joint proposals

International collaborations ("joint proposals"), where time is being sought from several participating countries, must be submitted using the Gemini PIT even if the proposal involves Participants that offer other submission mechanisms. 

DO NOT submit one proposal per participating country. Into a single proposal, split the requested time among the participating countries, and submit only once. 

Non-Gemini partner proposals

Proposals from outside of the Gemini community must be submitted through one of the NTACs. Links to Participant- and host-specific information are contained in the following table.

Support available to you

The Proposal creation and submission process is supported by staff in the National Gemini Offices (NGOs). The Gemini Help Desk should be used to contact NGO staff regarding proposal submission. Currently submissions cannot be overwritten. Submit your corrected proposal and contact your NGO office to let them know which proposal you want considered, and which omitted.

Time Allocation

Beside DD and PW Programs which are approved directly at the Gemini Observatory level, and the FT proposals that are evaluated by other proposers, the LLP and the Standard Semester Proposals are evaluated by Time Allocation Committees (TAC).

Evaluation of Standard Semester proposals

Gemini Time Allocation proceeds in two phases, the National TAC (NTAC) and the International TAC (ITAC).

NTAC

After the Call for Proposals deadline at the end of March or September, each National Gemini Office (NGO) collects the proposals from its community, resolves logistical issues and errors, and evaluates them for technical feasibility. Each Participant then peer reviews its proposals via a NTAC, and creates a ranked list based primarily on scientific merit, but also on strategic feasibility. NTAC meetings generally occur 4-6 weeks after the proposal deadline (late Oct/early Nov or late April/early May).

The outcomes of the NTAC meetings are ranked lists of approved programs and recommended time allocations to fill each Participant's time allocation, for consideration by the ITAC. These approved programs are forwarded to Gemini via an NTAC package, due ~6 weeks after the proposal deadline each semester (generally 10-13 May or 10-13 November). 

ITAC

The Gemini ITAC merges the proposals to create a single combined list of programs for execution. The ITAC only sees those proposals that are forwarded by each NTAC. Prior to the ITAC meeting in the third or fourth week of May or November, the ITAC chair merges all of the forwarded programs from each Participant to produce a draft queue. The merging process is a deterministic process that steps through each Participant's ranked program listing and distributes the time according to each Participant's share of the total time. The draft queue is shared with the ITAC members and with the Gemini Heads of Science Operations who construct a preliminary telescope schedule (incorporating engineering and commissioning work with classical time requests and other considerations) and communicate any issues or concerns for operations.

During the ITAC meeting, the ITAC representatives analyze the draft queue in detail; they consider instrument availability, Target of Opportunity (ToO) programs, duplicate observations and the distribution of programs across different observing condition constraints, and adjust the ensemble of proposals to construct the best "queue", both for the individual communities and for the observatory.

The outcome of the ITAC meeting is a recommended queue list (including scientific ranking bands), and classical program list for execution in the coming semester. This recommendation is forwarded to the Gemini Director for detailed consideration. The final allocation is at the Director's discretion and changes can be made, although these are generally minor in nature. The final approved list of programs is shared with the NGOs and ITAC members, and "NGO feedback" is generated which includes relevant comments intended for the Principal Investigators for proposals that were forwarded to ITAC.

Within a few days of sending the NGO feedback, the observatory notifies all successful PIs directly by email with information about their time allocation and how to start the Phase II process. At the same time, all the approved programs are published on the Gemini website and the "Phase II skeletons" are made available in the Observing Database. This generally occurs around 15 June or 15 December.

List of Phase I resources

This will list links to tutorials, statistics, support avenues, ... (coming soon)

Phase II : Program preparation, observations, quality control and archiving

If your proposal was successful, and you were granted a Gemini Program, then you enter in the Phase II. Congratulations!

This section covers:

Program preparation

It is now expected from you to access your program using the Gemini Observing Tool (OT), open your program (using your program key sent to you in your time allocation notification email), and prepare your observing sequences for all your targets. The deadlines this process are advertised each semester on the "special instructions" page. Failure to submit a defined program by the deadline will result in the program being canceled. Extensions are granted only in exceptional circumstances and only if requested prior to the deadline.

Note that you are not alone in this. Your program in the OT has already been pre-filled with your target list, and with one sequence template per instrument configuration, as defined in your proposal. You can consult the Phase II section of this website which offers a series of tutorial, procedures and checklists. Also, your program is attributed a Principal and a Secondary Support person (coming either from your country’s National Gemini Office (NGO) or from Gemini). Please do not hesitate to contact them would you have any question or doubts. Finally, please take the time to read carefully the notification email, as it contains lots of useful information.

Preparation work and checks

When you access your program for the first time, it is retrieved from a remote-access database located at the Observatory, the ODB. You may save your program locally as an XML file, although the completed sequences must be returned to the Observatory by storing them into the ODB (with the “Sync” button). 

After you mark a sequence “For Review”, it is returned to your NGO or Gemini Observatory for checking. For established instruments, the NGOs provide the first level of support (except for US PIs for which the Observatory staff does that). For new instruments or modes, and for higher-level queries, Gemini Observatory staff provide support.

After checking by NGO staff the Gemini Contact Scientist (CS) is notified that the observations are ready for final checks. The CS carries out a final verification before setting the observation status to "Ready", which prompts the ODB to notify the investigators that their observation is now a candidate for scheduling. (If there is a problem with the observation definition, then the observations are returned to the Primary Support Contact for further iterations with the investigators). At this point, the observations are available to be executed.

If your strategy changes

It is possible to make changes to your program during the preparation or even once the semester has started, as long as this does not substantially alter the science approved by the TAC. Certain changes, such as target swaps or improvements to observing conditions, require approval from the local head of science operations (see the change request process). However, if you wish to relax the IQ constraint for your program in order to increase the chances of getting data, for instance, this can normally be easily accommodated. 

Once a sequence is “Ready”, you cannot change observations in the ODB; any requests for subsequent changes to observation details must be made via their CS. On-site (classical, PV) observers may access their observations in the ODB directly using the OT, but non-approved changes of target or instrument setup are not allowed.

Now, what to expect?

What should you expect next? Here is what happens once your observations are in the queue, how to monitor their progress, and what to do if things aren't going as you expected.

Queue planning and nighttime operations

Every day a plan is made for the night's observing, with variations for different sets of weather conditions. The queue planning follows a set of principles aimed at making sure that programs are completed (following priority status) and telescope time used efficiently (i.e., getting the best out of the weather conditions). More detail is given on the Queue Planning and Execution page.

Monitoring your observations

If desired, the progress of a Gemini observing program can be monitored quite closely. There are several ways of doing so:

  • By default, PIs are signed up to receive automatic email notification the morning after each night that data are obtained for a given program. These emails are sent before any quality assessment is performed on the data.
  • The Gemini Observatory Archive (GOA) can be queried for data sets belonging to a program. Raw files usually go into the archive shortly after being written to disk.
  • The archive also sends periodic emails notifying PIs that data sets are available for collection. No sign-up is needed for these emails. These emails go out once Gemini staff have created a data package for the program, usually on Friday of each week for programs with data obtained during the preceding week.
  • The PI (or, in fact, anyone with the program "key") can fetch the program, Sync it, and see how it is progressing. As observations are executed, their status changes in the OT, time is charged to the program and/or partner country, and an "observation log" is created inside each observation. An example of a program with observations in progress can be found here.
  • Various pages in the Queue and Schedules section of the website give program execution statistics. See for instance the "Queue Summary" and "Database Snapshots" links.
  • When your program is completed or deactivated before completion, you get a notification email from a Gemini staff. This email contains post-observing (Phase III) useful information when data have been taken.

You thought you'd have (more) data by now. What's going on?

We make every effort to obtain as much data as possible of as high a standard as possible. After all, everybody benefits when a good publication is written based on a Gemini data set. However, sometimes programs aren't started, aren't completed, or take longer to complete than the PI anticipated. The software used by the ITAC to create the queue for the semester mean that the available programs are now more balanced in terms of observing condition requirements and RA distribution, but (for example) a long period of poor weather coinciding with an instrument being on the telescope for a limited time can have a large effect on program execution nonetheless. Similarly, the queue scheduling software is quite effective in ensuring that time-critical programs are observed as requested by the PI, but mistakes and misunderstandings can still occur. If you are concerned about progress on your program, you are welcome to contact your NGO support, Gemini contact scientists or the Head of Science Operations at the relevant site (contact details on this page).

The following information may be useful:

  • Regular queue programs in Band 1 are allowed to execute over the full subsequent semester and encouraged to execute late in the semester before their formal allocation begins. These programs, designated "persistent Band 1", have Band 1 priority, whichever semester they are executing in.
  • Semester boundaries are treated as flexible at both sites. To make best use of telescope time, we will start a new semester's observations before the start of that semester, and "balance" that by keeping some (non-Band 1) programs active after the end of the current semester.
  • Non-Band 1 programs not used for this balancing time are deactivated shortly after the end of the semester. Occasionally, programs from previous semesters may be reactivated to fill gaps in the queue. This is unusual, however.
  • Band 3 programs overfill the queue and not all of them will be started. See the Band 3 page for advice on maximising the likelihood of getting data for your Band 3 program. This information may also be useful for PIs of higher- ranked programs in semesters with high weather loss, for example.
  • If your program is not making good progress, it is recommended to work with your CS if the weather conditions could be relaxed, increasing the 

Data checking, quality assessment and repeat observations

The daytime data checking is limited to band 1 observations plus other data sets that may be requested by the queue coordinator. The data checking usually takes place the day after the observations are taken, but can take longer if there's a backlog of data after a long weekend, for example. The data checking includes making sure that the PI's requested observing condition constraints were met, checking for missed calibrations, and looking for signs of possible instrument or telescope issues that could affect the data. Data quality states are set in the file headers and can also be seen in the OT.

If a problem is found, then often a replacement observation will be set up by the queue coordinator and made available to be repeated at the next opportunity (NB: this does not apply to Band 4 programs). This is generally the case for simple issues such as an observation taken under poorer seeing conditions than requested, does not require PI interaction, and does not affect the program's chances of being completed. If you see new observations in your program that you didn't put there, this is probably why; look for a yellow note or comments in the OT observation log for the explanation. For more complex or ambiguous cases the queue coordinator or another staff member may get in touch with the PI to ask for more information about how to proceed. A prompt response to such emails helps ensure that data can be retaken in a timely manner, if that is what is needed.

The aim of the nighttime observer and many other observatory staff is to ensure that PIs receive data that meet their requirements. However, if you feel that this is not the case with your data, please get in touch with your NGO support, Gemini contact scientists or the Head of Science Operations for the site in question. If we are at fault we will endeavour to correct the mistake.

Gemini Observatory Archive

The Gemini Observatory Archive (GOA) located at https://archive.gemini.edu is the primary conduit to obtain data from Gemini Observatory. The Help and About pages also linked at the top of the archive page give more information.

The GOA provides a simple yet powerful interface to search for and download science data, calibrations and observation logs both for Gemini program PIs and Co-Is, and also for people searching for Gemini data by observation details such as instrument configuration, sky coordinates, observing date etc. 

List of Phase II resources

This will list links to tutorials, statistics, support avenues, ... (coming soon)

Phase III : Data reduction and analysis

If you enter Phase III when you need to reduce Gemini data. Maybe they are data from a program of yours, maybe you just got them from the archive and you want to use them for a new project. In either case, Gemini offers you support with a Data Processing software (currently based on IRAF), and a Help Desk system would you need help to overcome any obstacle with the data reduction.

What to expect?

The Gemini Observatory will support anyone that need to reduce Gemini data. That support consist of:

  • Distributing and maintaining a Data Processing software
  • Answering questions about issues encountered with the Data Processing software, including installation, troubleshooting error messages, and fixing bugs.
  • Repeating the observation of problematic data if you are the owner of the data (PI with proprietary access) and the corresponding program is currently active.

On the other hand, the support does NOT consist of:

  • Basic training on Data Reduction (e.g., What is a bias and flat, and what they are for).
  • Executing the Data Reduction in place of the PI or a Co-I.
  • Developing or modifying software to accommodate an unusual observation.

Any effort on the later list is expected to be done by a member of the project’s science team (PI or Co-I). There is no restriction regarding the possibility to include a Gemini staff as a Co-I in a science project.

Data formats

Gemini facility instruments

Data from Gemini facility instruments are stored as Multi-Extension FITS (MEF) files. The data from a single "OBSERVE" command from the Observing Tool creates one MEF file. Each MEF file consists of one Primary Header Unit (PHU) and one or more extensions that contain pixel data. The extensions in MEF files are indexed from [0], which is the PHU. The extension numbers ([0], [1], etc.) in the figure below are shown to the left on each of the panel.

[MEF]

Visitor instruments at Gemini

Data from the current visitor instruments are stored as simple FITS files. Each single FITS file consists of header information immediately followed by the pixel data, as shown in the figure below.

[Simple FITS]

How to handle MEF files with IRAF

The Gemini IRAF package is written to handle the MEF files produced by the Gemini facility instruments. Most IRAF tasks can handle MEF files if the extension number is specified. However, some IRAF tasks that attempt to derive the output file names from the name of the input image may get confused by the extension number and/or add the extension number to the output file names. In such cases it is recommended to specify the output file names rather than let IRAF choose these based on the name of the input image.

Below are some examples of simple IRAF commands to access, display and manipulate MEF files. All examples assume that the IRAF imtype is set to "fits". This is done with the command

set imtype="fits"

The user may either include this command in the "login.cl" file, or issue it on the IRAF command line. If the command is issued on the command line, it needs to be followed by the command

flpr

in order to take effect.

These examples assume that the MEF file example1 has one pixel extension, and that the MEF file example2 has three pixel extensions, as shown in the figure above.

  1. To list the PHU of the MEF file example1: 
    imhead example1[0] long+
  2. To list the header information of extension [1] of the MEF file example1 
    imhead example1[1] long+
  3. To display extension [1] of the MEF file example1 
    display example1[1] 1
  4. To copy extension [1] of the MEF file example1 to a simple FITS file 
    imcopy example1[1] newimage
  5. To add extension [1] of the MEF file example1 and extension [2] of the MEF file example2 and save the output to a simple FITS file. This requires that the dimensions of the two extensions match. 
    imarith example1[1] + example2[2] newimage
  6. To multiply each of the extensions in the MEF file example2 by extension [1] of the MEF file example1, overwriting the current extensions in the MEF file example2. This requires that the dimensions of the extensions match. 
    imarith example2[1] * example1[1] example1[1,overwrite]
    imarith example2[2] * example1[1] example2[2,overwrite]
    imarith example2[3] * example1[1] example3[3,overwrite]

Getting started

Getting started page?