Classes

Astromatic-Wrapper

This Package is a wrapper for E. Bertin’s AstrOmatic Software suite

astromatic_wrapper.api

API for E. Bertin’s Astromatic software suite

class astromatic_wrapper.api.Astromatic(code, temp_path=None, config={}, config_file=None, store_output=False, **kwargs)

Class to hold config options for an Astrometric code.

Initialize a particular astromatic code with a given set of configurations.

Parameters:

code: str

Name of the code to use

temp_path: str

Path to store temporary files generated by the astromatic code (such as resamp files in SWarp)

config: dict (optional)

Dictionary of configuration options to pass in the command line.

config_file: str (optional)

Name of the configuration file to use. If none is specified, the default config file for the given code is used

store_output: boolean (optional)

If store_output is False, the output of the code is printed to sys.stdout. If store_output is True the output is saved in a variable that is returned when the function is run.

build_cmd(filenames, **kwargs)

Build a command to run an astromatic code.

Parameters:

filenames: str or list

Name of a file or list of filenames to run in the command line statement

**kwargs: keyword arguments

The following are optional keyword arguments that may be used:

• code: str

  Name of the astromatic code to use. This should be contained in astrotoyz.astromatic.api.codes

• config: dict (optional)

  Dictionary of configuration options to pass in the command line.

• config_file: str (optional)

  Name of the configuration file to use. If none is specified, the default config file for the given code is used

Returns:

cmd: str

Commandline statement to run the given code

kwargs: dict

Dictionary of keyword arguments used in the build

get_version(cmd=None)

Get the version of the currently loaded astromatic code

Parameters:

cmd: str (optional)

Name of the command to run. If this isn’t specified it will use the cmd specified when the Astromatic class was initialized. This is usually just the code to run (for example ‘sex’, ‘scamp’, ‘swarp’, ‘psfex’, ...) but occationally if the user doesn’t have root privilages this may be another location (for example ~/astromatic/bin/sex).

Retruns

——-

version: str

Version of the specified astromatic code

date: str

Date associated with the specified astromatic code

run(filenames, store_output=False, raise_error=True, **kwargs)

Build the command and run the code with a given set of options. If one of the keyword arguments is store_output=True the output of the code is returned, otherwise the status of the codes execution is returned.

Parameters:

filenames: str or list

Name of a file or list of filenames to run in the command line statement

store_output: bool (optional)

Whether to store the output and return it to the user or print the output to the screen.

raise_error: bool (optional)

If raise_error==True, python will raise an error if the astromatic code fails due to an error

**kwargs: keyword arguments

The following are optional keyword arguments that may be used:

• code: str

  Name of the astromatic code to use. This should be contained in astrotoyz.astromatic.api.codes

• config: dict (optional)

  Dictionary of configuration options to pass in the command line.

• config_file: str (optional)

  Name of the configuration file to use. If none is specified, the default config file for the given code is used

Returns:

result: dict

Result of the astromatic code execution. This will minimally contain a status key, that indicates success or error. Additional keys: - error_msg: str

If there is an error and the user is storing the output or exporting XML metadata, error_msg will contain the error message generated by the code

• output: str

  If store_output==True the output of the program execution is stored in the output value.

• warnings: str

  If the WRITE_XML parameter is True then a table of warnings detected in the code is returned

run_frames(filenames, code=None, frames=[1], raise_error=True, **kwargs)

If the user is running sextractor on an individual frame, this command will correctly add the frame to the image filename, flag filename, and weightmap filename (if they are specified).

Parameters:

filenames: str or list

Name of a file or list of filenames to run in the command line statement

code: str (optional)

Name of the astromatic code to use. This should be contained in astrotoyz.astromatic.api.codes and defaults to Astromatic.code.

frames: list (optional)

Subset of AstrOmatic code to use. Defaults to [1].

raise_error: bool (optional)

If raise_error==True, python will raise an error if the astromatic code fails due to an error

**kwargs: keyword arguments

The following are optional keyword arguments that may be used:

• config: dict (optional)

  Dictionary of configuration options to pass in the command line.

• config_file: str (optional)

  Name of the configuration file to use. If none is specified, the default config file for the given code is used

Returns:

result: dict

Result of the astromatic code execution. This will minimally contain a status key, that indicates success or error. Additional keys: - error_msg: str

If there is an error and the user is storing the output or exporting XML metadata, error_msg will contain the error message generated by the code

• output: str

  If store_output==True the output of the program execution is stored in the output value.

• warnings: str

  If the WRITE_XML parameter is True then a table of warnings detected in the code is returned

exception astromatic_wrapper.api.AstromaticError

Bases: exceptions.Exception

args

message

astromatic_wrapper.api.run_psfex(pipeline, step_id, catalogs, api_kwargs={})

Run PSFEx with a specified set of parameters.

Parameters:

pipeline: `astromatic_wrapper.utils.pipeline.Pipeline`

Pipeline containing parameters that may be necessary to set certain AstrOmatic configuration parameters

step_id: str

Unique identifier for the current step in the pipeline

catalogs: str or list

catalog filename (or list of catalog filenames) to use

api_kwargs: dict

Keyword arguements to pass to PSFEx

Returns:

result: dict

Result of the astromatic code execution. This will minimally contain a status key, that indicates success or error. Additional keys: - error_msg: str

If there is an error and the user is storing the output or exporting XML metadata, error_msg will contain the error message generated by the code

• output: str

  If store_output==True the output of the program execution is stored in the output value.

• warnings: str

  If the WRITE_XML parameter is True then a table of warnings detected in the code is returned

astromatic_wrapper.api.run_scamp(pipeline, step_id, catalogs, api_kwargs={}, save_catalog=None)

Run SCAMP with a specified set of parameters

Parameters:

pipeline: `astromatic_wrapper.utils.pipeline.Pipeline`

Pipeline containing parameters that may be necessary to set certain AstrOmatic configuration parameters

step_id: str

Unique identifier for the current step in the pipeline

catalogs: list

List of catalog names used to generate astrometric solution

api_kwargs: dict

Dictionary of keyword arguments used to run SCAMP

save_catalog: str (optional)

If save_catalog is specified, the reference catalog used to generate the solution will be save to the path save_catalog.

Returns:

result: dict

Result of the astromatic code execution. This will minimally contain a status key, that indicates success or error. Additional keys: - error_msg: str

If there is an error and the user is storing the output or exporting XML metadata, error_msg will contain the error message generated by the code

• output: str

  If store_output==True the output of the program execution is stored in the output value.

• warnings: str

  If the WRITE_XML parameter is True then a table of warnings detected in the code is returned

astromatic_wrapper.api.run_sex(pipeline, step_id, files, api_kwargs={}, frames=[])

Run SExtractor with a specified set of parameters.

Parameters:

pipeline: `astromatic_wrapper.utils.pipeline.Pipeline`

Pipeline containing parameters that may be necessary to set certain AstrOmatic configuration parameters

step_id: str

Unique identifier for the current step in the pipeline

files: dict

Dict of filenames for fits files to use in sextractor. Possible keys are:

• image: filename of the fits image (required)
• dqmask: filename of a bad mixel mask for the given image (optional)
• wtmap: filename of a weight map for the given image (optional)

kwargs: dict

Keyword arguements to pass to atrotoyz.Astromatic.run or astrotoyz.Astromatic.run_sex_frames

frames: list of integers (optional)

Only run sextractor on a specific set of frames. The default value is an empty list, which runs SExtractor without specifying any frames

Returns:

result: dict

Result of the astromatic code execution. This will minimally contain a status key, that indicates success or error. Additional keys: - error_msg: str

If there is an error and the user is storing the output or exporting XML metadata, error_msg will contain the error message generated by the code

• output: str

  If store_output==True the output of the program execution is stored in the output value.

• warnings: str

  If the WRITE_XML parameter is True then a table of warnings detected in the code is returned

astromatic_wrapper.api.run_swarp(pipeline, step_id, filenames, api_kwargs, frames=[])

Run SWARP with a specified set of parameters

Parameters:

pipeline: `astromatic_wrapper.utils.pipeline.Pipeline`

Pipeline containing parameters that may be necessary to set certain AstrOmatic configuration parameters

step_id: str

Unique identifier for the current step in the pipeline

filenames: list

List of filenames that are stacked together

api_kwargs: dict

Keyword arguments used to run SWARP

frames: list (optional)

Subset of frames to stack. Default value is an empty list, which runs SWarp on without specifying any frames

Returns:

result: dict

Result of the astromatic code execution. This will minimally contain a status key, that indicates success or error. Additional keys: - error_msg: str

If there is an error and the user is storing the output or exporting XML metadata, error_msg will contain the error message generated by the code

• output: str

  If store_output==True the output of the program execution is stored in the output value.

• warnings: str

  If the WRITE_XML parameter is True then a table of warnings detected in the code is returned

astromatic_wrapper.ldac

Functions to convert FITS files or astropy Tables to FITS_LDAC files and vice versa.

astromatic_wrapper.utils.ldac.convert_hdu_to_ldac(hdu)

Convert an hdu table to a fits_ldac table (format used by astromatic suite)

Parameters:

hdu: `astropy.io.fits.BinTableHDU` or `astropy.io.fits.TableHDU`

HDUList to convert to fits_ldac HDUList

Returns:

tbl1: astropy.io.fits.BinTableHDU

Header info for fits table (LDAC_IMHEAD)

tbl2: astropy.io.fits.BinTableHDU

Data table (LDAC_OBJECTS)

astromatic_wrapper.utils.ldac.convert_table_to_ldac(tbl)

Convert an astropy table to a fits_ldac

Parameters:

tbl: `astropy.table.Table`

Table to convert to ldac format

Returns

——-

hdulist: `astropy.io.fits.HDUList`

FITS_LDAC hdulist that can be read by astromatic software

astromatic_wrapper.utils.ldac.get_table_from_ldac(filename, frame=1)

Load an astropy table from a fits_ldac by frame (Since the ldac format has column info for odd tables, giving it twce as many tables as a regular fits BinTableHDU, match the frame of a table to its corresponding frame in the ldac file).

Parameters:

filename: str

Name of the file to open

frame: int

Number of the frame in a regular fits file

astromatic_wrapper.utils.ldac.save_table_as_ldac(tbl, filename, **kwargs)

Save a table as a fits LDAC file

Parameters:

tbl: `astropy.table.Table`

Table to save

filename: str

Filename to save table

kwargs:

Keyword arguments to pass to hdulist.writeto

astromatic_wrapper.pipeline

Class and functions to define an astronomy pipeline

class astromatic_wrapper.utils.pipeline.Pipeline(paths={}, pipeline_name=None, next_id=0, create_paths=False, **kwargs)

Bases: object

Parameters:

paths: dict (optional)

Paths used for files generated by the pipeline. Each key will be added to the Pipeline class as the name of a path, and its corresponding value is the path to be used. If create_paths==True, the path will automatically be created on the disk if it does not exist, otherwise the user will be asked whether or not to create the path. At a minimum it is recommended to define a temp_path, used to store temporary files generated by the pipeline and a log_path, used to save any log files created by the pipeline and the pipeline itself after each step.

pipeline_name: str (optional)

Name of the pipeline (used when saving the pipeline). The default value is None, which results in the current date being used for the pipeline name in the form ‘year-month-day_hours-minutes-seconds’.

steps: list of `astromatic.utils.pipeline.PipelineStep` (optional)

If the user already has a list of steps to run they can be set when the pipeline is initialized

next_id: int (optional)

Next number to use for a pipeline step id. The default is 0

create_paths: bool (optional)

If create_paths==True, any path in paths that does not exist is created. Otherwise the user will be prompted if a path does not exist. The default is to prompt the user (create_paths==False).

kwargs: dict

Additional keyword arguments that might be used in a custom pipeline.

add_step(func, tags=[], ignore_errors=False, ignore_exceptions=False, **kwargs)

Add a new PipelineStep to the pipeline

Parameters:

func: function

A function to be run in the pipeline. All functions must return a dictionary with (at a minimum) a status key whose value is either success or error. It is also common to return a warnings key whose value is an astropy table that contains a list of warnings that may have occured during the step. The entire result dictionary returned from the function is saved in the pipeline’s log file.

tags: list (optional)

A list of tags used to identify the step. When running the pipeline the user can specify a set of conditions that will filter which steps are run (or not run) based on a set of specified tags

ignore_errors: bool (optional)

If ignore_errors==False the pipeline will raise an exception if an error occurred during this step in the pipeline (meaning it returned a result with result['status']=='error'). The default is False.

ignore_exceptions: bool (optional)

If ignore_exceptions==True the pipeline will set result['status']=='error' for the step that threw an exception and continue running. The default is ignore_exceptions==False, which will stop the pipeline and raise an exception.

kwargs: dict

Keyword arguments passed to the func when the pipeline is run

get_result_table(key, meta_fields=[])

Get a specific key from the results of each step in a pipeline that has already been run. This function expects the result for the given key to be an astropy Table that can be vstacked with the results from previous steps.

Parameters:

key: str

Name of the key in the result dictionary for each step to extract. For example, key='warnings' will return a table with the warnings generated by each step in the pipeline.

meta_fields: list (optional)

List of metadata fields to include in each record. For example, astromatic_wrapper warning tables have a ‘filename’ metadata field, which gives the name of the XML file generated by the AstrOmatic code.

Returns:

all_results: astropy.table.Table

A table with the results from

run(run_tags=[], ignore_tags=[], run_steps=None, run_name=None, resume=False, ignore_errors=None, ignore_exceptions=None, start_idx=None, current_step_idx=None)

Run the pipeline given a list of PipelineSteps

Parameters:

run_tags: list (optional)

Run all steps that have a tag listed in run_tags and not in ignore_tags. If len(run_tags)==0 then all of the steps are run that are not listed in ignore tags.

ignore_tags: list (optional)

Ignore all steps that contain one of the tags in ignore_tags.

run_steps: list of `PipelineStep` (optional)

Instead of running the steps associated with a pipeline, the user can specify a set of steps to run. This can be useful if (for example) mulitple criteria are used to select steps to run and the user wants to perform these cuts in some other function to generate the necessary steps to run.

run_name: str (optional)

Name of the current run. When a pipeline is run, if a logpath has been specified then a copy of the pipline with a record of all warnings and steps run is saved in the logpath directory. A run_name can be specified to distinguish between different runs of the same pipeline with the same logpath.

resume: bool (optional)

If resume==True and start_idx is None, the pipeline will continue where it left off. If resume==False and start_idx is None then the pipeline will start at the first step (Pipeline.run_step_idx=0). The default is resume=False.

ignore_errors: bool (optional)

If ignore_errors==False the pipeline will raise an exception if an error occurred during any step in the pipeline which returned a result with result['status']=='error'. The default is None, which will use the ignore_errors parameter for each individual step to decide whether or not to throw an exception.

ignore_exceptions: bool (optional)

If ignore_exceptions==True the pipeline will set result['status']=='error' for the step that threw an exception and continue running. The default is None, which will use the ignore_exception parameter for each individual step to decide whether or not to throw an exception.

start_idx: int (optional)

Index of Pipeline.run_steps to begin running the pipeline. All steps in Pipeline.run_steps after start_idx will be run in order. The default value is None, which will not change the current Pipeline.run_step_idx.

exception astromatic_wrapper.utils.pipeline.PipelineError

Bases: exceptions.Exception

Errors generated by running Pipeline

args

message

class astromatic_wrapper.utils.pipeline.PipelineStep(func, step_id, tags=[], ignore_errors=False, ignore_exceptions=False, func_kwargs={})

A single step in the pipeline. This takes a function and a set of tags and kwargs associated with it and stores them in the pipeline.

Initialize a PipelineStep object

Parameters:

func: function

The function to be run. All functions must return a dictionary with at a minimum a status key whose value is either success or error.

id: str

Unique identifier for the step

tags: list (optional)

A list of tags used to identify the step. When running the pipeline the user can specify a set of conditions that will filter which steps are run (or not run) based on a set of specified tags

ignore_errors: bool (optional)

If ignore_errors==False the pipeline will raise an exception if an error occurred during this step in the pipeline, which returned a result with result['status']=='error'. The default is False.

ignore_exceptions: bool (optional)

If ignore_exceptions==True the pipeline will set result['status']=='error' for the step that threw an exception and continue running. The default is ignore_exceptions==False, which will stop the pipeline and raise an exception.

func_kwargs: dict

Keyword arguments passed to the func when the pipeline is run

astromatic_wrapper.utils.pipeline.check_path(path, auto_create=False)

Check if a path exists and if it doesn’t, give the user the option to create it.

Parameters:

path: str

Name of the path to check

auto_create: bool (optional)

If the path does not exist and auto_create==True, the path will automatically be created, otherwise the user will be prompted to create the path

astromatic_wrapper.utils.pipeline.create_paths(paths)

Search for paths on the server. If a path does not exist, create the necessary directories. For example, if paths=['~/Documents/images/2014-6-5_data/'] and only the path ‘~/Documents’ exists, both ‘~/Documents/images/’ and ‘~/Documents/images/2014-6-5_data/’ are created.

Parameters:

paths: str or list of strings

If paths is a string, this is the path to search for and create. If paths is a list, each one is a path to search for and create

astromatic_wrapper.utils.pipeline.get_bool(prompt)

Prompt a user for a boolean expression and repeat until a valid boolean has been entered.

Parameters:

prompt: str

The text to prompt the user with.

astromatic_wrapper.utils.pipeline.str_2_bool(bool_str)

Case independent function to convert a string representation of a boolean ('true'/'false', 'yes'/'no') into a bool. This is case insensitive, and will also accept part of a boolean string ('t'/'f', 'y'/'n').

Raises a astromatic.utils.pipeline.PipelineError if an invalid expression is entered.

Parameters:

bool_str: str

String to convert to a boolean