What is Autosys? • An automated job control system for scheduling,monitoring and reporting jobs • The jobs can reside on an Autosys configured machine attached to a network
What is an Autosys Job? • A single action performed on a validated machine • Autosys jobs can be defined using GUI or JIL • Any single command,executable script or NT batch file.It includes a set of qualifying attributes ,conditions specifying when and where a job should be run.
Autosys jobs can be defined by assigning it a name and specifying attributes describing its behavior.
Two methods to define Autosys Jobs are:- 1. Using Autosys GUI • Autosys GUI allows to set the attributes that describe when,where and how a job should be run. • GUI Control Panel is used to define jobs.Contain fields that correspond to Autosys JIL sub-commands and attributes.
2. Using Job Information Language(JIL) • A specification language that has its own commands to describe when,where and how a job should be run. • The attributes are set b JIL sub-commands
Defining Autosys Jobs There are the three methods you can use to create job definitions:
# By Autosys Web interface
# By AutoSys Graphical User Interface (GUI).
# By using AutoSys Job Information Language (JIL) through a command-line interface. Autosys Job Types and Structure There are three types of jobs: Command, File Watcher Box. ----------------------------------------------------------- System Components Autosys system components are:- • Event Server • Event Processor • Remote Agent Event Server (or Autosys Database) # Data Repository that stores Autosys system information,events and job definitions. # The Autosys Db is termed ‘Data Server’ which describes a server instance
• Event Processor
# Interprets and processes all the events it reads from the Autosys Database
# A program that actually runs Autosys
# Scans the database for processing events. Checks if the events satisfy the starting conditions of the job and the determines the actions
• Remote Agent
# Temporary process started by the event processor to perform a specific task on a remote machine
# It starts the command specified for a given job, sends running and completion information about a task to the Event Server
# If it is unable to transfer the information, it waits and tries until it successfully communicates with the Db
Interaction Between System Components
• Step1: From the Autosys event server(that holds the Events and Job Definitions info),the Event Processor reads the new event,checks for the condition,reads the job definition and determines the actions.
• Step2: The Remote Agent receives the instructions from the Event Processor
• Step3: The Remote Agent performs resource checks,ensuring minimum specified number of processors are available and then initiates a child process that runs the specified command
• Step4: The command completes and exits,with the Remote Agent capturing the command’s exit code.
• Step5: The Remote Agent directly communicates the event(exit code,status) to the Event Server. Autosys Machines V/s Autosys Instances Autosys architecture has two types of machines:-
• Server Machine: # The machine on which the Event Processor and Event Server reside
• Client Machine: # The machine on which the Remote Agent resides and where Autosys jobs are run.
What is an Autosys Instance?
• A version of Autosys software running as an Autosys server,with one or more clients,on single machine or on multiple machines • An instance uses its own Event Server and Event Processor by operating independently of all other Autosys instances • Multiple instances can run and schedule jobs on the same machine without affecting other instances on that machine.
Events • Since Autosys in Event-driven,it requires an event to occur on which the job depends,for a job to be activated by the Event Processor.
• The sources of these events can be:-
# Jobs changing status such as starting,finishing # Internal Autosys verification Agents such as detected errors. # Events send with the SENDEVENT command
• While processing an event,the Event processor scans the database for jobs that are dependent on that event
• If the event satisfies another job’s starting condition,that job is run automatically and completion of that job can cause another job to be started,thereby making the jobs progress in a controlled sequence.
Alarms • Alarms are special events that send notifications during situations requiring attention
• Addresses incidents that require manual intervention
• For example,a set of jobs could be dependent on the arrival of a file and if the file is long overdue.It is important that someone investiage the situation,make a decision and resolve the problem.
• Aspects of alarms include but is not limited to:- # Alarms are informational only.Any action to be taken due to a problem is intiated by a separate action event. # Alarms are system messages about a detected problem # Alarms are sent through the system as an event
Utilities • Autosys provides a set of commands that run essential utility programs for defining,controlling and reporting on jobs .
• For example the autorep command allows generating a variety of reports about job execution,and the sendevent commands allow manually controlling job processing.
• Additional utility programs are provided to assist in troubleshooting running monitors and browsers,and starting/stoping Autosys and its components.
• Autosys also provides database maintenance utility that runs daily by default
Autosys Work-Flow Step1: The Event Processor scans the Event Server for the next event to processor.If no event is ready,the Event Processor scans again in 5 seconds.
• Step2: The Event Processor reads from the Event Server that an event is ready.The job definition and attributes are retrieved from the Event Server,including the command and the pointer to the profile file to be used for the job
• Step3: The Event Processor processes the event.The Event Processor attempts to establish a connection with the Remote Agent on the client machine and passes the job attributes to the client machne.The Event Processor sends a CHANGE_STATUS event marking in the Event Server that the job is in STARTING state
• Step4: The Remote Agent is invoked using the UserID and Password passed from the Event Processor.
• Step5: The Remote Agent receives the job parameters and sends an acknowledgement to the Event Processor
• Step6: The Remote Agent Starts a process and executes the command in the job definition.
• Step7: The Remote Agent issues a CHANGE_STATUS event marking in the Event Server that the job is in RUNNING state
• Step8: The client job process runs to completion,then returns an exit code to the Remote Agent and quits.
• Step9: The Remote Agent sends the Event Server a CHANGE_STATUS event corresponding to the completion status of the job.The Remote Agent quits ============================================================
delete_job: APT#gp#cmd#P0_initialize insert_job: APT#gp#cmd#P0_initialize job_type: c box_name: APT#gp#box#ps_1 command: /export/appl/PSdevl/src/run/pS001_initialize_id.ksh machine: pm_ps_clust owner: PSdevl profile: /appl/PSdevl/.profile permission: gx,ge description: cmd for DIN34APT std_out_file: /export/appl/PSdevl/serial/trans/log/$AUTO_JOB_NAME.log.$AUTORUN std_err_file: /export/appl/PSdevl/serial/trans/err/$AUTO_JOB_NAME.err.$AUTORUN alarm_if_fail: 1 ======================================================= Note: gx -- group exicution ge -- group edit ======================================================= AutoSys Cheatsheet ======================================================= SUB-COMMANDS
insert_job: Saves a brand-new job to the database update_job: PERMANENTLY changes the definition of a pre- existing job override_job: TEMPORARILY changes the definition of a pre- existing job delete_job: Deletes a single job from the database delete_box: Deletes a box as well as all the contents
job_type:command (c), file watcher (f), box (b)(command is default) machine: Name of machine (or IP) where job is to be run command: Command to be executed (.exe, .sh, .bat) watch_file: File being monitored by file watcher box_name: Used to nest a job inside a box std_out_file: Redirects output from a command job to a text file std_err_file: Redirects error messages to a text file condition: Used to structure job dependencies (success, failure, terminated, done, notrunning, exit code, and value) min_run_alarm: Causes job to issue an alarm if it finishes too quickly max_run_alarm: Causes a job to issue an alarm if it runs too long alarm_if_fail: States whether a job will issue an alarm if it fails date_conditions: Toggle which must be set in order for date/time attributes to be recognized by AutoSys run_calendar: Specifies the calendar a job will run off of [cannot be used with days_of_week] days_of_week: Specifies exact days a job will run [cannot be used with run_calendar] start_times: Exact time each day a job will run [cannot be used with start_mins] start_mins: Minutes after each hour a job will execute [cannot be used with start_times] exclude_calendar: Specifies a calendar with days specified upon which a job will not execute watch_interval: Steady state for file watchers watch_file_min_size: Minimum size a file must be before a file watcher can evaluate to success box_success: Specifies custom success condition for a box box_failure: Specifies custom failure condition for a box max_exit_success: Specifies maximum exit code which will be evaluated as a success box_terminator: “If I fail, I kill the box I’m in” job_terminator: “If the box I’m in fails or gets killed, I kill myself” term_run_time: “I kill myself after this many minutes” chk_files: Resource check that verifies a minimum amount of file space is available before running job heartbeat_interval: Specifies frequency in minutes at which job’s command is expected to issue a “heartbeat” profile: Specifies a file which contains custom environment variables to be used by a single job std_in_file: Specifies a file to be used as input for a job n_retrys: Specifies how many times a job should attempt to re-run itself after an application-level failure timezone: Specifies which timezone a job should use when running auto_delete: Specifies how many HOURS after successful completion a job should delete itself from the database auto_hold: Used only with jobs in boxes. When the box goes to a RUNNING state, the job will automatically go ON_HOLD permission: Extend edit or execute permissions to others run_window: Specifies when a job can start avg_runtime: *Only accessible through JIL* Specifies how long a job takes to run, on average
This Blog is a collection of topics related to Data Modeling, DBMS & Normalization, Database, Data Warehouse (ODS, Data Marts, EDW ), CRM, ERP , UNIX and Other applications.
Note: Some of the articles are grab from various websites/Blogs.