DataStage Job Execution: The Orchestra.....

Datastage Job Execution: The Orchestra
========================================

# conductor - Initial process
- Composes the Score
- Creates Section Leader process (one/node)
- Consolidates messages to DataStage log.
- Manages orderly shutdown

# Section Leader (one per Node)
- Forks Player process (one per stage)
- Manages up/down communication

# Players - The actual processes associated with stages
- Combined players: one process only
- Sends stderr, stdout to Section Leader
- Establish connections to other players for data flow.
- Clean up upon completion.

The Conductor nod e has the start-up process. Creates the score. Starts up section leaders.
Section leaders communicate with the conductor only. The Conductor communicates with the players.

================================================================
Runtime Control and Data Networks:
================================================================
Every player has to be able to communicate with every other player. There are separate communication channels(pathways) for control, messages, errors, and data. Note that the data channel does not go through the section leader/conductor, as this would limit scalability. Data flows directly from upstream operator to downstream operator using APT_Communicator class.