UNIX Commands

ls ------ list files and directories
ls -a ------ list all files and directories
mkdir ------ make a directory
cd directory ------ change to named directory
cd ------ change to home-directory
cd ~ ------ change to home-directory
cd .. ------ change to parent directory
pwd ------ display the path of the current directory
cp file1 file2 ------ copy file1 and call it file2
mv file1 file2 ------ move or rename file1 to file2
rm file ------ remove a file
rmdir directory ------ remove a directory
cat file ------ display a file
less file ------ display a file a page at a time
head file ------ display the first few lines of a file
tail file ------ display the last few lines of a file
grep 'keyword' file ------ search a file for keywords
wc file ------ count number of lines/words/characters in file
============================================================

cat
This is one of the most flexible Unix commands. We can use to create, view and concatenate files. For our first example we create a three-item English-Spanish dictionary in a file called "dict."

% cat >dict
red rojo
green verde
blue azul

%

stands for "hold the control key down, then tap 'd'". The symbol > tells the computer that what is typed is to be put into the file dict. To view a file we use cat in a different way:

% cat dict
red rojo
green verde
blue azul
%

If we wish to add text to an existing file we do this:
% cat >>dict
white blanco
black negro

%

Now suppose that we have another file tmp that looks like this:

% cat tmp
cat gato
dog perro
%

Then we can join dict and tmp like this:
% cat dict tmp >dict2

We could check the number of lines in the new file like this:

% wc -l dict2
8

The command wc counts things --- the number of characters, words, and line in a file.


--------------------------------------------------------------------------------

chmod
This command is used to change the permissions of a file or directory. For example to make a file essay.001 readable by everyone, we do this:

% chmod a+r essay.001

To make a file, e.g., a shell script mycommand executable, we do this

% chmod +x mycommand

Now we can run mycommand as a command.
To check the permissions of a file, use ls -l . For more information on chmod, use man chmod.


--------------------------------------------------------------------------------

cd
Use cd to change directory. Use pwd to see what directory you are in.

% cd english
% pwd
% /u/ma/jeremy/english
% ls
novel poems
% cd novel
% pwd
% /u/ma/jeremy/english/novel
% ls
ch1 ch2 ch3 journal scrapbook
% cd ..
% pwd
% /u/ma/jeremy/english
% cd poems
% cd
% /u/ma/jeremy

Jeremy began in his home directory, then went to his english subdirectory. He listed this directory using ls , found that it contained two entries, both of which happen to be diretories. He cd'd to the diretory novel, and found that he had gotten only as far as chapter 3 in his writing. Then he used cd .. to jump back one level. If had wanted to jump back one level, then go to poems he could have said cd ../poems. Finally he used cd with no argument to jump back to his home directory.


--------------------------------------------------------------------------------

cp
Use cp to copy files or directories.
% cp foo foo.2

This makes a copy of the file foo.
% cp ~/poems/jabber .

This copies the file jabber in the directory poems to the current directory. The symbol "." stands for the current directory. The symbol "~" stands for the home directory.


--------------------------------------------------------------------------------

date
Use this command to check the date and time.
% date
Fri Jan 6 08:52:42 MST 1995


--------------------------------------------------------------------------------

echo
The echo command echoes its arguments. Here are some examples:

% echo this
this
% echo $EDITOR
/usr/local/bin/emacs
% echo $PRINTER
b129lab1

Things like PRINTER are so-called environment variables. This one stores the name of the default printer --- the one that print jobs will go to unless you take some action to change things. The dollar sign before an environment variable is needed to get the value in the variable. Try the following to verify this:

% echo PRINTER
PRINTER



--------------------------------------------------------------------------------

ftp
Use ftp to connect to a remote machine, then upload or download files. See also: ncftp

Example 1: We'll connect to the machine fubar.net, then change director to mystuff, then download the file homework11:

% ftp solitude
Connected to fubar.net.
220 fubar.net FTP server (Version wu-2.4(11) Mon Apr 18 17:26:33 MDT 1994) ready.
Name (solitude:carlson): jeremy
331 Password required for jeremy.
Password:
230 User jeremy logged in.
ftp> cd mystuff
250 CWD command successful.
ftp> get homework11
ftp> quit

Example 2: We'll connect to the machine fubar.net, then change director to mystuff, then upload the file collected-letters:

% ftp solitude
Connected to fubar.net.
220 fubar.net FTP server (Version wu-2.4(11) Mon Apr 18 17:26:33 MDT 1994) ready.
Name (solitude:carlson): jeremy
331 Password required for jeremy.
Password:
230 User jeremy logged in.
ftp> cd mystuff
250 CWD command successful.
ftp> put collected-letters
ftp> quit

The ftp program sends files in ascii (text) format unless you specify binary mode:

ftp> binary
ftp> put foo
ftp> ascii
ftp> get bar

The file foo was transferred in binary mode, the file bar was transferred in ascii mode.


--------------------------------------------------------------------------------

grep
Use this command to search for information in a file or files. For example, suppose that we have a file dict whose contents are

red rojo
green verde
blue azul
white blanco
black negro

Then we can look up items in our file like this;
% grep red dict
red rojo
% grep blanco dict
white blanco
% grep brown dict
%

Notice that no output was returned by grep brown. This is because "brown" is not in our dictionary file.

Grep can also be combined with other commands. For example, if one had a file of phone numbers named "ph", one entry per line, then the following command would give an alphabetical list of all persons whose name contains the string "Fred".

% grep Fred ph | sort
Alpha, Fred: 333-6565
Beta, Freddie: 656-0099
Frederickson, Molly: 444-0981
Gamma, Fred-George: 111-7676
Zeta, Frederick: 431-0987

The symbol "|" is called "pipe." It pipes the output of the grep command into the input of the sort command.
For more information on grep, consult

% man grep


--------------------------------------------------------------------------------

head
Use this command to look at the head of a file. For example,

% head essay.001

displays the first 10 lines of the file essay.001 To see a specific number of lines, do this:

% head -n 20 essay.001

This displays the first 20 lines of the file.
--------------------------------------------------------------------------------

ls
Use ls to see what files you have. Your files are kept in something called a directory.

% ls
foo letter2
foobar letter3
letter1 maple-assignment1
%

Note that you have six files. There are some useful variants of the ls command:

% ls l*
letter1 letter2 letter3
%

Note what happened: all the files whose name begins with "l" are listed. The asterisk (*) is the " wildcard" character. It matches any string.


--------------------------------------------------------------------------------

lpr
This is the standard Unix command for printing a file. It stands for the ancient "line printer." See

% man lpr

for information on how it works. See print for information on our local intelligent print command.


--------------------------------------------------------------------------------

mkdir
Use this command to create a directory.
% mkdir essays

To get "into" this directory, do
% cd essays

To see what files are in essays, do this:
% ls

There shouldn't be any files there yet, since you just made it. To create files, see cat or emacs.


--------------------------------------------------------------------------------

more
More is a command used to read text files. For example, we could do this:

% more poems

The effect of this to let you read the file "poems ". It probably will not fit in one screen, so you need to know how to "turn pages". Here are the basic commands:

q --- quit more
spacebar --- read next page
return key --- read next line
b --- go back one page
For still more information, use the command man more.



--------------------------------------------------------------------------------

mv
Use this command to change the name of file and directories.

% mv foo foobar

The file that was named foo is now named foobar



--------------------------------------------------------------------------------

ncftp
Use ncftp for anonymous ftp --- that means you don't have to have a password.

% ncftp ftp.fubar.net
Connected to ftp.fubar.net
> get jokes.txt

The file jokes.txt is downloaded from the machine ftp.fubar.net.



--------------------------------------------------------------------------------

print
This is a moderately intelligent print command.
% print foo
% print notes.ps
% print manuscript.dvi

In each case print does the right thing, regardless of whether the file is a text file (like foo ), a postcript file (like notes.ps, or a dvi file (like manuscript.dvi. In these examples the file is printed on the default printer. To see what this is, do

% print

and read the message displayed. To print on a specific printer, do this:
% print foo jwb321
% print notes.ps jwb321
% print manuscript.dvi jwb321

To change the default printer, do this:
% setenv PRINTER jwb321


--------------------------------------------------------------------------------

pwd
Use this command to find out what directory you are working in.
% pwd
/u/ma/jeremy
% cd homework
% pwd
/u/ma/jeremy/homework
% ls
assign-1 assign-2 assign-3
% cd
% pwd
/u/ma/jeremy
%

Jeremy began by working in his "home" directory. Then he cd 'd into his homework subdirectory. Cd means " change directory". He used pwd to check to make sure he was in the right place, then used ls to see if all his homework files were there. (They were). Then he cd'd back to his home directory.


--------------------------------------------------------------------------------

rm
Use rm to remove files from your directory.
% rm foo
remove foo? y
% rm letter*
remove letter1? y
remove letter2? y
remove letter3? n
%

The first command removed a single file. The second command was intended to remove all files beginning with the string "letter." However, our user (Jeremy?) decided not to remove letter3.


--------------------------------------------------------------------------------

rmdir
Use this command to remove a directory. For example, to remove a directory called "essays", do this:

% rmdir essays

A directory must be empty before it can be removed. To empty a directory, use rm.


--------------------------------------------------------------------------------

rsh
Use this command if you want to work on a computer different from the one you are currently working on. One reason to do this is that the remote machine might be faster. For example, the command

% rsh solitude

connects you to the machine solitude. This is one of our public workstations and is fairly fast.

See also: telnet


--------------------------------------------------------------------------------

setenv
% echo $PRINTER
labprinter
% setenv PRINTER myprinter
% echo $PRINTER
myprinter


--------------------------------------------------------------------------------

sort
Use this commmand to sort a file. For example, suppose we have a file dict with contents
red rojo
green verde
blue azul
white blanco
black negro

Then we can do this:
% sort dict
black negro
blue azul
green verde
red rojo
white blanco

Here the output of sort went to the screen. To store the output in file we do this:
% sort dict >dict.sorted

You can check the contents of the file dict.sorted using cat , more , or emacs .
--------------------------------------------------------------------------------

tail
Use this command to look at the tail of a file. For example,

% tail essay.001

displays the last 10 lines of the file essay.001 To see a specific number of lines, do this:

% tail -n 20 essay.001

This displays the last 20 lines of the file.
--------------------------------------------------------------------------------

tar
Use create compressed archives of directories and files, and also to extract directories and files from an archive. Example:

% tar -tvzf foo.tar.gz

displays the file names in the compressed archive foo.tar.gz while

% tar -xvzf foo.tar.gz

extracts the files.
--------------------------------------------------------------------------------

telnet
Use this command to log in to another machine from the machine you are currently working on. For example, to log in to the machine "solitude", do this:

% telnet solitude

See also: rsh.


--------------------------------------------------------------------------------

wc
Use this command to count the number of characters, words, and lines in a file. Suppose, for example, that we have a file dict with contents

red rojo
green verde
blue azul
white blanco
black negro

Then we can do this
% wc dict
5 10 56 tmp

This shows that dict has 5 lines, 10 words, and 56 characters.

The word count command has several options, as illustrated below:

% wc -l dict
5 tmp
% wc -w dict
10 tmp
% wc -c dict
56 tmp
===============================================================

Access Control

exit - terminate a shell (see "man sh" or "man csh")
logout - sign off; end session (C shell and bash shell only; no man page)
passwd - change login password
rlogin - log in remotely to another UNIX system
slogin - secure version of rlogin

Communications

from - list senders of mail
Mail - send and receive mail
mesg - permit or deny terminal messages and talk requests
pine - send and receive mail (easiest for new users)
talk - talk to another logged-in user (full screen)
write - write to another logged-in user

Programming Tools

ar - archive and library maintainer
as - assembler, specific to each machine architecture
awk - pattern scanning and processing language
/bin/time - time a command
cc - C compiler
crontab - maintain periodic tasks
csh - C shell command interpreter
dbx - source-level debugging program
gdb - GNU Project debugger
gprof - display profile of called routines
kill - kill a process
ld - the UNIX loader
lex - generate lexical analysis programs
lint - check C source code
make - maintain large programs
maple - symbolic mathematics program
nice - run a command at low priority (see "man nice" or "man csh")
nohup - run a command immune to hangups
pc - Pascal compiler (xlp on ADS)
perl - Popular script interpreter
prof - display profile data
ranlib - convert archives to random libraries
sh - Bourne shell command interpreter
yacc - generate input parsing programs

Documentation

apropos - locate commands by keyword lookup
info - start the InfoExplorer program
man - find manual information about commands
/usr/bin/X11/dxbook - start the BookReader program
whatis - describe what a command is
whereis - locate source, binary, or man page for a program

Editors

ed - line-oriented text editor
emacs - screen-oriented text editor
ex - line-oriented text editor
pico - simple, screen-oriented text editor (easiest for new users)
sed - stream-oriented text editor
vi - full-screen text editor
vim - full-screen text editor ("vi-improved")

File and Directory Management

cd - change working directory
chmod - change the protection of a file or directory
cmp - compare two files
comm - select/reject lines common to two sorted files
compress - compress a file
cp - copy files
crypt - encrypt/decrypt files
diff - compare the contents of two ASCII files
file - determine file type
grep - search a file for a pattern
ln - make a link to a file
ls - list the contents of a directory
mkdir - create a directory
mv - move or rename files and directories
pwd - show the full pathname of your working directory
quota - display disk usage and limits
rm - delete (remove) files
rmdir - delete (remove) directories
sort - sort or merge files
tee - copy input to standard output and other files
tr - translate characters
umask - change default file protections
uncompress - restore compressed file
uniq - report (or delete) repeated lines in a file
wc - count lines, words, and characters in a file

File Display and Printing

cat - show the contents of a file; catenate files
fold - fold long lines to fit output device
head - show first few lines of a file
lpq - examine the printer spooling queue
lpr - print a file
lprm - remove jobs from the printer spooling queue
more - display a file, one screen at a time
page - like "more", but prints screens top to bottom
pr - paginate a file for printing
tail - show the last part of a file
zcat - display a compressed file

File Transfer

ftp - transfer files between network hosts
rcp - transfer files between networked UNIX hosts
scp - secure version of rcp
rz - receive files using ZMODEM protocol
sz - send files using ZMODEM protocol

Miscellaneous

alias - define synonym commands
chsh - change default login shell
clear - clear terminal screen
echo - echo arguments
pbm - portable bitmap manipulation programs
popd - pop the directory stack (C shell only)
pushd - push directory on stack (C shell only)
script - make typescript of terminal session
setenv - set an environment variable (C shell only)
stty - set terminal options

News/Networks

netstat - show network status
Pnews - submit USENET news articles
rlogin - login remotely on another UNIX system
slogin - secure version of rlogin
tin - read/post USENET news articles
trn - read/post USENET news articles using threaded news reader
rsh - run shell or command on another UNIX system
ssh - secure-shell version of rsh
telnet - run Telnet to log in to remote host

Process Control

(The following commands function under the C shell, bash, and ksh. They do not have separate man pages.)

bg - put suspended process into background
fg - bring process into foreground
jobs - list processes
^y - suspend process at next input request
^z - suspend current process

Status Information

date - show date and time
df - summarize free disk space
du - summarize disk space used
env - display environment
finger - look up user information
history - list previously issued commands (C shell, bash, and ksh
only)
last - indicate last login of users
lpq - examine spool queue
manpath - show search path for man pages
printenv - print out environment
ps - show process status
pwd - print full pathname of working directory
set - set shell variables (C shell, bash, and ksh only)
stty - set terminal options
uptime - show system load, how long system has been up
w - show who is on system, what command each job is executing
who - show who is logged onto the system
whois - Internet user name directory service

Text Processing

addbib - create or extend bibliographic database
checknr - check nroff/troff files
col - filter reverse line feeds
diction - identify wordy sentences
diffmk - mark differences between files
dvips - convert TeX DVI files into PostScript
eqn - typeset mathematics with troff
explain - explain phrases found by diction program
grap - pic preprocessor for drawing graphs
hyphen - find hyphenated words
indxbib - build inverted index for a bibliography, find references
ispell - check spelling interactively
latex - format text
lookbib - find bibliography references
macref - make cross-reference listing of nroff/troff macro files
ndx - create a subject-page index for a document
neqn - format mathematics with nroff
nroff - format text for simple display
pic - make simple pictures for troff input
psdit - filter troff output for Apple LaserWriter
ptx - make permuted index (not on CCWF)
refer - insert references from bibliographic databases
roffbib - run off bibliographic database
sortbib - sort bibliographic database
spell - find spelling errors
style - analyze surface characteristics of a document
tbl - format tables for nroff/troff
tex - format text
tpic - convert pic source files into TeX commands
troff - format text
=================================================

Shell Variables

Shell variables are used for storing and manipulating strings of characters. A shell variable name begins with a letter and can contain letters, digits, and underscores, such as
x
x1
abc_xyz

Shell variables can be assigned values like this:
x=file1
x1=/usr/man/man1/sh.1
abc_xyz=4759300

Notice that there are no spaces before or after the equals-sign. The value will be substituted for the shell variable name if the name is preceded by a $. For example,
echo $x1

would echo
/usr/man/man1/sh.1

Several special shell variables are predefined. Some useful ones are
$#, $*, $?, and $$.

Arguments can be passed to a shell script. These arguments can be accessed inside the script by using the shell variables $1, $2,...,$n for positional parameter 1,2,...,n. The filename of the shell script itself is $0. The number of such arguments is $#. For example, if file do_it is a shell script and it is called by giving the command
do_it xyz

then $0 has the value do_it, $1 has the value xyz, and $# has the value 1.
$* is a variable containing all the arguments (except for $0) and is often used for passing all the arguments to another program or script.

$? is the exit status of the program most recently executed in the shell script. Its value is 0 for successful completion. This variable is useful for error handling (see section 10.6).

$$ is the process id of the executing shell and is useful for creating unique filenames. For example,

cat $1 $2 >> tempfile.$$

concatenates the files passed as parameters 1 and 2, appending them to a file called tempfile.31264 (assuming the process id is 31264).
Flow Control
Most programming languages provide constructs for looping and for testing conditions to know when to stop looping. The shell provides several such flow control constructs, including if, for, and while.
if

The if command performs a conditional branch. It takes the form

if command-list1
then
command-list2
else
command-list3
fi

A command-list is one or more commands. You can put more than one command on a line, but if you do so, separate them by semicolons. If the last command of command-list1 has exit status 0, then command-list2 is executed. But if the exit status is nonzero, then command-list3 is executed.
for

The for command provides a looping construct of the form

for shell-variable in word-list
do command-list
done

The shell variable is set to the first word in word-list and then command-list is executed. The shell variable is then set to the next word in word-list and the process continues until word-list is exhausted. A common use of for-loops is to perform several commands on all (or a subset) of the files in your directory. For example, to print all the files in your directory, you could use
for i in *
do echo printing file $i
lpr $i
done

In this case, * would expand to a list of all filenames in your directory, i would be set to each filename in turn, and $i would then substitute the filename for i (in the echo and lpr commands).
while

The while command provides a slightly different looping construct:

while command-list1
do command-list2
done

While the exit status of the last command in command-list1 is 0, command-list2 is executed.
Test
The test command can be used to compare two integers, to test if a file exists or is readable, to determine if two strings are equal, or to test several other conditions. For example, to test whether the value of shell variable x is equal to 5, use
test $x -eq 5

If $x is equal to 5, test returns true.
Other useful tests include

test -s file (true if file exists and has a size larger than 0)
test -w file (true if file exists and is writable)
test -z string (true if the length of string is 0)
test string1 != string2
(true if string1 and string2 are not identical)

The test command is often used with the flow-control constructs described above. Here is an example of test used with the if command:
if test "$1" = "" (or if ["$1" = ""] )
then
echo usage: myname xxxx
exit 1
fi

This tests to see if the command line contains an argument ($1). If it does not ($1 is null), then echo prints a message.
A complete list of test operators can be found in the man page for test.

Error Handling

Each time a program is executed from within a shell script, a value is returned to indicate whether the program ran successfully or not. In most cases, a value of zero is returned on successful execution, and a nonzero number is returned if the program encountered an error. This exit status is available in the shell variable $?.
For example,

grep $1 phonelist
if test $? -ne 0
then
echo I have no phone number for $1
fi

will run a program (grep) and examine the exit status to determine if the program ran properly.
Traps
Some signals cause shell scripts to terminate. The most common one is the interrupt signal ^c typed while a script is running. Sometimes a shell script will need to do some cleanup, such as deleting temporary files, before exiting. The trap command can be used either to ignore signals or to catch them to perform special processing. For example, to delete all files called ``tmp.*'' before quitting when an interrupt signal is generated, use the command
trap 'rm tmp.*; exit' 2

The interrupt signal is signal 2, and two commands will be executed when an interrupt is received (rm tmp.* and exit). You can make a shell script continue to run after logout by having it ignore the hangup signal (signal 1). The command
trap ' ' 1

allows shell procedures to continue after a hangup (logout) signal.
Command Substitution
A useful capability in shell programming is to assign the output from a program to a shell variable or use it as a pattern in another command. This is done by enclosing the program call between accent grave (`) characters. For example, the command
where=`pwd`

will assign the string describing the current working directory (the results of the pwd command) to the shell variable where. Here is a more complicated example:
for i in `ls -t *.f`
do f77 $i
a.out >output
cat $i output | lpr -P$1
rm a.out output
done

In this case, the shell script executes a series of commands for each file that ends with ``.f'' (all Fortran programs). The `ls -t *.f` is executed and expands into all filenames ending with ``.f'', sorted by time, most recent to oldest. Each is compiled and executed. Then the source file and output file are sent to the printer identified by the first argument ($1) passed to the shell script. Then these files are deleted.
I/O Redirection
Besides the I/O redirection already described, there are a few additional forms of redirection. Under the Bourne shell, standard input is also associated with file descriptor 0, standard output with file descriptor 1, and standard error output with file descriptor 2. So both >file and 1>file redirect standard output to file, and 2>file redirects standard error output to file.
To merge standard output (file descriptor 1) and standard error output (file descriptor 2), then redirect them to another file, use this notation:

command >file 2>&1

Another method of redirecting input in shell scripts allows a command to read its input from the shell script itself without using temporary files. For instance, to run the editor ed to change all x's in a file to z's, you could create a temporary file of ed commands, then read it to perform those commands, and finally delete it, like this:
echo "1,$s/x/z/g" >edtmp.$$
echo "w" >>edtmp.$$
echo "q" >>edtmp.$$
ed filename rm edtmp.$$
echo "x's changed to z's"

The same thing can be accomplished without a temporary file by using the << symbol and a unique string, like this:
ed filename <<%
1,$s/x/z/g
w
q
%
echo "x's changed to z's"

The << symbol redirects the standard input of the command to be right here in the shell script, beginning from the next line and continuing up to the line that matches the string following the << (in this case %). The terminating string must be on a line by itself. The string is arbitrary: for example, <Debugging Shell Scripts
Two methods of tracing shell script execution are useful for debugging. The script may be called with a verbose flag (-v) or execution trace flag (-x). The -v flag causes each line of the shell script to be echoed as it is read but after all substitutions are performed. The -x flag causes each line to be echoed just before it is executed. For example:
sh -v do_it
sh -x do_it

To turn on both flags, use -vx.

Netezza Enterprise Stage in DataStage

Ascential PACK for SAP R/3

ABAP:- Advanced Business Application Programming. The
language developed by SAP for application development
purposes. All R/3 applications are written in ABAP

BAPI:- Business Application Programming Interface. A precisely
defined interface providing access to processes and data
in business application systems. BAPIs are defined as
API methods of SAP objects. These objects and their
methods are stored in the Business Objects Repository.

BOR:- Business Object Repository, which is the object-oriented
Repository in the SAP BW system. It contains, among
other objects, SAP Business Objects and their methods.

Business Object:-The representation of a business entity, such as an
employee or a sales order, in the SAP Enterprise system.

Control record:-A special administrative record within an IDoc, one for each IDoc. The control record contains a standard set of
fields that describe the IDoc as a whole.

ERP:- Enterprise Resource Planning business management
software.

IDoc:- Intermediate document. An IDoc is a report, that is, a
hierarchal package of related records, generated by R/3
Enterprise in an SAP proprietary format. An IDoc, whose
transmission is initiated by the source database,
exchanges data between applications.

IDoc type:-Named meta data describing the structure of an IDoc that
is shared across databases. It consists of a hierarchy of
segment record types.

MATMAS01:- An example of an IDoc type.

PACK:- Packaged Application Connection Kit. Accesses and
extracts data from and loads data to R/3 Enterprise.

PSA:- Persistent Staging Area.

R/3:- Real time/three tiers.

RFC:- Remote Function Call. The SAP implementation of RPC
(Remote Procedure Call) in ABAP. It calls a function
module that runs on a different system from the calling
function. The Remote Function Call can also be called
from within the same system, but usually the caller and
callee are dispersed.

RFM:- Remote Function Module. A function that belongs to a
BOR object type and has a BAPI method name.

SAP:- Systems, Applications, and Products in Data Processing.
SAP is a product of SAP AG, Walldorf, Germany.

SCM:- Supply Chain Management. The solution that tracks
financial, informational, and materials processes and
identifies processing exceptions.

Segment:- A record within an IDoc that is identified by a segment
number.

Segment type:-A named record definition for segments within an IDoc
that is one level in the hierarchy of segment types within
an IDoc type.

tRFC:- port Transactional RFC port.

Variant:- A collection of predefined criteria, similar to a group of
values used as parameters. Variants are attached to various processes used by
DataStage for the ABAP program process, for example. The ABAP Program referenced by the ABAP Program process itself has a variant attached to it.)

000-418 DataStage Test

Question # 1
You have a cluster of nodes available to run DataStage jobs. The network configuration between the servers is a private network with a 1 GB connection between each node. The public name is on a 100 MB network, which is what each hostname is identified with. In order to use the private network for communications between each
node you need to use an alias for each node in the cluster. The Information Server Engine
node (conductor node) is where the DataStage job starts.
Which environment variable must be used to identify the hostname for the Engine node?
A. $APT_SERVER_ENGINE
B. $APT_ENGINE_NODE
C. $APT_PM_CONDUCTOR_HOSTNAME
D. $APT_PM_NETWORK_NAME

Question # 2
When reading a delimited Sequential File, you are instructed to interpret two contiguous field delimiters as NULL for the corresponding field regardless of data type.
Which three actions must you take? (Choose three.)
A. Set the data type to Varchar.
B. Set the field to nullable.
C. Set the "NULL Field Value" to two field delimiters (e.g., "||" for pipes).
D. Set the "NULL Field Value" to ''.
E. Set the environment variable $APT_IMPEXP_ALLOW_ZERO_LENGTH_FIXED_NULL.

Question # 3
Which two attributes are found in a Data Set descriptor file? (Choose two.)
A. A copy of the job score.
B. The schema of the Data Set.
C. A copy of the partitioned data.
D. A copy of the configuration file used when Data Set was created.

Question # 4
Test A parallel job reads from a DB2 table containing 15 million records, and uses a Sparse Lookup against an Oracle table containing 30 million records. Both tables are indexed on their join key columns. The Oracle reference table is larger than available
shared memory. Only those records whose key values have matching reference data are sent to the target DB2 table.
Which redesign change would improve performance while still meeting functional requirements?
A. Replace the Sparse Lookup with custom SQL to join the two source tables.
B. Change the Sparse Lookup to a Normal Lookup.
C. Replace the Sparse Lookup with an Inner Join stage.
D. Replace the Sparse Lookup and Oracle read with custom SQL and a DB2
stored procedure to access the remote Oracle table.

Question # 5
A DataStage job contains a parallel Transformer with a single input link and a single output link. The Transformer has a constraint that should produce 1000 records,
however only 900 came out through the output link.
What should be done to identify the missing records?
A. Turn trace on using DataStage Administrator.
B. Add a Reject link to the Transformer stage.
C. Scan generated osh script for possible errors.
D. Remove the constraint on the output link.

Question # 6
You are assigned to write a job which reads a DB2 database, applies business logic, and writes the results to an Oracle database. However, the table name and layout may change from one run to another. You are guaranteed that a core set of columns required to perform the business logic will always be present, though not necessarily in
the same order. Which three features would you use to build this job? (Choose three.)
A. Table read method
B. Partition Table option
C. Load method
D. Partition Column option
E. Runtime Column Propagation

Question # 7
You are using the Change Capture stage in your job design to identify changes made to the input link (before image) to obtain the results in the output link. This job will be using a multi-node configuration file when executed. The Change Capture stage produces an output change link with which two characteristics? (Choose two.)
A. Table Definition based on the keys of the input link
B. Table Definition based on the after input link
C. added column named Change_Code
D. added column named Difference_Value

Question # 8
What are three characteristics of containers? (Choose three.)
A. Containers are used to execute jobs.
B. Containers are a group of stages and links.
C. Containers allow you to simplify complex job designs.
D. Containers allow you to provide pre-configured job design logic.
E. Containers allow multiple users to access a single job.

Question # 9
In the Repository Advanced Find interface, which three properties can be used to constrain the results of impact analysis? (Choose three.)
A. Type
B. Export category
C. Owner name
D. Dependencies of
E. Creation

Question # 10
You are building a job which uses Runtime Column Propagation throughout. However, at run-time, the job aborts with an error message indicating that one of the key columns for a sort is not present on the record when it arrives at the sort.
What would help debug the run-time environment?
A. $OSH_PRINT_DROPPED_COLUMNS
B. $OSH_PRINT_SCHEMAS
C. $OSH_SHOW_COMPONENT_CALLS
D. $OSH_SHOW_STARTUP_STATUS

Update Action in DataStage

Specifies which SQL statements are used to update the target table. Some update actions require key columns to update or delete rows. There is no default. Choose the option you want from the list:

Clear table then insert rows. Deletes the contents of the table and adds the new rows, with slower performance because of transaction logging. This is the default value.

Truncate table then insert rows. Truncates the table with no transaction logging and faster performance. For DB2/UDB and Informix, this option is the same as Clear table then insert rows.

Insert rows without clearing. Inserts the new rows in the table.

Delete existing rows only. Deletes existing rows in the target table that have identical keys in the source files.

Replace existing rows completely. Deletes the existing rows, then adds the new rows to the table.

Update existing rows only. Updates the existing data rows. Any rows in the data that do not exist in the table are ignored.

Update existing rows or insert new rows. Updates the existing data rows before adding new rows. It is faster to update first when you have a large number of records.

Insert new rows or update existing rows. Inserts the new rows before updating existing rows. It is faster to insert first if you have only a few records.

Insert new rows only. Inserts the new rows in the table but does not report duplicate rows to the log.

Truncate only. Ignores any incoming data and truncates the target table.

Important. A stage cannot stand alone on the canvas. Configure a dummy source stage. To keep the process simple and to preserve the resource, make sure the dummy source stage does not have any source data.

User-defined SQL. Writes the data using a user-defined SQL statement, which overrides the default SQL statement generated by the stage. If you choose this option, you enter the SQL statement on the SQL tab.

User-defined SQL file. Reads the content of the specified file to write the data.

DataStage Job Sequences

DataStage provides a graphical Job Sequencer which allows you to specify a sequence of server or parallel jobs to run. The sequence can also contain control information, for example, you can specify different courses of action to take depending on whether a job in the sequence succeeds or fails. Once you have defined a job sequence, it can be scheduled and run using the DataStage Director. It appears in the DataStage Repository and in the DataStage Director client as a job.

Ascential DataStage Parallel Transformer Stage Programming Guide

date Field Functions
Orchestrate performs no automatic type conversion of date fields. Either an input data set must match the operator interface or you must effect a type conversion by means of the transform or modify operator.

21 32 Orchestrate 7.5 Operators Reference date format The default format of the date contained in the string is yyyy-mm-dd. However, you can specify an optional format string that defines another format. The format string requires that you provide enough information for Orchestrate to determine a complete date (either day, month, and year, or year and day of year). The date components of a source string (date, month, and year) must be zero-padded to the character length specified by the format string. Orchestrate zero pads the date components of a destination string to the specified length. The possible format components are:

%dd: A two digit day.
%mm: A two digit month.
%yy: A two digit year derived from a year cutoff of 1900.
%year_cutoffyy: A two digit year derived from yy and the specified year cutoff. (For example, if you specify the year format as %2000yy, two-digit values which represent years from 2000-2099 are imported and exported.)
%yyyy: A four digit year.
%ddd: Day of year in three digit form (range of 1 - 366) The default date format is as follows:
%yyyy-%mm-%dd When you specify a date format string, prefix each component with the percent symbol (%). Separate the string’s components with any character except the percent sign (%).

For example, the format string %mm/%dd/%yyyy specifies that slashes separate the string’s date components; the format %ddd-%yy specifies that the string stores the date as a value from 1 to 366, derives the year from the current year cutoff of 1900, and separates the two components with a dash (-).

Function Description

date date_from_days_since
(int32, “date�? | format_variable) Returns date by adding the given integer to the baseline date. Converts an integer field into a date by adding the integer to the specified base date. The date must be in the format yyyy-mm-dd and must be either double quoted or a variable.

date date_from_julian_day(uint32) Returns the date given a Julian day.
date date_from_string
(string, "date_format" | date_uformat | format_variable) Returns a date from the given string formatted in the optional format specification. By default the string format is yyyy-mm-dd.

date date_from_ustring
(string, "date_format" | date_uformat | format_variable) Returns a date from the given ustring formatted in the optional format specification. By default the ustring format is yyyy-mm-dd.

string string_from_date
(date, "date_format" | date_uformat) Converts the date to a string representation using the given format specification. By default the ustring format is yyyy-mm-dd.

ustring ustring_from_date
(date, "date_format" | date_uformat) Converts the date to a ustring representation using the given format specification. By default the ustring format is yyyy-mm-dd.

date date_from_timestamp(timestamp) Returns the date from the given timestamp.
int32 days_since_from_date
(date, “source_date�? | format_variable) Returns a value corresponding to the number of days from source_date to date. source_date must be in the form yyyy-mm-dd and must be double quoted or be a variable.

uint32 julian_day_from_date(date) Returns a Julian date given the date.
int8 month_day_from_date(date) Returns the day of the month given the date. For
example, the date 07-23-2001 returns 23.

int8 month_from_date(date) Returns the month from the given date. For example,
the date 07-23-2001 returns 7.

date next_weekday_from_date
(date, “day�? | format_variable) The value returned is the date of the specified day of the week soonest after date (including the date). The day argument is optional. It is a string or variable specifying a day of the week. You can specify day by either the first three characters of the day name or the full day name. By default, the value is Sunday.

date previous_weekday_from_date
(date, "day" | format_variable) Returns the previous weekday date from date. The destination contains the closest date for the specified day of the week earlier than the source date (including the source date) The day argument is optional. It is a string or variable specifying a day of the week. You can specify day using either the first three characters of the day name or the full day name. By default, the value is Sunday.

int8 weekday_from_date
(date, "origin_day" | format_variable) Returns the day of the week from date. The optional argument origin_day is a string or variable specifying the day considered to be day zero of the week. You can specify the day using either the first three characters of the day name or the full day name. If omitted, Sunday is day zero.

int16 year_day_from_date(date) Returns the day of the year (1-366) from date.
int16 year_from_date(date) Returns the year from date. For example, the date 07-
23-2001 returns 2001.

int8 year_week_from_date(date) Returns the week of the year from date. For example,
the date 07-23-2001 returns 30.

decimal and float Field Functions
You can do the following transformations using the decimal and float field functions.

Assign a decimal to an integer or float or numeric string, or compare a decimal to an integer or float or numeric string.
Specify an optional fix_zero argument (int8) to cause a decimal field containing all zeros to be treated as a valid zero.
Optionally specify a value for the rounding type (r_type) for many conversions. The values of r_type are:
– ceil: Round the source field toward positive infinity. This mode corresponds to the IEEE 754 Round Up mode. Examples: 1.4 -> 2, -1.6 -> -1 – floor: Round the source field toward negative infinity. This mode corresponds to the IEEE 754 Round Down mode. Examples: 1.6 -> 1, -1.4 -> -2 – round_inf: Round or truncate the source field toward the nearest representable value, breaking ties by rounding positive values toward positive infinity and negative values toward negative infinity. This mode corresponds to the COBOL ROUNDED mode. Examples: 1.4 -> 1, 1.5 -> 2, -1.4 -> -1, -1.5 -> -2

– trunc_zero (default): Discard any fractional digits to the right of the rightmost fractional digit supported in the destination, regardless of sign. For example, if the destination is an integer, all fractional digits are truncated. If the destination is another decimal with a smaller scale, round or truncate to the scale size of the destination decimal. This mode corresponds to the COBOL INTEGER-PART function. Examples: 1.6 -> 1, -1.6 -> -1

decimal decimal_from_decimal
(decimal, "r_type" | format_variable) Returns decimal in decimal representation, changing the precision and scale according to the returned type. The rounding type, r_type, may be ceil, floor, round_inf, or trunc_zero as described above this table. The default rtype is trunc_zero.

decimal decimal_from_dfloat
(dfloat, "r_type" | format_variable) Returns dfloat in decimal representation. The rounding type, r_type, may be ceil, floor, round_inf, or trunc_zero as described above this table. The default is trunc_zero.

decimal decimal_from_string
(string, "r_type" | format_variable) Returns string in decimal representation. The rounding type, r_type, may be ceil, floor, round_inf, or trunc_zero as described above this table. The default is trunc_zero.

decimal decimal_from_ustring
(ustring, "r_type" | format_variable) Returns ustring in decimal representation. The rounding type, r_type, may be ceil, floor, round_inf, or trunc_zero as described above this table. The default is trunc_zero.

dfloat dfloat_from_decimal
(decimal, “fix-zero�? | format_variable) Returns decimal in dfloat representation.

int32 int32_from_decimal
(decimal, "r_type fix_zero") Returns int32 in decimal representation. The rounding type, r_type, may be ceil, floor, round_inf, or trunc_zero as described above this table. The default is trunc_zero.

int64 int64_from_decimal
(decimal, "r_type fix_zero") Returns int32 in decimal representation. The rounding type, r_type, may be ceil, floor, round_inf, or trunc_zero as described above this table. The default is trunc_zero.

uint64 uint64_from_decimal
(decimal, "r_type fix_zero") Returns uint64 in decimal representation. The rounding type, r_type, may be ceil, floor, round_inf, or trunc_zero as described above this table. The default is trunc_zero.

string string_from_decimal
(decimal, "fix_zero suppress zero") Returns string in decimal representation. fix_zero causes a decimal field containing all zeros to be treated as a valid zero. suppress_zero argument specifies that the returned ustring value will have no leading or trailing zeros. Examples: 000.100 -> 0.1; 001.000 -> 1; -001.100 -> -1.1

ustring ustring_from_decimal
(decimal, "fix_zero suppress_zero�? | format_variable) Returns ustring in decimal representation. fix_zero causes a decimal field containing all zeros to be treated as a valid zero. suppress_zero argument specifies that the returned ustring value will have no leading or trailing zeros. Examples: 000.100 -> 0.1; 001.000 -> 1; -001.100 -> -1.1

string string_from_decimal
(decimal, “fix_zero suppress_zero�? | format_variable) Returns ustring in decimal representation. fix_zero causes a decimal field containing all zeros to be treated as a valid zero. suppress_zero argument specifies that the returned ustring value will have no leading or trailing zeros. Examples: 000.100 -> 0.1; 001.000 -> 1; -001.100 -> -1.1

dfloat mantissa_from_dfloat(dfloat) Returns the mantissa (the digits right of the decimal point) from dfloat.
dfloat mantissa_from_decimal
(decimal) Returns the mantissa (the digits right of the decimal point) from decimal.

raw Field Functions
Use the raw field functions to transform a string into a raw data type and to determine the length of a raw value.

raw raw_from_string(string) Returns string in raw representation.
raw u_raw_from_string(ustring) Returns ustring in raw representation.
int32 raw_length(raw) Returns the length of the raw field.
time and timestamp Field Functions
Orchestrate performs no automatic conversions to or from the time and timestamp data types. You must use the modify or transform operator if you want to convert a source or destination field. Most field conversions extract a portion of the time, such as hours or minutes, and write it into a destination field. Time conversion to a numeric field can be used with any Orchestrate numeric data type. Orchestrate performs the necessary modifications to translate a conversion result to the numeric data type of the destination. For example, you can use the transformation function hours_from_time() to convert a time to an int8, or to an int16, int32, dfloat, and so on. The string_from_time() and time_from_string() conversion functions take a format as a parameter of the conversion. The default format of the time in the string is hh:nn:ss. However, you can specify an optional format string defining another time format. The format string must contain a specification for hours, minutes, and seconds.

time Uformat
The time uformat provides support for international components in time fields. It’s syntax is: string%macroString%macroString%macroString where %macro is a time formatting macro such as %hh for a two-digit hour. See “time Format�? below for a description of the date format macros. Only the String components of time uformat can include multi-byte Unicode characters.

timestamp Uformat
This format is a concatenation of the date uformat and time uformat which are described in “date Uformat�? on page 21--33 and “time Uformat�? on page 21--38. The order of the formats does not matter, but the two formats cannot be mixed.

time Format
The time components of a source string (hours, minutes, and seconds) must be zero padded to the character length specified by the format string; Orchestrate zero pads the time components of a destination string to the specified length. The components of the format string are:

%hh: A two-digit hours component.
%nn: A two-digit minutes component.
%ss: A two-digit seconds component.
%ss.n: A two-digit seconds component plus a fractional part, where n is the number of fractional digits with a maximum value of 6. If n is 0, no decimal point is printed as part of the seconds component. Trailing zeros are not suppressed. For example, a format string of %hh:%nn:%ss.4 indicates that the * string contains the seconds to four decimal places.
The default time format is as follows: %hh:%nn:%ss When you specify a time format string, prefix each component with the percent symbol (%). Separate the string’s components with any character except the percent sign (%).

time Field and timestamp Field Functions

int8 hours_from_time(time) Returns the hour portion of the given time.
int32 microseconds_from_time(time) Returns the number of microseconds from the
given time.

dfloat midnight_seconds_from_time
(time) Returns the number of seconds from midnight to

time.
int8 minutes_from_time(time) Returns the number of minutes from time.
dfloat seconds_from_time(time) Returns the number of seconds from time.
dfloat seconds_since_from_timestamp
(timestamp, "source_timestamp_string" | format_variable) Returns the number of seconds from timestamp to the base timestamp, or optionally the second timestamp argument for the number of seconds between timestamps. The source_timestamp_string argument must be double quoted or be a variable.

time time_from_midnight_seconds(dfloat) Returns the time given the number of seconds (dfloat) since midnight.
time time_from_string
(string, time_format | time_uformat | format_variable) Returns a time representation of string using the optional time_format, time_uformat, or format_variable. By default, the time format is hh:nn:ss.

time time_from_ustring
(ustring, time_format | time_uformat | format_variable) Returns a time representation of ustring using the optional time_format, time_uformat, or format_variable specification. By default, the time format is hh:nn:ss.

string string_from_time
(time, "time_format" | format_variable | time_uformat) Returns a string from time. The format argument is optional.The default time format is hh:nn:ss.

time time_from_timestamp(timestamp) Returns the time from timestamp.
date date_from_timestamp(timestamp) Returns the date from the given timestamp.
timestamp timestamp_from_date_time
(date, time) Returns a timestamp from date and time. The date specifies the date portion (yyyy-nn-dd) of the timestamp. The time argument specifies the time to be used when building the timestamp. The time argument must be in the hh:nn:ss format.

timestamp timestamp_from_seconds_since
(dfloat, "original_timestamp_string" | format_variable) Returns the timestamp from the number of seconds (dfloat) from the base timestamp or the original_timestamp_string argument. The original_timestamp_string must be double quoted or be a variable.

timestamp timestamp_from_string
(string, "timestamp_format" | timestamp_uformat | format_variable) Returns a timestamp from string, in the optional timestamp_format, timestamp_uformat, or format_variable. The timestamp_format must be double quoted or be a variable. The default format is yyyy-nn-dd hh:nn:ss.

timestamp timestamp_from_ustring
(ustring, "timestamp_format" | timestamp_uformat | format_variable) Returns a timestamp from ustring, in the optional format specification. The timestamp_format must be a double quoted string, a uformat, or a variable. The default format is yyyy-nn-dd hh:nn:ss.

string string_from_timestamp
(timestamp, "timestamp_format" | format_variable) Returns a string from timestamp. The formatting specification is optional. The default format is yyyy-mm-dd hh:mm:ss.

ustring ustring_from_timestamp
(timestamp, "timestamp_format" | format_variable) Returns a ustring from timestamp. The formatting specification is optional. The default format is yyyy-mm-dd hh:mm:ss.

timestamp timestamp_from_time(time, time_format | time_uformat)
Returns a timestamp from time. date date_from_timestamp(timestamp) Returns the date from the given timestamp.

timestamp timestamp_from_timet(int32) Returns a timestamp from the given UNIX time_t representation (int32).
int32 timet_from_timestamp(timestamp) Returns the UNIX time_t representation of timestamp.
Null Handling Functions
All Orchestrate data types support nulls. As part of processing a record, an operator can detect a null and take the appropriate action, for example, it can omit the null field from a calculation or signal an error condition. Orchestrate represents nulls in two ways.

It allocates a single bit to mark a field as null. This type of representation is
called an out-of-band null.

It designates a specific field value to indicate a null, for example a numeric
field’s most negative possible value. This type of representation is called an inband null. In-band null representation can be disadvantageous because you must reserve a field value for nulls and this value cannot be treated as valid data elsewhere.

The null-handling functions can change a null representation from an out-of-band null to an in-band null and from an in-band null to an out-of-band null.

destination_field handle_null
(source_field, value) Change the source_field NULL representations from out-ofband representation to an in-band representation. The value field assigns the value that corresponds to NULL.

destination_field make_null
(source_field, value) Changes source_field NULL representation from in-band NULL representation to out-of-band. The value field allows multiple valid NULL values to be inputted as arguments.

int8 notnull(source_field) Returns 1 if source_field is not NULL, otherwise returns 0.
int8 null(source_field) Returns 1 if source_field is NULL, otherwise returns 0.
set_null() This function is used with “=�? to set the left side output field, when it is nullable, to null. For example:
a-field = set_null();

int8 is_dfloat_inband_null
(dfloat) Returns 1 if dfloat is an inband null; otherwise it returns 0.

int8 is_int16_inband_null
(int16) Returns 1 if int16 is an inband null; otherwise it returns 0.

int8 is_int32_inband_null
(int32) Returns 1 if int32 is an inband null; otherwise it returns 0.

int8 is_int64_inband_null
(int64) Returns 1 if int64 is an inband null; otherwise it returns 0.

int8 is_sfloat_inband_null
(sfloat) Returns 1 if sfloat is an inband null; otherwise it returns 0.

int8 is_string_inband_null
(string) Returns 1 if string is an inband null; otherwise it returns 0.

int8 u_is_string_inband_null
(ustring) Returns 1 if ustring is an inband null; otherwise it returns 0. Note Null-handling functions cannot be used for subrecord fields.

Mathematical Functions
int32 abs(int32) Returns the absolute value of int32.
dfloat acos(dfloat) Returns the principal value of the arc cosine of dfloat.
dfloat asin(dfloat) Returns the principal value of the arc sine of dfloat.
dfloat atan(dfloat) Returns the principal value of the arc tangent of dfloat.
dfloat atan2(dfloat, dfloat) Returns the principal value of the arc tangent of y/x (where y is the first argument).
int ceil(decimal) Returns the smallest integer value greater than or equal to
decimal.

dfloat cos(dfloat) Returns the cosine of the given angle (dfloat) expressed in
radians.

dfloat cosh(dfloat) Returns the hyperbolic cosine of dfloat.
dfloat exp(dfloat) Returns the exponential of dfloat.
dfloat fabs(dfloat) Returns the absolute value of dfloat.
int floor(decimal) Returns the largest integer value less than or equal to decimal.
dfloat ldexp(dfloat, int32) Reconstructs dfloat out of the mantissa and exponent of int32.
uint64 llabs(int64) Returns the absolute value of int64.
dfloat log(dfloat) Returns the natural (base e) logarithm of dfloat.
dfloat log10(dfloat) Returns the logarithm to the base 10 of dfloat.
int32 max(int32, int32) Returns the larger of the two integers.
int32 min(int32, int32) Returns the smaller of the two integers.
dfloat pow(dfloat, dfloat) Returns the result of raising x (the first argument) to the power y (the second argument).
uint32 rand() Returns a pseudo-random integer between 0 and 232 - 1. The function uses a multiplicative congruential random-number generator with period 232. See the UNIX man page for rand for more details.
dfloat sin(dfloat) Returns the sine of dfloat expressed in radians.
dfloat sinh(dfloat) Returns the hyperbolic sine of dfloat.
dfloat sqrt(dfloat) Returns the square root of dfloat.
int32 quotient_from_dfloat
(dfloat1, dfloat2) Returns the value of the quotient after dfloat1 is divided bydfloat2.

srand(uint32) Sets a new seed (uint32) for the frand() or srand() random number generator.
srandom(uint32) Sets a random seed for the random() number generator. See the UNIX man page for srandom for more details.
dfloat tan(dfloat) Returns the tangent of the given angle (dfloat) expressed in
radians.

dfloat tanh(dfloat) Returns the hyperbolic tangent of dfloat.
random() Returns a random integer between 0 and 231 - 1. The function uses a nonlinear additive feedback random-number generator employing a default state array size of 31 long integers to return successive pseudo-random numbers. The period of this random-number generator is approximately 16 x (231 - 1).
Compared with rand, random is slower but more random. See the UNIX man page for random for more details.

string Field Functions
Strings can be assigned (=), compared (==, <, >=, etc.), and concatenated (+) in the Transformation Language. In addition, the functions described in the first list below are available for string manipulations, and the functions described in second list are available for ustring manipulations. When a long string is assigned to a short string, the long string is truncated to the length of the short string. The term white space refers to spaces, tabs, and any other blank space.

string Conversions and Lookup Tables
You can construct a string lookup table to use when default conversions do not yield satisfactory results. A string lookup table is a table of two columns and as many rows as are required to perform a conversion to or from a string as shown below. Each row of the lookup table specifies an association between a 16-bit integer or unsigned 32-bit integer value and a string or ustring. Orchestrate scans the Numeric Value or the String or Ustring column until it encounters the value or string to be translated. The output is the corresponding entry in the row. The numeric value to be converted may be of the int16 or the uint32 data type. Orchestrate converts strings to values of the int16 or uint32 data type using the same table. If the input contains a numeric value or string that is not listed in the table, Orchestrate operates as follows:

If a numeric value is unknown, an empty string is returned by default. However, you can set a default string value to be returned by the string lookup table.
If a string has no corresponding value, 0 is returned by default. However, you can set a default numeric value to be returned by the string lookup table.
A table definition defines the rows of a string or ustring lookup table and has the following form: {propertyList} ('string' | 'ustring' = value; 'string' | 'ustring'= value; ... ) where:

propertyList is one or more of the following options; the entire list is enclosed in braces and properties are separated by commas if there are more than one:
case_sensitive: perform a case-sensitive search for matching strings; the default is case-insensitive.
default_value = defVal: the default numeric value returned for a string that does not match any of the strings in the table.
default_string = defString: the default string returned for numeric values that do not match any numeric value in the table.
A String Lookup Table Numeric Value String or Ustring numVal1 string1 | ustring1 numVal2 string2 | ustring1 ... ... numVal3 stringn | ustringn

string or ustring specifies a comma-separated list of strings or ustrings associated with value; enclose each string or ustring in quotes.
value specifies a comma-separated list of 16-bit integer values associated with string or ustring.
string Field Functions

int8 is_alnum(string) Returns 1 (true) if string consists entirely of alphanumeric
characters.

int8 is_alpha(string) Returns 1 (true) if string consists entirely of alphabetic
characters.

int8 is_numeric(string) Returns 1 (true) if string consists entirely of numeric
characters, including decimal and sign.

int8 is_valid
("type_string", "value_string") Returns 1 (true) if value_string is valid according to type_string, including NULL. The type_string argument is required. It must specify an Orchestrate schema data type. integer types are checked to ensure the value_string is numeric (signed or unsigned), a whole number, and a valid value (for example, 1024 can not be assigned to an int8 type). Decimal types are checked to ensure the value_string is numeric (signed or unsigned) and a valid value. Float types are checked to ensure the value_string is numeric (signed or unsigned) and a valid value (exponent is valid). String is always valid with the NULL exception below. For all types, if the field cannot be set to NULL and the string is NULL, 0 (false) is returned. Date, time, and timestamp types are checked to ensure they are correct, using the optional format argument, and valid values. Raw cannot be checked since the input is a string.

int16 lookup_int16_from_string
(string , "table_definition" | table_variable) Returns an integer corresponding to string using table_definition string or variable.

string lookup_string_from_int16
(int16 , "table_definition" | table_variable) Returns a string corresponding to int16 using table_definition string or variable.

string lookup_string_from_uint32
(uint32 , "table_definition" | table_variable) Returns a string corresponding to uint32 using table_definition string or variable. * uint32 lookup_uint32_from_string (string , "table_definition" | table_variable) Returns an unsigned integer from string using table_definition string or variable.

string lower_case(string) Converts string to lowercase. Non-alphabetic characters
are ignored in the transformation.

string string_from_date
(date, "date_format" | format_variable | date_uformat) Converts date to a string representation using the specified optional formatting specification. By default, the date format is yyyy-mm-dd.

string string_from_decimal
(decimal, “fix_zero suppress_zero�? | format_variable) Returns a string from decimal. fix_zero causes a decimal field containing all zeros to be treated as a valid zero. suppress_zero argument specifies that the returned ustring value will have no leading or trailing zeros. Examples: 000.100 -> 0.1; 001.000 -> 1; -001.100 -> -1.1 The formatting specification is optional.

string string_from_time
(time, "time_format" | format_variable | time_uformat) Returns a string from time. The format argument is optional.The default time format is hh:nn:ss.

string string_from_timestamp
(timestamp, "timestamp_format" | format_variable) Returns a string from timestamp. The formatting specification is optional. The default format is yyyy-mmdd hh:mm:ss.

string soundex
(input_string, length, censusOption) Returns a string which represents the phonetic code for the string input word. Input words that produce the same code are considered phonetically equivalent. The empty string is returned if the input string is empty. length is an int8 and can be any value between 4 and 10. The default is 4. censusOption can be 0, 1, or 2 where 0 (the default) is enhanced soundex and not a census code; 1 are the normal census codes used in all censuses from 1920 on; and 2 are special census codes used intermittently in 1880, 1900, and 1910.

string upper_case(string) Converts string to uppercase. Non-alphabetic characters
are ignored in the transformation.

string compact_whitespace
(string) Returns a string after reducing all consecutive white space in string to a single space.

string pad_string
(string, pad_string, pad_length) Returns the string with the pad_string appended to the bounded length string for pad_length number of characters. pad_length is an int16. When the given string is a variable-length string, it defaults to a bounded-length of 1024 characters. If the given string is a fixed-length string, this function has no effect.

string strip_whitespace(string)
Returns string after stripping all white space in the string.

string trim_leading_trailing
(string) Returns string after removing all leading and trailing white space.

string trim_leading(string) Returns a string after removing all leading white space.
string trim_trailing(string) Returns a string after removing all trailing white space.
int32 string_order_compare
(string1, string2, justification) Returns a numeric value specifying the result of the comparison. The numeric values are: -1: string1 is less than string2 0: string1 is equal to string2 1: string1 is greater than string2 The string justification argument is either 'L' or 'R'. It defaults to 'L' if not specified. 'L' means a standard character comparison, left to right. 'R' means that any numeric substrings within the strings starting at the same position are compared as numbers. For example an 'R' comparison of “AB100�? and “AB99�? indicates that AB100 is great than AB99, since 100 is greater than 99. The comparisons are case sensitive.

string replace_substring
(expression1, expression2, string) Returns a string value that contains the given string, with any characters in expression1 replaced by their corresponding characters in expression2. For example: replace_substring (“ABC:, “abZ�?, “AGDCBDA�?) returns “aGDZbDa�?, where any “A�? gets replaced by “a�?, any “B�? gets replaced by “b�? and any “C�? gets replaced by “Z�?. If expression2 is longer than expression1, the extra characters are ignored. If expression1 is longer than expression2, the extra characters in expression1 are deleted from the given string (the corresponding characters are removed.) For example: replace_substring("ABC", "ab", "AGDCBDA") returns "aGDbDa".

int32 count_substring
(string, substring) Returns the number of times that substring occurs in

string. If substring is an empty string, the number of
characters in string is returned.

int32 dcount_substring
(string, delimiter) Returns the number of fields in string delimited by delimiter, where delimiter is a string. For example, dcount_substring(“abcFdefFghi�?, “F�?) returns 3. If delimiter is an empty string, the number of characters in the string + 1 is returned. If delimiter is not empty, but does not exist in the given string, 1 is returned.

string double_quote_string
(expression) Returns the given string expression enclosed in double quotes.

string substring_by_delimiter
(string, delimiter, occurrence, numsubstr) The string and delimiter argumets are string values, and the occurrence and numsubstr arguments are int32 values. This function returns numsubstr substrings from string, delimited by delimiter and starting at substring number occurence. An example is: substring_by_delimiter (“abcFdefFghiFjkl�?, “F�?, 2, 2) The string “defFghi�? is returned. If occurence is < 1, then 1 is assumed. If occurence does not point to an existing field, the empty string is returned. If numsubstr is not specified or is less than 1, it defaults to 1.

int32 index_of_substring
(string, substring, occurrence) Returns the starting position of the nth occurrence of substring in string. The occurrence argument is an integer indicating the nth occurrence. If there is no nth occurrence or string doesn’t contain any substring, -1 is returned. If substring is an empty string, -2 is returned.

string left_substring
(string, length) Returns the first length characters of string. If length is 0, it returns the empty string. If length is greater than the length of the string, the entire string is returned.

string right_substring
(string, length) Returns the last length characters of string. If length is 0, it returns the empty string. If length is greater than the length of string, the entire string is returned.

string string_of_space(count) Returns a string containing count spaces. The empty string is returned for a count of 0 or less.
string single_quote_string
(expression) Returns the expression string enclosed in single quotes.

string string_of_substring
(string, count) Returns a string containing count occurrences of string. The empty string is returned for a count of 0 or less.

string trimc_string
(string [,character [,option]]) If only string is specified, all leading and trailing spaces and tabs are removed, and all multiple occurrences of spaces and tabs are reduced to a single space or tab. If string and character are specified, option defaults to 'R' The available option values are: 'A' remove all occurrences of character 'B' remove both leading and trailing occurrences of character. 'D' remove leading, trailing, and redundant whitespace characters. 'E' remove trailing white-space characters 'F' remove leading white-space characters 'L' remove all leading occurrences of character 'R' remove all leading, trailing, and redundant occurrences of character 'T' remove all trailing occurrences of character

string system_time_date() Returns the current system time in this 24-hour format:
hh:mm:ss dd:mmm:yyyy

int32 offset_of_substring
(string, substring, position) Searches for the substring in the string beginning at character number position, where position is an uint32. Returns the starting position of the substring.

int8 string_case_compare
(string, string) This is a case-insensitive version of string_compare() below.

int8 string_compare
(string, string) Compares two strings and returns the index (0 or 1) of the greater string.

int8 string_num_case_compare
(string, string, uint16) This is a case-insensitive version of string_num_compare() below.

string string_num_concatenate
(string, string, uint16) Returns a string after appending uint16 characters from the second string onto the first string.

int8 string_num_compare
(string, string, uint16) Compares first uint16 characters of two given strings and returns the index (0 or 1) of the greater string.

string string_num_copy
(string, uint16) Returns the first uint16 characters from the given string int32 string_length(string) Returns the length of the string.

string substring
(string, starting_position, length) Copies parts of strings to shorter strings by string extraction. The starting_position specifies the starting location of the substring; length specifies the substring length. The arguments starting_position and length are uint16 types and must be positive (>= 0).

string char_from_num(int32) Returns an ASCII character from the given int32. If given a value that is not associated with a character such as -1, the function returns a space.
An example use is: char_from_num(38) which returns "&"

int32 num_from_char(string) Returns the numeric value of the ASCII-character in the string. When this function is given an empty string, it returns 0; and when it is given a multi-character string, it uses the first character in the string.
An example use is: num_from_char("&") which returns 38.

ustring Field Functions
Orchestrate provides the ustring type for multi-byte Unicode-character strings. ustrings can be assigned (=), compared (==, <, >=, etc.), and concatenated (+) in the Transformation Language. The term white space refers to spaces, tabs, and any other blank space.

ustring ustring_from_date
(date, "date_format" | date_format | format_variable) Converts date to a ustring representation using the optional format specification. By default, the format is yyyy-mm-dd.

ustring ustring_from_decimal
(decimal, "fix_zero suppress_zero�? | format_variable) Returns a ustring from decimal. fix_zero causes a decimal field containing all zeros to be treated as a valid zero. suppress_zero argument specifies that the returned ustring value will have no leading or trailing zeros. Examples: 000.100 -> 0.1; 001.000 -> 1; -001.100 -> -1.1 The format specification is optional. ustring ustring_from_time (time, "time_format" | time_uformat | format_variable) Returns a ustring from time using an optional format specification.The default time format is hh:nn:ss.

ustring ustring_from_timestamp
(timestamp, "timestamp_format" | format_variable) Returns a ustring from timestamp. The format specification is optional. The default format is yyyy-mm-dd hh:mm:ss.

int8 u_is_alnum(ustring) Returns 1 (true) if ustring consists entirely of alphanumeric characters.
int8 u_is_alpha(ustring) Returns 1 (true) if ustring consists entirely of alphabetic characters.
int8 u_is_numeric(ustring) Returns 1 (true) if ustring consists entirely of numeric
characters, including decimal and sign.

int16 lookup_int16_from_ustring
(ustring , "table_definition" | table_variable) Returns an integer corresponding to ustring using table_definition string or variable.

ustring lookup_ustring_from_int16
(int16 , "table_definition" | table_variable) Returns a ustring corresponding to int16 using table_definition string or variable. * ustring lookup_ustring_from_uint32 (uint32 , "table_definition" | table_variable) Returns a ustring corresponding to uint32 using table_definition string or variable. * uint32 lookup_uint32_from_ustring (string , "table_definition" | table_variable) Returns an unsigned integer from ustring using table_definition string or variable. int8 u_is_valid ("type_ustring", "value_ustring") Returns 1 (true) if value_ustring is valid according to type_ustring, including NULL. The type_ustring argument is required. It must specify an Orchestrate schema data type. Integer types are checked to ensure the value_ustring is numeric (signed or unsigned), a whole number, and a valid value (for example, 1024 can not be assigned to an int8 type). Decimal types are checked to ensure the value_ustring is numeric (signed or unsigned) and a valid value. Float types are checked to ensure the value_ustring is numeric (signed or unsigned) and a valid value (exponent is valid). String is always valid with the NULL exception below. For all types, if the field cannot be set to NULL and the string is NULL, 0 (false) is returned. Date, time, and timestamp types are checked to ensure they are correct, using the optional format argument, and valid values. Raw cannot be checked since the input is a string. ustring u_lower_case(ustring) Converts ustring to lowercase. Non-alphabetic characters are ignored in the transformation.

ustring u_upper_case(ustring)
Converts ustring to uppercase. Non-alphabetic characters are ignored in the transformation.

ustring u_compact_whitespace (ustring)
Returns the ustring after reducing all consecutive white space in ustring to a single space.

ustring u_pad_string
(ustring, pad_ustring, pad_length) Returns the ustring with pad_ustring appended to the bounded length string for pad_length number of characters. pad_length is an int16. When the given ustring is a variable-length string, it defaults to a bounded-length of 1024 characters. If the given ustring is a fixed-length string, this function has no effect.

ustring u_strip_whitespace
(ustring) Returns ustring after stripping all white space in the string.

ustring u_trim_leading_trailing
(ustring) Returns ustring after removing all leading and trailing white space.

ustring u_trim_leading(ustring) Returns ustring after removing all leading white space.
ustring u_trim_trailing
(ustring) Returns a ustring after removing all trailing white space.

int32 u_string_order_compare
(ustring1, ustring2, justification) Returns a numeric value specifying the result of the comparison. The numeric values are: -1: ustring1 is less than ustring2 0: ustring1 is equal to ustring2 1: ustring1 is greater than ustring2 The string justification argument is either 'L' or 'R'. It defaults to 'L' if not specified. 'L' means a standard character comparison, left to right. 'R' means that any numeric substrings within the strings starting at the same position are compared as numbers. For example an 'R' comparison of “AB100�? and “AB99�? indicates that AB100 is great than AB99, since 100 is greater than 99. The comparisons are case sensitive.

ustring u_replace_substring
(expression1, expression2, ustring) Returns a ustring value that contains the given ustring, with any characters in expression1 replaced by their corresponding characters in expression2. For example: u_replace_substring (“ABC", “abZ�?, “AGDCBDA�?) returns “aGDZbDa�?, where any “A�? gets replaced by “a�?, any “B�? gets replaced by “b�? and any “C�? gets replaced by “Z�?. If expression2 is longer than expression1, the extra characters are ignored. If expression1 is longer than expression2, the extra characters in expression1 are deleted from the given string (the corresponding characters are removed.) For example: u_replace_substring("ABC", "ab", "AGDCBDA") returns "aGDbDa".

int32 u_count_substring
(ustring, sub_ustring) Returns the number of times that sub_ustring occurs in ustring. If sub_ustring is an empty string, the number of characters in ustring is returned.

int32 u_dcount_substring
(ustring, delimiter) Returns the number of fields in ustring delimited by delimiter, where delimiter is a string. For example, dcount_substring(“abcFdefFghi�?, “F�?) returns 3. If delimiter is an empty string, the number of characters in the string + 1 is returned. If delimiter is not empty, but does not exist in the given string, 1 is returned.

ustring u_double_quote_string
(expression) Returns the given ustring expression enclosed in double quotes.

ustring u_substring_by_delimiter
(ustring, delimiter, occurrence, numsubstr) The delimiter argument is a ustring value, and the occurrence and numsubstr arguments are int32 values. This function returns numsubstr substrings from ustring, delimited by delimiter and starting at substring number occurence. An example is: u_substring_by_delimiter (“abcFdefFghiFjkl�?, “F�?, 2, 2) The string “defFghi�? is returned. If occurence is < 1, then 1 is assumed. If occurence does not point to an existing field, the empty string is returned. If numsubstr is not specified or is less than 1, it defaults to 1.

int32 u_index_of_substring
(ustring, sub_ustring, occurrence) Returns the starting position of the nth occurrence of sub_ustring in ustring. The occurrence argument is an integer indicating the nth occurrence. If there is no nth occurrence, 0 is returned; if sub_ustring is an empty string, -2 is returned; and if ustring doesn’t contain any sub_ustring, -1 is returned.

ustring u_left_substring
(ustring, length) Returns the first length characters of ustring. If length is 0, it returns the empty string. If length is greater than the length of the ustring, the entire ustring is returned.

ustring u_right_substring
(ustring, length) Returns the last length characters of ustring. If length is 0, it returns the empty string. If length is greater than the length of ustring, the entire ustring is returned.

ustring u_string_of_space(count)
Returns a ustring containing count spaces. The empty string is returned for a count of 0 or less.

ustring u_single_quote_string
(expression) Returns expression enclosed in single quotes.

ustring u_string_of_substring
(ustring, count) Returns a ustring containing count occurrences of ustring. The empty string is returned for a count of 0 or less

ustring u_trimc_string
(ustring [,character [,option]]) If only ustring is specified, all leading and trailing spaces and tabs are removed, and all multiple occurrences of spaces and tabs are reduced to a single space or tab. If ustring and character are specified, option defaults to 'R' The available option values are: 'A' remove all occurrences of character 'B' remove both leading and trailing occurrences of character. 'D' remove leading, trailing, and redundant whitespace characters. 'E' remove trailing white-space characters 'F' remove leading white-space characters 'L' remove all leading occurrences of character 'R' remove all leading, trailing, and redundant occurrences of character 'T' remove all trailing occurrences of character

ustring u_system_time_date() Returns the current system time in this 24-hour format:
hh:mm:ss dd:mmm:yyyy

int32 u_offset_of_substring
(ustring, sub_ustring, position) Searches for the sub_ustring in the ustring beginning at character number position, where position is an uint32. Returns the starting position of the substring.

int8 u_string_case_compare
(ustring, ustring) This is a case-insensitive version of u_string_compare() below. int8 u_string_compare (ustring, ustring) Compares two ustrings and returns the index (0 or 1) of the greater string. int8 u_string_num_case_compare (ustring, ustring, uint16) This is a case-insensitive version of u_string_num_compare() below.

ustring u_string_num_concatenate
(ustring, ustring, uint16) Returns a ustring after appending uint16 characters from the second ustring onto the first ustring.

int8 u_string_num_compare
(utring, ustring, uint16) Compares first uint16 characters of two given ustrings and returns the index (0 or 1) of the greater ustring.

ustring u_string_num_copy
(ustring, uint16) Returns the first uint16 characters from the given ustring. int32 u_string_length(ustring) Returns the length of the ustring.

ustring u_substring
(ustring, starting_position, length) Copies parts of ustrings to shorter strings by string extraction. The starting_position specifies the starting location of the substring; length specifies the substring length. The arguments starting_position and length are uint16 types and must be positive (>= 0).

ustring u_char_from_num(int32) Returns a ustring character value from the given int32. If given a value that is not associated with a character such as -1, the function returns a space. An example use is: u_char_from_num(38) which returns
"&"

int32 u_num_from_char(ustring) Returns the numeric value of the character in the ustring.
When this function is given an empty string, it returns 0; and when it is given a multi-character string, it uses the first character in the string. An example use is: u_num_from_char("&") which returns 38

Bit Manipulation Functions
string bit_expand(uint64) Expands the given uint64 to a string containing the binary representation.
ustring u_bit_expand(uint64) Expands the given uint64 to a ustring containing the
binary representation.

uint64 bit_compress(string) Converts the string binary representation to an uint64
field.

uint64 u_bit_compress(ustring) Converts the ustring binary representation to an uint64 field.
uint64 set_bit
(uint64, list_of_bits, bit_state) Turns the uint64 bits that are listed by number in the string list_of_bits on or off, depending on whether the value of the bit_state integer is 1 or 0. bit_state is an optional argument, and has a default value of 1 which turns the list of bits on. An example use is: set_bit(0, "1,3,5,7") which returns 85.

uint64 u_set_bit
(uint64, list_of_bits, bit_state) This function is a internationalized version of set_bit() above.