Chapter 3
Using UNIX

 

Introduction

A Systems Administrator not only has to look after the computers and the operating system, they also have to be the expert user (or at least a very knowledgeable user) of their systems. When other users have problems where do they go? The documentation? Online help facilities? No, they usually go to the Systems Administrator.

The following reading aims to start you on the road to becoming an expert UNIX user.  Becoming a UNIX guru can only be achieved through a great deal of experience so it is important that you spend time using the commands introduced in this chapter.

Introductory UNIX

Basic UNIX

You will find an introduction to some very basic UNIX concepts under the Resource Materials section for week 2.

Exercises

3.1         What UNIX commands would you use to
- change to your home directory
- display the list of files in the current directory
- display my name is fred onto the screen
- copy the file tmp.dat from the current directory to the directory data underneath your home directory and after the file has been copied delete it

3.2         What will the following UNIX commands do? Don't execute a UNIX command if you aren't sure what it is going to do. In particular do not try to execute the first command below.
rmdir ~
cat /etc/passwd
ls ../../fred/doc/tmp


UNIX Commands are programs

The UNIX commands that have been introduced so far are stored on a UNIX computer as executable files. Most of the commands you will use in this chapter are stored in standard binary directories such as /bin /usr/bin /usr/local/bin.  On a system running RedHat version 5.0 there are over 1000 different files in the directories /bin, /usr/bin and /usr/sbin.  Which means over 1000 different commands.

vi

A major task of any user of a computer is editing text files.  For a Systems Administrator of a UNIX system manipulation of text files is a common task due to many of the system configuration files being text files.  The most common, screen-based UNIX editor is vi.  The mention of vi sends shudders through the spines of some people, while other people love it with a passion. vi is difficult to learn, however it is also an extremely powerful editor which can save a Systems Administrator a great deal of time.

As you progress through this subject you will need an editor. vi is an anachronistic antique of an editor hated by most people. So why should you use it? Reasons include

§         it is very powerful,
How many editors do you know that can take the first 20 characters of every line in a file and swap them with the second set of 20 characters (something I've had to do)

§         it is the only screen editor available on every UNIX system

§         There will be times when a Systems Administrator cannot use a full screen editor. At times like this you must resort to single line editors like ed and ex. vi grew out of the ex editor and so uses many of the same commands. Learning and using these commands in vi can save you time later on.

As a result of all this it is strongly suggested that you use vi whereever possible in studying for this unit.  Early on you will find using vi a hassle but sticking with it will be worthwhile in the end.

An introduction to vi

Linux comes with vi as standard. Most distributions also provide you with an option to install vim. vim is an improved version of vi that includes features like multiple levels of undo.

 

Using vi

Under the resource materials section for week 2 (on the 85321 CD-ROM and Web site) contains a number of resources to introduce you to vi including an introduction and a number of references.

UNIX commands

A UNIX system comes with hundreds of executable commands and programs (it is quite easy to get to a count of 600 without really looking hard). Typically each of these programs carries out a particular job and will usually have some obscure and obtuse name that means nothing to the uninitiated.

Philosophy of UNIX commands

There are no set rules about UNIX commands however there is a UNIX philosophy that is used by many of the commands.

§         small is beautiful,
UNIX provides the mechanisms to join commands together so commands should do one thing well.

§         10 percent of the work solves 90 percent of the problems,
UNIX was never designed to solve all problems, it was designed to solve most requirements without too much hassle on the programmer's part.

§         solve the problem, not the machine,
Commands should ignore any machine specific information and be portable.

§         solve at the right level, and you will only have to do it once.
The key to UNIX problem solving is only to do it once e.g. pattern matching is only implemented once, in the shell, not in every command.

UNIX command format

UNIX commands use the following format

 command_name -switches parameter_list

Component

Explanation

command_name

the name of the actual command, generally this is the name of the executable program that is the command

-switches

The - symbol is used to indicate a switch. A switch modifies the operation of a command.

parameter_list

the list of parameters (or arguments) that the command will operate on, could be 0, 1 or more parameters, parameters are separated by white space characters (space, TAB)

Table 3.1
UNIX command format


Example commands

§         ls -l
The switch -l is used to modify the action of the ls command so that it displays a long listing of each file.

§         ls -l /etc/passwd / /var
Commands can take multiple parameters.

§         ls -ld /etc/passwd / /var
Multiple switches can also be used.

 

Linux commands take multiple arguments

Unlike MS-DOS, UNIX commands can take multiple arguments.

Exercises

3.3         One of your users has created a file called -tmp? (The command cat /etc/passwd > -tmp will do it.) They want to get rid of it but can't. Why might the user have difficulty removing this file? How would you remove the file?

A command for everything

A fairly intelligent and experienced would be computer professional has just started using UNIX seriously (he was a student in the very first offering of this subject). He gets to a stage where he wants to change the name of some files.

Being an MS-DOS junkie from way back what command does he look for? The rename command of course. It doesn't work! "That's a bit silly!", he thinks, "You would think that UNIX would have a rename command."

It just so happens that this person has just completed a C programming subject in which one of the assignments was to write a rename command. So he spends the next day trying to write and compile this program. After much toil and trouble he succeeds and follows good administration policy and informs all the other students of this brand new wonderful program he has written. He goes into great detail on how to use the command and all the nice features it includes.

They all write back to tell him about the UNIX command mv (the move command) that is the UNIX command that is equivalent to rename.

The moral of the story

The moral of this story is that if you want to do something under UNIX, then chances are that there is already a command to do it. All you have to do is work out what it is.


Online help

UNIX comes with online help called man pages . Man pages are short references for commands and files on a UNIX system.  They are not designed as a means to learn the commands.

The man pages are divided into different sections. Table 3.2 shows the sections that Linux uses. Different versions of Linux use slightly different sections.

Section number

Contents

1

user commands

2

system calls

3

Library functions

3c

 standard C library

3s

standard I/O library

3m

arithmetic library

3f

Fortran library

3x

special libraries

4

special files

5

file formats

6

games

7

miscellaneous

8

administration and privileged commands

Table 3.2
Manual Page Sections

Using the manual pages

To examine the manual page for a particular command or file you use the man command. For example if you wanted to examine the man page for the man command you would execute the command man man.

Is there a man page for...

The command man -k keyword will search for all the manual pages that contain keyword in its synopsis. The commands whatis and apropos perform similar tasks.

Rather than search through all the manual pages Linux maintains a keyword database in the file /usr/man/whatis. If at any stage you add new manual pages you should rebuild this database using the makewhatis command.

If there is a file you wish to find out the purpose for you might want to try the –f option of the man command.


man page format

Each manual page is stored in its own file formatted (under Linux) using the groff command (which is the GNU version of nroff). The files can be located in a number of different directories with the main manual pages located under the /usr/man directory.

Under /usr/man you will find directories with names mann and catn. The n is a number that indicates the section of the manual. The files in the man directories contain the groff input for each manual page. The files in the cat directories contain the output of the groff command for each manual page.

Generally when you use the man command the groff input is formatted and displayed on the screen. If space permits the output will be written into the appropriate cat directory.

Some UNIX commands

There are simply too many UNIX commands for this chapter to introduce all, or even most of them.  The aim of the following is to show you some of the basic commands that are available.  To find the remainder you will have to discover them for yourself.  One method for becoming more familiar with the available commands is to

§         look at the filenames in the /bin /usr/bin /usr/local/bin directories,
These are the “binary” directories which contain the exectuable programs which are the UNIX commands.

§         take the filename and look at the manual page
Each of the commands will have a manual page which will explain what the command does and how you can use it.

The commands introduced in this table can be divided into categories based on their purpose

§         identification commands,
These commands identify the user or the system.

§         simple commands,
Perform some simple, non-specific task.

§         filters.
Filters take some input, modify it and then output it.

 


 

Command

Purpose

Command

Purpose

date

Display the current time and date

who

display who is currently on the computer

banner

Display a large banner

cal

display a calendar

whoami

Displays your current username

cat

display the contents of a file

more and less

Display the contents of a file a page at a time

head

display the first few lines of a file

tail

Display the last few lines of a file

sort

sort the content of a file into order

uniq

Remove duplicate lines from a file

cut

remove columns of characters from a file

paste

join columns of files together

tr

translate specific characters

grep

Display all lines in a file containing a patter

wc

count the number of characters, words and lines in a file

Table 3.3
Basic UNIX commands

Identification Commands

who

Displays a list of people currently logged onto the computer.

dinbig:/$ who
david    tty1     Feb  5 14:27  
         

whoami

Displays who the computer thinks you are currently logged in as.

dinbig:/$ whoami
david    

uname

Displays information about the operating system and the computer on which it is running

[david@beldin david]$ uname
Linux
[david@beldin david]$ uname –a
Linux beldin.cqu.edu.au 2.0.31 #1 Sun Nov 9 21:45:23 EST 1997 i586 unknown

Simple commands

The following commands are simple commands that perform one particular job that might be of use to you at some stage. There are many others you'll make use of.

Only simple examples of the commands will be shown below. Many of these commands have extra switches that allow them to perform other tasks. You will have to refer to the manual pages for the commands to find out this information.

date

Displays the current date and time according to the computer.

dinbig:/$ date
Thu Feb  8 16:57:05 EST 1996

banner

Creates a banner with supplied text.

dinbig:/$ banner -w30 a
          ##
        ######  ##
       ##    ## ###
        #    #     #
        ##  ##   ##
       ###########
       ##

cal

Display a calendar for a specific month. (The Linux version might not work).

bash$ cal 1 1996
   January 1996
 S  M Tu  W Th  F  S
    1  2  3  4  5  6
 7  8  9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27
28 29 30 31    

Filters

Filters are UNIX commands that take input or the contents of a file, modify that content and then display the result on output. Later on in this chapter you will be shown how you can combine these filters together to manipulate text.

cat

The simplest filter. cat doesn't perform any modification on the information passed through it.

bash$ cat /etc/motd
Linux 1.2.13. 

more and less

These filters display their input one page at a time. At the end of each page they pause and wait for the user to hit a key. less is a more complex filter and supports a number of other features. Refer to the man page for the commands for more information.

head and tail

head and tail allow you to view the first few lines or the last few lines of a file.

Examples

§         head chap1.html
Display the first 10 lines of chap1.html

§         tail chap1.html
display the last 10 lines of chap1.html

§         head -c 100 chap1.html
display the first 100 bytes of chap1.html

§         head -l 50 chap1.html
display the first 50 lines of chap1.html

§         tail -c 95 chap1.html
display the last 100 bytes of chap1.html sort

sort

The sort command is used to sort data using a number of different criteria outlined in the following table.

Switch

Result

-r

sort in descending order (default is ascending)

-n

sort as numbers (default is as ASCII characters)
When sorting numbers as numbers 100 is greater than 5. When sorting them as characters 5 is greater than 100.

-u

eliminate duplicate lines

+numbern

skip number fields

-tcharacter

specify character as the field delimiter

Table 3.4
Switches for the sort command

Examples

The following examples all work with the /etc/passwd file. /etc/passwd is the file that stores information about all the users of a UNIX machine. It is a text file with each line divided into 7 fields. Each field is separated by a : character. Use the cat command to view the contents of the file.

§         sort /etc/passwd
sort in order based on the whole line

§         sort -r /etc/passwd
reverse the order of the sort

§         sort +2n -t: /etc/passwd
sort on third field, where field delimiter is : (skip the first two fields)

§         sort +2n -t: -n /etc/passwd
same sort but treat the field as numbers not ASCII characters

uniq

uniq is used to find or remove and duplicate lines from a file and display what is left onto the screen. A duplicate to uniq is where consecutive lines match exactly. sort is often used to get the duplicate lines in a file into consecutive order before passing it to uniq.  Passing a file from one command to another is achieved using I/O redirection which is explained in a later chapter.

Examples

§         uniq names
remove duplicate lines from names and display them on the screen

§         uniq names uniq.names
remove duplicates lines from names and put them into uniq.names

§         uniq -d names
display all duplicate lines

tr

Used to translate specified characters into other characters. tr is used in conjunction with I/O redirection which is explained in the next chapter. In the examples below the < character is an I/O redirection character.

Examples

§         tr a z < /etc/passwd
translate all a's to z's in /etc/passwd and display on the screen

§         tr '[A-Z]' '[a-z]' < /etc/passwd
translate any character in between A-Z into the equivalent character between a-z. (make all upper-case characters lower case)

§         tr -d ' ' < /etc/passwd
delete any single space characters from the file


cut

Is used to "cut out" fields from a file. Try cut -c5-10 /etc/passwd. This will display all the characters between the 5th and 10th on every line of the file /etc/passwd. The following table explains some of the switches for cut

Switch

Purpose

-cRANGE

cut out the characters in RANGE

-dcharacter

specify that the field delimiter is character

-fRANGE

cut out the fields in RANGE

Table 3.5
Switches for the cut command

RANGE used by the -f and -c switches can take the following forms

§         number-
get all from character or field number to the end of the line

§         number-number2
get all from character or field number to character or field number2

§         number,number2
get characters or fields number and number2

And combinations of the above.

Examples

§         cut -c1 /etc/passwd
get the first character from every line

§         cut -c1,5,10-20 /etc/passwd
get the first, fifth character and every character between 10 and 20

§         cut -d: -f2 /etc/passwd
get the second field

§         cut -d: -f3- /etc/passwd
get all fields from the third on

paste

This command performs the opposite task to cut. It puts lines back together.

Assume we have two files

names
george
fred
david
janet

addresses
55 Aim avenue
1005 Marks road
5 Thompson Street
43 Pedwell road

 

To put them back together we'd use the command

bash$ paste names addresses
george  55 Aim avenue
fred    1005 Marks road
david   5 Thompson Street
janet   43 Pedwell road

The two fields have been separated by a tab character. To use a different character you use the -d switch.

bash$ paste -d: names addresses
george:55 Aim avenue
fred:1005 Marks road
david:5 Thompson Street
janet:43 Pedwell road

To paste together lines from the same file you use the -s switch.

bash$  paste -s names
george  fred    david   janet     

grep

grep stands for Global Regular Expression Pattern match. It is used to search a file for a particular pattern of characters.

§         grep david /etc/passwd
display any line from /etc/passwd that contains david

To get the real power out of grep you need to be familiar with regular expressions which are discussed in more detail in a later chapter.

wc

Used to count the number of characters, words and lines in a file. By default it displays all three. Using the switches -c -w -l will display the number of characters, words and lines respectively.

bash$ wc /etc/passwd
  19      20     697 /etc/passwd
bash$ wc -c /etc/passwd
   697 /etc/passwd
bash$ wc -w /etc/passwd
   20 /etc/passwd
bash$ wc -l /etc/passwd
   19 /etc/passwd
 

For the following exercises create a file called phone.book that contains the following

george!2334234!55 Aim avenue
fred!343423!1005 Marks road
david!5838434!5 Thompson Street
janet!33343!43 Pedwell road
     

The field delimiter for this file is ! and the fields are name, phone number, address.

Exercises

3.4         What command would you use to (assume you start from the original file for every question)
1.  sort the file on the names
2.  sort the file in descending order on phone number
3.  display just the addresses
4.  change all the ! characters to :
5.  display the first line from the file
6.  display the line containing david's information

7. 
What would effect would the following command have paste -d: -s phone.book

Getting more out of filters

The filters are a prime example of good UNIX commands. They do one job well and are designed to be chained together. To get the most out of filters you combine them together in long chains of commands.  How this is achieved will be examined in a later chapter when the concept of I/O redirection is introduced.

I/O redirection allows you to count the number of people on your computer who have usernames starting with d by using the grep command to find all the lines in the /etc/passwd file that start with d and pass the output of that command to the wc command to count the number of matching lines that grep found.

How you do this will be explained next week.

Conclusions

In this chapter you have been provided a brief introduction to the philosophy and format of UNIX commands. In addition some simple commands have been introduced including

§         account related commands
login passwd exit

§         file and directory manipulation commands
cd ls rm mv mkdir

§         some basic commands
date who banner cal

§         some filters
cat more less head tail sort uniq cut paste tr grep

A Systems Administrator has to be a "guru". An expert user of their system. A Systems Administrator should not only to be able to get the most out of the system but also to be able to explain and assist other users.