• View
• Edit
• History
• Print

LectureNotes

Advanced Linux Notes

Notes from Lectures 1, 2 and 3

You connect to a system either over a network or via a serial cable, or using the screen and keyboard attached to that system.

To log in to a system you usually supply a user name and password. This needs to be initially set up by your system administrator.

 Other methods of authentication include onetime keypads, smart cards and biometric devices. these are configured with the PAM subsystem. In fact all authentication and authorization is configured with PAM the default is username and password.

To change your password, type passwd You are prompted first for your old password, then twice for the new one you are changing it to. The system administrator can change any password with the command passwd username

You are interacting on the COMMAND LINE with a SHELL. Bash is the popular shell in Linux whose prompt is a dollar sign. However, if you work with online tutorials, they may show the % sign as a prompt because they might asume you are using a different shell.

In general on a command line, you type a command, followed by the field separator which is usually a space, followed by one or more options. Typically an option begins with a minus sign or two minus signs, which many humans and synthesizers pronounce as "hyphen" or "dash". Options are also called arguments. Commands, Options and files in Linux are case sensitive.

Note that most commands will accept --help and --version as options to show a help summary and version number respectively.

The echo command simply echoes whatever is typed on the command line to standard output. echo also has options to output escape characters.

 man echo

for details.

To display the files in the current directory, including subdirectories, type

 ls

Note this shows subdirectories in the current directory but does not recourse them.

Like most Linux commands, ls has a variety of options. The -a displays all files. the -F places a slash after directory names. The -l is a "long" listing showing multiple columns of information about each file. The -ld parameter displays information about a directory. The F, by the way is upper case.

The who command displays information about who is currently logged in. who can also show your user name try

 who mom likes
 who am i

The cp command copies a file. The first argument to cp is the source, and the second argument is the destination. Note that options to cp allow copying file permitions and folder structure see

 cp --help

The cat command regurgitates the contents of files or standard input if no filenames are given. Cat stands for concatinate Due to this flexibility it is used to display, copy or create files, though displaying files is its most common use. cat can also be used to show special characters in files using the --show-all option. See the manpage for details

The cd command changes to a new directory. a directory can be specified as either relative or absolute. assume the current directory is your home directory.

 cd usr/backups

changes you to the usr/backups folder inside your home directory. this is a relative path, relative to where you are now. execute

 pwd

to find this out.

 cd /usr/backups

changes you to the usr/backups folder directory off the root of the filesystem which is represented by / This is an absolute path as it specifies the intire path from the root to the folder.

The rm command deletes (removes) a file or files. this command has options, see the manpage. -i for interactive, -r for recursive and -f for force are worth investigating.

The man command displays "manual pages", documentation on commands and other topics. man -k lets you search for a keyword.

Typing man man gives options for the man command. Typing man who documents options for the who command.

The mesg command has either a y or an n option. It tells the system whether you wish to receive messages from other users. It does this by changing the permitions on the controlling tty explained later in the lectures.

Another fun command to experiment with is date. The date command has many formatting options which need to be flagged with a plus sign or by using the preferred long options.

The fortune command displays a fun quote.

In Linux, everything is a file. A directory is a file. A sound device is a file. A printer is a file.

Unless Linux is told otherwise, it accepts input from Standard Input, and sends output to Standard Output. by default Standard Output is the screen and Standard Input is the keyboard. actually standard output and input are connected to the controlling tty which can be the console, a serial port or a pseudo tty. To terminate standard input, type CTRL-D which is the EOT character.

Commands that expect multi-line input wait for you to type in those lines and press ctrl-D to end the input. An example of this is wall, which displays a message to all users. Type "wall" and press enter. Then type multiple lines and complete your input with CTRL-D.

Commands normally send output to the screen, because by default that's where standard output is "directed". However, both standard input and standard output can be redirected.

To redirect input, use the < sign. To redirect output use the > sign.

To append redirected output to an existing file use a double greater than, that is two right angle greater signs together.

For example each time you type

    ls > test.txt

a new test.txt gets created with the output from ls. However if you type

    ls >> test.txt

new directory listings are appended to test.txt.

You can also use the right angle greater sign to create a new file as in
>file1.txt

this redirects nothing to file1.txt, thus creating a new empty file.

Another way to accomplish the same thing is to do
touch file2.txt

this creates an empty file2.txt

 Sometimes you have long scrolling output that you want to save. Redirecting it to a file is a good way to do that. However, if you simply want to display it one screen at a time, you can use the more or the less commands. Both of them are called "pagers" because they divide the output in to "pages" which simply means screenfulls of output. 

More is a simple command. Less is more complex; it lets you scroll both backwards and forwards, as well as search. Output of the man command is automatically piped through less without you needing to tell it to do so. most is even more fully featured than less which does more than more.

To redirect input, the less-than (<) sign tells Linux input is coming from a source other than the keyboard.

Input for one command is more commonly sent from output of another command. To do this the concept of a pipe is used. The vertical bar character indicates a pipe. For example

    ls|sort

displays the directory listing in alphabetical order.

    cat longfile | less

displays the long file one screen at a time.

The grep command searches a file or files for a pattern. A pattern is enclosed in single or double quotes if it contains spaces or needs to be protected from the shell. Due to redirection, the output of ls can be piped through grep to filter out files and show only those files which are directories themselves. This is the most complex command we have studied so far:

    ls -F | grep '/'
 the single quotes are not strictly necessary on this command in bash.

Lecture 4

Most of this lecture explored working with directories, pathnames and files. See Understanding Linux Path Structure and Understanding Permissions Permissions were explored further in the study group discussion, and redirection, which is covered earlier.

We are now regularly connecting to
67.23.234.157
also known as
drongo.cucat.org

This is a cloud-based virtual machine (called a VPS) which Kerry set up for us to practice with.

We can also use the virtual machine downloaded from the Cavi site and its primary user is
noddy
with default password of
user

An experienced user can get away with copying and pasting commands. However, beginners should always type out commands to develop a muscle memory and to ensure they are keying them in correctly. Because it is easy to make mistakes -- copying a file to an unwanted destination, or deleting a directory with hundreds of valuable files -- unintentionally, having a firm grasp of the command syntax is vital to becoming a competent system administrator.

It is also crucial to carefully read over what you typed before pressing ENTER. Beginners should also not forget to read the system response after a command executes. You may need to return to the previous lines above the cursor using screen review in order to fully explore whether your last command executed without errors. Linux does things without asking if you are sure.

Passwords used to be stored in a world-readable file called /etc/passwd. Now this file is primarily used to look up information about users, their full names, user ids and home directories. Passswords are saved in /etc/shadow which is encrypted and not accessible to the outside so a box cannot be so easily cracked.

Lecture 5

Lecture 5 was such a large collection of topics, I've created a separate page for its info. Please go to Terminals, Screen Sessions and Command Line Editing Basics for that documentation.

In Study Group 5, we installed Dwww, which gives access to documentation on a Linux system from a web browser. On Ubuntu and Debian documentation can be found in man pages, info pages and in various html and gzipped text files in /usr/share/doc/PACKAGENAME. the dwww package, which is available on Ubuntu consolidates these disparate documentation sources in to a web-based interface that's easier to browse. To Install it simply type
sudo apt-get instal dwww

Lecture 6

The screen program was covered in more detail, and I simply added the new information to my existing page at: Terminals, Screen Sessions and Command Line Editing Basics

We also discussed Processes and Jobs which I've also covered on a separate page. In the following study group, we fleshed out some of these concepts more, and I edited their corresponding pages accordingly. We also discussed Linux Screen Access, a page I am in the process of creating now. It currently contains an explanation of screen access solutions under Linux but needs links to the web pages for all the software discussed.

In Study group 6, we also covered
Rsnapshot,
a Perl script that does incremental backups. It uses hard links to track unchanged versions of files, keeping all but the first backup as small as possible. Restoring a system to its previous state with rsnapshot is easy, and rsnapshot can be automated with cron jobs. There was also some discussions of package management and comparing various editors which being discussed more in Lecture 7, will be linked to from there. I submitted a script called sls, for speech-friendly ls which we also discussed. I will link to a final version of that from this page when I finish working on it. Typically a small script like this one would be installed in /usr/local/bin. This is where programs that are outside the distribution's official packaging system are usually kept.

We also discussed several initialization files in a user's home directory.
.bashrc
runs each time a new shell is invoked. and
.bash_profile
runs for a login shell. So .bashrc is examined each time any new shell is invoked.

Environment variables control how the system behaves and are created for a user in a particular shell. For example, the variable
EDITOR
tells programs which text editor to use by default. The export command changes an environment variable and also "exports" it to subshells that might be called. as in
export EDITOR=/bin/ed

To see all the environment variables, type
set | less

and to temporarily change one without it affecting other shells or future sessions type
set VARIABLE=value

the variable is always in upper-case. The syntax for set and export is identical, and there is no space between the variable the equals sign and the value.

To make a variable change permanent, it must be set with export and this is done in either .bashrc or .bash_profile.

When screen is run, its subshell inherits all the environment variables that the user already has set either through .bashrc, .bash_profile or temporarily with the set command.

Lecture 7

Now that we know our final assignment is to install Pm Wiki on our virtual machines, we began to cover the skills necessary to perform this assignment.

We had some introduction to Basic Package Management which has been expanded on with a separate page.

For our assignment we'll need to install
libapache2-mod-php5
which enables Apache to handle web pages with php.

If instead, we'd installed php5-cli, we would not have pulled in the needed web module for Apache. Php though it's most commonly a web language can also be used in the console (at the command line) with the php5-cli packages.

If php is on a page in a directory accessed by a web server, and users are asked if they want to download the file, instead of the php being run, then the php interpreter on the server needs to be fixed, probably by reconfiguring something or by installing a package that is missing.

We discussed script, a simple utility that starts a sub-shell, and logs all input and output to a text file. as in
script myfile.txt

You are in a sub-shell, and all your commands and their results can be examined by perusing the log created later on.

when you are done logging, type exit to return to the calling shell. Your script file is saved.

One problem with script is that it saves all the terminal control codes along with the text. For example, a single carriage return which sends the cursor back to the start of the line to overwrite that line with new status information, will display in the script file as ^M along with the repeatedly overwritten line, replicated as many times as it was repainted onscreen.

Tera Term also can log to a file but suffers from the same limitations. A better solution if using screen is to toggle logging with ctrl-A H. That H is upper-case.

We also discussed the rammifications of becoming SuperUser and using the SuDo command, discussed in the page corresponding to the link given here.

In study group, We discussed the importance of the /var directory and to help me remember I created the page WhatsInSlashVar.

In lecture and studygroup, we reviewed Archive And Compression Utilities a page I have fleshed out complete with a sample backup script.

To create a new user on Ubuntu type
adduser username
as in
adduser joseph
and you'll be prompted for the remaining user information, such as full name and phone numbers. You'll have to enter the password for the newly created user twice.

To add a user to a group, simply type
adduser username groupname
as in
adduser bob audio
to add Bob to the audio group.

We spent quite a bit of time on learning the different ways to update the system, and this is covered in Basic Package Management.

We discussed wget, a command-line tool for fetching files from web servers. It makes it easy to mirror part or all of a site, or grab a few files you need to download from the web. For an example:
wget www.sample.com/files/samples/sample.zip

would download sample.zip from sample.com or
wget http://example.edu/docs/example.txt
will download example.txt from example.edu.

You need to specify the exact url for a single file if that is all you wish wget to download. The web is full of tutorials on using wget with more advanced parameters.

We examined files using several tools. We're reminded that
ls -ld

shows directory information and
ls -la
shows file information and
ls -l
shows all information and that this information is in the so-called "long" format, with permissions, owner and group, size and various dates.

The -a for ls shows the "hidden" files, those that start with a period. These dot files are typically configuration text files, for example .lynxrc is the user-specific configuration for Lynx.

The "file" command, tells us about a file, for example that it is text with long lines, an MP3 or an executable binary or a Perl or bash script.

The flip command will change line endings in place, converting to and from Mac, DOS, Windows and Unix/Linux. Linux/Unix use line-feedcharacters. Window/DOS use carriage returns fillowed by line feeds. The Mac and incidentally the Braille 'N' Speak use carriage returns only.

You can also use The fmt command which formats (word-wraps) long lines in a file and depending on how the paragraphs are formatted, it can make a file easier to read in Linux, or mess up formatting.

The need to master the basics of regular expressions was emphasized and so I've started this page on Regular Expression Basics and will continue to refine and improve it as time goes on.

Finally the ability to efficiently edit configuration files on Linux was emphasized and it was suggested we truly attempt to master the power of the Ed editor.

For this effort, I'm working on the three-part tutorial:
Getting Your Feet Wet With Ed