Linux Intro Wiki | Main / Lecture9

We spent the first half hour reviewing the regular expression homework and how to solve some of the trickier problems.

We were reminded that
www.regexp.info
has excellent information. Also the info pages for grep and man are helpful.

Bookshare has the book on Awk and Sed, at
https://www.bookshare.org/browse/book/396053

as well as Unix Power Tools at
https://www.bookshare.org/browse/book/390459

There is the Ced and Awk pocket reference at
https://www.bookshare.org/browse/book/391266

and of course, Mastering Regular Expressions at
https://www.bookshare.org/browse/book/456342

Also useful, is the Regular Expression Pocket Reference
https://www.bookshare.org/browse/book/395300/

For more advanced users, the Regular Expressions Cookbook
https://www.bookshare.org/browse/book/396052/

We reviewed substitution in Ed:
s is for substitute and g is for global. An ed command line has three parts, the range, the command and the expression. So
1,3s/dog/cat

would replace dog with cat in just the first three lines. But only the first dog on each line would be replaced.

If the range 1,3 were replaced with 1,$, then this substitution would occur on every line in the file, but only the first occurance of dog on each found line would be replaced because the /g wasn't included in the command.

Note that a shorthand for 1,$ is to simply type a comma. so
,s/dog/cat
is the same as
1,$s/dog/cat
The command
s/dog/cat/g

would do a global replace, but only on a single line, the current line which is referenced by dot. Since dot is the default when no range is specified, this command would replace three occurances of dog with cat if the current line indeed contained the word dog three times.

To actually replace all occurrances of dog throughout the file, you'd need this command:
1,$s/dog/cat/g

For a more complex expression, the ampersand is used in ed, to substitute for the matched text. The ampersand is not a meta-character; it's an ed-specific indicator for the match.

For example, you want each line to begin with a number sign because you are going to make this text in to an extended comment for a script. You'd do your global search and replace like this:
1,$s/^/&#/g

We reviewed the expression
[A-Z]

to express all upper case characters and
[a-z]
to express all lower-case characters. We reviewed that the caret matches the start of the line and the dollar sign the end of the line.

We reviewed alternation and the use of parenthesis as in
(dog|cat)
to express either dog or cat.

We explored more territory with Processes And Jobs including the top command, and using grep as in
ps aux | grep kerry
to show only processes with kerry in the displayed lines. We learned how we could pipe two grep commands together, to better filter what's displayed. For example:
ls -c1
displays all the files, each in its own column (-c1 is the parameter for one column.)

Now to filter out only files beginning with an uppercase letter, this command would be
ls -c1 |grep '^[A-Z]'
But let's suppose we had some MP3 files that we didn't care about. We could use the grep -v which tells grep to *NOT* display matching lines to filter out those mp3 files. So that command would be
ls -c1 |grep '^[A-Z]'| grep -v '\.MP3'
which tells Linux to display the files that are not MP3 files and that have an uppercase letter at the start of their names.

Kerry used the double grep examples to demonstrate filtering process listings with ps -aux. He mistakenly suggests the -n option; it's in fact the -v option to "invert" the selection as the man page explains. -v says do not match lines with the given expression.

For a simple example:
ps aux | grep 'root'
displays all processes with root in their display line.
But
ps aux | grep -v 'root'
displays all processes that don't include root in their display line.

So to display all processes that have an upper-case letter somewhere in the name, but are not owned by root, you could pipe grep in to grep as in
ps aux | grep [A-Z] | grep -v root

We briefly covered the difference between a shell glob and a regular expression. A glob is also a syntax that can filter out filenames, similar but not exactly like the wildcards used in Windows and DOS.

We also defined some topics for future lectures:

apache is the most commonly used open-source web server.

postfix and exim are mail transfer agents -- programs which manage mail on Linux systems.

Samba is the set of programs which let a Linux file server pretend to the network that it is a Windows server sharing files. Smb is the network file sharing protocol used by Windows.

We also chatted in general about these types of programs and what would be covered in upcoming lectures.

We discussed web-based administrative interfacs including Webmin and CPanel. The simplify administration, for example enabling a user to change permissions or add a new user by clicking a few buttons, but they also lock users out of more complex administrative tasks.

Virtual hosting refers to running more than one website on the same server.

There are two types of virtual hosting schemes: IP-based and name-based.

In IP-based virtual hosting, each site has its own IP which is mapped in the A records of the DNS server. In name-based virtual hosting, the browser asks the web server which site is being requested. The server hands out pages based on the name that the browser specified. The page request from the browser comes with a "host header" that tells the web server which virtual domain the browser is requesting.

Returning to mail transfer agents, we learned they were responsible for sending, receiving email, plus transferring email to and from the internet and other networks. It is not a pop or imap server. It is not the program that actually displays the inbox and lets users compose mail. Programs that display email and let users compose, save and filter messages are called Mail User Agents.