diff

Diff remote files using ssh in Linux:

-----------------------------------------------------

I have already discussed how we can edit a remote file using vi and scp in one of my previous post ; today we will see how we can find or show differences between a local file and a remote file using ssh.

Suppose we have to find the differences between local file "/tmp/filepurge.sh.old" and remote file "/root/scripts/filepurge.sh" located in remote host 172.21.16.11.

This is how can do it:

$ ssh root@172.21.16.11 "cat ~/scripts/filepurge.sh" | diff - /tmp/filepurge.sh.old

And using vimdiff:

$ vimdiff scp://root@172.21.16.11//root/scripts/filepurge.sh /tmp/filepurge.sh.old

** We would need ssh to work using public key authentication so that we can do remote commands execution without being prompted for passwords.

cat

Bash cat command space issue explained:

-----------------------------------------------------------

Input file contains some 4 student names like this:

$ cat file.txt

Alex C M

Peter S

Dhiren K

Prahlad G N

Required: I was trying to produce the following output:

1) Alex C M [3]

2) Peter S [2]

3) Dhiren K [2]

4) Prahlad G N [3]

i.e. a serial number, Name of the student, number of words in his name.

Lets try with bash for loop like this:

$ cat lp1.sh

#!/bin/sh

c=0

for line in $(cat file.txt)

    do

        ((c+=1))

        numfields=$(echo $line | awk '{print NF}')

        echo "$c) $line [$numfields]"

done

And the output it produced !

$ ./lp1.sh

1) Alex [1]

2) C [1]

3) M [1]

4) Peter [1]

5) S [1]

6) Dhiren [1]

7) K [1]

8) Prahlad [1]

9) G [1]

10) N [1]

So what went wrong ?

I tried echo "$line" as well, same output.

In the above example, we need to take care of the Bash IFS environmental variable. From Bash man page:

IFS:

The Internal Field Separator that is used for word splitting after expansion and to split lines into words with the read built in command.

The default value is

<space><tab><newline>.

And since the lines in the input file got lines with spaces in between, above script is behaving in that way.

We can temporarily change the IFS in the shell script like this:

$ cat lp3.sh

#!/bin/sh

OLD_IFS=$IFS

IFS=$'\n'

c=0

for line in $(cat file.txt)

    do

        ((c+=1))

        numfields=$(echo $line | awk '{print NF}')

        echo "$c) $line [$numfields]"

done

IFS=$OLD_IFS

Output:

$ ./lp3.sh

1) Alex C M [3]

2) Peter S [2]

3) Dhiren K [2]

4) Prahlad G N [3]

Bash 'while loop' used in the below way also works without changing the IFS:

$ cat lp2.sh

#!/bin/sh

c=0

while read line

    do

        ((c+=1))

        numfields=$(echo $line | awk '{print NF}')

        echo "$c) $line [$numfields]"

done < "file.txt"

Output:

$ ./lp2.sh

1) Alex C M [3]

2) Peter S [2]

3) Dhiren K [2]

4) Prahlad G N [3]

Note: The above example is taken mainly to show the use of Bash IFS variable. Using awk, the above can be done easily like this:

$ awk '{print NR")",$0,"["NF"]"}' file.txt

$ awk '{++c}{print c")",$0,"["NF"]"}’ file.txt

less

Print line number with unix less command :

--------------------------------------------------------------

Less command introduction:

As you know Unix less command writes the contents of a file onto the screen a page at a time and this is one of the utilities using which one can view the content of a file without opening it in an editor.

Press the [space-bar] if you want to see another page, and type [q] if you want to quit reading.

This is how we can print line number with Unix less command.

1)

From man page of CAT(1):

       -n, --number

              number all output lines

So,

$ cat -n file.txt | less

will print the line number in-front of each line.

2)

From man page of LESS(1):

       -N or --LINE-NUMBERS

              Causes a line number to be displayed at the beginning of each line in the display.

So,

$ less -N file.txt

will do the same as 1) above.

3) One can set the following to print line number with less by default:

$ export LESS='-RS#3NM~g

exec,find

Execute multiple commands with exec and find:

--------------------------------------------------------------------

Is it possible to execute multiple commands using exec on the Linux/UNIX find command output ?

The answer is "yes". Each -exec action is to be associated with a escaped semi-colon (\;)

e.g.

I had to find files named "1251936000.log" and then need to perform two actions on it:

- Count number of lines in the file.

- Do a "ls -l" listing of the file.

And the find command I wrote with exec:

$ find . -name 1251936000.log -exec wc -l {} \; -exec ls -l {} \;

Output:

   6924 ./lv1/1251936000.log

-rw-r--r-- 1 root root 977264 Sep  4 00:17 ./lv1/1251936000.log

List empty directories using find in bash:

---------------------------------------------------------------

Linux command find gives an option called "empty" using which we can list empty regular files or empty directories.

e.g.

To list all the empty directories

$ find . -type d -empty

Output: 

./bdb/prac

./sim/old/data

./prac/testd

Linux find command and logical operators :

---------------------------------------------------------------------

Some of the example to show how we can use logical operators with linux find command.

To find any file whose name ends with either 'sh' or 'pl'

$ find . -type f \( -iname "*.sh" -or -iname "*.pl" \)

To find .txt files that are writeable by "others"

$ find . -type f \( -iname "*.txt" -and -perm -o=w \)

To find .txt files but exclude the ones which are writeable by "others"

$ find . -type f \( -iname "*.txt" ! -perm -o=w \)

or

some find versions support "-not" as well

$ find . -type f \( -iname "*.txt" -not -perm -o=w \)

Remember: The parentheses must be escaped with a backslash, "\(" and "\)", to prevent them from being interpreted as special shell characters.

wait

Bash wait command:

-----------------------------------

wait command stop script execution until all jobs running in background have terminated, or until the job number or process id specified as an option terminates.

It returns the exit status of waited-for command.

wait can take the job-id or the process number. i.e.

wait%1 or wait $PID

_________

wait ${!}

_________

wait ${!} means "to wait till the last background process is completed" ($! being the PID of the last background process)

________________________

An example on wait command.

________________________

Suppose

- you have a script called sort_db.sh which sorts some data files and takes a lot of time to complete(you definitely want it to run as background process in your script)

- One more script called bkptmp.sh, which does some job of backing up some tmp files(nowhere related to the above sort_db.sh)

- You have to perform some tasks in your script after sort_db.sh and bkptmp.sh complete their individual tasks (Note: both the .sh should be completed before you perform the said operation)

$ cat waittest.sh

#!/bin/sh

./sort_db.sh &

echo "1st Line"

./bkptmp.sh &

echo "2dn line"

wait

echo "Some operation will follow this"

...

...

In such situations the "bash wait command" is useful. It will wait till sort_db.sh and bkptmp.sh get complete their execution.

You might be thinking, we could have run sort_db.sh and bkptmp.sh in foreground, so that the execution of the operation will follow them. The problem is that you don't want to wait for sort_db.sh to complete for bkptmp.sh to start (I told earlier, they are not dependent). So using wait, the time sort_db.sh gets completed, we will be done(or almost done) with bkptmp.sh.