Stufftar Revisited

In Overload 132, I presented stufftar, a program used by me to backup key parts of my filesystem [Bruntlett16]. Since then, I have occasionally had the need to find out: what has changed in my filesystem since a particular backup file was created? It remained a ‘would like to have’ option until I was experimenting with the GNU find command line utility. It has a -newer option which, while searching a sub-directory, causes it to list any files newer than a reference file. So this command will list any files in my TECH-Manuals folder newer than the file TECH-Manuals.tar.gz:

  find ~/TECH-Manuals/ -newer ~/TECH-Manuals.tar.gz

Because I wasn’t sure I could remember the name of the -newer option, I wrote a bash shell script inspired by the above command. And wrote it so that I could pass additional parameters to find. The script is in Listing 1 and is called like this:

#!/usr/bin/env bash
# Ian Bruntlett, 20th April 2021 - 7th July 2021
# newtar - file to list entries in a tar file 
# newer than the date of a 2nd file
# Based on this command: find ~/TECH-Manuals/ 
# -newer ~/TECH-Manuals.tar.gz
# 2021-07-07 - updated using advice from
# shellcheck
function Usage
{
  echo Usage: "$0" dir-of-source-files tar-file-that-backed-up-that-dir
  echo Example: "$0" Desktop Desktop_02_April_2021.tar.gz
  echo Note: Extra parameters passed on command line are passed on to find
}
if [ $# -lt 2 ]; then
  Usage >&2
  echo Incorrect number of parameters - at least 2 expected, received $# >&2
  exit 1
fi

if [ ! -d "$1" ]; then
  Usage >&2
  echo ERROR: "$1" is not a directory >&2
  exit 1
fi

if [ ! -f "$2" ]; then
  ERROR_CODE=$?
  Usage >&2
  echo ERROR: "$2" is not a file \($ERROR_CODE\) >&2
  exit 1
fi

STARTING_POINT=$1
TIME_REFERENCE=$2
shift
shift
find "$STARTING_POINT" -newer "$TIME_REFERENCE" "$@"
			

  newtar ~/TECH-Manuals ~/TECH-Manuals.tar.gz

The script requires two parameters – the path to the backed up directory and the name of the backup file. For example, given the directory TECH-Manuals and a backup of it, TECH-Manuals.tar.gz, running:

  find ~/TECH-Manuals/ -newer ~/TECH-Manuals.tar.gz

currently gives this output:

  TECH-Manuals
  TECH-Manuals/tm-processes.html
  TECH-Manuals/tm-index.html
  TECH-Manuals/tm-processes.html~
  TECH-Manuals/tm-index.html~
  TECH-Manuals/tm-platforms.html
  TECH-Manuals/tm-platforms.html~

That is useful – but what if you want to know more about those files? You can pass in printf options. This example lists when the new files were last updated:

  newtar TECH-Manuals TECH-Manuals.tar.gz -printf
  "%t %p\n"

which gives this output:

  Sun Jul 18 21:16:08.7651258860 2021 TECH-Manuals
  Wed Jul  7 18:35:57.6009844760 2021 TECH-Manuals/tm-processes.html
  Sun Jul 18 21:16:08.7651258860 2021 TECH-Manuals/tm-index.html
  Wed Jul  7 18:06:16.9915321630 2021 TECH-Manuals/tm-processes.html~
  Fri Jul  9 16:30:47.2928147850 2021 TECH-Manuals/tm-index.html~
  Wed Jul  7 18:34:51.5270892980 2021 TECH-Manuals/tm-platforms.html
  Wed Jul  7 18:05:39.4933397510 2021 TECH-Manuals/tm-platforms.html~

Since I wrote stufftar, I have learned about the shellcheck program that critiques shell programming style. This new script is quietly accepted by shellcheck.

A system call surprise

It seems that every Linux expert says “it’s in the man files” but I was always curious to know: how do you find out about what you don’t know exists? The apropos command (a synonym for man -k) can be used to find about things related to a topic. For example, to find out things related to the Hewlett Packard Printing and Scanning utilities, I use this command:

  apropos hp-

which yields this (abbreviated) output::

  hp-align (1)        - Printer Cartridge Alignment Utility
  hp-check (1)        - Dependency/Version Check Utility
  hp-check-plugin (1) - AutoConfig Utility for Plug-in Installation

So I thought: what if I used a regular expression? I tried this command:

  man -k "[a-z]"

and got a lot of output – 7,519 lines worth. And then I thought… what if I restricted the output to a particular section of the man pages using the -s option? To refresh the reader’s memory, here is a quick summary of the section numbers:

1. Executable programs or shell commands
2. System calls (functions provided by the kernel)
3. Library calls (functions within program libraries)
4. Special files (usually found in /dev)
5. File formats and conventions, e.g. /etc/passwd
6. Games
7. Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)
8. System administration commands (usually only for root)
9. Kernel routines [Non standard]

So running this command lists all the pages on system calls:

  man -k "[a-z]"  -s 2

yielding 496 lines of output, abbreviated here:

  _newselect (2) - synchronous I/O multiplexing
   _Exit (2)      - terminate the calling process
  __clone2 (2)   - create a child process
  _exit (2)      - terminate the calling process
  _llseek (2)    - reposition read/write file offset
  _syscall (2)   - invoking a system call without library support (OBSOLETE)
  _sysctl (2)    - read/write system parameters
  accept (2)     - accept a connection on a socket
  <snip!>
  vserver (2)    - unimplemented system calls
  wait (2)       - wait for process to change state
  wait3 (2)      - wait for process to change state, BSD style
  wait4 (2)      - wait for process to change state, BSD style
  waitid (2)     - wait for process to change state
  waitpid (2)    - wait for process to change state
  write (2)      - write to a file descriptor
  writev (2)     - read or write data into multiple buffers

Again, I took this as an opportunity to write a supporting shell script so I wrote the one in Listing 2:

#!/usr/bin/env bash
# 2021-07-11 Ian Bruntlett
# 2021-07-14 Tidying up heredoc in "Usage" 
# function.
# Name   : man-section
# Purpose: To dig around the man pages

function Usage
{
cat <<END-OF-USAGE-MESSAGE
Usage: $0 section-number (from 1 to 9, optional)
       1   Executable programs or shell commands
       2   System calls (functions provided by 
the kernel)
       3   Library calls (functions within 
program libraries)
       4   Special files (usually found in /dev)
       5   File formats and conventions, e.g. /etc/passwd
       6   Games
       7   Miscellaneous (including macro 
packages and conventions), e.g. man(7), groff(7)
       8   System administration commands 
(usually only for root)
       9   Kernel routines [Non standard]
END-OF-USAGE-MESSAGE
}

if [ $# -eq 0 ]; then
  Usage >&2
  exit 0
fi

while [ "$1" != "" ]; do
  man -k "[a-z]" -s "$1"
  shift
done
			

The added bonus of the above bash script is that it can list the contents of multiple sections. So the command man-section 1 8 will list executables for the user (section 1) and for administrators (section 8).

The command man-section {1..9} will list pages for all 9 sections. This gives a strange warning message on my system as there are no installed pages for section 9 on my system.

I know its a cliché but I am perpetually hopping inside a small collection of languages, learning new things and reinforcing existing knowledge. I have been advised that one way to achieve this is to have side projects to exercise knowledge to avoid losing it. I bought some revision flash cards from WH Smith’s and am collecting bash questions for me to answer in the future. As time goes by, I’ll collect questions for the other languages. I am aware that the shell scripts here are quite basic but I believe that developing them has helped improve my bash skills and possibly provided people with a couple of useful tools.

References

[Bruntlett16] Ian Bruntlett (2016) ‘Stufftar’ in Overload 132, published April 2016, available from https://accu.org/journals/overload/24/132/bruntlett_2226/

Ian Bruntlett Ian has been programming for some years. He also reads a lot to improve his skills. He volunteers for a mental health charity called Contact (www.contactmorpeth.org.uk) doing various things including running a Computer Wombling Project (refurbishing old computers with Linux for the members) and, perhaps, running Mongoose Traveller games there again one day.