Viewing Files

You can print the contents of a file to the shell with the cat command. However unless it is a very short file, it is most likely going to scroll off the top of the terminal and you'll only see the end of it. If you want to view a file a screenfull at a time, and be able to move back and forth, and search for things then use either less or more. I'm sure at one point there was major feature differences between these, but they seem pretty interchangeable to me these days. When you first run either one on a file, you'll see the first 'page' (screen or terminal window full) of the file. Press the up/down keys to move a line at a time, or the space key to go a page at a time. The 'q' key will quit out. If you want to search or jump to some known text, hit the forward slash key ('/') and enter some text and press return.

If you're only interested in the first few lines of a file you can run the head command on the file, and it will show you the first dozen lines. You can give it a -NN flag, where 'NN' is a number, and it will show you that many lines instead. The tail command works exactly the same way, but shows you the end section of a file, very useful when looking at output/error logs.

If you want to know if two files are the same, you can run the diff command, listing both files as arguments. This will tell you if the contents of the two files match, and show you any differences in a pretty cryptic way. Most systems now have a graphical diff utility installed, such as xxdiff or kompare which will show you the two files side by side, with the differences highlighted in a more intuitive way. Diffing is pretty handy when you have one thing that you know works, and one that doesn't, and you want to see what the difference is (for example, an ascii RIB file that renders correctly and one that produces a black frame).

Editing Files

You can create or modify text files with one of the many text editors installed on a unix system (it seems everyone has their own favourite). There are two types of editors: those that run within the shell, and those that bring up a new window and can be used with the mouse. You start all editors the same way, with the name of the editor, and the name of the file to create or edit, e.g. 'nedit some.rib'

One of the simplest and most widely available graphical editors is nedit. Anyone familiar with Windows or the Mac OS should be able to use nedit with no problems at all, but if you look through the menus you'll find it also has some quite advanced functionality .

If you're running Linux with the KDE desktop installed, then a good alternative is kate, which I prefer, as it allows you to have many different files open at once, and easily move between them.

If you don't have one of these two editors installed, or if you need a non-graphical editor, then you'll have to use one of the "old-school" unix text editors. Although quite powerful, they can be very difficult for new unix users. The standard editor is vi, and a much more elaborate editor is emacs. Both of these are very powerful, but rely on the user knowing how to switch between text input and command modes, and various keyboard shortcuts for running commands, which is all a bit daunting when you're just starting out. There is a very simple alternative, which is the pico editor, installed on most systems.

Finding and Searching within Files

If you're looking for a file somewhere on disk, you can use the find command to locate it. This command is a little tricker to use than you might expect, because it is very flexible, and can be used to find files based on all sorts of attributes, not just their name. To use it to find files by name, use the syntax 'find DIR -name 'NAME'', where 'DIR' is the name of a directory to start the search from (use '.' for the current directory), and 'NAME' is an exact filename or wildcard pattern. Because we want the wildcard pattern to be passed through to the command, rather than being expanded in the shell and passed through as multiple filenames we wrap it in single or double quote characters. If you only want the search to operate on the local physical drives of your computer, rather than across network drives, add the -mount flag. You can add many more flags to this command to search by a file's date, owner, permissions and other things too. Use the 'man find' command to get further info.

Though the find command can be used to locate files based on their name or attributes, if you want to match files based on their content then you'll need to use the grep command, specifying the text to search for and then one or more filenames, like this: 'grep shadowMap *.rib' - this will search for the word 'shadowMap' in all the .rib files of the current directory. The output of the grep command will list each line in the file(s) matching the text. You can also pass it flags to have it just output the name of the file it was found in (-l), or files it was not found in (-L). Use 'man grep' to get more complex usage information.

File Permissions

Hopefully if your work environment is setup well, you won't need to worry too much about file permissions, and the files you and your colleagues create will automatically be readable and editable by the right people. However, it pays to know a little about permissions, most commonly to be able to protect important files from being accidently overwritten.

If you do an ls -l on a file or directory, you'll see the permissions of the files listed in the first column:

drwxr-xr-x    2 chapman  users        4096 Jan  9  2005 backup
-r--r--r--    1 chapman  users           0 Jan  9  2005 important.txt
-rwxrwxrwx    1 chapman  users           0 Jan  9  2005 some.rib
-rw-r--r--    1 chapman  users           0 Jan  9  2005 some.sl

The first character in the permissions column is usually either a 'd', meaning it is a directory, or otherwise '-' for a normal file (commonly also 'l' for a file link, but we won't cover links in this article). The next 9 characters specify the read, write and execute permissions, first for the owner of the file, then for those in the same user group as the owner, and finally for everyone else. The 'r' or 'read' permission on files means you can read their contents, 'w' or 'write' permission means you can modify them, and 'x' or 'execute' permission means you can run them if they are executable files or scripts.

You can change the permissions of a file with the chmod command, but you can only change permissions on a file you have 'write' access to, otherwise it would defeat the whole purpose. You pass a flag with a '+' to add or '-' to revoke a certain permission, followed by the type of permission (i.e. 'r', 'w' or 'x') and then the name of one or more files to affect. For example, to remove write permission from all files in the current directory you can run 'chmod -w *'. Specifying permissions this way will change them for all three access levels (owner, group and everyone). You can be a little more precise by placing 'u' for owner, 'g' for group or 'o' for everyone in front of the '+'/'-' flags. You can add one, two or all three of these letters together. This example is the same as the previous one, but it will only protect the files against writing for 'group' and 'everyone' - the owner will still be able to write to the files: 'chmod go-w *'. You can also specify permissions with a numerical system (e.g. 'chmod 777 *') but I won't go into that here.

Linked with the three levels of permissions, files also have the name of their owner and group associated with them. In the 'ls -l' output above, all the files are owned by 'chapman', and are in the group 'users'. To change the owner of a file, use 'chown OWNER FILES', where 'OWNER' is the login name of the new owner, and you can specify one or more files to change after that. Similarly, you can run 'chgrp GROUP FILES' to transfer one or more files to another ownership group. Be careful though: if you transfer ownership so that you no longer have write access to the file, you won't be able to transfer it back to yourself or change permissions on it.

You can change permissions or ownership of files in a recursive way, but passing the -R flag to chmod/chown/chgrp and naming a directory.

The permissions that newly created files have by default are determined by the umask command, which is a bit complex to explain here (it uses the numerical permissions system) but if you need to modify that behaviour run 'man umask' to get more info.

'Foreach' Looping

This is a fairly advanced command, which normally I wouldn't be covering in an introductory guide like this, except that it is so incredibly powerful when you're dealing with sequences of images, or other file sequences we commonly have to deal with in our industry.

With the tcsh shell, you can run the foreach command to operate on a bunch of files at a time. After running the command, specifying a list of files, it then asks you for one or more commands to run on each file. You stop specifying commands by entering 'end'. Here is a simple example:

foreach file (a.jpg b.jpb c.jpg)
  foreach? cp $file $file.bak
  foreach? end

So, we have called the foreach command with the names of three files ('a.jpg', 'b.jpg' and 'c.jpg'). The second argument is telling the command what we want the name of the file to be as we loop through the list. In this case we say to call it 'file', and in the commands below we refer to it by '$file'. If you need to embed the variable in the middle of some text, wrap it in curly braces, so {$file} in this example. After running the command it starts prompting us for things to do with each file as it loops through them, and we say we want to copy the file to a file with the same name, but with '.bak' tacked on the end. We then tell it we're done specifying commands to run by typing in 'end'. The end result of all this is that the following commands will be run:

cp a.jpg a.jpg.bak
cp b.jpg b.jpg.bak
cp c.jpg c.jpg.bak

In other words, we're making backup copies of the files specified. That is a simple example, but not too useful, as we need to specify all the files to the command. However, you can put wildcard patterns within the parentheses, or even the output of other commands like ls or find by wrapping them in single back-ticks ('`'). Here is a more complex example, which uses the find and grep commands we mentioned earlier to list all the RIB files under the '/tmp/' directory that contain the text '/job/troy/':

foreach file (`find /tmp -name '*.rib'`)
  foreach? grep -l /job/troy $file
  foreach? end

So hopefully you're starting to see the power of the unix shell - that various commands can be hooked together to solve all sorts of problems. In this example we're just searching through files, but we could be doing literally anything to them - copying, converting, rendering, etc.