Tutorial: The Unix Shell for VFX Arteeests

If you're using a Unix based operating system such as Linux then you're going to need to know your way around a unix shell. The shell is where you manage your files, and run utilities and applications.

As an artist you may think you don't need to know about the shell, that it is 'too techie' or a waste of your time. However, the truth is that with a bit of basic knowledge you'll be able to get your work done much more efficiently than you ever could with a GUI based file manager, especially when dealing with image sequences. And with OSX now providing the option of a slick interface over the top of a Unix core and shell, you can have the best of both worlds.

This guide starts at the very basics, but there are some nifty 'power tips' later on, some real time savers I wish I had known earlier. So if you've already got some experience with unix shells don't be afraid to skip forward.

Tcsh and Other Shells

This tutorial is written for the 'tcsh' shell, which is not the default unix shell, but seems to be very popular, particularly in the VFX industry, and it is the shell I prefer because of a few handy features. Other shells include the original 'sh', and 'bash' which is the default shell under most Linux systems. The shell you use is a setting stored along with your password, which is either in the /etc/passwd file or on a special server on your network (ask your sysadmins for help here).

To find out which shell you are currently using, run echo $SHELL, which should then report something like '/bin/tcsh' or '/bin/bash'.

Most things work exactly the same between all the shells, but certain things such as setting environment variables are slightly different.

Directory Navigation

The shell has a 'current directory' that you are working in. The commands you execute will assume you're operating on the files contained within the current directory (unless you supply full path names). To see what directory you are currently in, use the pwd command (pwd is short-hand for 'print working directory' - most unix commands have very short names as you are frequently typing them). When you execute pwd it will display the full path name of the directory you're currently working in. When you start a new shell the current directory will be set to your home account directory.

You navigate to other directories using the cd command (change directory). If you run this command by itself it will return you to your account's home directory, otherwise you specify a path name after the command to move to that location, for example: cd /usr/local/aw/maya6.0. That example used an 'absolute path' which is a full path name starting from the root directory (i.e. the path starts with the '/' character, which is the top of your directory tree). You can also cd by a 'relative path', which is a partial path name relative to your current directory. For example, if you were in a directory and it contained a sub-directory called 'shots' you could do: cd shots and you would then be in that directory.

There are two special directory names used in relative paths. The first one is '..' (two dots) which represents the parent directory, and the other is '.' (single dot) which represents the current directory. So, in our previous example you had moved into a sub-directory named 'shots' and you wanted to go back one level you simply type: cd ... There is another handy directory name called '~' (tilda) which represents your home account, so running 'ls ~' lists the files in your home directory. You can also add a username to this to represent someone else's home account, for example '~jsmith'. Note though, that the '~' character only has meaning within the shell, so you won't be able to use it in file browsers of applications.

Finally, there is a handy shortcut for returning to the directory you were previously in, which is cd -. For example, if you are in your home account, then do cd /usr/local/aw/maya6.0, you can then type cd - and you will be returned to your home account again.

Listing Files

You list files with the ls command, which by default shows you a list of all non-hidden files in the current directory. You can also give it a directory as an argument (e.g. ls /tmp), and it will show you the files in that directory instead.

The listing will only contain file and directory names by default, so to get more info you add the -l flag (for 'long' output), which will then show you file sizes, last modification dates, file owner and permissions:

-rw-r--r--    1 chapman  liquidmaya     9707 Mar 30  2004 liquid.jpg
-rw-r--r--    1 chapman  liquidmaya     5076 Mar 30  2004 sflogo.png

You can also sort the list based on most of these attributes. The most common one I end up using is 'ls -ltr' which lists the files with all their details (-l), in order of their time/date stamp (-t) but listed in reverse order (-r), so you can see the most recently modified files at the bottom of the list.

The file sizes are shown in bytes by default, all very technical, precise and unixy, but not so helpful, so try adding the -h flag, for 'human' mode, which will show you file sizes in kilobytes, megabytes and gigabytes.

Finally, if a file starts with the period/dot character, it is considered to be a hidden file, and isn't normally shown in an ls listing. If you want to see these files, add the -a (all) flag. Most configuration files are 'dot files'.

Copying, Moving and Deleting Files

To copy a file from one place to another, use the cp command (short for copy), with two arguments, the first is the source file, the second is the destination directory or filename. You can copy more than one file at a time, in this case the last argument must be a directory, and any preceding filenames are the source files to copy. See the section on wildcard matching below for specifying many files to copy by pattern matching.

If you want to copy whole directories, including all the files and subdirectories they contain, you need to specify the -R argument to copy, e.g. 'cp -R /tmp/some_directory /home/users/me/'

If you want to move files instead of copying them, just substitute mv (short for move) instead of cp.

To delete files, use the rm command (short for remove), with one or more names of files to delete as arguments. To delete an empty directory, use the rmdir command. To recursively delete a directory and everything under it, use rm -rf and one or more files or directories as arguments. The flags passed here are -r for recursive, and -f for force, which will delete all files, even if they are write protected.

Wildcard Matching

Most unix commands can take multiple files as arguments, and rather than listing each filename you can specify a pattern to match, known as a wildcard. You might already be familiar with the simplest form of wildcard, the asterisk (*) which matches everything, most commonly used to list, delete or copy all files with a certain extension, e.g. 'rm *.jpg' will delete all files with the jpg extension.

It is important to understand though, that the commands themselves do not usually know anything about wildcards, they just know how to accept multiple filenames as arguments. It is the shell itself that expands the wildcards into a list of filenames, and passes that list off to the commands. This idea is commonly overlooked, but helps to explain a lot of things and makes the process of dealing with wildcards a lot easier to understand in the long run, for example knowing when to wrap your wildcards in quotes so that they are not expanded on their way to the commands, such as with the find command mentioned below.

Here is a quick outline of the most common forms of wildcard matching, and some examples:

Any combination of the above symbols can be strung together to create some quite complex wildcard patterns. Note however, that these wildcard expressions are quite simple, and vary considerably compared to regular expressions (or just regexps) which can be used to match very complex expressions, usually in search and replace functions of text editors or scripting languages. We won't be covering regexps in this article though.

If you're making anything but the simplest pattern to match files for deletion or something else that is potentially destructive, you should always test the match with an ls command first.