Disk Usage

From time to time you'll want to check on how much of your local or network drives you've still got free. By running the df command (disk free) you can see the size, usage and free space on all your local physical drives, and any network share drives that are currently mounted. If you run it with the -h option ('human' format output) you'll get sizes in kb/mb/gb. The output will look something like this, which should be pretty self-explanatory:

Filesystem            Size  Used Avail Use% Mounted on
/dev/md0              514M  293M  195M  61% /
/dev/md1              514M   95M  394M  20% /tmp
/dev/md2              2.0G  1.5G  444M  78% /var
/dev/md3              4.0G  1.3G  2.5G  35% /usr
/dev/md4               27G   12G   14G  45% /var/local

That lets you know about your overall disk usage situation, but if you want to know how much disk a certain section of your file hierarchy is using, you'll need the du command (disk usage). Running it with no arguments will list the size of all directories within the current directory, or you can pass it the name of one or more other directories to list. The number next to each entry is normally in kilobytes by default, but again you can pass it the -h flag for 'human' output. If you're interested in the total size of a single directory, pass the -c flag, which will print the total size in addition to the size of any subdirectories.

Processes

Unix gives you quite a lot of control over the processes that are currently running on your machine (provided you have permission to mess with them - usually only your own processes). You can easily kill off processes (e.g. if they aren't behaving), pause and restart processes, move processes from the background to the foreground in the shell, and easily monitor how much system resources processes are taking.

To list the processes running on your machine, run the ps command. By default this will show you all the processes your login is running in the current shell. By specifying the -A flag it will list all processes running on the machine you're logged into. This command is mostly useful for checking whether a certain process is running (e.g. "I'm using a Maya license, but I don't see Maya open - is it hung or running in the background?"), or seeing which shell you launched something from.

To see what is using resources on your machine, you'll want to run the top utility. This shows you the top CPU hogging processes currently running, with info about them. By pressing the 'M' key (capital 'm') you can see the top processes ranked by memory usage instead of CPU usage. If you only want to see your own processes, hit the 'u' key and type in your login name. You can see how much memory and percentage of the CPU(s) each task is taking, and also up the top some overall summary information (system load, free memory). Press 'q' to quit.

When you run a command in the shell, it normally will keep control of that shell, so you can't do anything else there until the command has exited. If you want to run a command and retain control of the shell, all you need to do is append the '&' character to the end of the command before pressing return. If you've already run a command which has taken the shell, you can temporarily suspend it by pressing 'CTRL-Z'. You then have a choice of starting the command back up again in the 'foreground' mode (taking the shell) by running the fg command, or running it in the background with the bg command.

If you have a command running that you want to abort, you can press CTRL-C which should kill it.

If there is a process running on your machine that you want to stop, you can abort it with the kill command. In order to issue this command to a process though, you'll need to know the process ID, which you can get from either ps or top. Then just run kill PROC_ID, where 'PROC_ID' is the process ID to kill. By default the command tries to kill the process off in a nice and polite way, sending it a signal asking it to close itself down. However, if you have a stubborn process which is perhaps out of control, you might want to run kill -9 PROC_ID instead. The '-9' is a type of kill signal, which means the operating system will intervene and forcibly remove the process. However, you should be careful with this, as it doesn't give the process any chance to shutdown gracefully, save data, etc.

You can also stop all processes with a certain name by using the killall command. So running something like killall prman will interrupt all running instances of the PRMan renderer currently running. This is handy if something gets out of control and starts spawning lots of the same type of process (perhaps you've just done 'nedit *' in a directory full of files), or if you want to stop a process and you're sure there is only one with that name running, and you don't want to have to go and find its process ID.

If you have a graphical application running that you want to kill, you can run xkill, then when the cursor changes shape to a small rectangular icon just click on the window to kill. Running the normal kill or killall utilities also works for graphical programs, but using xkill you don't need to know the process name or ID, which is much easier.

File Archives: tar, gzip and unzip

Most archives under unix are in the TAR format, which was initially developed for tape archiving. A tar file can either be compressed (using gzip compression), or just be a collection of files mashed together. A compressed archive normally has the extension '.tar.gz' or '.tgz', and an uncompressed archive has '.tar'.

To extract a tar archive, run 'tar -xf FILE'. The 'x' flag means to extract, an the 'f' specifies that we're dealing with a file (without it tar assumes you're dealing with a tape drive), and 'FILE' is the name of your archive, which must come right after the '-f' flag. If it is a compressed archive you'll also need to pass the 'z' flag too.

To create a TAR archive, you'll need to use the '-c' flag (for 'create' mode) instead of the 'x' flag above, and you'll also need to specify the names of one or more files or directories to put into the archive. This example creates a compressed archve (because of the 'z' flag) of all the files and directories under '/tmp' and puts them into an archive called 'tmp_backup.tar.gz': 'tar -czf tmp_backup.tar.gz /tmp/*'

You might also have a single file which has been compressed, and this would normally have a '.gz' extension. To extract the file from this just run 'gzip -d FILE'. The -d flag means you're decompressing a file. If you want to compress a file, leave that flag off and just run 'gzip FILE' which will yield you a compressed file, now with a '.gz' extension.

If you've got a ZIP file from the Windows world, you can decompress it with unzip FILE. You can create a zip file with the zip command. The same example from above, but to create a ZIP file this time is: 'zip tmp_backup.zip /tmp/*'

Redirecting and Piping

Normally when you run a command, the output is printed to the shell. However, you can tell the shell to redirect that output to a file, by adding the '>' character to the end of the command with a filename to write to. For example, this will create a list of all the files in the current directory, and put that list in a file called 'my_files': 'ls > my_files'. If you want the redirection to add to the end of an existing file, rather than overwriting the file, just use '>>' instead of a single '>'.

Redirecting is handy if you want to save a copy of some output. Another case is if you just want to suppress all output you can redirect it to a special file device called '/dev/null'. Anything sent there will simply disappear into the digital abyss. A handy use of redirecting to a file is to produce small batch files. You can redirect an ls command like in the example above, then edit that file to add commands to be run on the files, then execute that script. This is similar to the foreach looping mentioned previously, but a little safer, because you can see and edit exactly what is to be run, and also it can be saved off for running again in the future.

As well as redirecting the output to a file, you can redirect the output straight to the input of another command. This is known as piping commands together, and is done with the '|' character. For example, if you wanted to see if the user 'jbloggs' was logged into your computer, you could run who, and manually scan the output, or you could run 'who | grep jbloggs' and it will pass the output of the who command to the grep command, which will then search for 'jbloggs'. There is no limit to the number of commands that can be strung together, and unix has many handy little utilities which are all designed to be used in this way.

Environment Variables

An environment variable (a.k.a env-var) is a single variable name and value that you can set to control programs that are run in the shell. Common uses are to tell programs where to find the files they need (e.g. MAYA_SCRIPT_PATH tells Maya where to find MEL scripts) or to tell programs to behave in a slightly different way to normal.

To set an env-var under the tcsh shell, you run setenv NAME VALUE. That variable can then be used anywhere in the shell by prefixing the variable name with the dollar sign ('$'). So here's an example where we'll set a new variable and print its value in the shell:

  >> setenv MOL 42
  >> echo The meaning of life is $MOL
  The meaning of life is 42

You can also get the value of a variable with 'getenv NAME', or remove a variable with 'unsetenv NAME'. To see a list of all the env-vars currently operating in the shell, run env.

Environment variables only exist within the current shell process, and are propogated down to any new processes launched by that shell. If you want to set an env-var permanently, you'll need to run the setenv command within your .cshrc file (see the 'Saving Settings in your .cshrc File' section later).

Command Aliases

Although most of the standard unix commands are very short, some other commands or scripts you run might be longer, or perhaps you want to always run a certain command with some extra flags instead of its default behaviour. In these cases it can be really handy to set a command alias. You can create an alias with 'alias NAME COMMANDS', and 'unalias NAME' to remove an alias.

Here is an example where we'll setup an alias for the ls command, so that it always outputs the file details, and lists in reverse time order:

  >> ls
  list.gz  some.rib  some.sl
  >> alias ls ls -ltr
  >> ls
  -r--r-----    1 chapman  users       12218 Jan  9 12:28 some.sl
  -rwxrwxrwx    1 chapman  users        3453 Jan  9 13:23 some.rib
  -rw-r--r--    1 chapman  users      618358 Jan 11 13:51 list.gz

You can even string multiple commands together to be run by a single alias, just separate them with ';' characters, e.g: 'alias mkshot mkdir maya; mkdir maya/scenes; mkdir maya/images'. Everything after the first word is run together when you run the aliased command.

You can list all current aliases by running alias with no arguments. Also handy is 'which COMMAND', that tells you the full path to a command, or what it is aliased to if it is an alias.

Command Editing Shortcuts

People who aren't used to using the shell think there is a lot of typing involved, but once you know a few tricks you'll find that most of the time you really aren't typing very much at all.

The most handy feature is filename completion. By typing a small section of a filename and pressing the '' key the shell will automatically fill in the rest of the name for you if there is only a single match. If there is more than one potential match it will fill in as much as it can, and wait for you to type another character or two, then press '' again to continue the matching.

Next on the list of most handy shortcuts is being able to scroll back through commands you've run previously. So unlike under a GUI interface where to perform an action twice requires exactly the same amount of effort the second time, to perform something again in the shell all you do is press the up arrow key to scroll back to the command you want, and press '' to run it again. If there is a small change to be made (e.g. a different filename), you can edit the command before pressing ''. If you want to see a list of previous commands you've run, use the history command.

If you are using the tcsh shell, there are a couple of options you can set to make these previous commands even easier to find and run:

bindkey -k up   history-search-backward
bindkey -k down history-search-forward

After putting these lines in your .cshrc file (see later section) and starting a new shell, you can start typing a few letters of a previous command, then press the up/down arrows to scroll through any matching commands you've previously run.

It is also easy to move around within a long command, making changes. Pressing 'CTRL-A' will move the cursor to the beginning of the line, 'CTRL-E' will move to the end of the line. 'ALT-F' will move forward a word, 'ALT-B' will move back a word. 'CTRL-K' will delete everything on the command line from the cursor to the end of the line.

There are a couple of symbols which you can use in a command to refer to previous commands. '!!' refers to the last command you ran, and '!$' refers to the last word of the last command you run. Most people wouldn't immediately recognise the power of this last one, but because of the way unix works, you quite often run a series of commands on the same file, and the filename is usually the last argument. So instead of having to specify the name of the same file many times over, you can just use '!$'. Here is an example:

  mkdir /tmp/imgconvert
  cd !$
  convert /tmp/image.bmp image.tif
  fcheck !$

In that example we made a directory, moved into it, converted an image into that directory and viewed the image using Maya's 'fcheck' utility. Although we ran multiple commands on the directory and on the file, we only needed to actually type the name of each once. Another way of re-using previous commands is with the '^' character. This tells the shell to run the previous command, but to replace a word in it with something else before running it. Here is another example:

  mkdir sequence001
  ^001^002
  ^002^003

This makes a new directory called 'sequence001', then the next line does the same, but replaces '001' with '002', and the third replaces '002' with '003'. A trivial example, but this is another handy little shortcut you may find yourself using over and over again.

Misc

Here's a few useful tips and commands that don't really warrant their own sections.

• It is worth explaining a little about quotes and escaping. Because everything is entered as text, the shell needs to know which part of the text is the command being run, and where each one of the arguments to that command starts and finishes. Normally this is very easy, the command is the first word, the first argument to that command is the second word, the next argument is the third word, etc, etc. However, if you have files with spaces in the name (NEVER put spaces in filenames, it will cause you no end of trouble, and you deserve every bit of it!) then each word will be considered to be a separate argument. In order to have multiple whitespace separated words considered as a single argument, you'll need to wrap them in quotation marks, either single or double (' or ").
• Escaping is a similar problem, and stems from the fact that certain characters (such as quotation marks) have a special meaning to the shell, so if you want any of these special characters to actually be used as part of a filename, you need to 'escape' them, which just means putting a single backslash character ('') in front of the special character. Note that because the backslash itself is a special character, if you want a literal backslash to be passed through to the command, you will have to escape it with another backslash.

• If you are transferring files between unix and Windows, then you will find that any text files created or edited on the Windows side will have funny characters at the end of the lines when viewed in unix. Most unix installations come with a handy little utility for correcting these files, called 'dos2unix', and the opposite called 'unix2dos'. Just run them with the name of the file to convert as the only argument.

• Normally you only run one command at a time in your shell, but sometimes it can be handy to run a couple at once. You can do this by separating each command to run with a semi-colon character (';'), and each command will be run in turn. This is most immediately handy for when you want to reorganise some files that other people might be working on, and so by running all the commands at once then the files might be in a state of change for only a fraction of a second, rather all the time that you run the commands one at a time. Perhaps even more useful is stringing together commands with the '&&' symbols. These can be used just like the ';' character, except that the subsequent commands will only be run if the preceding one was successful. For example prman my.rib && fcheck my.tif will render the RIB file using PRMan, then show you the resulting image using Maya's fcheck utility, but it will only view the image if the render was successful.

Saving Settings in your .cshrc File

If there are any commands you would like to have run each time you open a new shell (for example setting environment variables or aliases) then you should add those commands to your .cshrc file. This file exists in your home directory (just create it if need be) and any commands listed in it are run in sequence when you start a new shell.

In addition to your .cshrc file you can create other script files (also known as batch files) containing common sequences of commands you might want to run together. All you need to do is create a new file (any name you like), edit it to contain the commands you want, and give the file executable permissions (with 'chmod +x FILENAME'). You can then run the script by typing its filename.

Finding Help

This brief introduction is an attempt to get you started, and only really scratches the surface of what you can do. As well as books or other tutorials you can find online, unix itself has a few ways of providing its own help for you as you go.

There are online docs for most commands, which you can access with the man command, so in order to see the help for the ls command for example, you simply run 'man ls'. The resulting help shows up in a text viewer (see 'Viewing Files' section earlier for help). The 'man' pages are usually pretty dry, but are good for finding additional flags and options to commands, and the better pages have example uses.

The 'man' system also has a search functionality, for when you are looking for something but you don't know the name of the command. By running 'man -k KEYWORD' it will list all commands matching the keyword you enter.

Whilst the standard unix commands will have 'man' pages, many additional commands and utilities may not. However, most commands will show you some basic usage information if you run them with either -help, --help, or -h flags.

Finally, the tcsh shell has a very useful feature similar to filename completion, but instead for commands. By typing the start of a command's name then pressing 'CTRL-D' it will display a list of all the commands you can run that match that name.

Summary

This tutorial is just intended to be a very brief overview of the most commonly used commands and topics of unix shell usage. There are many weighty books you can buy on the topic, and countless more thorough tutorials online. Hopefully though this will get you comfortable in using a unix environment so you can get on with churning out your shots rather than worrying about how to deal with your files.

If you can think of something obvious I've left out here please let me know.

This article originally posted on Andrew's own website www.technolumiere.com

---