How to Automate FTP

Programming with the File Transport Protocol Macros

Automating FTP - Mark Alexander Bain
Automating FTP - Mark Alexander Bain
FTP (File Transport Protocol) has been around for a long time, but it's still as useful as it ever was - especially when it comes to automating processes with FTP macros.

Most people's experience of FTP (File Transport Protocol) is through a form based GUI (Graphical User Interface) - the kind of interface that allows a user to drag and drop files from their pc to their web site; however, for the programmer, FTP has much more to offer:

  • FTP can be run from the command line and can, therefore, be used in scripts
  • FTP actually has its own programming language.

Using FTP from the Command Line

The majority of FTP users will be happy to use the graphical interface, but that will not suffice for any programmers or web site owners who want to automate the upload (or download) process - and that's where the command line FTP can be useful.

At it's simplest level the command line FTP works in the same way as the GUI FTP, and the user will:

  • connect to the FTP server using their own user name
  • enter a password
  • upload (or download) selected files

A typical FTP session will look like:

$ ftp ftp.linuxtalk.co.uk
Connected to ftp.linuxtalk.co.uk.
220 FTP server ready
Name (ftp.linuxtalk.co.uk:bainm): linuxtalk
331 Password required for linuxtalk
Password:
230 User linuxtalk logged in
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> put testfile.txt
local: testfile.txt remote: testfile.txt
200 PORT command successful
150 Opening BINARY mode data connection for testfile.txt
226 Transfer complete
ftp>

However, there is a limitation with using FTP in this way - the user's password must be entered manually, preventing the process from being automated; and that's where the .netrc file is useful.

Using the .netrc File

The .netrc file does not exist by default - it needs to be created manually and placed in the user's home directory; it needs to contain the connection details for any FTP accounts to be used:

machine ftp.linuxtalk.co.uk
login linuxtalk
password abcDEF123

The .netrc must always end with a blank line (and a blank line must also be inserted between any FTP account definitions).

Obviously, before continuing, there is also a security risk to be considered here - the user's password is being saved in a text file; however, FTP reduces this risk by only running if the .netrc file has its permissions set to user read only (and unreadable by anyone else); therefore the most important job is to do:

$ chmod 600 .netrc

With the .netrc file in place FTP access becomes very simple:

$ ftp ftp.linuxtalk.co.uk
Connected to ftp.linuxtalk.co.uk.
220 FTP server ready
331 Password required for linuxtalk.co.uk
230 User linuxtalk.co.uk logged in
Remote system type is UNIX.
Using binary mode to transfer files.
ftp>

And now any file processing can be automated:

echo put testfile1.txt | ftp ftp.linuxtalk.co.uk
echo get testfile2.txt | ftp ftp.linuxtalk.co.uk

FTP Macros

The automation process is made much easier by the fact that macros can be used with FTP - they can use any standard FTP command and are defined in the .netrc file by using the macdef keyword:

macdef updateWeb
prompt off
LCD $1
mkdir $2
cd $2
put *
macdef myWebUpdate
$ updateWeb /home/bainm/webfiles webdirectory

In the example above all of the files in a directory are uploaded on to a server (creating a new directory if necessary) - and it's worth noting that one macro can call another (by using the $ symbol), and that the macros can also be called from the command line:

echo "$ myWebUpdate" | ftp ftp.linuxtalk.co.uk

Conclusion

FTP has always been a powerful tool, enabling even the least technically minded person upload and download files with ease; but the .netrc file and macros turn FTP into a useful programming tool - especially when combined with other tools such as cron. FTP may be quite old technology but it's still as vital as it ever was.

Mark Alexander Bain - Mark Alexander Bain is a writer, Mo Bro and consultant for all aspects of software development at dsquared. He has also written regularly ...

rss
Advertisement

Comments

Sep 27, 2009 1:25 AM
Guest :
Don't bother posting this. I just want to say that I'm bamboozled at how perfect this article is. You people are the modern day equivalent of the Gods on Mt. Olympus.
1
Advertisement
Advertisement