SDSU CS 596 Client-Server Programming
FTP Protocol

[To Lecture Notes Index]
San Diego State University -- This page last updated April 11, 1996
----------

Contents of FTP Protocol Lecture

  1. FTP Protocol
    1. Data Transfer Issues
    2. Protocol Commands
      1. ACCESS CONTROL COMMANDS
      2. TRANSFER PARAMETER COMMANDS
      3. FTP SERVICE COMMANDS
    3. Establishing Data Connection

FTP Protocol


Reference:
File Transfer Protocol. J. Postel, J.K. Reynolds. Oct-01-1985, rtf 959


Port Numbers
20 default data
21 for control

Terms

These definitions are taken directly from rfc 959

control connection
The communication path between the USER-PI and SERVER-PI for the exchange of commands and replies.
This connection follows the Telnet Protocol.


data connection
A full duplex connection over which data is transferred, in a specified mode and type.
The data transferred may be a part of a file, an entire file or a number of files.
The path may be between a server-DTP and a user-DTP, or between two server-DTPs.


data port
The passive data transfer process "listens" on the data port for a connection from the active transfer process in order to open the data connection.
DTP
The data transfer process establishes and manages the data connection.
The DTP can be passive or active


PI
The protocol interpreter.
The user and server sides of the protocol have distinct roles implemented in a user-PI and a server-PI.


server-DTP
The data transfer process, in its normal "active" state, establishes the data connection with the "listening" data port.
It sets up parameters for transfer and storage, and transfers data on command from its PI.
The DTP can be placed in a "passive" state to listen for, rather than initiate a connection on the data port.

user
A person or a process on behalf of a person wishing to obtain file transfer service.

Motivation for Two Connections



Data Transfer Issues


Data is represented differently on different machines
ASCII verses EBCDIC
Word sizes can differ
32 bits
36 bits
Byte order can be different
End-of-line character(s) can differ

Binary data of various formats

Basic Client-Server Interaction

Client sends command to server
Server always returns a reply

Command and reply are on command connection

Commands are not case sensitive

reply consists of:
three digit code <sp> text <telnet end of line>
three digit code indicates status to program
text for humans

Multi-line replies
three digit code-text <telnet end of line>
second line
etc.
three digit code <sp> whatever

No data is sent on the command connection between client and server

Details from rfc 959
Replies to File Transfer Protocol commands are devised to ensure the synchronization of requests and actions in the process of file transfer, and to guarantee that the user process always knows the state of the Server. Every command must generate at least one reply, although there may be more than one; in the latter case, the multiple replies must be easily distinguished. In addition, some commands occur in sequential groups, such as USER, PASS and ACCT, or RNFR and RNTO. The replies show the existence of an intermediate state if all preceding commands have been successful. A failure at any point in the sequence necessitates the repetition of the entire sequence from the beginning.

An FTP reply consists of a three digit number (transmitted as three alphanumeric characters) followed by some text. The number is intended for use by automata to determine what state to enter next; the text is intended for the human user. It is intended that the three digits contain enough encoded information that the user-process (the User-PI) will not need to examine the text and may either discard it or pass it on to the user, as appropriate. In particular, the text may be server-dependent, so there are likely to be varying texts for each reply code.

A reply is defined to contain the 3-digit code, followed by Space <SP>, followed by one line of text (where some maximum line length has been specified), and terminated by the Telnet end-of-line code. There will be cases however, where the text is longer than a single line. In these cases the complete text must be bracketed so the User-process knows when it may stop reading the reply (i.e. stop processing input on the control connection) and go do other things. This requires a special format on the first line to indicate that more than one line is coming, and another on the last line to designate it as the last. At least one of these must contain the appropriate reply code to indicate the state of the transaction. To satisfy all factions, it was decided that both the first and last line codes should be the same.

Thus the format for multi-line replies is that the first line will begin with the exact required reply code, followed immediately by a Hyphen, "-" (also known as Minus), followed by text. The last line will begin with the same code, followed immediately by Space <SP>, optionally some text, and the Telnet end-of-line code.

For example:
123-First line
Second line
234 A line beginning with numbers
123 The last line

FTP Protocol Commands

USER <SP> <username> <CRLF>
PASS <SP> <password> <CRLF>
ACCT <SP> <account-information> <CRLF>
CWD <SP> <pathname> <CRLF>
CDUP <CRLF>
SMNT <SP> <pathname> <CRLF>
QUIT <CRLF>
REIN <CRLF>
PORT <SP> <host-port> <CRLF>
PASV <CRLF>
TYPE <SP> <type-code> <CRLF>
STRU <SP> <structure-code> <CRLF>
MODE <SP> <mode-code> <CRLF>
RETR <SP> <pathname> <CRLF>
STOR <SP> <pathname> <CRLF>
STOU <CRLF>
APPE <SP> <pathname> <CRLF>
ALLO <SP> <decimal-integer>
[<SP> R <SP> <decimal-integer>] <CRLF>
REST <SP> <marker> <CRLF>
RNFR <SP> <pathname> <CRLF>
RNTO <SP> <pathname> <CRLF>
ABOR <CRLF>
DELE <SP> <pathname> <CRLF>
RMD <SP> <pathname> <CRLF>
MKD <SP> <pathname> <CRLF>
PWD <CRLF>
LIST [<SP> <pathname>] <CRLF>
NLST [<SP> <pathname>] <CRLF>
SITE <SP> <string> <CRLF>
SYST <CRLF>
STAT [<SP> <pathname>] <CRLF>
HELP [<SP> <string>] <CRLF>
NOOP <CRLF>

Protocol Commands


Note following descriptions are from the rfc 595

ACCESS CONTROL COMMANDS


The following commands specify access control identifiers (command codes are shown in parentheses).
USER NAME (USER)
The argument field is a Telnet string identifying the user. The user identification is that which is required by the server for access to its file system. This command will normally be the first command transmitted by the user after the control connections are made (some servers may require this). Additional identification information in the form of a password and/or an account command may also be required by some servers. Servers may allow a new USER command to be entered at any point in order to change the access control and/or accounting information. This has the effect of flushing any user, password, and account information already supplied and beginning the login sequence again. All transfer parameters are unchanged and any file transfer in progress is completed under the old access control parameters.
PASSWORD (PASS)
The argument field is a Telnet string specifying the user's password. This command must be immediately preceded by the user name command, and, for some sites, completes the user's identification for access control. Since password information is quite sensitive, it is desirable in general to "mask" it or suppress typeout. It appears that the server has no foolproof way to achieve this. It is therefore the responsibility of the user-FTP process to hide the sensitive password information.
ACCOUNT (ACCT)
The argument field is a Telnet string identifying the user's account. The command is not necessarily related to the USER command, as some sites may require an account for login and others only for specific access, such as storing files. In the latter case the command may arrive at any time.
There are reply codes to differentiate these cases for the automation: when account information is required for login, the response to a successful PASSword command is reply code 332. On the other hand, if account information is NOT required for login, the reply to a successful PASSword command is 230; and if the account information is needed for a command issued later in the dialogue, the server should return a 332 or 532 reply depending on whether it stores (pending receipt of the ACCounT command) or discards the command, respectively.
CHANGE WORKING DIRECTORY (CWD)
CHANGE TO PARENT DIRECTORY (CDUP)
This command is a special case of CWD, and is included to simplify the implementation of programs for transferring directory trees between operating systems having different syntaxes for naming the parent directory. The reply codes shall be identical to the reply codes of CWD. See Appendix II for further details.
STRUCTURE MOUNT (SMNT)
This command allows the user to mount a different file system data structure without altering his login or accounting information. Transfer parameters are similarly unchanged. The argument is a pathname specifying a directory or other system dependent file group designator.
REINITIALIZE (REIN)
This command terminates a USER, flushing all I/O and account information, except to allow any transfer in progress to be completed. All parameters are reset to the default settings and the control connection is left open. This is identical to the state in which a user finds himself immediately after the control connection is opened. A USER command may be expected to follow.
LOGOUT (QUIT)
This command terminates a USER and if file transfer is not in progress, the server closes the control connection. If file transfer is in progress, the connection will remain open for result response and the server will then close it. If the user-process is transferring files for several USERs but does not wish to close and then reopen connections for each, then the REIN command should be used instead of QUIT.
An unexpected close on the control connection will cause the server to take the effective action of an abort (ABOR) and a logout (QUIT).

TRANSFER PARAMETER COMMANDS


All data transfer parameters have default values, and the commands specifying data transfer parameters are required only if the default parameter values are to be changed. The default value is the last specified value, or if no value has been specified, the standard default value is as stated here. This implies that the server must "remember" the applicable default values. The commands may be in any order except that they must precede the FTP service request. The following commands specify data transfer parameters:
DATA PORT (PORT)
The argument is a HOST-PORT specification for the data port to be used in data connection. There are defaults for both the user and server data ports, and under normal circumstances this command and its reply are not needed. If this command is used, the argument is the concatenation of a 32-bit internet host address and a 16-bit TCP port address. This address information is broken into 8-bit fields and the value of each field is transmitted as a decimal number (in character string representation). The fields are separated by commas. A port command would be:
PORT h1,h2,h3,h4,p1,p2
where h1 is the high order 8 bits of the internet host address.
PASSIVE (PASV)
This command requests the server-DTP to "listen" on a data port (which is not its default data port) and to wait for a connection rather than initiate one upon receipt of a transfer command. The response to this command includes the host and port address this server is listening on.
REPRESENTATION TYPE (TYPE)
The argument specifies the representation type as described in the Section on Data Representation and Storage. Several types take a second parameter. The first parameter is denoted by a single Telnet character, as is the second Format parameter for ASCII and EBCDIC; the second parameter for local byte is a decimal integer to indicate Bytesize. The parameters are separated by a <SP> (Space, ASCII code 32).
FILE STRUCTURE (STRU)
The argument is a single Telnet character code specifying file structure described in the Section on Data Representation and Storage.
The following codes are assigned for structure:
F - File (no record structure) R - Record structure P - Page structure
The default structure is File.
TRANSFER MODE (MODE)
The argument is a single Telnet character code specifying the data transfer modes described in the Section on Transmission Modes.
The following codes are assigned for transfer modes:
S - Stream B - Block C - Compressed
The default transfer mode is Stream.

FTP SERVICE COMMANDS


The FTP service commands define the file transfer or the file system function requested by the user. The argument of an FTP service command will normally be a pathname. The syntax of pathnames must conform to server site conventions (with standard defaults applicable), and the language conventions of the control connection. The suggested default handling is to use the last specified device, directory or file name, or the standard default defined for local users. The commands may be in any order except that a "rename from" command must be followed by a "rename to" command and the restart command must be followed by the interrupted service command (e.g., STOR or RETR). The data, when transferred in response to FTP service commands, shall always be sent over the data connection, except for certain informative replies. The following commands specify FTP service requests:
RETRIEVE (RETR)
This command causes the server-DTP to transfer a copy of the file, specified in the pathname, to the server- or user-DTP at the other end of the data connection. The status and contents of the file at the server site shall be unaffected.
STORE (STOR)
This command causes the server-DTP to accept the data transferred via the data connection and to store the data as a file at the server site. If the file specified in the pathname exists at the server site, then its contents shall be replaced by the data being transferred. A new file is created at the server site if the file specified in the pathname does not already exist.
STORE UNIQUE (STOU)
APPEND (with create) (APPE)
This command causes the server-DTP to accept the data transferred via the data connection and to store the data in a file at the server site. If the file specified in the pathname exists at the server site, then the data shall be appended to that file; otherwise the file specified in the pathname shall be created at the server site.
ALLOCATE (ALLO)
RESTART (REST)
The argument field represents the server marker at which file transfer is to be restarted. This command does not cause file transfer but skips over the file to the specified data checkpoint. This command shall be immediately followed by the appropriate FTP service command which shall cause file transfer to resume.
ABORT (ABOR)
This command tells the server to abort the previous FTP service command and any associated transfer of data.
DELETE (DELE)
REMOVE DIRECTORY (RMD)
MAKE DIRECTORY (MKD)
PRINT WORKING DIRECTORY (PWD)
LIST (LIST)
This command causes a list to be sent from the server to the passive DTP. If the pathname specifies a directory or other group of files, the server should transfer a list of files in the specified directory. If the pathname specifies a file then the server should send current information on the file. A null argument implies the user's current working or default directory. The data transfer is over the data connection in type ASCII or type EBCDIC. (The user must ensure that the TYPE is appropriately ASCII or EBCDIC). Since the information on a file may vary widely from system to system, this information may be hard to use automatically in a program, but may be quite useful to a human user.
NAME LIST (NLST)
This command causes a directory listing to be sent from server to user site. The pathname should specify a directory or other system-specific file group descriptor; a null argument implies the current directory. The server will return a stream of names of files and no other information. The data will be transferred in ASCII or EBCDIC type over the data connection as valid pathname strings separated by <CRLF> or <NL>. (Again the user must ensure that the TYPE is correct.) This command is intended to return information that can be used by a program to further process the files automatically. For example, in the implementation of a "multiple get" function.
SITE PARAMETERS (SITE)
This command is used by the server to provide services specific to his system that are essential to file transfer but not sufficiently universal to be included as commands in the protocol. The nature of these services and the specification of their syntax can be stated in a reply to the HELP SITE command.
SYSTEM (SYST)
This command is used to find out the type of operating system at the server. The reply shall have as its first word one of the system names listed in the current version of the Assigned Numbers document [4].
STATUS (STAT)
This command shall cause a status response to be sent over the control connection in the form of a reply. The command may be sent during a file transfer (along with the Telnet IP and Synch signals--see the Section on FTP Commands) in which case the server will respond with the status of the operation in progress, or it may be sent between file transfers. In the latter case, the command may have an argument field. If the argument is a pathname, the command is analogous to the "list" command except that data shall be transferred over the control connection. If a partial pathname is given, the server may respond with a list of file names or attributes associated with that specification. If no argument is given, the server should return general status information about the server FTP process. This should include current values of all transfer parameters and the status of connections.
HELP (HELP)
This command shall cause the server to send helpful information regarding its implementation status over the control connection to the user. The command may take an argument (e.g., any command name) and return more specific information as a response. The reply is type 211 or 214. It is suggested that HELP be allowed before entering a USER command. The server may use this reply to specify site-dependent parameters, e.g., in response to HELP SITE.
NOOP (NOOP)
This command does not affect any parameters or previously entered commands. It specifies no action other than that the server send an OK reply.

Sample FTP Protocol Session

rohan 13-> telnet saturn 21
Trying 130.191.229.1...
Connected to saturn.sdsu.edu.
Escape character is '^]'.
220 saturn FTP server (SunOS 4.1) ready.

user whitney
331 Password required for whitney.

pass getRealMan;
230 User whitney logged in.

ls
500 'LS': command not understood.

list
425 Can't build data connection: Connection refused.

cd ..
500 'CD ..': command not understood.

cwd ..
250 CWD command successful.

Sample Multi-Line Response

help
214-The following commands are recognized (* =>'s unimplemented).
  USER  PORT  RETR  MSND*  ALLO  DELE  SITE*  XMKD  CDUP
  PASS  PASV  STOR  MSOM*  REST*  CWD   STAT*  RMD   XCUP
  ACCT*  TYPE  APPE  MSAM*  RNFR  XCWD  HELP  XRMD  STOU
  REIN*  STRU  MLFL*  MRSQ*  RNTO  LIST  NOOP  PWD
  QUIT  MODE  MAIL*  MRCP*  ABOR  NLST  MKD   XPWD
214 Direct comments to bugs@Sun.COM.


Sample FTP Client-User Interaction

rohan 14-> ftp saturn
Connected to saturn.sdsu.edu.
220 saturn FTP server (SunOS 4.1) ready.
Name (saturn:whitney):<return>

331 Password required for whitney.

Password:
230 User whitney logged in.

ftp> ls
200 PORT command successful.
150 ASCII data connection for /bin/ls (130.191.143.100,36027) (0 bytes).
...
mbox
paragon
public_html
smalltalk
226 ASCII Transfer complete.
585 bytes received in 0.14 seconds (4.2 Kbytes/s)

ftp> cd ..
250 CWD command successful.

FTP Client Commands

ftp> help
Commands may be abbreviated.  Commands are:

!               cr              macdef          proxy           send
$               delete          mdelete         sendport        status
account         debug           mdir            put             struct
append          dir             mget            pwd             sunique
ascii           disconnect      mkdir           quit            tenex
bell            form            mls             quote           trace
binary          get             mode            recv            type
bye             glob            mput            remotehelp      user
case            hash            nmap            rename          verbose
cd              help            ntrans          reset           ?
cdup            lcd             open            rmdir
close           ls              prompt          runique

Establishing Data Connection


Server Default Data Port: 20

Client Default Data Port: same port used for command connection

Client can request server use different port for server data connection with PASV command

Client can tell server to use different port for client data connection via PORT command

User agent "listens" on data port

Server connects to user agents data port, starts data transmission

No error checking is done by FTP

Transmission Modes
Stream Mode

Passes the data with little or no processing

If sending files send raw bytes, EOF is indicated by closing connection

If sending records, EOR and EOF are indicated by two bytes of control information
Escape byte is all ones
EOR is 1
EOF is 2
If data contains escape byte, repeat the escape byte
Block Mode

Data is sent in blocks

Allows for marking data for restarts of data transmission

Can mark original data as "suspect"
Compressed Mode

Does some simple data compression
----------