|
|
|
| Reference Series |
Table of Contents For This Issue
|
|
| |
| How Computers Work, Part I | |
|
August 2001• Vol.5 Issue 3 Page(s) 200-207 in print issue | |
En Route With Electronic Mail Sending & Receiving E-mail Takes More Than Just A Click Of A Button |
E-mail has long been the killer application of the Internet. Even the graphically stunning World Wide Web has not been able to eclipse the popularity of e-mail. Yet, despite the continued popularity of e-mail, few of us really understand how it works.As with most really good technologies, e-mail has almost become transparent, requiring little knowledge about how it works to use it effectively. For example, you probably don't understand how the pictures on your television reach you; you simply turn on the tube and allow it to inform and entertain you. Similarly, you send e-mail without understanding the process by which your messages transmit from you to your recipients. Most e-mail software, such as QUALCOMM's Eudora and Microsoft's Outlook Express, lets you easily check for new messages and send e-mail worldwide. But behind the scenes is a complicated network of machines that transmit each message to its destination within seconds.  Our Favorite Tool.
E-mail is a favorite tool of the Internet community. Your inbox is often brimming with unsolicited e-mail, jokes you’ve read a thousand times, and assorted hoaxes and chain letters. Yet, despite e-mail debris, it is still a valuable collaboration tool.E-mail is simply plain text encoded for transmission across a network (a group of connected computers). The method of transportation does not really matter, as long as the same method is used throughout the transport of the message. Because the Internet has come of age, TCP/IP (Transmission Control Protocol/Internet Protocol; a protocol that provides a language by which computers can communicate) has become the standard way to transport data, including e-mail, across a network. TCP/IP alone, though, is not enough to transport data. Every type of data has special needs and requires certain protocols. E-mail is widely transported through SMTP (Simple Mail Transfer Protocol), which delivers the message to a remote server. Additional protocols, such as POP3 (Post Office Protocol version 3), are then used to download the message from the remote server to the recipient’s computer. (Both SMTP and POP3 run on TCP/IP and specify commands that let each protocol complete its task.) Sending A Message.
When someone sends you an e-mail message, it isn't sent directly to your computer because the standard dial-up user isn't constantly connected to the Internet. Instead, your message is sent to a computer known as a mail server, which your ISP (Internet service
provider) or company owns. The message is stored there until you log on to retrieve your messages. SMTP is the set of commands that passes a message from the sender to your service provider's mail server.SMTP is good for transporting a single message to one or more users. It is bad at organizing and delivering several messages for a single user. For this reason, SMTP is used primarily to deliver a message to your service provider's e-mail server. POP3 is used for retrieving messages from your e-mail server. Just as an e-mail server holds inbound messages for your retrieval, an SMTP host, also owned by your service provider, processes outgoing messages. This SMTP host is an intermediary between your computer and the mail server for the user you are attempting to reach. An SMTP session is used to transfer the message to each stop, which means that a minimum of two SMTP sessions are needed to transfer a message from your computer to the appropriate mail server. SMTP is a command/ response-based protocol. Unlike other mail protocols, SMTP must issue its commands in a sequence. The sending host not only issues the commands for the message transport, but also receives responses from the receiving host. Each response has a three-digit code for a prefix and a line of text, such as “OK,” that clarifies the response. The response lets the sending host know if the command was properly executed and that it can send the next command. The receiving host waits for commands from the sending host, executes each command, and issues a response stating the status of the last command. Eventually, the sending host will send the last message to the receiving host, and the SMTP session will end. If the receiving host is only an intermediary stop, it will contact the next host and become the sending host for the new SMTP session. We’ll use a simulated (and simplified) example of an SMTP session to explain the process. An “SH:” designates the sending host and an “RH:” designates the receiving host. Only lines proceeded by an “SH:” or an “RH:” are part of the session. We’ve added additional text to explain and comment on the process in the proceeding portion of the session. This text is not part of the session. In our example, Judy Smith (jsmith@local isp.com) sends a message to her supervisors, Roger Jones (rjones@corporate.com) and Laurie Rob (lrob@corporate.com). Smith’s computer establishes a connection with her assigned SMTP host and transfers the message. Her SMTP host then relays the message to the corporate SMTP server, which files the message in the appropriate mailboxes. For the first SMTP session, Smith’s PC will be the sending host and her service provider’s SMTP host will be the receiving host. RH:220 smtp.localisp.com ready SH:HELO 167.66.72.189 RH:250 smtp.localisp.com smtp initiated Here, Smith’s computer (identified as 167.66.72.189) is contacting the receiving host (her service provider’s SMTP host). Both machines identify themselves to one another. The 250 reply in this section indicates the receiving host is ready to process commands. SH:MAIL FROM <jsmith@localisp.com> RH:250 OK Smith’s system has issued the MAIL FROM command, which includes the reverse-path. The reverse-path is like a trail of bread crumbs. If the receiving host cannot deliver the mail because of some error, it can follow the reverse-path to return the error to the originator of the message. So far, this path only consists of the mailbox and domain name from which the message originated. As the message relays to other SMTP hosts, the reverse-path will contain more information on additional SMTP hosts. In the above example, the receiving host accepts the command and is ready for the next command. SH:RCPT TO <smtp.localisp.com: rjones@corporate.com> RH:250 OK SH:RCPT TO <smtp.localisp.com: lrob@corporate.com> RH:250 OK The sending host uses the RCPT TO command for each of the intended recipients, and the receiving host responds positively. The RCPT TO command contains a forward-path (the opposite of the reverse-path in the MAIL FROM command). The forward-path contains what host the message will pass through on its way to its destination. A colon separates each host on the list. The first entry in the forward-path is the name of the receiving host (smtp .localisp.com). Any intermediate hosts need only a domain name. (Domain names are used to locate computers by name on the Internet.) The final entry in the forward-path contains a mailbox (rjones or lrob) in addition to a domain name. As the message progresses to another SMTP host, the first entry in the forward-path will become the first entry in the reverse-path. SH:DATA RH:354 Start mail input The sending host indicates it’s ready to send the body of the message and the receiving host has responded appropriately. This command tells the receiving host the following lines are part of the message. Without this command, the receiving host would interpret each line of the message as a command. SH:(Headers) SH: SH:Roger and Laurie SH: SH:I’ll have the reports done by tomorrow. SH: SH:-Judy SH: SH:. RH:250 OK The sending host sends each line of the message to the receiving host. In the previous example, the Headers line would consist of several lines for To, From, Cc, Date, and Time data. The final line contains the period, signaling the end of the message. The sending host adds a period on a line by itself to the end of the message. This period is removed before it reaches the reader, but most protocols use this character to signal the end of the message and a return to the command/response model. SH:QUIT RH:221 smtp.localisp.com closing communications channel The channel between Judy's computer and her SMTP host closes. The local SMTP host will contact the next host, in this case the corporate SMTP host, in the forward-path. The intended recipients have mailboxes on this host. Notice that the local ISP SMTP host is now the sending host, and the receiving host is the corporate SMTP host. RH:220 smtp.corporate.com ready SH:HELO smtp.localisp.com RH:250 smtp.corporate.com smtp initiated The localisp SMTP host contacts the corporate SMTP host. SH:MAIL FROM <smtp.localisp.com: jsmith@localisp.com> RH:250 OK Notice smtp.localisp.com has moved from the forward-path to the reverse-path. The first entry in the reverse-path is always the sending host. SH:RCPT TO <rjones@corporate.com> RH:250 OK Only one field remains in the forward-path. Because this is the last field, a mailbox (rjones) must be specified. SH:RCPT TO <lrob@corporate.com> RH:251 User not local; will forward to <lrob@island.com> If the corporate SMTP host can’t find a mailbox for lrob, an error message is returned to Smith stating the message to lrob@corporate .com was undeliverable. Occasionally, a host might have an idea of where to find the user. In this case, the corporate SMTP host believes lrob’s mailbox is on a host with the island.com domain. When this occurs, the host forwards the message to the domain or issues an error message. In this case, it forwards the message. If it returns the message to the sending host, the sending host will decide whether to forward the message to the new domain or issue an error message to the user. SH:DATA RH:354 Start mail input SH:(Headers) SH: SH:Roger and Laurie SH: SH:I’ll have the reports done by tomorrow. SH: SH:-Judy SH: SH:. RH250 OK SH:QUIT RH:221 smtp.corporate.com closing connection channel The above was a fictional process, but a real process would closely resemble it. Messages are waiting for the user rjones (and hopefully lrob) in their respective mailboxes. When Jones logs into his mailbox, he’ll use a different protocol to transfer the message from his remote mailbox to the local mailbox on his office computer. The Download Process.
Downloading e-mail relies on different protocols. These protocols have to be able to process multiple messages for a single user and organize them efficiently for delivery. By far, the most popular protocol for this today is POP3. Another powerful alternative
is IMAP (Internet Message Access Protocol). POP3 transfers all messages to a user's local machine, then deletes all the messages on the server; IMAP lets users administer their remote mailboxes as they would their own mailboxes on their computers. Most
consumer-based service providers use the POP3 protocol. Although IMAP is more powerful, we won't cover it here because it is obscure when compared to POP3.
POP3 provides the set of commands needed to retrieve e-mail from the remote server. Much like SMTP, this process is based on commands and responses. The commands are usually three- or four-letter commands, some of which require an argument (additional information that can accompany a command). The response is either affirmative (+OK) or negative (-ERR). A POP3 connection passes through three states. The Authorization State requires a client to identify itself and prove it has permission to access the server. In the Transaction State, the client downloads the e-mail to the local computer and marks the transferred messages on the server for deletion. In the Update State, the server deletes messages marked for deletion during the Transaction State. You can abort the process at any time, but this will skip the Update State. The Authorization State helps to protect your privacy by ensuring that only you can retrieve messages intended for you. The authorization can be in one of two forms: a username and password format or the APOP format. The username and password method is straightforward. Every user has a username (usually the same name as his mailbox name) and a password. The server issues the USER command followed by the username as the argument. If the server has a mailbox on the system by that name, it returns a +OK response. The client will then send a password as an argument following the PASS command. If the issued password matches the password on record for that user, it will grant the client access to the mailbox. APOP is a more complicated, albeit more secure, authorization method. This method eliminates the need to send password data over the Internet in an insecure fashion. When the client logs on to the server, the server issues a greeting that includes a date and time stamp. This stamp could be in several formats. For our example, we’ll denote the date and time stamp as <timestamp>. (The angled brackets are part of the stamp.) This date and time stamp proceeds a secret string, a key, known by the client and server. The key can be anything, but the longer it is, the more secure it is. For our purposes, we’ll use the secret string “smartcomputing” (without the quotes). This results in the string “<timestamp>smartcomputing” on both the client and server. Both systems perform a special type of encryption on this unique string using the same algorithm (or simple mathematical program). The client then sends the encrypted string to the server. If the client’s encrypted string matches the server’s encrypted string, the server grants access to a specified mailbox. The Transaction State follows the completion of the Authentication State. In this stage, the client established an access lock with the mailbox on the server. This access lock prevents access to the mailbox by another party. This includes SMTP servers attempting to deliver new mail to the mailbox. POP3 requires a specific ordering and numbering of the messages. Any change to the mailbox, including the addition of new messages, could cause errors in transferring messages from the POP3 server. Your remote mailbox will receive any messages that happen to arrive during an access lock when the lock is broken. After the client and server establish an access lock, it numbers each message in ascending order. The numbers assigned to each message can’t change during the course of the POP3 session, but may (in fact, most likely will) change from session to session. As mentioned above, the client issues commands to the server and waits for a response before sending its next command. Unlike SMTP, a POP3 client can issue a command at any time as long as it is valid within the current state. Some commands can make use of an argument to provide extra information in conjunction with the command (such as usernames, passwords, and message numbers). Here’s a rundown of some of the more common POP3 commands. Some commands (in capital letters) support arguments but do not require them; others require an argument (lowercase letters). Parentheses indicate an optional argument. •USER username: Usually the first command given in password-authenticated servers. The USER command requires an argument (a username) and is only valid in the Authentication State. •PASS password: This command follows the USER command on password-authenticated sites. The argument for this command is a valid password. PASS is only valid in Authentication State. •STAT: This command will return a message indicating the number of messages in the mailbox and their total size in octets. Octets are a collection of 8 bits (a bit is a single portion of information). Although 8 bits are usually referred to as a byte, networks refer to them as octets because 7-bit bytes also exist in the network world. A common response may be +OK 4 460. This command is only available in the Transaction State. •LIST (n): The LIST command will list the size (in octets) of messages individually. If used without an argument, it will return data on every message in the mailbox. If used with a message number (n), it will display information on the specific message if message “n” exists in the mailbox and has not been marked for deletion. This is only valid in the Transaction State. •UIDL (n): The server gives every message a unique ID number. The server cannot reissue the number while another message has that number. You can use the UIDL command to retrieve the UID for each message. UIDL requires the connection to be in the Transaction State. •RETR n: This command retrieves the message and downloads it to the local system. The argument provides the server with data on the message the client is requesting for retrieval. The client expects a multiple line response for this command. It will issue another command when it receives the escape character (a period on a single line). The Transaction State is the only valid state for this command. •DELE n: This command marks a message for deletion during the Update State (and requires an argument). DELE is valid in the Transaction State. •QUIT: Ends the session, or ends the Transaction State and begins the Update State on the server. This command is valid in any state, but will not delete messages unless used from the Transaction State. There are more POP3 commands available, but the ones here are the most common during a POP3 session. As you can see, this protocol is more flexible than SMTP in terms of when it can issue commands. The Update State breaks the access lock, removes any messages marked for deletion from the server, and closes the connection. On many POP3 clients, you can choose not to delete the messages from the server. This will prevent your client from issuing the DELE command after it transfers a message. The message will remain on the server for future reference. In the following example, rjones@corporate .com is logging into the POP3 server (pop.corporate.com) to retrieve any e-mail it has for him. S:+OK pop.corporate.com POP3 server ready C:USER rjones S:+OK C:PASS 0498clA S:+OK user access granted The server grants access to the user. The server and client establish an access lock with the appropriate mailbox (rjones), and the process moves to the Transaction State. C:STAT S:+OK 3 400 In this example, three messages contain a combined 400 octets. The client needs to either issue the STAT or LIST command before it can retrieve info. Otherwise it would have no way of knowing what message numbers it needs to issue with the RETR commands. The LIST command returns data for each message. C:RETR 1 S:+OK S:(message header and body sent) The actual message consists of several lines, each representing a line of the message. We shortened the message transfers to save space. C:DELE 1 S:+OK message deleted C:RETR 2 S:+OK S:(message header and body sent) C:DELE 2 S:+OK message deleted C:RETR 3 S:+OK S:(message header and body sent) C:DELE 3 S:+OK message deleted C:QUIT S:+OK pop.corporate.com shutting down connection After the QUIT command, the server enters the Update State and deletes all messages marked for deletion. Protocol Magic.
Although physical network connections provide the roadways, SMTP and POP3 provide the primary vehicles to transport messages worldwide at record speeds. Understanding how these protocols communicate is a key to understanding how your message transmits from
San Francisco to New York in seconds. These protocols have made e-mail so reliable and easy to use that the phrase "I'll e-mail you" has become as popular as "I'll call you." The next time a helping of
spam (unsolicited e-mail) makes you groan, think of the little miracles that make it possible.
 by Chad Denton
View the graphics that accompany this article.
|
