SIMPLE MESSAGING PROTOCOL (SMP) 1. INTRODUCTION 1.1 Layering Protocol Layering +---------------------+ | higher-level | +---------------------+ | SMP | +---------------------+ | internet protocol | +---------------------+ |communication network| +---------------------+ Figure 1 1.2 Operation The primary purpose of the SMP is to provide reliable securable messaging between pairs of threads. To provide this service on top of a less reliable internet communication system requires facilities in the following areas: Basic Data Transfer Reliability Precedence and Security The basic operation of the SMP in each of these areas is described in the following paragraphs. Basic Data Transfer: The SMP is able to transfer messages of different sizes. It's up to the receiver to decide if a large message can be received or not. To be able to support efficient synchronization between two threads, a reply can be requested by the client thread. Reliability: The SMP must recover from data that is damaged, lost, duplicated, or delivered out of order by the internet communication system. This is achieved by having the offset from the start message in each packet transmitted, and requiring a positive acknowledgment (ACK) from the receiving SMP. If the ACK is not received within a timeout interval, the data is retransmitted. A different connection number is used for each message. The connection number is used to avoid problems with old duplicates. The initial connection number is determined by the receiver. At the receiver, the offset in message is used to correctly order segments that may be received out of order and to eliminate duplicates. Damage is handled by adding a checksum to each segment transmitted, checking it at the receiver, and discarding damaged segments. As long as the SMPs continue to function properly and the internet system does not become completely partitioned, no transmission errors will affect the users. SMP recovers from internet communication system errors. Precedence and Security: The users of SMP may indicate the security and precedence of their communication. Provision is made for default values to be used when these features are not needed. 1.3. Model of Operation The model of internet communication is that there is an internet protocol module associated with each SMP which provides an interface to the local network. This internet module packages SMP segments inside internet datagrams and routes these datagrams to a destination internet module or intermediate gateway. To transmit the datagram through the local network, it is embedded in a local network packet. The packet switches may perform further packaging, fragmentation, or other operations to achieve the delivery of the local packet to the destination internet module. At a gateway between networks, the internet datagram is "unwrapped" from its local packet and examined to determine through which network the internet datagram should travel next. The internet datagram is then "wrapped" in a local packet suitable to the next network and routed to the next gateway, or to the final destination. A gateway is permitted to break up an internet datagram into smaller internet datagram fragments if this is necessary for transmission through the next network. To do this, the gateway produces a set of internet datagrams; each carrying a fragment. Fragments may be broken into smaller ones at intermediate gateways. The internet datagram fragment format is designed so that the destination internet module can reassemble fragments into internet datagrams. A destination internet module unwraps the segment from the datagram (after reassembling the datagram, if necessary) and passes it to the destination SMP. 1.4. Interfaces The SMP/internet interface provides calls to send a message and wait for a reply to SMP modules in hosts anywhere in the internet system. 1.5. Relation to Other Protocols The following diagram illustrates the place of the SMP in the protocol hierarchy: +-----+ +------+ +-----+ +-----+ | SMP | |Telnet| | FTP | ... | | Application Level +-----+ +------+ +-----+ +-----+ | | | | +-----+ +-----+ | | TCP | ... | | Host Level | +-----+ +-----+ | | | +-----------------------------------+ | Internet Protocol | Gateway Level +-----------------------------------+ | +---------------------------+ | Local Network Protocol | Network Level +---------------------------+ | Protocol Relationships Figure 2. 1.6. Robustness Principle SMP implementations should follow a general principle of robustness: be conservative in what you do, be liberal in what you accept from others. 1.7. Protocol number This is protocol 121 when used in the Internet Protocol 1.8. Extensions Extensions to this protocol are NOT allowed under any circumstances. All protocols that uses protocol number 121 must comply with this specification. A small implementation might decide not to implement certain functions, such as arbitrary size, but all implemented functions must comply with this document. This ensures the SMP protocol remains open. 2. FUNCTIONAL SPECIFICATION 2.1. Header Format SMP segments are sent as internet datagrams. The Internet Protocol header carries several information fields, including the source and destination host addresses [1]. A SMP header follows the internet header, supplying information specific to the SMP protocol. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Connection Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Message Offset / Total Message Size / Max Receive Size | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Mailslot | Length | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |S|E|R|R|N|R| | | Number | | |O|O|E|P|A|S|0|0| Of | Checksum | |M|M|Q|Y|M|T| | | Responses | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Responses | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Message Data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ SMP Header Format Note that one tick mark represents one bit position. Figure 3. Connection Number: 32 bits Connection number. A unique number used to avoid problems with old duplicates. The connection number is kept per destination mailslot. It should be cached on both sides to avoid the name resolution requests on every message exchange. The connection number is incremented for every message exchanged. Message Offset / Total Message Size / Max Receive Size: 32 bits If this is a name resolution reply, this field contains the maximum message size the receiver can handle. If this is the first part of a data message, this field contains the total message size. Otherwise, it contains the offset of the data in this datagram from the start of the message. Mailslot: 16 bits The receiver mailslot. Length: 16 bits Size of the SMP header, responses and data. SOM: 1 bit If set, this is the first block in a message. EOM: 1 bit If set, this is the last block in a message. REQ: 1 bit If set, this is a request message. RPY: 1 bit If set, this is a reply message. NAM: 1 bit If set, this is a name resolution request or reply. The data part contains name resolution specific information. RST: 1 bit If set, this is a reset message. Data part is ignored for reset messages. Number Of Responses: 8 bits The number of response records following this header. Checksum: 16 bits The checksum field is the 16 bit one's complement of the one's complement sum of all 16 bit words in the header and text. If a segment contains an odd number of header and text octets to be checksummed, the last octet is padded on the right with zeros to form a 16 bit word for checksum purposes. The pad is not transmitted as part of the segment. While computing the checksum, the checksum field itself is replaced with zeros. The checksum also covers a 96 bit pseudo header conceptually prefixed to the SMP header. This pseudo header contains the Source Address, the Destination Address, the Protocol, and SMP length. This gives the SMP protection against misrouted segments. This information is carried in the Internet Protocol and is transferred across the SMP/Network interface in the arguments or results of calls by the SMP on the IP. +--------------------------+ | Source Address | +--------------------------+ | Destination Address | +--------------------------+ | zero | PTCL | SMP Length | +--------------------------+ The SMP Length is the SMP header plus the data length in octets (this is not an explicitly transmitted quantity, but is computed from the total length, and the header length). 2.2 Name resolution specific data Name resolution is used to convert mailslot names to mailslot numbers. It also synchronizes connection numbers, and informs the sender of the maximum message size. 2.2.1 Name resolution request format 0 1 2 3 4 5 6 7 +-+-+-+-+-+-+-+-+ | Name | +-+-+-+-+-+-+-+-+ Name: Variable size The mailslot name. It's a null terminated string. 2.2.2 Name resolution response format 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Maximum Outstanding Messages | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Maximum Outstanding Messages: 16 bits The maximum of non-replied to messages allowed by the receiver host. The sender host shouldn't try to send more messages when this limit is reached. If this field is zero, the mailslot is invalid. 2.3. Response format To obtain optimal performance, a SMP SHOULD piggy-back as many responses to previous messages as possible when it sends data. SMP allows responses to different connections to be put in the same datagram. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Connection Number | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Mailslot | Size | Action | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Response data | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Connection Number: 32 bits Connection number. Mailslot: 16 bits The receiver mailslot. Size: 8 bits Size of the variable data for the response Action: 8 bits Defines the response action. The following response actions are currently defined: 2 Data accepted 3 Message is too large 4 Receiver is busy. 2.4 Response variable data formats The following section describes the variable data field for different action codes. 2.4.1 Data accepted Data accepted is used to acknowledge parts or whole messages. Requests only needs to be ACKed if the response cannot be sent before the sender times out. 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ACK Offset | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ACK Offset: 32 bits ACKed offset within message. 2.4.2 Message too large This response is returned if a sender tries to send a message that is too large for the receiver. This message has no additional data. 2.4.3 Receiver is busy This is an advisoratory message. It tells the sender the receiver is still busy with this or another request. This message has no additional data. 2.5 Data flow Basic 3 message data exchange SMP A SMP B 1. SEND --> --> 2. <-- <-- REPLY 3. Timeout 4. --> --> Basic 2 message data exchange SMP A SMP B 1. SEND --> --> 2. <-- <-- REPLY 3. SEND --> --> 4. <-- <-- REPLY 5. Timeout 6. --> --> Large message data exchange (3 blocks) SMP A SMP B 1. SEND --> --> 2. --> --> 3. <-- <-- 4. --> --> 5. <-- <-- REPLY 6. <-- <-- 7. --> --> 8. <-- <-- 9. Timeout 10. --> --> Duplicate from old connection SMP A SMP B 1. SEND --> --> ... 2. ... --> --> (??) 3.Ignore<-- <-- RESET 4.Ignore<-- <-- ... 5. <-- <-- REPLY 6. Timeout 7. --> --> First time send to mailslot SMP A SMP B 1. --> --> Flush 2. <-- <-- 3. SEND --> --> 4. <-- <-- REPLY 5. Timeout 6. --> --> Receiver has rebooted SMP A SMP B 1. SEND --> --> (??) 2. <-- <-- RESET 3.Flush --> --> Flush 4. <-- <-- 5. --> --> 6. <-- <-- REPLY 7. Timeout 8. --> --> Sender has rebooted SMP A SMP B 1. --> --> Flush 2. <-- <-- 3. SEND --> --> 4. <-- <-- REPLY 5. Timeout 6. --> --> Receiver is busy SMP A SMP B 1. --> --> 2. Timeout 3. --> --> 3. Timeout 4. --> --> 5. Lost <-- <-- Busy 6. Timeout 7. --> --> 8. <-- <-- Busy 9. <-- <-- REPLY 10. Timeout 11. --> --> Lost packets SMP A SMP B 1. --> --> Lost! 2. Timeout 3.Resend--> --> 4. Lost <-- <-- REPLY 5. Timeout 6.Resend--> --> 7. <-- <-- Resend 8. Timeout 9. --> --> Duplicated packets SMP A SMP B 1. --> --> ... 2. Timeout 3. --> --> ... 4. ... --> --> 5. <-- <-- REPLY 6. ... --> --> 7.Ignore<-- <-- Resend 8.Timeout 9. --> --> Simultanous requests to the same port SMP A SMP B 1. SEND --> --> 2. SEND --> --> 3. <-- <-- REPLY 4. <-- <-- REPLY 5. Timeout 6. --> --> Simultanous requests arriving out-of order to the same port SMP A SMP B 1. SEND --> --> ... 2. SEND --> --> Queue! 3. ... --> --> 4. ... <-- <-- REPLY 5. <-- <-- REPLY 6. <-- <-- ... 7. Timeout 8. --> --> Simultanous requests arriving and first request lost SMP A SMP B 1. SEND --> --> Lost! 2. SEND --> --> Queue! 3. Timeout 4.Resend--> --> 5. <-- <-- REPLY 6. Timeout 7. ... <-- <-- REPLY 8.Resend--> --> 9. <-- <-- ... 10.Ignore<- <-- Resend 11. Timeout 12. --> --> 2.5.1 Rules on how a SMP should react The examples above lead to the following rules that must be observed by all SMP implementations. a) A SMP should always wait for a timeout before it sends the final ACK to the receiver. If it doesn't, the efficient 2 packet exchange can never happen. b) The reply doesn't need to ACK the request. The ACK is implicit. c) For large messages, the same congestion handling as TCP uses is required. At least every 2nd packet should be ACKed. On timeouts, the first unacked packet should be resent. If packets comes in out-of-order, an immediate ACK should be sent to indicate a packet is missing to the sender. d) The receiver side should send a RST if a connection number is outside of the current window. The sender side would then decide if a new name resolution is required or not. The sender side should just drop invalid connection packets. e) If a connection isn't cached or if it's reset, the sender needs to send a name resolution request before sending any messages. f) When the receiver is busy, it shouldn't send a busy response immediately. It should wait for the sender to timeout and resend the request. After the second resend, it should answer with a busy response. g) The receiver should resend the first packet of data if the same connection request is encountered again after a reply has already been sent. In effect, this is the same handling as if it received an ACK with position 0 in the message. h) In the request phase of the message exchange, the sender is responsible for resending. After the sender has got the first part of the reply, the receiver is responsible for resending. This means that for multi- packet messages, both sender and receiver can resend before the reply is sent, but before it's delivered to the sender. If no resend requests comes from the side beeing responsible for this, the non-responsible side might send some ACKs, and if no response is still seen, should flush and reset all connections to the offending host. i) A SMP should allow at least 16 simultanous REQUESTs to the same mailslot from the same host. This implies it must have a window of allowed connection numbers. A receiver should always process connections in order. This implies it must not start servicing an out-of-order request until all requests preceding it are processed. A sender should always send connection requests in order. j) Invalid packets should be dropped. Headers with both REQ and RPY set are considered invalid. k) Before a sender sends a name resolution it should flush all connections to the destination host. When a receiver gets the name resolution request it should flush all connections to the source host, select a new connection number and send a name resolution reply. When the sender gets the name resolution reply, it should resend all unhandled connections that where flushed previously. This ensures the system will work properly even though the sender or receiver is restarted. 2.6 Interfaces SMP User Commands The following sections functionally characterize a USER/SMP interface. The notation used is similar to most procedure or function calls in high level languages. The function calls and parameters should be treated as an example. A real implementation might decide to support another set of calls. 2.6.1 Sender side calls Get Format: GET ([host], mailslot name) -> mailslot Gets a mailslot. This function can be used to get a local mailslot, or a mailslot on another host. Send Format: SEND (mailslot, message buffer, message size, reply buffer, max reply size, [timeout]) -> reply size, status Sends a message to a mailslot and waits for a reply. If the receipient is local, it would queue the message on the receiver and suspend the sender. If the message is to another host, the SMP would try to locate the cached entry for the remote mailslot. If it cannot find one, allocate a cache entry and send a UDP query The message would then be sent to the destination SMP and the sender should be suspended. Free Format: FREE (mailslot) Free the mailslot handle obtained with get. 2.5.2 Receiver side calls Define Format: DEFINE (mailslot name, max size) Defines a new mailslot for the current thread. Receive Format: RECEIVE (message buffer) -> size Receive a message from the local mailslot. The mailslot should have been defined previously, unless the OS supports default names for mailslots (such as the thread name) Reply Format: REPLY (reply buffer, reply size) Reply to a received message. If the reply is local, it can be copied directly to the sender. After copying it, the sender should be resumed. If the reply is to another host, the SMP will send it to the destination SMP. The destination SMP will then resume the sender. References [1] Postel, J., "Internet Protocol," RFC 760, USC/Information Science Institute, January 1980