Creating an Ethernet Messaging Application

Chris Robinson Advanced Applications Engineer, Embedded Systems

Altera Corporation When I was asked to create an Ethernet messaging application for work, I thought to myself, “It can’t be that hard, but where do I start?”. I think this is a concern of anyone who hasn’t worked with Ethernet. I associate Ethernet with the Internet and the World Wide Web, so the first thing that popped into my head was hosting a web page from an embedded system. Then I found out we would just be transferring data over an Ethernet network from one master board to several slave boards. After spending a little time researching and implementing the application, I was surprised at how easy adding Ethernet capabilities to an embedded system could be. Much of the complexity is taken care of by the TCP/IP stack and the ease of use it provides. My biggest surprise was that I didn’t need to be a TCP/IP expert, and all I needed to know was how to correctly implement a series of API routines. In this article, I will discuss the basics of creating your own Ethernet messaging application. There will be a brief introduction to the TCP/IP protocol. We will look at the “Sockets” API, since most TCP/IP stacks use this as a template for their own API. Using an example design, we will discuss creating our own messaging protocol for an application, which will be included on top of the standard TCP/IP protocol. Terminology Before we begin it’s necessary to define some of the terminology I will be using: TCP/IP stands for “Transmission Control Protocol/Internet Protocol”. It is the most widely used protocol suite for Ethernet networking TCP/IP Stack is used to refer to the embedded software library and associated API that implements the TCP/IP protocol Datagram is the term used to refer to the packets that contain the header and data. Ethernet is used to refer to the physical medium of the network Client is a system, which initiates a connection to a server Server is a system which responds to connection requests and serves data to clients requesting it. TCP/IP Basics The TCP/IP stack hides most of the details. There are many books and articles available to learn the in-depth details of TCP/IP and they do a much better job than I could ever do. However, there are certain things you should be aware of when working with TCP/IP. The TCP/IP protocol suite is a software-based networking protocol. Its most popular use is for allowing us to connect to the Internet. TCP/IP stands for Transmission Control CF-EMA031105-1.0

Protocol and Internet Protocol. The name is actually misleading since TCP and IP are only two of the many different protocols included in this protocol suite. There are numerous different protocols included in TCP/IP. The protocols we will be most concerned with are:

IP: The Internet Protocol provides basic datagram service for transport protocols such as UDP and TCP. IP does all the work when it comes to sending and receiving datagrams. IP is responsible for getting data to the destination host and network. However, IP is not reliable, so delivery is not guaranteed. (3)

ARP: The Address Resolution Protocol provides a mechanism for IP devices to

locate the hardware addresses of other devices on the local network. This service is required in order for IP-enabled systems to communicate with each other. (3)

UDP: The User Datagram Protocol provides a low-overhead transport services

for application protocols that do not need (or cannot use) the connections-oriented services offered by TCP. UDP is most often used with applications that make use heavy use of broadcasts or multicasts, as well as applications that need fast turnaround times on lookups and queries. (3)

TCP: The Transmission Control Protocol provides a reliable, connection-

oriented transport protocol for transaction-oriented application to use. TCP is used by almost all of the applications protocols found on the Internet today, as most of them require a reliable, error-correcting transport layer in order to ensure that data does not get lost or corrupted. (3)

Figure 1 represents the architecture for an Ethernet system, which consists of the following Hardware and Software components:

Figure 1: Ethernet System Architecture Ethernet LAN Controller major components consist of a MAC and a PHY. You will need drivers to interface with your controller; most TCP/IP stacks have support for certain controllers. PHY: The PHY is the “physical” interface, it is the analog front-end hardware used for sending and receiving transmissions. It is essentially, an Ethernet to MAC converter, allowing us to take high-speed serial data and transferring this data into a more digital readable. MAC: The Media Access Control handles aspects of frame transmission and reception. It provides features such as collision detection, preamble generation and detection and CRC generation. The MAC also provides a unique address, which is used by the Internet. The software interface consists of three layers: Internet Protocol (IP): IP provides basic datagram service. This is a “best-effort” service which allows for sending and receiving a datagram. However, there is no guarantee of delivery. Transport (UDP or TCP): These are transport protocols that provide features on top of IP. These features include connection management, checksum, and error correction. Application: The application protocol is used to manage and interpret the data being sent and received. Commonly known application protocols for TCP/IP are HTTP (Hyper-Text Transport Protocol) and SMTP (Simple Message Transfer Protocol). In the case of this article we will be defining our own application protocol.

Addressing Each Ethernet-enabled device requires a unique ID or address. In reality there are two unique addresses for each device in an Ethernet networked system, the IP address and MAC address. The IP address provides information concerning its position in the Internet. The IP address consists of a four-byte number, which is expressed in dotted decimal notation (e.g. 137.57.192.153). The IP address consists of three parts, which help identify its location

1. network address, identifies the organization/network 2. subnet address, identifies a subnetwork associated with the main network 3. system address, identifies a single node on the subnet

This is very similar to that of a postal address, where you have a zip code describing the city/state (network), street (subnet) and street number (system). The MAC address (a.k.a. Ethernet Address) uniquely identifies the hardware in a system. The manufacturer incorporates this address into a network adapter. The MAC address is represented by a six-byte address, which is expressed in hexadecimal (e.g. 00-B0-D0-3D-C7-30). The ARP protocol allows for the correlation of IP addresses with the Ethernet MAC addresses of hardware on the same subnet. A Subnet Mask is used to see if an IP address is on the same subnet. If not, the transmission will be sent to a “Default Gateway”, which will then take care of the transmission. Sockets In this section we will discuss the Berkeley Sockets Interface, a.k.a. “Sockets”. “Sockets” is the standard API for networking in UNIX. It is also the standard in which most, if not all, embedded TCP/IP stacks API are based upon. For simplicity, we will only be looking at a high-level overview of the Sockets API, for a more detail look at the API or for code examples, I recommend “TCP/IP Sockets in C” by Donahoo and Calvert. The basic component used is called a “socket”. A socket is a software abstraction of a physical connection to an application. This abstraction contains the following components that describe where the application is located:

1. IP address 2. Transport protocol (i.e. TCP or UDP) 3. port number

Since your system may need to run more than one application per transport protocol, we need a way to identify which application to send the payload of the transmission. This is the purpose of ports: to identify the ultimate destination of the transmitted data. Table 1 provides a list of some commonly used ports.

Table 1: Commonly Used Ports Port Service Application

7 Echo Echoes back all incoming characters

13 Daytime Returns time and date strings

21 FTP File transfer protocol

23 Telnet Remote login

25 SMTP Simple mail transfer protocol

37 Time Binary value of date and time

80 HTTP Hypertext transfer protocol Essentially, a socket provides us with the physical address, the protocol, and application to which we want to communicate with (Figure 2).

TCP UDP

I P

TCP Sockets UDP Sockets

UDP Ports(1-65535)

TCP Ports(1-65535)

Applications

Figure 2: IP to Application Interface (1) TCP Client and Server Interaction With TCP, an interaction between the client and server involves a “connection”. A connection is an acknowledgement from both systems to open a communication channel between each other. The client initiates the connection and the server accepts or declines

the connection. Once the communication is completed the client should close the connection and the server should do the same. The TCP client and server interaction (Figure 3) starts with the creation of two sockets, one on the client and one on the server. A port is automatically assigned on the client’s socket when it tries to connect. However, the server must bind a port to the socket. This is because the port must be known by the client application so that it knows where to make a connection. Once the port is bound to the socket, the socket must be told to listen before the client will be able to open a connection. This will allow incoming connection requests to be heard and accept or denied. The user is able to provide a routine for the socket, which determines whether or not to accept a connection. Once the connection is accepted, the Client and Server can communicate freely, sending data back and forth. Since we are using TCP, the protocol will guarantee the delivery and integrity of the packets. Once the transaction is complete, the client closes the connection. The server then closes its connection and returns to listening for new connection requests.

Connect

Accept

Communicate

Create Socket Create Socket

Bind to port

Listen

Communicate

Close Connection Close Connection

Client Server

socket()

bind()

listen()

accept()

recv()/send()

close()

socket()

connect()

send()/recv()

close()

Figure 3: TCP Client and Server Interaction (1) UDP Client and Server Interaction With UDP the client and server interaction (Figure 4) is much simpler than TCP. Similar to TCP, both the client and server must create a socket and the server must bind its socket to a port known by the client. However, a UDP transaction is connectionless.

This means the client just sends data to a socket not knowing if it is available. In order for the packet to be received the client must have the socket open and ready to receive. Unlike TCP, there is no response from the server telling the client that the data is received. Another difference is that communication is in only one direction, client to server. If this communication is required in the other direction, the client and server must change roles.

Send to

Create Socket Create Socket

Bind to port

Receive from

Client Server

socket()

sendto() recvfrom()

socket()

bind()

Figure 4: UDP Client and Server Interaction (1) Advanced Usage The previous interactions show the basic usage of sockets assuming the client and server are waiting for responses. For a system, this becomes inefficient since the processor is not able to do other operations. And if a transmission fails, we could be stuck waiting forever. Instead of waiting for a response, an application can use an interrupt from the MAC/PHY to signal an incoming packet. An Interrupt Service Routine can be created to handle the incoming packets, or set a semaphore if using an RTOS. The use of timeouts provides an indication that a transaction failed. This is very useful in the TCP protocol, when trying to connect or when you are waiting for a confirmation of a packet receipt. A timeout is a simple use of a timer, allowing it to countdown a reasonable amount of time before indicating a failure. Creating a Messaging Protocol TCP/IP doesn’t care about the payload it is transporting, all it knows is that it is transporting a packet of bytes of a certain length. The interpretation of these bytes is up to the receiving application. For the bytes to be interpreted correctly, both the client and server application have to be expecting an agreed upon a message protocol. The user will define the message protocol in accordance to their particular application. One should be aware that the message protocol is incorporated entirely in the payload of the TCP/IP packet. We will take a look at an example message protocol created to allow basic data transfer between two embedded systems.

Our example message protocol allows four basic functions: block read, block write, multi-address read, and multi-address write. This will allow us to read and write to memory buffers that are used to transfer data. Which functionality we choose will be controlled in our message header. The Type describes which data transaction will occur. The Sequence is optional and can be used if we need to send sequential data packets. The length is the number of bytes of the entire payload, message header and data. Message Header:

Type (byte) Sequence (byte) Length (2 bytes) Message packet:

msg hdr (4 bytes) Data (6-1496 bytes) The total length of the message packet cannot exceed 1500 bytes; this is limited due to the payload section of Ethernet. The data section format depends on the type of transaction being performed. The server will respond with an acknowledgement for writes and with address/data information for reads. Random Write The Random Write data structure consists of a series of address-data pairs. Each address-data pair will consist of an address and the data to be written at that location. The number of pairs is variable and only limited to size of the data section. Random-Write Data Section:

addr0 data0 addr1 data1 … addrN dataN Once the writes are completed, the server will send a 4-byte acknowledgement: Random-Write Acknowledgement:

‘A’ ‘C’ ‘K’ 0 Random Read The Random Read data structure consists of a variable number of addresses for which it is requesting the data stored. Random-Read Data Section:

addr0 addr1 … addrN The server will read the data stored at the memory locations and send a response of address and data pairs. Random-Read Response:

addr0 data0 addr1 data1 … addrN dataN

Block Write The Block Write data structure consists of a starting and ending address and the corresponding data that needs to be written in that address space. The data will be written in 2-byte blocks in sequential order starting at the starting address. Care must be taken when creating the data section to ensure the data properly fits in the address range. Block-Write Data Section:

addr0 addrN data0 data1 … dataN Once the writes are completed, the server will send a 4-byte acknowledgement is sent: Block-Write Acknowledgement:

‘A’ ‘C’ ‘K’ 0 Block Read The Block Read data structure consists of a starting and ending address for which it is requesting the data stored between the two addresses sent back to the client. Some error checking is needed to ensure the address range will not cause the response to exceed its packet size limitation. Block-Read Data Section:

addr0 addrN The server will read the data stored between the two addresses and send back a response. Block-Read response:

addr0 addrN data0 data1 … dataN The above messaging protocol may or may not serve your needs. This is simply used as an example of a method for a device to request reads from and writes to memory on a remote device. Features can easily be added to serve your needs. When creating you own protocol there some things to look out for. Byte-Padding Byte-Padding occurs when the compiler adds extra bytes to a structure. The compiler does this to for the ease of addressing the members of a structure. Problems can occur when placing the structure into the payload. The extra padding could get stuffed into the payload changing the length and how the packet is read. struct test1{ short x;

long y; short z; };

struct test2{ long x;

short y; short z; };

Structure test1:

short pad Long short pad Structure test2:

Long short short From the example above, you can see the byte-padding added to the structure test1. Careful attention must be made when accessing this structure, or we can rearrange the structure to avoid the padding that is inserted by the compiler. Byte-Swapping The IP protocol treats the payload only as a series of bytes; it doesn’t understand any other type. However, the application may interpret these bytes as integers, shorts or another type. Problems can occur when different systems have different endianness. Endianness is the order of bytes that construct a type. The network byte-order is always Big-endian; however the processors connecting to the network could be big or little endian. Data bytes could be swapped when transferred between the two, which cause subtle bugs that are difficult to track down. To ensure the data is in the correct endian, the data should be converted using macros provided by the TCP/IP stack to convert the between the host and network. htons() and htonl()convert from host to network, short and longs respectively. ntols and ntohl convert from network to host. It is recommended to convert data right before creating the packet and immediately after receiving a packet. Code Examples There are two code examples provide, receiver.c and transmitter.c. This code is written for use with Altera’s soft-core Nios processor and uses the TCP/IP “Plugs” library. The basic functionality of transmitter.c is to test the messaging application and measure the time required for transfer. Receiver.c processes the requests from transmitter.c using the messaging protocol and sends back a response. Conclusion This paper describes the basics of using TCP/IP in an embedded networking application. We provide a simple example of a messaging protocol embedded in a TCP packet that is transferred from a requesting system to a responding system. Although this example is simple, extensions of it to meet your particular need are straightforward.

References: (1) Donahoo, Michael J. and Calvert, Kenneth L. TCP/IP Sockets In C: Practical Guide for Programmers. Morgan Kauffman Publishers, 2001 (2) Bentham, Jeremy. TCP/IP Lean: Web Servers for Embedded Systems. Second Edition. CMP Books, 2002 (3) Hall, Eric A. Internet Core Protocols: The Definitive Guide. O’Reily, 2000