Tuesday, 11 January 2022

Nichestack OS tutorial 4 - Handling packets using the Device Driver

                                 Nichestack TCP/IP stack will also include the device drivers such as Ethernet, SLIP, PPP, PPPoE and Loopback. Each network interface will be associated with a network device structure. 

This network structure contains the  prepare function pointer. This prepare function will be called during the NicheStack initialization. There is another structure called NET structure which will contain all the device specific details. During the prepare function call, required parameters of the NET structure will be initialized. The NET structure also contains the function pointer to the driver specific routines and this will be assigned during this call. 

Next step is the initialization of devices and during this step, the stack will check whether the interface associated with the driver is OK and if it is working fine then change the device MIB status to UP.

After the driver initialization, the driver can send the packets using the packet send function. The packet will be send in the same order, which is provided to the driver. Once the packet is send, the driver will frees the packet. If the driver is busy, then the data to send will be queued and later when the driver is available, it will send the data.

Above paragraph describes the packet sending process, next we can dig into the packet receiving process. When the data is received by the device driver, the data must be stored in either chained or contiguous packet. Then the packet is placed in the stacks receive queue(rcvdq) and send the signal to the main task which is waiting for the incoming data. The signal will unlock the main task and dequeue the data and send to the upper layers based on the type of packet.

If the application layer task need to avoid the overhead during sending and receiving, then the task should use another mechanism called TCP-Zero-Copy. This feature can be used to avoid the overhead of having the stack copy data between application-owned buffers and stack-owned buffers.

We can stop the device driver using the close function. This will frees the memory resources associated with a driver and also changes the network interface status to DOWN.


Sunday, 9 January 2022

Nichestack OS tutorial 3 - Semaphores

                                In NicheStack, the semaphore acts as a signaling mechanism to notify a event occurrence. This notification will allow the waiting task to resume it's execution. Nichestack only supports the binary semaphore. Nichestack supports both signaling from ISR and task. 

Nichestack contains a main semaphore, which will be signaled while we receive any incoming data on the drivers. This will unlock the waiting semaphore inside the main task and thereby dequeues the incoming packet and based on the type it will send to relevant upper layers. The main semaphore will be created during the main task module initialization.

During signaling the task ID will be passed along with the semaphore ID and using this, the stack will understood the consumer task. Semaphore wait can be blocking or non-blocking based on the timeout value passed along with the semaphore ID. Unlike the mutexes, the semaphores are created during the module initialization.

Use case Diagram of the Interniche Semaphore Signaling





Tuesday, 4 January 2022

Nichestack OS tutorial 1 - Process of Starting a Module

Brief Introduction about NicheStack
                                                                Nichestack is primarily a TCP/IP stack with an inbuilt OS called Nichestack OS. Using the Nichestack OS, the TCP/IP stack can be directly run on a hardware or can be run on top of different third party RTOS. For this the Nichestack OS API should be mapped to the relevant RTOS API. There are different Nichestack ports already available including UCOS ii, FREE RTOS etc. Nichestack also supports another feature called Super-loop, particularly targeting the non-OS based systems. In Super-loop, there will be only one task with a single call-stack.
Nichestack was initially developed by Interniche technologies and it is taken over by HCC-Embedded and now by TUXERA.   
                                            
This post excludes the details about the Super-loop and Nichetsack OS on a third party RTOS.

Modules:
                This TCP/IP stack consists of different application layer protocols like FTP, HTTP etc. Each application layer protocol will be called as a "module"

Devices:
                There are different communication drivers present in the Nichestack such as Ethernet, PPP etc. These drivers associated with an interface is called as a "Device".

The below diagram describes the step by step process, while starting a module using Nichestack. Under the section initialize the modules, the socket section mentioned is with respect to server code.



Monday, 3 January 2022

Nichestack OS tutorial 2 - NET Resource method and Critical Section

There are two types of mutexes present in the Nichestack OS. They are: 

     1. NET Resource Method: This allows the programmer to obtain and release a mutex for accessing the shared resources. The API's used to obtain and release the mutexes are LOCK_NET_RESOURCE() and UNLOCK_NET_RESOURCE() respectively

In-order to use this mutex for a project we must create it first. Usually, the mutexes are created during the OS initialization phase by calling the mutex create API. All the mutexes used in a project will be available in the ipport.h file. Based on the maximum mutex number ID the mutexes will be created during the OS initialization phase.

The nichestack OS also has the TRY_NET_RESOURCE () and UNLOCK_NET_RESOURCE() locking mechanism. The difference between LOCK_NET_RESOURCE and TRY_NET_RESOURCE is the former will wait until we acquire the mutex, but the latter checks if we can get the mutex or not else skip.

Few mandatory mutexes present in the Nichestack OS are NET_RESID, RXQ_RESID and FREEQ_RESID. The NET_RESID must be obtained by the higher-level application while accessing the Sockets, TCP, UDP and the IP layers of the stack. RXQ_RESID must be obtained while accessing the data queue structure (getq() and putq()). FREEQ_RESID must be obtained while accessing the free packet buffer queue structure (PK_ALLOC() and PK_FREE()).

Points to consider while porting, if one task is obtained the NET_RESID mutex then the other task needs to wait till the first task release it. There won’t be any effect between different mutexes that means if one task is locking the NET_RESID then other task can obtain other mutexes except the NET_RESID mutex. If there is a case in which a task needs both the NET_RESID and FREE_RESID then NET_RESID should be locked first followed by FREE_RESID and for releasing FREE_RESID followed by NET_RESID mutex. Same will be applicable in the case of RX_RESID and FREE_RESID

The porting engineer can add a new mutex, if they want to protect the shared resources. One use case scenario for this, consider there are multiple client instances accessing the shared memory variables, then it must be protected using a mutex.

Never try to do nesting on same mutex calls. For example:

Task1()

{

LOCK_NET_RESOURCE(NET_RESID);

          ……………

          LOCK_NET_RESOURCE(NET_RESID);

}

Here we are trying to access the same           mutex twice in the task function.

Use case Scenario of NET Resource       method implementation:

Consider there are 2 tasks, FTP

Task and Telnet Task and the FTP task needs to get the received data on a blocking socket. The FTP task locked the NET_RESID mutex and because it is configured as blocking socket, it will wait till the data arrives, but the data is delaying for a long time, now inside the lower TCP layer the tcp_sleep() function will be automatically called and tcp_sleep() will release the NET_RESID lock from the FTP task and change the state to suspended. Also tcp_sleep() function will wait for the receive signal(SIGWAIT).
Now NET_RESID is free and the TELNET task needs to access the socket for getting its received data. If the telnet data is already available then read the data and does the processing and release the NET_RESID mutex, if not telnet task needs to follow the same steps like mentioned in the FTP case and release the mutex.

2. Critical Section Method: The difference between the Net resource method and      critical section is the latter will be used in the lower layers of the OS like getting and  putting the queue and also other task related functions(tk_XXX)

The API's used for the entering and exiting the critical sections are: ENTER_CRIT_SECTION() and EXIT_CRIT_SECTION() respectively While entering the critical section, all the interrupts will be disabled and enable it back while exiting the critical section.

The Nichestack OS and TCP/IP stack can run on top of a RTOS or directly on the hardware. If there is no RTOS and the ISR will not access any shared resource, then the Enter and Exit critical section API's were no-ops. If the ISR is accessing the shared resource, then the Critical section method should be used.

In the case of hard real time system projects care should be taken that the ISR will not access any shared resources. So while entering the critical section only other tasks needs to be waited not the ISR.

The entering and exiting process sometimes can be nested. For example:     

Func1()

{

          ENTER_CRIT_SECTION()

          …………

          EXIT_CRIT_SECTION ()

}

Func2()                                       

{

          ENTER_CRIT_SECTION()

          Func1();

          ……….

          EXIT_CRIT_SECTION ()

}


The main difference between the critical section and mutex is the former will completely disables all the interrupts but the latter only block other tasks to access the shared resource. The critical section method is mostly used in the low levels and due to the reason that it is disabling the interrupts, the execution should not delay too much.


Please check out this project to get more details about the implementation.

https://github.com/songwenshuai/NICHESTACK

Wednesday, 8 December 2021

PPP Server - Link Termination Phase

                                 Consider our device is PPP Server and other peer is PPP Client. There are 5 states present in the PPP communication. They are ESTABLISH, AUTHENTICATE, NETWORK, TERMINATE and DEAD states. If the client or server moved to the PPP Terminate State, then that device can no longer accept the LCP packets so re-start is not possible. We will discuss more about this later in this post. As the topic is about the Link termination phase we are not going into the details about the other states.

In PPP communication both the parties can initiate the Link(Connection) Termination. The link termination is achieved through the exchange of 2 messages - Terminate Request and Terminate Acknowledgement. 

If the client needs to terminate the connection, a terminate-REQ message is send and wait for the terminate-ACK message from the server. Upon receiving the ACK message the PPP state will be changed to TERMINATE and LCP state to CLOSED subsequently. Same will be applicable if the server initiates the link termination. In the server side, after sending the ACK message to the client, the server should wait until at least one restart timer has passed for changing the state to DEAD. After that if same or different client tries to re-connect, then the server will move from the DEAD state to ESTABLISH state upon receiving the LCP Config-REQ message from the Client.

If the client is initiating the link terminate, then server will move to DEAD state instead of TERMINATE state so, later it can re-establish another connection. But for the opposite scenario the state will be changed to TERMINATE and cannot re-establish the connection.

Monday, 8 March 2021

Big Endian Read and Write


/* This demo program shows how to read */
/* and write big-endian data  */

#include  stdio.h
#include  stdint.h

static inline uint32_t read_32bit_be( const uint8_t * const ptr_buf )
{
  uint32_t  byte0;
  uint32_t  byte1;
  uint32_t  byte2;
  uint32_t  byte3;

  byte0 = ptr_buf[0];
  byte1 = ptr_buf[1];
  byte2 = ptr_buf[2];
  byte3 = ptr_buf[3];

  byte0 <<= 24;
  byte1 <<= 16;
  byte2 <<= 8;

  return  ( byte0 | byte1 | byte2 | byte3 );
}

static inline void  write_32bit_be( uint8_t * ptr_buf, uint32_t value )
{
  ptr_buf[0] = value >> 24;
  ptr_buf[1] = value >> 16;
  ptr_buf[2] = value >> 8;
  ptr_buf[3] = value;
}

int main()
{
  uint8_t buffer[4];
  uint32_t val_32bit;

  val_32bit = UINT32_MAX;

  /*write the 32 bit value to the buffer*/
  write_32bit_be( buffer, val_32bit );

  val_32bit = 0;
  /*Retrieve the big endian value from the buffer*/
  val_32bit = read_32bit_be( buffer );

  printf( "Big-Endian value - %X", val_32bit );

  return 0;
}

Static and Dynamic Configuration in Embedded C Programming


/* This demo program shows the static and dynamic */
/* configuration with respect to IPv4 and IPv6 */
/* Compiler Used - Visual Studio */
#include stdio.h
#include stdint.h

/* IP_ENABLE == 0: none */
/* IP_ENABLE == 1: IPv4 */
/* IP_ENABLE == 2: IPv6 */
/* IP_ENABLE == 3: IPv4 and IPv6 */
#define IP_ENABLE     0

#define TRUE          1
#define FALSE         0

typedef struct
{
  int ipv4_enable;
  int ipv6_enable;
}ipconfig;

ipconfig ip_config;

static void set_ip_config( ipconfig * ptr_config )
{
  ip_config.ipv4_enable = ptr_config->ipv4_enable;
  ip_config.ipv6_enable = ptr_config->ipv6_enable;
}

int main()
{
  ipconfig ip; /*for getting the input*/
  /*Static configuration*/
#if ( IP_ENABLE == 0 )
  ip_config.ipv4_enable = FALSE;
  ip_config.ipv6_enable = FALSE;
#elif ( IP_ENABLE == 1 )
  ip_config.ipv4_enable = TRUE;
  ip_config.ipv6_enable = FALSE;
#elif ( IP_ENABLE == 2 )
  ip_config.ipv4_enable = FALSE;
  ip_config.ipv6_enable = TRUE;
#elif ( IP_ENABLE == 3 )
  ip_config.ipv4_enable = TRUE;
  ip_config.ipv6_enable = TRUE;
#endif

  printf( "IPv4: %d\t", ip_config.ipv4_enable );
  printf( "IPv6: %d\n", ip_config.ipv6_enable );

  /*Dynamic Configuration*/
  (void)scanf( "%d", &(ip.ipv4_enable) );
  (void)scanf( "%d", &(ip.ipv6_enable) );

  set_ip_config( &ip );

  printf( "IPv4: %d\t", ip_config.ipv4_enable );
  printf( "IPv6: %d\n", ip_config.ipv6_enable );

  if ( ip_config.ipv4_enable == TRUE )
  {
    /*Get the IPv4 address from the DHCP server*/
  }

  if ( ip_config.ipv6_enable == TRUE )
  {
    /*Get the IPv6 address from the DHCPv6 server*/
  }
  
  return 0;
}

Friday, 5 March 2021

Point to Point Protocol Communication(PPP)

 

                                PPP(Point to Point Protocol) is a layer 2 protocol(Datalink), commonly used for the communication between switches, routers etc.  PPP can communicate over Serial link(PPPd) or Ethernet(PPPoE) or USB CDC ACM. The PPP peers should pass through different phases to establish the connection. These phases are- 

  • LCP (Link Control Protocol)  - 
                                                          This is the starting phase of the PPP communication and initially both the peers send the config request messages. By receiving the config request message the peer came to know about the feature supported by other side. If some of the features are not supported then it will send config NAK message to other side. This negotiation will continue until both sides agree about the features that is going to be used in the further communication. This LCP negotiation will ends with sending the Config ACK message to the other side which declares about the features which are agreed.
Below you can see the wireshark packet which shows LCP communication. Here no Config NAK can be seen because both sides support the features mentioned in the config request packet. 

  • Authentication(PAP, CHAP...) - 
                                                          This is an optional phase that means we can also directly move from the LCP phase to IPCP phase bypassing authentication phase. Two commonly used authentication method in PPP communication are PAP(Password Authentication Protocol) and CHAP(Challenge Handshake authentication Protocol). Both the peers will make an agreement about which protocol should be used for authentication in the LCP phase itself. More about the PAP and CHAP protocols
    1. PAP - User can select either of these authentication protocols to use in their PPP communication but comparing PAP with CHAP, CHAP will provide more security. In the PAP authentication the peer will send the username and password to the other side. Then the other side will verify the username and password with its database. If verification is successful then the access will be granted. This PAP authentication process can be done in both ways.
    2. CHAP - I mentioned earlier that the CHAP authentication is more secure compared to PAP and the reason is, in the CHAP password will not send through network to the other side.  During the CHAP communication the peer will receive a challenge message from other side and using the challenge message and the stored password the peer will create a hash value. This hash value will be send to the other side and in the other side the same process(password + challenge message) will be done to generate the hash value. This hash value will be compared with the received hash value from the peer, if both matches then the authentication is success. Like PAP, CHAP can also be done in both ways. Below wireshark packet shows 2 way CHAP authentication.


  • NCP(Network Control Protocol) or IPCP(IP Control Protocol
                                                                 If the authentication process is successful then the PPP communication will be move to the IPCP phase. Similar to the LCP phase, in the IPCP phase also different negotiations will be happen between the peers. Some features may be rejected and the peer will send the Config NAK message to the other side. Some negotiations happen in this phase includes features such as DNS Server address, IP Compression, WINS Server etc. In the IPCP phase the peer acting as server will assign the IP address to the client. This IP address will be assigned in the Config ACK send from the server to client and thereby the client and server move to the connected state. Below wireshark shows the IPCP packet exchange.  
    
                                                                    Once the NCP phase is completed the IP communication can be started between the client and the server. PPP support both the IPv4 and IPv6 and it also supports IP compression(reduces the overhead which is good for low bandwidth networks) protocol such as Van Jacobson Compression.
                                                                    If you are having 2 raspberry pi then you can easily try out the working of PPP communication. I had experience in PPP over serial, so I will only mention the PPP serial commands. Make the serial connection between 2 raspberry pi's. Install the pppd services and packages. For this you can find lots of tutorials in the internet. 
PPP Client Command:

sudo pppd -detach lock /dev/ttyAMA0 115200 debug auth dump record client.pcap +chap local noipdefault defaultroute 0.0.0.0:0.0.0.0

PPP Server Command:

sudo pppd -detach lock 192.168.151.101:192.168.151.203 /dev/ttyAMA0 115200 debug auth local dump record server.pcap +chap

You can find more information about these commands from this website - PPPd commands


Monday, 15 February 2021

Ways to Obtain DNS Server Address in IPv6 Client

                                                   I assume you had a brief idea about DHCPv6 and ICMPv6.

                                                   DHCPv6 Client will usually obtain the dynamic IP address through 4 messages. Those messages are SOLICIT, ADVERTISE, REQUEST and REPLY. So during these message transaction the DHCPv6 client can also request the DNS server address using the ORO(Option Request Option) option. If the client request is accepted by the server then reply packet will contains the DNS Server address. Like wise the client can also request other ORO options.

                                                    The IPv6 client can also obtain the dynamic IPv6 address using other mode called SLAAC mode(Stateless Auto Address Configuration). The IPv6 client will usually perform the SLAAC mode when the managed bit flag is not set in the received router advertisement(RA) message  whereas if the managed bit is set then client will use DHCPv6 client. In the SLAAC mode itself there are 2 ways to obtain the DNS Server address. First one is through RA message itself and it is called as RDNSS and other method will be used when the received ICMP RA message contains the other bit flag set. If the other bit flag is set then DHCPv6 client will be used for getting the additional information's like DNS Server address.

                                                    The final method is of-course the static configuration of the DNS Server Address.

Friday, 27 November 2020

Process of finding a file from the EXFAT filesystem

                                     I hope the reader will have the basic knowledge about the EXFAT filesystem. For finding a file from the SD card formatted with EXFAT filesystem, we need to read the file entries.   

                                     Basically there are 3 or more file entries associated with a single file. These file entries are File directory entry, Stream Extension directory entry and Filename Directory entry. Filename directory entry contains the filename and a single file can have more than one filename entry depending on the length of the filename but only can have one file and stream entry. Here I am not going into the detailed explanation of the file entries. But briefly I can say these file entries hold all the details about a file including starting cluster, file attributes, timestamp etc.

                                     Lets assume we have to find a file called "myfile" which exists in directory "mydir/myfile" and the "mydir" contains different other files. Our PWD is "mydir" and we are staring a file search then file entry read will starts from the first cluster of  "mydir". Initial process is to read the file directory entry located at the starting cluster of the "mydir". From the file directory entry we will get the secondary count value. The secondary count denotes the number of secondary directory entries(stream and filename entry) following the primary directory entry(File directory entry). 

                                     Next step is to read the stream extension directory entry and filename directory entry from the media. After reading these entries, from the filename directory entry we will extract the filename. As you know that EXFAT filesystem is case insensitive, we need to convert the filename to upcase and then compares with the required filename(already up-cased). If both filename matches we have successfully found the file we are looing for, otherwise again repeat the previous steps - reading the directory entries and comparing the filename. There are several other internal process involved during the search but in this post I am trying to give a high level point of view of the search process.


Thursday, 26 November 2020

How to make the TCP server unreachable(NO response) on the fly

                                   I would like to share one of my previous experience with TCP Client-Server testing. My use case was to test the behavior of TCP client while the server is unreachable. Before going into the details, I will briefly introduce the test setup which I used:

  • TCP Client-Server with 3-way handshake mechanism.
  • TCP Client-Server  IP version- IPv4 
  • Code to test - TCP Client 
  • Platform used - Windows 10 PC with additional ethernet adapter

                                   So now my PC has got 2 ethernet interfaces and the client will run on the Ethernet interface 1 and server will run on the Ethernet interface 4.

                                   More about the scenario, after the connection is established, client tries to send some data[PSH, ACK]to the server when the server is unreachable. The server should go to the unreachable state without sending any notification to the client. This is where the issue exists because when I tried to disconnect or disable or close the server, it will send either [RST, ACK] or [FIN, ACK]message to the client. I also tried to disable the ethernet interface where the server is running but no success. 

                                    Finally I figured out there is a simple technique to make the server unreachable without sending any notification to the client. For this go to the specific ethernet adapter where your server is running and click on the properties and uncheck the Internet Protocol Version 4(TCP/IPv4) and that's it. I hope this trick will be helpful for some one with similar test environment.


Sunday, 11 October 2020

Useful Tools for Software Developers - Part 2

                        This is the continuation of the previous post. Again I would introduce some other open source tools with I used for my previous projects

  • Active DISK EDITOR:                                                                                                                     This tool will allow you to have a detailed look into the internal structure of the filesystem. This will support different filesystems including the FAT32, EXFAT, NTFS etc. You can perform an internal walkthrough for both the physical disk as well as the software image file. As the tool name suggests, you can edit different file system fields.  https://www.disk-editor.org/index.html

  • Frhed:                                                                                                                                                 This is a light weight binary editor for windows. You can edit different kinds of files including the pcap file, image file etc. This also allows you to even truncate the file as well.http://frhed.sourceforge.net/en/
  • Win 32 Disk Imager:                                                                                                                          I think lot of people are familiar with this tool. Win 32 Disk Imager is used for writing, reading and verifying the image to and from the SD cards.  https://sourceforge.net/projects/win32diskimager/
  • Whack Whack Terminal:                                                                                                                  Another visual studio plugin, this will allow you to open different terminals including bash, windows in the visual studio project window. The main advantage of this tool is you can execute CLI commands without switching the Visual Studio Application. https://marketplace.visualstudio.com/items?itemName=dos-cafe.WhackWhackTerminal

Saturday, 10 October 2020

Useful tools for Software Developers - Part 1

I would like to introduce some useful open source tools which I used for my previous projects.   
  • Dibbler Dhcpv6 Server
A light weight command line Dhcpv6 server. This software will actually include both dhcpv6 client and server. Useful for testing both client and server code.http://klub.com.pl/dhcpv6/
  • Technitium DNS Server
Good looking DNS Server with online dashboard available for configuration. This will support both IPV4 and IPV6, so it can be used with either DHCP or DHCPV6 server for testing.https://technitium.com/dns/
  • DHCP Server for Windows
Small DHCP server with decent UI and easily configurable.https://www.dhcpserver.de/cms/download/
  • TCP REPLAY:
Command line tool used for sending pre captured packets to a particular interface. This will be very handy when you want to verify the behavior of client or server with erroneous packets. Installation is bit tricky in windows, you require libpcap and cygwin.                                         https://tcpreplay.appneta.com/#:~:text=Tcpreplay%20is%20a%20suite%20of,to%20replay%20to%20web%20servers.
  • PCATTCP or TTCP:
Command line tool used for testing the performance of TCP and UDP. If you want to verify your TCP or UDP code,  then this will a good tool for this. Both windows and linux compatible. Simple Usage: Open 2 command lines one as a reciever and other as a transmitter. Official page is not working but you can download from this website.
  • UNCRUSTIFY:
An open source code beautifier for C, C++, JAVA etc.                                          https://github.com/uncrustify/uncrustify