Demystifying Network Sockets (Part 1): Understanding Sockets Using IoT

Published Feb 03, 2017Last updated Mar 18, 2017
Demystifying Network Sockets (Part 1): Understanding Sockets Using IoT

This tutorial was originally posted by the author on his github page. This version has been edited for clarity and may appear different from the original post.


I’m David Gatti, and my goal with this post is to demystify Sockets. I’ll try to explain them in the simplest possible way.

In these examples, I’m going to use Node.js and Particle (any version will do) to show how hardware can talk with NodeJS, and vice versa. But make no mistake, this doesn’t mean the tools that I choose are the only way to go about this. These tools are simply what I personally know best.

Any embedded device with network connectivity will work similarly, and any language with sockets support will also do.

The Repo Structure

These folders can be found on my github page.

  • Sockets folder: Here you can find the simplest form of sockets examples. Files in this folder should help you understand how to use TCP, UDP, and TLS (Node.js only for now). These are basically templates.
  • Examples folder: Here you will find specific examples that provide more of a working solution for a specific problem.
    • NodeJS2Hardware: Using Node.js to control an LED.
    • Hardware2NodeJS: Read values from an analog PIN and stream the data to Node.js. This example also explains how to send data as chars or binary.
    • BetterTCPStreamHandling: Node.js buffers the received data until the END character is detected.
    • WebServer:
      • StaticPage: Learn how to turn your Particle into a web server.
      • DynamicPages: Learn how to turn your Particle into a web server with multiple pages.
      • DynamicContent: Learn how to turn your Particle in to a web server with dynamic content.

The End Goal

I believe there is a large measure of mystery surrounding network sockets. Many people make them sound scary — my goal is to prove that they aren’t that mysterious or complicated. I hope that by the end of this post, you will view sockets as simpler solutions to problems — especially in embedded systems, where every byte counts.

What are Sockets

Sockets are the foundations for all network connectivity. Every connected device uses sockets. On top of sockets, you'll have protocols, which are nothing other than rules that specify how data should be sent or processed after it's received. The popular HTTP protocol is an example of this. Knowing this, you can mimic any device with any language that has Socket support. How? By sending bytes that adhere to specific protocols (rules). These rules are freely available online.

You could make an app that pretends to be a:

  • Network printer
  • DNS server
  • FTP server
  • SMTP server

Why do people think Sockets are complicated?

Probably because people tend to use words that convey complexity, like:

  • I’m crafting packets.
  • It is the lowest software layer.
  • I just sent the right amount of bytes.
  • I’m implementing this protocol.

After reading these few points, you might think, "this is not for me." In reality, Sockets are actually very straightforward. For example, to get a response from a web server, you simply need to send the following text:

GET/HTTP/1.1

That's it. The server will take this text, parse it, and understand that you are making a:

  • GET request: You want something from the server.
  • /: is the root address of the site. Another example would be to write /contact to get the contact page.
  • HTTP/1.1: Tells the server which HTTP version you can understand.

There isn’t much more to it. A printer will understand another header, similarly a DNS server will need something specific to its protocol (rules).

Hard to believe? Use a telnet app to connect to your favorite site using this command (only an insecure connection will be supported through port 80).

  • telnet SITE 80
  • Type GET / HTTP/1.1
  • Press enter twice, and you’ll get the page.

For a secure connection, you can use openssl as follows:

  • Openssl s_client -connect google.com:443
  • Admire all the security that is going on.
  • GET / HTTP/1.1
  • Press enter twice. You’ll get the page.

Another example would be to send an email by connecting directly to a SMTP server. Most current SMTP servers are secured by passwords and use encryption, which makes it hard to quickly test this. However, if you had access to a plain SMTP server, you could just type the following:

  • telnet example.com smtp
    and type
  • HELO client.example.com
  • MAIL from: <sender@example.com>
  • RCPT to: <recipient@example.com>
  • DATA
  • From: sender@example.com
  • To: recipient@example.com
  • Subject: Test message empty space
  • This is my awesome message
  • . just a single dot to tell the server that we finished our message
  • QUIT

As you can see, this isn't 🚀🔬.

How to Make Your Own Rules (Protocol)

Now that we have a better understanding of protocols, you’ll need to design a common structure for communication. Let's say you want to send the temperature of your house to your Node.js server. Your stream of bytes could look like this:

45,40.1,50,90,100,102.5

The comma acts as the separator for each measurement. You can choose any character you want, but, just so you know, the comma will make your data compatible with the CSV (Comma Separated Values) format. On the other end, you'll need some code that will check for the separator, and when that happens - you get your value.

As you can see from this example, there is no header or optional data. You decide what goes in your protocol.

Based on the example above, you could add humidity to your protocol, like this:

45:80,40:85,32.1:82,50:89

Again, the comma separates your data, while the colon differentiates your data set. Remember, protocols need good documentation, so other developers can make sense of them.

Types - be aware of how you send your data.

Computers work in 1s and 0s — this is a fact. There is no way for the example to distinguish a compiled application from regular data. Everything is stored as a series of bits, which means that even the data that is sent over the Internet is in 1s and 0s.

You are probably thinking, then why should I care about types — depending on your type, your binary data will be different. For example: an integer of 1 will be 00000001, where a char of 1v in UTF8 is 31, which will become 00011111.

This means that, at the other end of the connection, you need to know what you're getting. Let's say you want to make a simple comparison.

if(data == 1) {
    true
}

If you converted your data as if it were an integer, but you sent it as a char, then you'll be comparing int 31 to int 1. But if you know that you're sending a character, you can compare it with the right type:

if(data == '1') {
    true
}

Now char 1 is actually 31, and the comparison will work. In the Hardware2NodeJS example folder, you'll find simple code that explains the difference in practice.

Which one: TCP or UDP?

On the Internet, the two most popular protocols for sending data are TCP and UDP. You'll see that people add /IP at the end of the name. IP stands for (Internet Protocol) — you can think of it as the Internet's address system. TCP and UDP are different from IP. TCP and UDP do not only apply to the Internet, they are simply the way the data is packaged. What address system you use is up to you.

TCP and UDP are simply protocols. By now, you should know that protocols simply mean rules — these two protocols are nothing more than rules explaining how to package data. The other end understands the same rules, and can unpack the received information, and vice versa.

Difference between TCP and UDP

  • TCP: A stream of data that always arrives at the other end no matter what. The rules say, if some data is lost during the transmission, resend it. This way you get high fidelity, "slower" speeds, and more data over all.
  • UDP: Sends data as a standalone packet. Once the data is sent, there is no way to tell if it reached its destination. The rules here are: I'll send it, but I won't guarantee that you'll receive it. With this protocol you get faster speeds because you won't be spending time on resending lost data. Additionally, you will not be sending information, needed to determine whether anything was lost, back to the source.

Use cases

Most of the Internet uses TCP because we want to assure that all that we send will be received on the other end. For example, if you send a document, you don't want letters or words to be missing. With many formats, if parts are missing from the header, which tells the system how to understand the file, the system will inform you that the file is corrupted and can't be read.

Of course, there are formats that are more resilient and don't need 100 percent of the data. Some movies, images, and music formats, for example, can have some content missing and we mostly just live with that! An image or movie might have some weird artifacts; music might have some blips and noises.

When should I use UDP?

This protocol is very useful when sending data from hardware products. Let's say we're collecting sensor data from many devices through a cellular modem and we have to pay for each byte we send. In this case, we should consider the following: Is it worth it to pay more for the higher fidelity we'd have with TCP? Or is it better to accept the loss of some measurements at a lower cost for our data transfer?

Normally, I would default towards TCP; however, if you have a good case against using it, UDP is your next best option.

The End

If you've enjoyed this article/project, please consider giving it a 🌟. Also check out my GitHub account, where I have other articles and apps that you might find interesting.

Where to follow

You can follow me on social media 🐙😇, at the following locations:

More about me

I don’t only live on GitHub, I try to do many things not to get bored 🙃. To learn more about me, you can visit the following links:

Discover and read more posts from David
get started
Enjoy this post?

Leave a like and comment for David

1