Network Programming in Elixir and Erlang

Write High-Performance, Scalable, and Reliable Apps with TCP and UDP

by Andrea Leopardi

TCP, UDP, DNS, HTTP, and more: these are the network protocols that make up the fabric of the Internet. Erlang and Elixir are the perfect fit for building network-intensive applications—the BEAM’s actor model perfectly mirrors the way nodes operate in a network. Learn about networking and the power of the BEAM to write performant and reliable network applications. Create systems that are scalable, resilient, and efficient, thanks to language primitives and OTP. Take advantage of an ecosystem that has been solving network problems for more than thirty years. Learn about design patterns and common pitfalls for network applications on the BEAM.

From smart appliances to gigantic data centers, from phones to satellites, networks are the way computers talk to each other. Learn how to use network protocols, choose or design serialization protocols, and architect systems with servers and clients.

Start with the most widely-used protocol on the Internet: TCP. Build a chat server where multiple clients can chat in real time. Explore client-side TCP by building a client for Redis. Scale and harden server and client, thanks to features of the BEAM. Then dig into UDP, TCP’s “looser” sibling. Code a system for reporting metrics capable of rivaling hardened software like StatsD. Learn about DNS, which powers domain resolution for the Internet. Next, secure your TCP traffic with TLS. Lastly, explore HTTP, the protocol that even your microwave could be using. Build a JSON API, client, and server. Learn the differences between HTTP/1.1 and HTTP/2, and add real-time features via WebSockets.

The BEAM was built for networks and has been honed for more than three decades. It’s the perfect fit. Leverage years of real-world experience building network-intensive applications from a member of the Elixir core team, and become an expert at network programming in Elixir and Erlang.

• Errata, typos, suggestions

• Source Code (zip file)

Releases:

• B3.0 2024/10/14
• B2.0 2024/09/17
• B1.0 2024/08/19

Note: Contents and extracts of beta books will change as the book is developed.

Introduction

What Is Network Programming Anyway?

• Erlang and Elixir: The Perfect Fit
• Which One, Though, Erlang or Elixir?
• Where Do We Go Next?

TCP

• TCP: Exploring the Basics
  • TCP 101
  • Building a TCP Echo Server
  • Testing with TCP
  • Becoming Socket Pros with Modes and Packet Parsing
  • Wrapping Up
• Designing a Chat Protocol and Its TCP Server
  • Designing a Simple Chat Protocol
  • Writing Code For the Chat Protocol
  • Moving to the Server Side
  • Chatting with Clients
  • Wrapping Up
• Scaling TCP on the Server Side
  • Increasing Scalability with Multiple Acceptors
  • Rewriting the Server with ThousandIsland
  • Wrapping Up
• Building TCP Clients
  • The Redis Protocol
  • Building Our First Client
  • Queuing Requests
  • Wrapping Up
• Scaling and Optimizing TCP Clients
  • Pooling TCP Client Sockets
  • Building State-Machine-Like TCP Clients
  • Pooling Connections Through Registry
  • Wrapping Up
• Securing Protocols: TLS

UDP

• Same Layer, Different Protocol: Introducing UDP excerpt
  • The Basics of the Protocol
  • Building a Metrics Ingestion Daemon
  • Wrapping Up
• Adding Guarantees to UDP
  • Ordering, Splitting, and Dropping Packets
  • Adding Guarantees
  • Wrapping Up
• UDP in the Wild: DNS
  • Understanding How DNS Works
  • Writing a Simple DNS Client
  • inet_res
  • Writing a DNS Server Is So Easy
  • Wrapping Up

HTTP

• Talking the Internet Protocols: HTTP/1.1 and HTTP/2
• Communicating in Real-Time with WebSockets

Appendices

• The OSI Model
• Comparing Elixir and Erlang Syntaxes
• TCP Protocol Details
• UDP Broadcasting
• DNS Protocol Details
  • DNS Message Header
• Controlling Other Machines with SSH