Let's learn "WhatsApp (or any real-time messaging system ) system design"

-

 

In this tutorial, we'll learn about the system design of WhatsApp, which is the most popular real-time messaging application today. 

Features we'll be discussing in this tutorial

  1. Detection of message status ( which can be sent, delivered, and seen )
  2. User's status ( online/ last seen)
  3. Media sharing ( includes image, videos, etc )
  4. Message encryption ( for security reasons )
  5. Support of telephony ( including audio/video calls )
  6. Group Messaging

The general design for a messaging system



In any messaging system, the general idea is the client connects to the messaging server which knows the addresses of all the clients. When any client sends any message, the messaging server is responsible for forwarding the message to the correct client.

WhatsApp Architecture



For a messaging system like WhatsApp which has billions of users, a single messaging server will not be sufficient. So the architecture will look something like the above figure. All the connections are duplex. That means data can flow in both directions.

Dataflow

Let's discuss the steps that happen when client A sends a message to client B.
  1. Initially say client A, and client B are not connected to internet.
  2. Now client A wants to send a message to client B.
  3. But because its not connected to internet the message will be saved in local database (say in sqlite) in phone of client A.
  4. Once client A connects to the internet, it creates a connection to messaging server using the loadbalancer.
  5. Once the connection is established, it pushes the message to message server.
  6. Message server checks if there is already a connection to client B. In our case since client B is not connected to internet, there is no such connection. Hence message is saved to the DB.
  7. Now say client B connects to the internet. Then it creates a connection to messaging server.
  8. Messaging server found there are some messages in DB for client B, it will send those messages to client B and delete from DB once it is received by client B.
  9. Hence message reaches from client A to client B successfully.
NOTE: If client B is already connected by the time client A sends the message, then message will never be saved in DB.

Here's is a short animation describing the message flow.


How Messaging Server works?

Lets learn about the internals of messaging server.

  • Every client when connects to messaging server starts a process/thread which maintains its message queue for message buffering.
  • The information which client is connected to which messaging server ( since we may have multiple messaging servers in different regions ) is maintained in Metadata database.
  • Every client has unique id <uid>.
  • Messages are exchanged between the clients using IPC techniques.
Lets discuss the flow inside messaging server. The aim is client A should be able to send the message to client B.
  1. Client A open connect to internet which creates a connection to messaging server.
  2. Client A connects to messaging server which starts a process with buffered queue.
  3. The initial steps the started process/thread do:
    1. Create a DB entry in Metadata DB.
    2. Checks for the messages available in Message DB that belongs to client A. If there are such messages it is pushed to the queue hence to the client A app.
    3. Messages that are pushed to client A's app also send the messages to source client updating the message status.
  4. Now client A sends a message to messaging server destined for client B.
  5. The proces/thread (<pid1>) in above figure. will check in metadata db if there is a open connection for client B ( with <uid2> ).
  6. If there is an open connection, process( <pid1> ) will send the message to process(<pid2>) for client B which ultimately is sent to client B.
  7. If there is no open connection, process( <pid1> ) will save the message to Message database. So that when client B connects to messaging server, it call pull the messages.

Details of WhatsApp features

Detection of message status ( Sent/Delivered/Seen )

We know that to tell the users the status of the message, WhatsApp use different kind of tick icons as seen in below image.


Let's learn how what's app detect a message is at which stage.

Recall we've discussed the message flow from source client to destination client. Message stage detection is done using acknowledgement messages as follows:

1. First of all, each and every message sent/received has a unique id.
2. Client A sends a message to messaging server (say id = M1).
3. Once messaging server received the message, it sent back and acknowledgement message (for M1) to client A that it has received the message. At this point message status on client A's phone gets changed to "Message sent" (with single grey tick)
4. Now when messaging server is able to send the message to client B and receive the acknowledgement message from client B, that it has received the message, messaging server send acknowledgement message back to client A for M1 that message has been delivered. At this point status on client A's for message M1 gets changed to "Message delivered" (with double grey tick).
5. Now finally when client B opens the app and see the message, it sends an acknowledgement message to messaging server that it has read the message, and that acknowledgement gets sent to client A changing the status of message to "Message read" (with double blue tick).

User Status ( LastSeen/Online )

Lets talk about the WhatApp feature which allows to see the User's status as in weather the user is active right now or when is the last time when user was active.

This one is very simple, so basically every client periodically sends its liveness status which is maintained in a separate table like:

UID

LastSeen time (epoch)

uid1

83748375

uid2

83748398

uidk

83787375

Media Sharing( includes image, videos, etc )

This is a very useful feature in WhatsApp that you can send images, videos, documents etc. to your friends.


The thing is we cannot send large files like images, videos etc. over the existing connection with the messaging server because that connection is for very light weight messages. So instead of directory sending the files over existing connection we'll be using a HTTP server for storing media files. Then HTTP server will return a HASH of the file saved in the storage. So as a message client can send the hash value and the file type to the destination client.

Message Encryption

Messages are exchanged between the different clients on any messaging system should be secure. So there must be some encryption mechanism used while exchanging the messages. WhatsApp can use either symmetric of asymmetric encryption. The basic difference between these two types of encryption is that symmetrc encryption uses one key for both encryption and decryption, and the asymmetric encryption use public key for encryption and private key for decryption.

Telephony & Group Messaging

Keeping this tutorial simple and lets talk about this in a separate tutorial.

Technology that WhatsApp use in reality

  • WhatsApp code is written in Erlang due to its good performance. It is manage to maintain 10Million connections to a single messaging server which is great.
  • Messaging server WhatsApp use is freeBSD.
  • Database used is amnesia which works great with Erlang.

That's it! Hope you like this tutorial. Feel free to comment below if you have any doubts. And Stay tuned for more tutorials :)

Comments

Post a Comment

Popular posts from this blog

Lets learn "About kube proxy in iptables mode"

-
Image
In this tutorial, we'll learn about the Kube-proxy in iptables mode. Basically, we'll see how Kube-proxy create iptables that help in load balancing and service discovery. If you are not familiar with Linux IPTables I would recommend to check out my tutorial on IPTables here  first and then come back to read this tutorial. Kube-proxy Kube-proxy when running in iptables mode basically watch kube-apiserver for new endpoints and create corresponding iptables rules and iptables are fully responsible for service discover and proper routing of traffic to corresponding pods. Kube-proxy needs to run on all k8s worker nodes. Below is a small video describing the working of kube-proxy in iptables mode.  src:  https://www.slideshare.net/CJCullen/kubernetes-networking-55835829 Now that we know what Kube-proxy is and how it Kube-proxy works, let deep dive into the types of iptables Kube-proxy adds to the firewall. Kube-proxy added IPTables To learn this let's consider an example. Consid
Read more

Lets learn "System design for paste bin (or any text sharing website)"

-
Image
  In this tutorial, we'll learn about the system design for PasteBin. If you don't know what Pastebin is here goes its definition. Pastebin is a type of online content hosting service where users can store texts to share with others over the internet.  Pastebin system design requirements  Functional Basic - User should be able to paste the text, generate the link and share the link with other users. There should be the max size of the text that our system will support (say 10Mb) Users should be able to provide the custom URL path for the link created. Like if a user wants to create the links  www.pb.com/letlearn, he can give this URL and pastebin should create a link using this URL to share with others. The link or paste expiry feature should be there for each stored text. There can be default expiry time say 1 year but this should be configurable. Personalization means the user should be able to login to track all the pastes created by him and manage them. Non-Functional Durab
Read more

Lets learn "Factory design pattern"

-
 In this tutorial, we'll learn about the factory design pattern as in what it is, in which use cases this is being used and how it works. If you've are not familiar with what are software design patterns and what are its different types check out this tutorial. Introduction A factory design pattern is a creational design pattern that is related to object creation. In this pattern, we'll have a method to create an object without exposing its creation logic to the client. If a class method is creating an object it has to be static.  The idea is to create a static method ( factory method ) which instantiates a class without exposing its logic to the client and the client can use that method every time it needs to create an object of the class. The reason why the factory pattern is useful is that as it does not expose the logic of object creation, so if logic is changed, the client code need not to be changed. Usecases The place where this pattern is mostly used is library cod
Read more