Transparent Communication between Microservices: Understanding RPC and Service Discovery
As we delve into the world of distributed systems, we often encounter complex scenarios where services need to communicate with each other across different machines and teams. In this article, we will explore the concept of RPC (Remote Procedure Call) and service discovery, and how they can be used to enable transparent communication between microservices.
Calling a Remote Service: The RPC Way
When we need to call a remote service, we want to do so in a way that is transparent to the user. This means that the communication details should be encapsulated, allowing the caller to focus on the business logic without worrying about the underlying network communication. This is where RPC comes in.
RPC is a widely used protocol in large Internet companies, such as Alibaba’s HSF, Dubbo (open source), Facebook’s Thrift (open source), Google’s gRPC (open source), and Twitter’s Finagle (open source). The RPC framework encapsulates the communication details, making it possible for the caller to invoke a remote service as if it were a local call.
How RPC Works
Let’s take a closer look at the RPC process:
- Service Consumer (Client): The client invokes a method on the remote service, just as if it were a local call.
- Client Stub: The client stub receives the call and assembles the method parameters and other details into a message body.
- Message Transmission: The client stub finds the service address and sends the message to the server.
- Server Stub: The server stub receives the message, decodes it, and calls the local service based on the result of decoding.
- Local Service Execution: The local service executes and returns the results to the server stub.
- Result Transmission: The server stub returns the result message to the client stub.
- Client Stub: The client stub receives the result message, decodes it, and returns the result to the service consumer.
Encoding and Decoding Messages
To enable transparent communication, we need to encode and decode messages. The message structure generally includes the following:
- Interface Name: The name of the interface being called.
- Method Name: The name of the method being called.
- Parameter Type and Value: The type and value of the parameters being passed.
- Timeout: The time limit for the call to complete.
- Request ID: A unique identifier for the request.
Serialization is the process of converting a data structure or object into a binary string, making it possible to transmit over the network. Deserialization is the reverse process, converting a binary string back into a data structure or object.
Communication Models
There are two commonly used communication models: IO (Input/Output) and NIO (Non-Blocking Input/Output). RPC frameworks need to support both models.
Using Java NIO, we can develop our own communication framework, but this approach can be complex and prone to hidden bugs. Some companies use this approach, but it’s not widely adopted.
Mina and Netty are popular RPC frameworks that use NIO to handle communication. Netty is widely used in many RPC frameworks, including Alibaba’s HSF, Dubbo, and Twitter’s Finagle.
Request ID: Ensuring Thread Safety
When using Netty, messages are transmitted using the channel.writeAndFlush() method. This approach can lead to thread safety issues, as the client may receive multiple responses from the server, making it difficult to determine which response corresponds to which request.
To address this issue, we can use a unique identifier, known as a Request ID, to ensure that the correct response is associated with the correct request.
Publishing Your Own Service
To let others use your service, you need to publish it in a way that is transparent to the caller. This can be achieved using a service registry, such as ZooKeeper.
ZooKeeper acts as a service registry, allowing multiple service providers to form a cluster and allowing service consumers to access specific services by registering with the service registry.
Service Discovery with ZooKeeper
When a service provider deploys its services, it registers with ZooKeeper by creating a path on the registry. The path includes the service name, version, and IP address.
ZooKeeper provides a “heartbeat” function, which periodically sends a request to the service provider. If the response is not received, the service provider is considered “hung up,” and the path is removed from the registry.
Service consumers monitor the registry and update their list of service providers when the data on the path changes.
In conclusion, RPC and service discovery are essential components of distributed systems, enabling transparent communication between microservices. By understanding how RPC works, encoding and decoding messages, and using service discovery with ZooKeeper, we can build scalable and fault-tolerant systems.