Introduction
In this article, we will see how we can use gRPC implementation of client and server in Go language. If you are new to gRPC, I would recommend you to read my previous post on Introduction to gRPC.
We will build a simple client and server in Go and make them talk to each other. The source code mentioned in this post is available at Github.
Assumption
This article assumes that you understand Go programming and have all the required setup done in your system. It assumes that you have protobuf package and protoc command available. It would also need you to have Go dependencies such as the go implementation for protobuf. If you’re new to Go, then try out playing with the code using Go Playground and article on how to write go code.
Project Structure
In general, the workspace/project should be created under GOPATH/src directory, but its not mandatory after the introduction of Go modules in Go 1.11. Its good enough to have one main.go under a directory for simple projects. A real project may have more directories based on the need, and there are many project-layouts.
The project at its root from GOPATH contains three directories and the repository github.com/jobets/grpc-golang is placed under GOPATH/src directory as seen below:
bin:
The bin directory contains all the executable binary. The command go build and install executed against a main package generates the executable binary and places it in the bin folder.
pkg:
The pkg directory contains archived files in .a format. These are shared libraries that cannot be executed directly as it is not in a binary format. They are executed as part of other projects. External projects normally imports these libraries to get the required functionality.
src:
The src directory contains the versioned repository “github.com/jobets/grpc-golang” where each directory is a package containing .go files.
# Project Structure
bin/
hello # Executable binary
pkg/
linux_amd64/
github.com/ # Package archive
go-kit/
kit/
auth/
basics.a
casbin.a
jwt.a
src/
github.com/jobets/grpc-golang/ # Project repository
.git/
api/
generated/
hello-service.pb.go # Generated file from hello-service.proto
proto/
hello-service.proto # Protobuf file containing the api definition
client/
client.go # Client program
server/
server.go # Server program
Makefile # To generate protobuf files using protoc compiler
#In general, the repository may have third party folder containing third-party packages
Define Protobuf
The protobuf file provides an API contract which is exposed for other services to implement and consume the service. It defines request and response of the service with service definition. The protobuf provides messages and services. The messages consist of request and response attributes which are objects, and services are defined by SayHelloUnary method as seen below:
syntax = "proto3";
package api;
option go_package="api";
//message Hello
message Hello {
string first_name = 1;
string last_name = 2;
}
//Hello Request
message HelloRequest {
Hello hello = 1;
}
//Hello Request Multiple Times
message HelloRequestMultipleTimes {
Hello hello = 1;
}
//Hello Response
message HelloResponse {
string result = 1;
}
//Hello Response Multiple Times
message HelloResponseMultipleTimes {
string result = 1;
}
//Create a Hello Service
service HelloService{
// Unary
rpc SayHelloUnary(HelloRequest) returns (HelloResponse) {};
//Client Streaming RPC
rpc SayHelloClientStreaming(stream HelloRequestMultipleTimes) returns (HelloResponse) {};
//Server Streaming RPC
rpc SayHelloServerStreaming(HelloRequest) returns (stream HelloResponseMultipleTimes) {};
//Bidirectional Streaming RPC
rpc SayHelloBidirectionalStreaming(stream HelloRequestMultipleTimes) returns (stream HelloResponseMultipleTimes) {};
}
NOTE: We have used proto3 syntax for this article.
Generate (Client & Server Stubs)
The client and server stubs for the above protobuf definition is auto-generated by using the protoc compiler. I have used Makefile and added a goal to generate the stubs as seen below. In this example, the client and server stubs are auto-generated by issuing command make from the root of the repository.
#Path to .proto files PROTO_PATH := api/proto # Output directories. GRPC_OUT := api/generated protoc: # Generate proto stubs. protoc \ -I $(PROTO_PATH) \ --go_out=plugins=grpc:$(GRPC_OUT) \ $(PROTO_PATH)/*.proto
Once successfully executed, it will generate a file “hello-service.pb.go” that will have both client and server stubs which needs to be implemented. Now lets see the implementation (client and server) of 4 types of gRPC service methods:
- Unary RPC
- Client Streaming RPC
- Server Streaming RPC
- Bidirectional Streaming RPC
NOTE: The auto-generated file hello-service.pb.go in practice should never be modified or changed manually.
Create Server (Unary RPC)
The Unary RPC is similar to a request sent by a client, and server responds back with a response. The below code implements the Unary RPC server implementation as seen in repository file server.go.
package main
import (
"context"
"fmt"
"io"
api "github.com/grpc-golang/api/generated"
"log"
"net"
"google.golang.org/grpc"
)
type server struct{}
func (*server) SayHelloUnary(ctx context.Context, req *api.HelloRequest) (*api.HelloResponse, error) {
fmt.Printf("SayHelloUnary function was invoked with %v\n", req)
firstName := req.GetHello().GetFirstName()
LastName := req.GetHello().GetLastName()
result := "Hello " + firstName + LastName + "!"
res := &api.HelloResponse{
Result: result,
}
return res, nil
}
func main() {
fmt.Println("Hello World!!!")
//Start creating TCP listener to bind the gRPC server on port 5051.
lis, err := net.Listen("tcp", "0.0.0.0:5051")
if err != nil {
log.Fatal("Failed to listen: %v", err)
}
//Create instance of gRPC server
s := grpc.NewServer()
//Register the service and start the server
api.RegisterHelloServiceServer(s, &server{})
if err := s.Serve(lis); err != nil {
log.Fatal("failed to server: %v", err)
}
}
In the main() method, server struct{} is an abstraction of the server. It allows to attach the resources on the server for gRPC calls. The SayHelloUnary() method is defined as rpc call for gRPC service in hellp-service.proto file. The SayHelloUnary() method takes the request api.HelloRequest and returns the response of type api.HelloResponse. It can also return error in case of any.
Create Client (Unary Client)
Now lets write the gRPC implementation of client code seen below for the Unary RPC as seen in repository client.go
package main
import (
"context"
"fmt"
"google.golang.org/grpc"
"io"
api "github.com/grpc-golang/api/generated"
"log"
"time"
)
func main() {
fmt.Println("Hello I'm a client")
// Instantiate a client connection to bind with the server listening on TCP port 5051
cc, err := grpc.Dial("localhost:5051", grpc.WithInsecure())
if err != nil {
log.Fatal("could not connect: %v", err)
}
// defer closes the connection after the function returns
defer cc.Close()
// Create the client HelloServiceClient
c := api.NewHelloServiceClient(cc)
//Invoke the unary client function
checkUnary(c)
}
func checkUnary(c api.HelloServiceClient) {
fmt.Println("Starting a Unary RPC...")
req := &api.HelloRequest{
Hello: &api.Hello{FirstName: "Jobet",
LastName: " Samuel",
},
}
res, err := c.SayHelloUnary(context.Background(), req)
if err != nil {
log.Fatal("error while calling Hello RPC: %v", err)
}
log.Printf("Respone from Unary Server: %v", res.Result)
}
The checkUnary() function sets the request HelloRequest by setting FirstName and LastName and pass it to Unary server by invoking c.SayHelloUnary(context.Background(), req). Now lets run the server and then client to see the results.
sam:~/goLang/src/github.com/grpc-golang$ go run server/server.go Hello World!!! SayHelloUnary function was invoked with hello:<first_name:"Jobet" last_name:" Samuel" > sam:~/goLang/src/github.com/grpc-golang$ go run client/client.go Hello I'm a client Starting a Unary RPC... 2020/06/24 16:10:53 Respone from Unary Server: Hello Jobet Samuel!
Create Server (Client Streaming RPC)
The Client Streaming RPC is similar to Unary RPC, except the client sends streams of requests instead of single request. We will now implement the server-side code for processing a stream of request from the client-side.
func (*server) SayHelloClientStreaming(stream api.HelloService_SayHelloClientStreamingServer) error {
fmt.Printf("SayHelloClientStreaming function was invoked : \n")
result := ""
for {
req, err := stream.Recv()
if err == io.EOF {
return stream.SendAndClose(&api.HelloResponse{
Result: result,
})
}
if err != nil {
log.Fatalf("Error while reading client stream: %v", err)
return err
}
firstName := req.GetHello().GetFirstName()
result += "Hello " + firstName + "! "
}
}
The above function starts with a for–loop to receive a stream of request by using the method stream.Recv(). Then, it checks for EOF ( End-of-File, i.e the stream of requests are completed) and returns the result or err if any.
Create Client (Client Streaming RPC)
Lets now write the gRPC implementation of client code to push a stream of requests to the server.
func checkClientStreaming(c api.HelloServiceClient) {
fmt.Println("Starting a Client Streaming RPC...")
requests := []*api.HelloRequestMultipleTimes{
&api.HelloRequestMultipleTimes{
Hello: &api.Hello{
FirstName: "Jobet",
},
},
&api.HelloRequestMultipleTimes{
Hello: &api.Hello{
FirstName: "Mark",
},
},
&api.HelloRequestMultipleTimes{
Hello: &api.Hello{
FirstName: "John",
},
},
&api.HelloRequestMultipleTimes{
Hello: &api.Hello{
FirstName: "Matthew",
},
},
}
stream, err := c.SayHelloClientStreaming(context.Background())
if err != nil {
log.Fatal("Error while calling HelloClient: %v", err)
}
for _, req := range requests {
fmt.Printf("Sending request: %v\n", req)
stream.Send(req)
time.Sleep(1000 * time.Millisecond)
}
res, err := stream.CloseAndRecv()
if err != nil {
log.Fatal("Error while receiving response from Server: %v", err)
}
fmt.Printf("Response received from Client Streaming Server: %v\n", res)
}
The above code starts with creating request array requests := []*api.HelloRequestMultipleTimes. Then we implement the client method c.SayHelloClientStreaming(context.Background()) with context that returns stream and error. In for-loop we send stream of requests by stream.Send(req). Then close the stream and handle the errors if any, by using the method stream.CloseAndRecv(). We will run both server and client to see the response as seen below:
sam:~/goLang/src/github.com/grpc-golang$ go run server/server.go Hello World!!! sam:~/goLang/src/github.com/grpc-golang$ go run client/client.go Hello I'm a client Starting a Client Streaming RPC... Sending request: hello:<first_name:"Jobet" > Sending request: hello:<first_name:"Mark" > Sending request: hello:<first_name:"John" > Sending request: hello:<first_name:"Matthew" > Response received from Client Streaming Server: result:"Hello Jobet! Hello Mark! Hello John! Hello Matthew! "
Create Server (Server Streaming RPC)
In Server Streaming RPC similar to Unary RPC, except the server sends streams of requests to the client. we will now implement the server-side code by creating a stream of responses from the server to the client as seen below:
func (*server) SayHelloServerStreaming(req *api.HelloRequest, stream api.HelloService_SayHelloServerStreamingServer) error {
fmt.Printf("SayHelloServerStreaming function was invoked with %v\n", req)
firstName := req.GetHello().GetFirstName()
for i := 0; i < 10; i++ {
result := "Hello " + firstName + " number " + strconv.Itoa(i)
res := &api.HelloResponseMultipleTimes{
Result: result,
}
err := stream.Send(res)
if err != nil {
log.Fatalf("Error while sending to client stream: %v", err)
return err
}
time.Sleep(1000 * time.Millisecond)
}
return nil
}
The above function creates a response and in a for–loop sends a stream of response by using the method stream.Send(res). We print the loop iteration count by method strconv.Itoa(i) and returns err if any.
Create Client (Server Streaming RPC)
Lets now write the gRPC implementation of client code to send a request to the server.
func checkServerStreaming(c api.HelloServiceClient) {
fmt.Println("Starting a Server Streaming RPC...")
req := &api.HelloRequest{
Hello: &api.Hello{FirstName: "Jobet",
LastName: " Samuel",
},
}
res, err := c.SayHelloServerStreaming(context.Background(), req)
if err != nil {
log.Fatal("error while calling Hello RPC: %v", err)
}
for {
msg, err := res.Recv()
if err == io.EOF {
// Reached end of stream
break
}
if err != nil {
log.Fatal("Error while reading froms stream: %v", err)
}
log.Printf(" Response from Server Streaming RPC: %v", msg)
}
}
The above code starts with creating request api.HelloRequest. Then we implement the client method c.SayHelloServerStreaming(context.Background(), req) with context that returns res or error. In for-loop it process stream of messages received by res.Recv() method and handle the errors if any. We will run both server and client to see the response as seen below:
sam:~/goLang/src/github.com/grpc-golang$ go run server/server.go Hello World!!! SayHelloServerStreaming function was invoked with hello:<first_name:"Jobet" last_name:" Samuel" > sam:~/goLang/src/github.com/grpc-golang$ go run client/client.go Hello I'm a client Starting a Server Streaming RPC... 2020/06/25 13:17:20 Response from Server Streaming RPC: result:"Hello Jobet number 0" 2020/06/25 13:17:21 Response from Server Streaming RPC: result:"Hello Jobet number 1" 2020/06/25 13:17:22 Response from Server Streaming RPC: result:"Hello Jobet number 2" 2020/06/25 13:17:23 Response from Server Streaming RPC: result:"Hello Jobet number 3" 2020/06/25 13:17:24 Response from Server Streaming RPC: result:"Hello Jobet number 4" 2020/06/25 13:17:25 Response from Server Streaming RPC: result:"Hello Jobet number 5" 2020/06/25 13:17:26 Response from Server Streaming RPC: result:"Hello Jobet number 6" 2020/06/25 13:17:27 Response from Server Streaming RPC: result:"Hello Jobet number 7" 2020/06/25 13:17:28 Response from Server Streaming RPC: result:"Hello Jobet number 8" 2020/06/25 13:17:29 Response from Server Streaming RPC: result:"Hello Jobet number 9"
Create Server (Bidirectional Streaming RPC)
In Bidirectional Streaming RPC, the client initiates the request by sending request to the server. The server can respond initially with some metadata and then wait until client sends all the requests. Both client and server streams are independent, so it can read or write messages in any order. we will now implement the server-side code by creating a stream of responses from the server to the client as seen below:
func (*server) SayHelloBidirectionalStreaming(stream api.HelloService_SayHelloBidirectionalStreamingServer) error {
fmt.Printf("SayHelloBidirectionalStreaming function was invoked with \n")
for {
req, err := stream.Recv()
if err == io.EOF {
return nil
}
if err != nil {
log.Fatalf("Error while reading client stream: %v", err)
return err
}
firstName := req.GetHello().GetFirstName()
result := "Hello " + firstName + "! "
sendErr := stream.Send(&api.HelloResponseMultipleTimes{
Result: result,
})
if sendErr != nil {
log.Fatalf("Error while sending data to client: %v", sendErr)
return sendErr
}
}
}
The above function starts with a for–loop to receive a stream of request by using the method stream.Recv(). Then it checks for EOF ( End-of-File, i.e the stream of requests are completed) and returns the result or err if any.
Create Client (Bidirectional Streaming RPC)
Lets now write the gRPC implementation of client code to send stream of request to the server.
func checkBidirectionalStreaming(c api.HelloServiceClient) {
fmt.Println("Starting a Bidirectional Streaming RPC...")
stream, err := c.SayHelloBidirectionalStreaming(context.Background())
if err != nil {
log.Fatal("Error while creating stream: %v", err)
}
requests := []*api.HelloRequestMultipleTimes{
&api.HelloRequestMultipleTimes{
Hello: &api.Hello{
FirstName: "Jobet",
},
},
&api.HelloRequestMultipleTimes{
Hello: &api.Hello{
FirstName: "Mark",
},
},
&api.HelloRequestMultipleTimes{
Hello: &api.Hello{
FirstName: "John",
},
},
&api.HelloRequestMultipleTimes{
Hello: &api.Hello{
FirstName: "Matthew",
},
},
}
waitc := make(chan struct{})
go func() {
//Send the above bunch of requests created
for _, req := range requests {
fmt.Printf("Sending requests: %v\n", req)
stream.Send(req)
time.Sleep(1000 * time.Millisecond)
}
stream.CloseSend()
}()
go func() {
for {
res, err := stream.Recv()
if err == io.EOF {
break
}
if err != nil {
log.Fatalf("Error while receiving: %v", err)
break
}
fmt.Printf("Response Received from Bidirectional Server: %v\n", res.GetResult())
}
stream.CloseSend()
}()
<-waitc //Block until both are done.
}
The above code starts with creating request array requests := []*api.HelloRequestMultipleTimes. Then we implement the client method c.SayHelloBidirectionalStreaming(context.Background()) with context that returns stream and error.
In this example, we use channels that connect concurrent goroutines defined by go func() above. The two goroutines (light-weight thread) run asynchronously and concurrently, where one of the goroutine sends stream of requests and the other to process the responses received from the server. Then close the stream by stream.CloseSend().The code <-waitc blocks until both goroutines are completed. We will run both server and client to see the response as seen below:
sam:~/goLang/src/github.com/grpc-golang$ go run server/server.go Hello World!!! SayHelloBidirectionalStreaming function was invoked sam:~/goLang/src/github.com/grpc-golang$ go run client/client.go Hello I'm a client Starting a Bidirectional Streaming RPC... Sending requests: hello:<first_name:"Jobet" > Response Received from Bidirectional Server: Hello Jobet! Sending requests: hello:<first_name:"Mark" > Response Received from Bidirectional Server: Hello Mark! Sending requests: hello:<first_name:"John" > Response Received from Bidirectional Server: Hello John! Sending requests: hello:<first_name:"Matthew" > Response Received from Bidirectional Server: Hello Matthew!
Conclusion
In this article, we learned how to structure the code in golang, define the protobuf and the gRPC implementation of client and server in Go with different gRPC service methods. I hope this article was helpful.
References
gRPC Concepts: https://grpc.io/docs/what-is-grpc/core-concepts/
Protocol Buffers: https://developers.google.com/protocol-buffers