caddy & grpc adds a reverse proxy plug-in for caddy

caddy-grpc adds a reverse proxy plug-in for caddy

Project address: https://github.com/yhyddr/caddy-grpc

Preface

Last time we learned how to extend the plug-in we wanted in Caddy. Blogs provide only a general framework. This time, let's learn from the specific plug-in caddy-grpc.

The reason for choosing it is that it is a stand-alone application, which is made into a Caddy plug-in. Maybe you have a better understanding of Caddy's good design.

Plug-in role

The purpose of the plug-in is to Improbable-eng/grpc-web/go/grpcwebproxy The purpose is the same, but as a Caddy middleware plug-in rather than a stand-alone Go application.

And what is the role of this project?

This is a small reverse proxy that can support existing gRPC servers using the gRPC-Web protocol and expose their functions, allowing gRPC services to be used from browsers.
Features:

  • Structured Recording (log la) Agent requests to stdout (standard output)
  • Debuggeable HTTP port (default port 8080)
  • Prometheus monitors proxy requests (/ metrics at debugging endpoints)
  • Request (/ debug/requests) and connection tracking endpoints (/ debug/events)
  • TLS 1.2 Service (default port 8443):

    • Options to enable client certificate validation
  • Secure (plain text) and TLS gRPC backend connections:

    • Connect using customizable CA certificates

In fact, this reverse proxy is implemented in the middleware of caddy server.

Use

When you need it, you can go through it.

example.com 
grpc localhost:9090

The first line, example.com, is the hostname/address of the site to be served. The second line is an instruction called grpc, which specifies the address of the back-end gRPC service endpoint (that is, localhost: 9090 in the example). (Note: The above configuration defaults to TLS 1.2 to back-end gRPC service)

Caddyfile grammar

grpc backend_addr {
    backend_is_insecure 
    backend_tls_noverify
    backend_tls_ca_files path_to_ca_file1 path_to_ca_file2 
}

[](https://github.com/yhyddr/cad...

By default, the proxy will connect to the back end using TLS, but if the back end provides services in plaintext, this option needs to be added.

backend_tls_noverify

By default, the backend TLS is validated. If you do not want to validate, you need to add this option

backend_tls_ca_files

The PEM Certificate Link Path (separated by commas) used to verify back-end certificates. If it is empty, the host host host CA chain will be used.

[](https://github.com/yhyddr/cad...

Source code

directory structure

caddy-grpc
├── LICENSE
├── README.md
├── proxy // Functional Implementation of Agent grpc proxy
│   ├── DOC.md
│   ├── LICENSE.txt
│   ├── README.md
│   ├── codec.go
│   ├── director.go
│   ├── doc.go
│   └── handler.go
├── server.go // Handle logical file
└── setup.go // install files

Setup.go

In the order in which we wrote the plug-ins last time, if you don't remember, see: How to add plug-in extensions to caddy

First, look at the setup.go file installed

init func

func init() {
    caddy.RegisterPlugin("grpc", caddy.Plugin{
        ServerType: "http",
        Action:     setup,
    })
}

As you can see, the plug-in registers an http server named grpc

setup func

Then we see the most important setup function, which is responsible for analyzing the options in caddyfile. It will also configure its own plug-in by leaving the directive analyzed to Caddy's controller.

// setup configures a new server middleware instance.
func setup(c *caddy.Controller) error {
    for c.Next() {
        var s server

        if !c.Args(&s.backendAddr) { //loads next argument into backendAddr and fail if none specified
            return c.ArgErr()
        }

        tlsConfig := &tls.Config{}
        tlsConfig.MinVersion = tls.VersionTLS12

        s.backendTLS = tlsConfig
        s.backendIsInsecure = false

        //check for more settings in Caddyfile
        for c.NextBlock() {
            switch c.Val() {
            case "backend_is_insecure":
                s.backendIsInsecure = true
            case "backend_tls_noverify":
                s.backendTLS = buildBackendTLSNoVerify()
            case "backend_tls_ca_files":
                t, err := buildBackendTLSFromCAFiles(c.RemainingArgs())
                if err != nil {
                    return err
                }
                s.backendTLS = t
            default:
                return c.Errf("unknown property '%s'", c.Val())
            }
        }

        httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
            s.next = next
            return s
        })

    }

    return nil
}
  1. We noticed that c.Next() still started to read the configuration file. In fact, here it read the token grpc and proceeded to the next step.
  2. Then we see that the next thing that the grpc reads is the listening address.
if !c.Args(&s.backendAddr) { //loads next argument into backendAddr and fail if none specified
            return c.ArgErr()
        }

This corresponds to the configuration in caddyfile. grpc localhost:9090

  1. Note that c.Next(), c.Args(), c.NextBlock(), are all functions that read the configuration in caddyfile, which we call token in caddy.
  1. Another thing to note is the configuration of tls, which, as mentioned earlier, is the service that opens tls 1.2
        tlsConfig := &tls.Config{}
        tlsConfig.MinVersion = tls.VersionTLS12

        s.backendTLS = tlsConfig
        s.backendIsInsecure = false
  1. Then the configuration reading in the above-mentioned caddyfile grammar
//check for more settings in Caddyfile
        for c.NextBlock() {
            switch c.Val() {
            case "backend_is_insecure":
                s.backendIsInsecure = true
            case "backend_tls_noverify":
                s.backendTLS = buildBackendTLSNoVerify()
            case "backend_tls_ca_files":
                t, err := buildBackendTLSFromCAFiles(c.RemainingArgs())
                if err != nil {
                    return err
                }
                s.backendTLS = t
            default:
                return c.Errf("unknown property '%s'", c.Val())
            }
        }

You can see that each new token is analyzed by c.NextBlock(), and different configurations are made after reading with c.Val().

  1. Finally, don't forget that we're going to add it to the whole caddy Middleware
httpserver.GetConfig(c).AddMiddleware(func(next httpserver.Handler) httpserver.Handler {
            s.next = next
            return s
        })

server.go

Next step.

struct

First, look at the core structure of this plug-in. Which data is stored?

type server struct {
    backendAddr       string
    next              httpserver.Handler
    backendIsInsecure bool
    backendTLS        *tls.Config
    wrappedGrpc       *grpcweb.WrappedGrpcServer
}
  • Backend Addr is the listening address of the grpc service
  • next is Handler's Processing for the next Plug-in
  • Backend Is Insecure and backend TLS are both background services that have different security policies enabled.
  • Wrapped Grpc is the key to this plug-in. It implements grpc web protocol to make grpc services accessible by browsers.

serveHTTP

In our last article, this is the second most important part. The implementation of server HTTP represents specific functions. Last time, our content was only used to pass logic to the next Handle.

func (g gizmoHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
  return g.next.ServeHTTP(w, r)
}

Now let's see what logic is added to this grpc.

// ServeHTTP satisfies the httpserver.Handler interface.
func (s server) ServeHTTP(w http.ResponseWriter, r *http.Request) (int, error) {
    //dial Backend
    opt := []grpc.DialOption{}
    opt = append(opt, grpc.WithCodec(proxy.Codec()))
    if s.backendIsInsecure {
        opt = append(opt, grpc.WithInsecure())
    } else {
        opt = append(opt, grpc.WithTransportCredentials(credentials.NewTLS(s.backendTLS)))
    }

    backendConn, err := grpc.Dial(s.backendAddr, opt...)
    if err != nil {
        return s.next.ServeHTTP(w, r)
    }

    director := func(ctx context.Context, fullMethodName string) (context.Context, *grpc.ClientConn, error) {
        md, _ := metadata.FromIncomingContext(ctx)
        return metadata.NewOutgoingContext(ctx, md.Copy()), backendConn, nil
    }
    grpcServer := grpc.NewServer(
        grpc.CustomCodec(proxy.Codec()), // needed for proxy to function.
        grpc.UnknownServiceHandler(proxy.TransparentHandler(director)),
        /*grpc_middleware.WithUnaryServerChain(
            grpc_logrus.UnaryServerInterceptor(logger),
            grpc_prometheus.UnaryServerInterceptor,
        ),
        grpc_middleware.WithStreamServerChain(
            grpc_logrus.StreamServerInterceptor(logger),
            grpc_prometheus.StreamServerInterceptor,
        ),*/ //middleware should be a config setting or 3rd party middleware plugins like for caddyhttp
    )

    // gRPC-Web compatibility layer with CORS configured to accept on every
    wrappedGrpc := grpcweb.WrapServer(grpcServer, grpcweb.WithCorsForRegisteredEndpointsOnly(false))
    wrappedGrpc.ServeHTTP(w, r)

    return 0, nil
}
  • First of all, the configuration part of grpc. If you know about grpc, you will know that it is an option for configuring grpc clients. Codec encoding and decoding and different security policy options are added to our client.
    //dial Backend
    opt := []grpc.DialOption{}
    opt = append(opt, grpc.WithCodec(proxy.Codec()))
    if s.backendIsInsecure {
        opt = append(opt, grpc.WithInsecure())
    } else {
        opt = append(opt, grpc.WithTransportCredentials(credentials.NewTLS(s.backendTLS)))
    }
    backendConn, err := grpc.Dial(s.backendAddr, opt...)
    if err != nil {
        return s.next.ServeHTTP(w, r)
    }
  • Then the grpc server options are set
director := func(ctx context.Context, fullMethodName string) (context.Context, *grpc.ClientConn, error) {
        md, _ := metadata.FromIncomingContext(ctx)
        return metadata.NewOutgoingContext(ctx, md.Copy()), backendConn, nil
    }
    grpcServer := grpc.NewServer(
        grpc.CustomCodec(proxy.Codec()), // needed for proxy to function.
        grpc.UnknownServiceHandler(proxy.TransparentHandler(director)),
        /*grpc_middleware.WithUnaryServerChain(
            grpc_logrus.UnaryServerInterceptor(logger),
            grpc_prometheus.UnaryServerInterceptor,
        ),
        grpc_middleware.WithStreamServerChain(
            grpc_logrus.StreamServerInterceptor(logger),
            grpc_prometheus.StreamServerInterceptor,
        ),*/ //middleware should be a config setting or 3rd party middleware plugins like for caddyhttp
    )
  • Finally, we use grpcweb.WrapServer to implement the invocation of web Services
// gRPC-Web compatibility layer with CORS configured to accept on every
    wrappedGrpc := grpcweb.WrapServer(grpcServer, grpcweb.WithCorsForRegisteredEndpointsOnly(false))
    wrappedGrpc.ServeHTTP(w, r)

Proxy

Notice that proxy.TransparentHandler is used above as a function defined in handler.go of proxy. Agents used to implement gRPC services. This involves the implementation of gRPC interaction, focusing on the stream ing transmission of Client and Server, which has little to do with this article and is interesting to learn about.

epilogue

Think about what this brings as a Caddy plug-in?

Did you get a lot of scalable configurations in a flash?
Instead of putting some of the plug-ins you want in Caddy into the standalone application project you started with.

If you're also doing HTTP services, and you're still looking at some of the features of Caddy and its ecology, access it like this.

It also covers grpc-web, and if you're interested, you can expand your learning.

grpc-web client implementations/examples:

Vue.js
GopherJS

Reference resources

caddy: https://github.com/caddyserver/caddy
How to write middleware: https://github.com/caddyserver/caddy/wiki/Writing-a-Plugin:-HTTP-Middleware
caddy-grpc plug-in: https://github.com/pieterlouw/caddy-grpc

Tags: Go github codec encoding Vue

Posted on Sat, 10 Aug 2019 03:36:53 -0700 by Edison