Register Microservices to Consul Out-of-the-box Using Scala Macro Annotations

Shivam Kapoor
Thursday, March 21, 2019

Off late at my work which currently involves moving our platform code from monolithic to microservices architecture, I have come to realize that there is a lot of boilerplate code that not only needs to be implemented in every other service but also involves maintenance pertaining to tribal knowledge within the team. This means that there isn't any standard way of implementing a service, raising the possibility of a lot of code duplication, thereby resulting in a maintenance nightmare. Also, implementing a new service every time requires code from other services to be referred to in order to figure out the implementation of the basic infrastructure layout.

While pondering over possible solutions to fix this issue it occurred to me that for a particular domain the implementation (the code design) is very much coupled to the solution (the architectural design) for the domain itself which is why every other service seems to share a lot of boilerplate infrastructure code. This however needs to be standardized. Guess what? Turns out that it is the similar problem that most of the frameworks try to solve by abstracting a lot of potential boilerplate code using metaprogramming capabilities of the language they provide support for. To me this seems like a reasonable approach to get rid of potential boilerplate code in platform microservices at work too and bring some order in the way services are built.

Our platform microservices need to register themselves to consul for service discovery. To me this seems like a potential boilerplate code that could possibly be abstracted with Scala macro annotations, one of Scala's metaprogramming capabilities. Macro annotations is however an experimental feature as of now and may probably be included in future Scala releases. Since macro annotations are only available as part of macro paradise compiler plugin, services would need to define Scala compiler and macro paradise plugin as their dependencies in order to use them.

 

libraryDependencies += "org.scala-lang" % "scala-compiler" % "2.12.8"

 

libraryDependencies += compilerPlugin("org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full)

view rawMacro Annotations Dependencies.scala hosted with ❤ by GitHub

I propose to implement a macro annotation called @EnableServiceDiscovery which would abstract service registration to a Consul agent, allowing services to register themselves out-of-the-box by annotating their main object like so,

 

@EnableServiceDiscovery

 

object MyService extends App {}

view rawMacro Annotations Usage.scala hosted with ❤ by GitHub

Let's see how we would go about implementing it.

@EnableServiceDiscovery macro annotation

Before we learn the implementation details, it is important to understand that Scala macro annotations are compile time metaprogramming capability. That is, they are intended to modify the AST of a program to which they are added to at compile time. In the context of our problem what it would mean is that the service registration to Consul Agent logic will unfold into our service's main object definition that implements @EnableServiceDiscovery macro annotation, at compile time. Also, macro annotations need to be compiled as a separate module/project before they could be used in any other module/project.

 

In order to implement service registration to Consul, I have chosen to use a Java Consul client by rickfast for having not found a better alternative lib in Scala. This library to me seemed to be well maintained and popular enough to be used safely in production code.

Here is what it would look like:

 

@compileTimeOnly("enable macro paradise to expand macro annotations")

 

class EnableServiceDiscovery extends StaticAnnotation {

 

  def macroTransform(annottees: Any*): Any = macro EnableServiceDiscovery.impl

 

}

   
 

object EnableServiceDiscovery {

 

  def impl(c: blackbox.Context)(annottees: c.Expr[Any]*): c.Expr[Any] = {

 

    import c.universe._

   
 

    val result =  {

 

      annottees.map(_.tree).toList match {

   
 

        case q"$mods object $tname extends { ..$earlydefns } with ..$parents { $self => ..$stats }" :: Nil => {

 

          q"""$mods object $tname extends { ..$earlydefns } with ..$parents {

 

            $self => ..$stats

   
 

            def getConsulUrl = {

 

              val consulInterface = RuntimeEnvironment.appConfig.getString("consul.interface")

 

              val consulPort = RuntimeEnvironment.appConfig.getString("consul.port")

   
 

              val consulUrl = s"http://$$consulInterface:$$consulPort"

 

              consulUrl

 

            }

   
 

            def getServiceName = {

 

              val pkg = getClass.getPackage

 

              val serviceTitle = pkg.getImplementationTitle

 

              val serviceVersion = pkg.getImplementationVersion

   
 

              val serviceName = s"v$${serviceVersion}_$$serviceTitle"

 

              serviceName

 

            }

   
 

            def getServicePort = {

 

              val servicePort = RuntimeEnvironment.appConfig.getInt("myservice.http.port")

 

              servicePort

 

            }

   
 

            def getHealthCheckScript = {

 

              val healthCheckScript = s"$${RuntimeEnvironment.getHomeDir}/bin/status.sh"

 

              healthCheckScript

 

            }

   
 

            def getTags = {

 

              val tags = RuntimeEnvironment.appConfig.getList("myservice.tags").unwrapped().toArray().map(_.toString).toList

 

              tags

 

            }

   
 

            def registerService() = {

 

              import com.orbitz.consul.Consul

 

              import com.orbitz.consul.model.agent.ImmutableRegistration

 

              import com.orbitz.consul.model.agent.Registration

 

              import scala.collection.JavaConverters._

   
 

              val consulClient = Consul.builder().withUrl(getConsulUrl).build()

 

              val agentClient = consulClient.agentClient

   
 

              val hostName = java.net.InetAddress.getLocalHost.getHostName

 

              val serviceId = Util.md5(hostName + getServiceName)

   
 

              val service =

 

                ImmutableRegistration.builder().

 

                  id(serviceId).

 

                  name(getServiceName).

 

                  port(getServicePort).

 

                  check(Registration.RegCheck.args(List(getHealthCheckScript).asJava, 5L)).

 

                  tags(getTags.asJava).

 

                  build()

   
 

              agentClient.register(service)

 

            }

   
 

            registerService()

 

          }"""

 

        }

 

        case _ => c.abort(c.enclosingPosition, "Annotation @EnableServiceDiscovery can be used only with case classes which extends Scalar trait")

 

      }

 

    }

   
 

    c.Expr[Any](result)

 

  }

 

}

view raw@EnableServiceDiscovery Macro Annotation.scala hosted with ❤ by GitHub

Macro annotation is created by extending the StaticAnnotation trait and providing the macro implementation for macroTransform(annottees: Any*): Anymethod. The implementation iterates over a Seq of annotated definitions and extracts object declaration using quasiqouted extracter pattern as the only supported definition that could be annotated with @EnableServiceDiscoverymacro annotation. Further, the Abstract Syntax Tree for service registration to Consul logic is created and returned by wrapping it around quasiqoute string interpolator. Quasiquotes are yet another Scala's cool metaprogramming capability that lets you manipulate Scala syntax trees with ease.

 

As part of service registration to Consul, we have to first build service definition before submitting it to Consul for registration. The service name is created by pulling title and version info from the META-INF/MANIFEST.MF of the service's jar artifact. Service porttags and Consul info are pulled from application.conf. Health check is however commissioned via status.sh script. (Note that we would need to start Consul agent with enable-script-checks set to true for health check scripts to work.) Besides, every service is given a unique service id as a function of md5 of service name and host name.

myservice

Let's see now how we could use @EnableServiceDiscovery macro annotation in our services.

 

@EnableServiceDiscovery

 

object MyService extends App {

   
 

  implicit val system = ActorSystem("myservice-actor-system")

 

  implicit val materializer = ActorMaterializer()

 

  implicit val executionContext: ExecutionContext = system.dispatcher

   
 

  val interface = RuntimeEnvironment.appConfig.getString("myservice.http.interface")

 

  val port = RuntimeEnvironment.appConfig.getString("myservice.http.port").toInt

   
 

  val serverBinding =

 

    Http().bindAndHandle(

 

      pathSingleSlash {

 

        get {

 

          complete(StatusCodes.OK, "Microservices can use @EnableServiceDiscovery macro-annotation to register themselves to Consul out of the box.")

 

        }

 

      },

 

      interface,

 

      port

 

    )

   
 

  serverBinding.onComplete {

 

    case Success(bound) =>

 

      println(s"MyService started @ http://$interface:$port")

 

    case Failure(e) =>

 

      Console.err.println(s"MyService could not start!")

 

      e.printStackTrace()

 

      system.terminate()

 

  }

   
 

  Await.result(system.whenTerminated, Duration.Inf)

 

}

view rawMyService.scala hosted with ❤ by GitHub

Here, we have defined a simple service called MyService that hosts a trivial rest API endpoint. For this service to register itself to Consul upon startup, we have annotated the object definition with the @EnableServiceDiscovery annotation. To see it in practice, let's first build our project.

Clone Repo

 

$ git clone git@github.com:codingkapoor/consul-scala-macro-annotations.git

view rawClone Repo consul-scala-macro-annotations.sh hosted with ❤ by GitHub

Build Package

 

$ cd consul-scala-macro-annotations

 

$ sbt> project myservice

 

$ sbt> universal:packageBin

view rawBuild Package consul-scala-macro-annotations.sh hosted with ❤ by GitHub

Start Service

Before starting service make sure consul agent is up and running with script checks enabled. You can find details on how to start consul agent here.

 

$ cd consul-scala-macro-annotations/myservice/target/universal

 

$ unzip myservice-0.1.0-SNAPSHOT.zip

 

$ cd myservice-0.1.0-SNAPSHOT

 

$ bin/start.sh

 

$ tail -f logs/stdout.log

view rawStart Service consul-scala-macro-annotations.sh hosted with ❤ by GitHub

Direct Browser @localhost:8080

Accessing root at @localhost:8080 should return following message.

Microservices can use @EnableServiceDiscovery macro-annotation to register themselves to Consul out of the box.

Verify Registration

We can use HTTP endpoints to talk to Consul.

 

$ curl http://localhost:8500/v1/catalog/services?pretty

 

{

 

    "consul": [],

 

    "v0.1.0-SNAPSHOT_myService": [

 

        "tag1",

 

        "tag2"

 

    ]

 

}

view rawVerify Registration consul-scala-macro-annotations.sh hosted with ❤ by GitHub

We can even get details of a registered service, like so:

 

$ curl http://localhost:8500/v1/catalog/service/v0.1.0-SNAPSHOT_myService?pretty

 

[

 

    {

 

        "ID": "da6b578f-7842-891a-d38d-352e7bd9aa13",

 

        "Node": "inblrlt-shivam",

 

        "Address": "172.20.20.1",

 

        "Datacenter": "dc1",

 

        "TaggedAddresses": {

 

            "lan": "172.20.20.1",

 

            "wan": "172.20.20.1"

 

        },

 

        "NodeMeta": {

 

            "consul-network-segment": ""

 

        },

 

        "ServiceKind": "",

 

        "ServiceID": "6537f9694e553ec2912e19175aa09978",

 

        "ServiceName": "v0.1.0-SNAPSHOT_myService",

 

        "ServiceTags": [

 

            "tag1",

 

            "tag2"

 

        ],

 

        "ServiceAddress": "",

 

        "ServiceWeights": {

 

            "Passing": 1,

 

            "Warning": 1

 

        },

 

        "ServiceMeta": {},

 

        "ServicePort": 6565,

 

        "ServiceEnableTagOverride": false,

 

        "ServiceProxyDestination": "",

 

        "ServiceProxy": {},

 

        "ServiceConnect": {},

 

        "CreateIndex": 764,

 

        "ModifyIndex": 764

 

    }

 

]

view rawService Defintion consul-scala-macro-annotation.sh hosted with ❤ by GitHub

Health Check

Last but not the least health of your service gets monitored by Consul out of the box. Look for "Status": "passing" for a healthy service.

 

$ curl http://localhost:8500/v1/health/checks/v0.1.0-SNAPSHOT_myservice?pretty

 

[

 

    {

 

        "Node": "inblrlt-shivam",

 

        "CheckID": "service:2fd993e4931427b89f1bc832a6192ee0",

 

        "Name": "Service 'v0.1.0-SNAPSHOT_myService' check",

 

        "Status": "passing",

 

        "Notes": "",

 

        "Output": "Sun Dec 23 20:37:06 IST 2018: MyService is running\n",

 

        "ServiceID": "2fd993e4931427b89f1bc832a6192ee0",

 

        "ServiceName": "v0.1.0-SNAPSHOT_myService",

 

        "ServiceTags": [

 

            "tag1",

 

            "tag2"

 

        ],

 

        "Definition": {},

 

        "CreateIndex": 13070,

 

        "ModifyIndex": 13102

 

    }

 

]

view rawHealth Check Status Up consul-scala-macro-annotation.sh hosted with ❤ by GitHub

Let's turn our service down to see updated health check status. Look for "Status": "warning" or "Status": "critical" for unhealthy service.

 

$ curl http://localhost:8500/v1/health/checks/v0.1.0-SNAPSHOT_myservice?pretty

 

[

 

    {

 

        "Node": "inblrlt-shivam",

 

        "CheckID": "service:2fd993e4931427b89f1bc832a6192ee0",

 

        "Name": "Service 'v0.1.0-SNAPSHOT_myService' check",

 

        "Status": "warning",

 

        "Notes": "",

 

        "Output": "Sun Dec 23 20:40:18 IST 2018: MyService is not running\n",

 

        "ServiceID": "2fd993e4931427b89f1bc832a6192ee0",

 

        "ServiceName": "v0.1.0-SNAPSHOT_myService",

 

        "ServiceTags": [

 

            "tag1",

 

            "tag2"

 

        ],

 

        "Definition": {},

 

        "CreateIndex": 13070,

 

        "ModifyIndex": 13124

 

    }

 

]

view rawHealth Check Status Down consul-scala-macro-annotation.sh hosted with ❤ by GitHub