Servers

Finagle servers implement a simple interface:

def serve(
  addr: SocketAddress,
  factory: ServiceFactory[Req, Rep]
): ListeningServer

When given a SocketAddress and a ServiceFactory, a server returns a ListeningServer. The ListeningServer allows for management of server resources. The interface comes with variants that allow for serving a simple Service as well. Typical usage takes the form Protocol.serve(...), for example:

import com.twitter.finagle.Service
import com.twitter.finagle.Http
import com.twitter.finagle.http.{Request, Response}
import com.twitter.util.{Await, Future}

val service: Service[Request, Response] = new Service[Request, Response] {
  def apply(req: Request): Future[Response] = Future.value(Response())
}

val server = Http.server.serve(":8080", service)
Await.ready(server) // waits until the server resources are released

Note

finagle-thrift servers expose a rich API because their interfaces are defined via a Thrift IDL. See the protocols section on Thrift for more details.

Server Modules

Finagle servers are simple; they are designed to serve requests quickly. As such, Finagle minimally furnishes servers with additional behavior. More sophisticated behavior lives in the clients.

_images/serverstack.svg

Fig. 1: A visual representation of each module in a default Finagle server. Requests flow from left to right.

Many of the server modules act as admission controllers that make a decision (based on either a dynamic or static property) whether this server can handle the incoming request while maintaining some SLO (Service Level Objective).

Note

An Endpoint module represents a user-defined Service passed to the Server.serve(...) method.

Observability

A Finagle server comes with some useful modules to help owners observe and debug their servers. This includes Monitoring, Tracing, and Stats.

All the unhandled exceptions from a user-defined service flow through the Monitor object used by the given Finagle server. See this example on how to override the default instance that simply logs exceptions onto a the standard output.

Response Classification

To give Finagle visibility into application level success and failure developers can provide classification of responses on clients and servers by using response classifiers. This gives Finagle the proper domain knowledge and improves the efficacy of failure accrual and more accurate success rate stats.

For HTTP clients and servers, using HttpResponseClassifier.ServerErrorsAsFailures often works great as it classifies any HTTP 5xx response code as a failure. For Thrift/ThriftMux clients you may want to use ThriftResponseClassifier.ThriftExceptionsAsFailures which classifies any deserialized Thrift Exception as a failure. For a large set of use cases these should suffice. Classifiers get wired up to your client and server in a straightforward manner, for example, in a ThriftMux client:

import com.twitter.finagle.ThriftMux
import com.twitter.finagle.thrift.service.ThriftResponseClassifier

ThriftMux.client
  ...
  .withResponseClassifier(ThriftResponseClassifier.ThriftExceptionsAsFailures)

In an HTTP server:

import com.twitter.finagle.Http
import com.twitter.finagle.http.service.HttpResponseClassifier

Http.server
  ...
  .withResponseClassifier(HttpResponseClassifier.ServerErrorsAsFailures)

If a classifier is not specified on a client or server or if a user’s classifier isn’t defined for a given request/response pair then ResponseClassifier.Default is used. This gives us the simple classification rules of responses that are Returns are successful and Throws are failures.

Writing a custom classifier requires understanding of the few classes used. A ResponseClassifier is a PartialFunction from ReqRep to ResponseClass.

Let’s work our way backwards through those, beginning with ResponseClass. This can be either Successful or Failed and those values are self-explanatory. There are three constants which will cover the vast majority of usage: Success, NonRetryableFailure and RetryableFailure. While as of today there is no distinction made between retryable and non-retryable failures, this lays the groundwork for use in the future.

A ReqRep is a request/response struct with a request of type Any and a response of type Try[Any]. While all of this functionality is called response classification, you’ll note that classifiers make judgements on both a request and response.

Creating a custom ResponseClassifier is fairly straightforward for HTTP as the ReqRep is an http.Request and Try[http.Response] pair. Here is an example that counts HTTP 503s as failures:

import com.twitter.finagle.http
import com.twitter.finagle.service.{ReqRep, ResponseClass, ResponseClassifier}
import com.twitter.util.Return

val classifier: ResponseClassifier = {
  case ReqRep(_, Return(r: http.Response)) if r.statusCode == 503 =>
    ResponseClass.NonRetryableFailure
}

Note that this PartialFunction isn’t total which is ok due to Finagle always using user defined classifiers in combination with ResponseClassifier.Default which will cover all cases.

Thrift and ThriftMux classifiers require a bit more care as the request and response types are not as obvious. This is because there is only a single Service from Array[Byte] to Array[Byte] for all the methods of an IDL’s service. To make this workable, there is support in Scrooge and Thrift/ThriftMux.newService and Thrift/ThriftMux.newClient code to deserialize the responses into the expected application types so that classifiers can be written in terms of the Scrooge generated request type, $Service.$Method.Args, and the method’s response type. Given an IDL:

exception NotFoundException { 1: string reason }

service SocialGraph {
  i32 follow(1: i64 follower, 2: i64 followee) throws (1: NotFoundException ex)
}

One possible classifier would be:

import com.twitter.finagle.service.{ReqRep, ResponseClass, ResponseClassifier}

val classifier: ResponseClassifier = {
  // #1
  case ReqRep(_, Throw(_: NotFoundException)) =>
    ResponseClass.NonRetryableFailure

  // #2
  case ReqRep(_, Return(x: Int)) if x == 0 =>
    ResponseClass.NonRetryableFailure

  // #3
  case ReqRep(SocialGraph.Follow.Args(a, b), _) if a <= 0 =>
    ResponseClass.NonRetryableFailure
}

If you examine that classifier you’ll note a few things. First (#1), the deserialized NotFoundException can be treated as a failure. Next (#2), a “successful” response can be examined to enable services using status codes to classify errors. Lastly (#3), the request can be introspected to make the decision.

If you have a response classifier that categorizes Returns as failures, note that they will be counted in the StatsFilter as a com.twitter.finagle.service.ResponseClassificationSyntheticException in the StatsReceiver to indicate when this happens. See the FAQ for more details.

Concurrency Limit

The Concurrency Limit module is implemented by RequestSemaphoreFilter and maintains the concurrency of the Finagle server.

By default, this module is disabled, which means a Finagle server’s requests concurrency is unbounded. To enable the Concurrency Limit module and put some bounds in terms of maximum number of requests that might be handled concurrently by your server, use the following example [2].

import com.twitter.finagle.Http

val server = Http.server
  .withAdmissionControl.concurrencyLimit(
    maxConcurrentRequests = 10,
    maxWaiters = 0
  )
  .serve(":8080", service)

The Concurrency Limit module is configured with two parameters:

  1. maxConcurrentRequests - the number of requests allowed to be handled concurrently
  2. maxWaiters - the number of requests (on top of maxConcurrentRequests) allowed to be queued

All the incoming requests on top of (maxConcurrentRequests + maxWaiters) will be rejected [1] by the server. That said, the Concurrency Limit module acts as static admission controller monitoring the current concurrency level of the incoming requests.

See Requests Concurrency metrics for more details.

Rejecting Requests

A service may explicitly reject requests on a case-by-case basis. To generate a rejection response, have the server return a Future.exception of Failure with the Rejected flag set. A convenience method, Failure.rejected exists for this. By default, this also sets the Restartable flag which indicates the failure is safe to retry. The client’s RequeueFilter will automatically retry such failures. For requests that should not be retried, the server should return a Failure with the NonRetryable flag set.

import com.twitter.finagle.Failure

val rejection = Future.exception(Failure.rejected("busy"))
val nonRetryable = Future.exception(Failure("Don't try again", Failure.Rejected|Failure.NonRetryable))

These responses will be considered rejected [1] by Finagle.

Request Timeout

The Request Timeout module is implemented by TimeoutFilter and simply fails all the requests that a given server hasn’t be able to handle in the given amount of time. As well as for Finagle clients, this module is disabled by default (the timeout is unbounded). See this example to override this behaviour.

Note

The Request Timeout module doesn’t reject the incoming request, but fails it. This means it won’t by default be retried by a remote client given it’s not known whether the request has been timed out being in the queue (waiting for processing) or being processed.

Footnotes

[1](1, 2) Depending on the protocol, a rejected request might be transformed into a nack (currently supported in HTTP/1.1 and Mux) message.
[2](1, 2) Configuration parameters/values provided in this example are only demonstrate the API usage, not the real world values. We do not recommend blindly applying those values to production systems.

Session Expiration

In certain cases, it may be useful for the server to control its resources via bounding the lifetime of a session. The Session Expiration module is attached at the connection level and expires a service/session after a certain amount of idle time. The module is implemented by ExpiringService.

The default setting for the Expiration module is to never expire a session. Here is how it can be configured [2].

import com.twitter.conversions.time._
import com.twitter.finagle.Http

val twitter = Http.server
  .withSession.maxLifeTime(20.seconds)
  .withSession.maxIdleTime(10.seconds)
  .newService("twitter.com")

The Expiration module takes two parameters:

  1. maxLifeTime - the maximum duration for which a session is considered alive
  2. maxIdleTime - the maximum duration for which a session is allowed to idle (not sending any requests)

See Expiration metrics for more details.