Servers

Finagle servers implement a simple interface:

def serve(
  addr: SocketAddress,
  factory: ServiceFactory[Req, Rep]
): ListeningServer

When given a SocketAddress and a ServiceFactory, a server returns a ListeningServer. The ListeningServer allows for management of server resources. The interface comes with variants that allow for serving a simple Service as well. Typical usage takes the form Protocol.serve(...), for example:

import com.twitter.finagle.Service
import com.twitter.finagle.Http
import com.twitter.finagle.http.{Request, Response}
import com.twitter.util.{Await, Future}

val service: Service[Request, Response] = new Service[Request, Response] {
  def apply(req: Request): Future[Response] = Future.value(Response())
}

val server = Http.server.serve(":8080", service)
Await.ready(server) // waits until the server resources are released

Note

finagle-thrift servers expose a rich API because their interfaces are defined via a Thrift IDL. See the protocols section on Thrift for more details.

Server Modules

Finagle servers are simple; they are designed to serve requests quickly. As such, Finagle minimally furnishes servers with additional behavior. More sophisticated behavior lives in the clients.

Session Expiration Request Timeout Request Deadline Concurrency Limit Observe Endpoint

Fig. 1: A visual representation of each module in a default Finagle server. Requests flow from left to right.

Many of the server modules act as admission controllers that make a decision (based on either a dynamic or static property) whether this server can handle the incoming request while maintaining some SLO (Service Level Objective).

Note

An Endpoint module represents a user-defined Service passed to the Server.serve(...) method.

Observability

A Finagle server comes with some useful modules to help owners observe and debug their servers. This includes Monitoring, Tracing, and Stats.

All the unhandled exceptions from a user-defined service flow through the Monitor object used by the given Finagle server. See this example on how to override the default instance that simply logs exceptions onto a the standard output.

Response Classification

To give Finagle visibility into application level success and failure developers can provide classification of responses on clients and servers by using response classifiers. This gives Finagle the proper domain knowledge and improves the efficacy of failure accrual and more accurate success rate stats.

For HTTP clients and servers, using HttpResponseClassifier.ServerErrorsAsFailures often works great as it classifies any HTTP 5xx response code as a failure. For Thrift/ThriftMux clients and servers you may want to use ThriftResponseClassifier.ThriftExceptionsAsFailures which classifies any deserialized Thrift Exception as a failure. For a large set of use cases these should suffice. Classifiers get wired up to your client and server in a straightforward manner, for example, in a ThriftMux client:

import com.twitter.finagle.ThriftMux
import com.twitter.finagle.thrift.service.ThriftResponseClassifier

ThriftMux.client
  ...
  .withResponseClassifier(ThriftResponseClassifier.ThriftExceptionsAsFailures)

In an HTTP server:

import com.twitter.finagle.Http
import com.twitter.finagle.http.service.HttpResponseClassifier

Http.server
  ...
  .withResponseClassifier(HttpResponseClassifier.ServerErrorsAsFailures)

If a classifier is not specified on a client or server or if a user’s classifier isn’t defined for a given request/response pair then ResponseClassifier.Default is used. This gives us the simple classification rules of responses that are Returns are successful and Throws are failures.

ResponseClassifier is a PartialFunction from ReqRep to ResponseClass. Custom classifiers allow the user to tell Finagle what constitutes a failed outcome, and also what to do about it. Users define classifiers in terms of ReqRep and Try.

rc, defined below, is a classifier that tells Finagle that Throw means failure, and Return means success.

val rc: ResponseClassifier = {
  case ReqRep(req, Throw(exc)) => ResponseClass.RetryableFailure
  case ReqRep(req, Return(rep)) => ResponseClass.Success
}

A ReqRep is a request-response pair. This is so that classifiers can make judgements on both a request and response.

More than just telling Finagle if this ReqRep is a successful or failed outcome, it also gives Finagle a hint about what it should do next. Finagle can respond to failed outcomes with some nuance, for example, it may retry the operation.

ResponseClass defines three classes of failure:

  • NonRetryableFailure: Something went wrong, don’t retry.
  • RetryableFailure: Something went wrong, consider retrying the operation.
  • Ignorable: Something went wrong, but it can be ignored.

And, of course, Success means that the operation succeeded. Ignorable does not apply to Success because it is a mapping from FailureFlags.Ignorable which only applies to Failure and not any arbitrary response.

It’s important to note that classifiers are only consulted but not obeyed. For example, a classifier may emit Ignorable for a given ReqRep but what actually happens depends on how the caller chooses to interpret Ignorable. Similarly, just because a classifier emits RetryableFailure does not mean the caller will retry the operation.

Now that we’ve covered the basics, let’s look at an example in HTTP. Here is an example that counts HTTP 503s as failures:

import com.twitter.finagle.http
import com.twitter.finagle.service.{ReqRep, ResponseClass, ResponseClassifier}
import com.twitter.util.Return

val classifier: ResponseClassifier = {
  case ReqRep(_, Return(r: http.Response)) if r.statusCode == 503 =>
    ResponseClass.NonRetryableFailure
}

Note that this PartialFunction isn’t total which is ok due to Finagle always using user defined classifiers in combination with ResponseClassifier.Default which will cover all cases.

Thrift and ThriftMux classifiers require a bit more care as the request and response types are not as obvious. This is because there is only a single Service from Array[Byte] to Array[Byte] for all the methods of an IDL’s service. To make this workable, there is support in Scrooge, Thrift/ThriftMux.newService, Thrift/ThriftMux.newClient and Thrift/ThriftMux.serve code to deserialize the responses into the expected application types so that classifiers can be written in terms of the Scrooge generated request type, $Service.$Method.Args, and the method’s response type. Given an IDL:

exception NotFoundException { 1: string reason }
exception InvalidQueryException {
  1: i32 errorCode
}

service SocialGraph {
  i32 follow(1: i64 follower, 2: i64 followee) throws (
    1: NotFoundException ex1,
    2: InvalidQueryException ex2
  )
}

One possible classifier would be:

import com.twitter.finagle.service.{ReqRep, ResponseClass, ResponseClassifier}

val classifier: ResponseClassifier = {
  // #1
  case ReqRep(_, Throw(_: NotFoundException)) =>
    ResponseClass.NonRetryableFailure

  // #2
  case ReqRep(_, Return(x: Int)) if x == 0 =>
    ResponseClass.NonRetryableFailure

  // #3 *Caution*
  case ReqRep(SocialGraph.Follow.Args(a, b), _) if a <= 0 =>
    ResponseClass.NonRetryableFailure

  // #4
  case ReqRep(_, Throw(_: InvalidQueryException)) =>
    ResponseClass.Success
}

If you examine that classifier you’ll note a few things. First (#1), the deserialized NotFoundException can be treated as a failure. Second (#2), a “successful” response can be examined to enable services using status codes to classify errors. Next (#3), the request can be introspected to make the decision - HOWEVER - if an exception is thrown at the Mux layer (ex: c.t.f.mux.ClientDiscardedRequestException) there will NOT be a match against (#3). This style (#3) should be avoided for Thrift and ThriftMux. Instead, prefer to handle request specific details at the application layer, such as creating a Filter to reject the request, and reserve Response Classification to deal with wire level response concerns. Lastly (#4), the deserialized InvalidQueryException can be treated as a successful response.

If you have a response classifier that categorizes non-Exceptions as failures, this includes Thrift Responses (#2) or embedded Thrift Exceptions (#1), note that they will be counted in the StatsFilter as a com.twitter.finagle.service.ResponseClassificationSyntheticException in the StatsReceiver to indicate when this happens. See the FAQ for more details.

Concurrency Limit

The Concurrency Limit module is implemented by both RequestSemaphoreFilter and PendingRequestFilter and maintains the concurrency of the Finagle server.

By default, this module is disabled, which means a Finagle server’s requests concurrency is unbounded. To enable the Concurrency Limit module and put some bounds in terms of maximum number of requests that might be handled concurrently by your server, use the following example [2].

import com.twitter.finagle.Http

val server = Http.server
  .withAdmissionControl.concurrencyLimit(
    maxConcurrentRequests = 10,
    maxWaiters = 0
  )
  .serve(":8080", service)

The Concurrency Limit module is configured with two parameters:

  1. maxConcurrentRequests - the number of requests allowed to be handled concurrently
  2. maxWaiters - the number of requests (on top of maxConcurrentRequests) allowed to be queued.
    The value of this parameter determines which filter is used; if maxWaiters is 0, PendingRequestFilter is used (saving the overhead of the waiters queue in AsyncSemaphore); otherwise, RequestSemaphoreFilter is used.

All the incoming requests on top of (maxConcurrentRequests + maxWaiters) will be rejected [1] by the server. That said, the Concurrency Limit module acts as static admission controller monitoring the current concurrency level of the incoming requests.

See Requests Concurrency metrics for more details.

Rejecting Requests

A service may explicitly reject requests on a case-by-case basis. To generate a rejection response, have the server return a Future.exception of Failure with the Rejected flag set. A convenience method, Failure.rejected exists for this. By default, this also sets the Restartable flag which indicates the failure is safe to retry. The client’s RequeueFilter will automatically retry such failures. For requests that should not be retried, the server should return a Failure with the NonRetryable flag set.

import com.twitter.finagle.Failure

val rejection = Future.exception(Failure.rejected("busy"))
val nonRetryable = Future.exception(Failure("Don't try again", Failure.Rejected|Failure.NonRetryable))

These responses will be considered rejected [1] by Finagle.

Request Timeout

The Request Timeout module is implemented by TimeoutFilter and simply fails all the requests that a given server hasn’t be able to handle in the given amount of time. As well as for Finagle clients, this module is disabled by default (the timeout is unbounded). See this example to override this behaviour.

Note

The Request Timeout module doesn’t reject the incoming request, but fails it. This means it won’t by default be retried by a remote client given it’s not known whether the request has been timed out being in the queue (waiting for processing) or being processed.

Footnotes

[1](1, 2) Depending on the protocol, a rejected request might be transformed into a nack (currently supported in HTTP/1.1 and Mux) message.
[2](1, 2) Configuration parameters/values provided in this example are only demonstrate the API usage, not the real world values. We do not recommend blindly applying those values to production systems.

Session Expiration

In certain cases, it may be useful for the server to control its resources via bounding the lifetime of a session. The Session Expiration module is attached at the connection level and expires a service/session after a certain amount of idle time. The module is implemented by ExpiringService.

The default setting for the Expiration module is to never expire a session. Here is how it can be configured [2].

import com.twitter.conversions.DurationOps._
import com.twitter.finagle.Http

val twitter = Http.server
  .withSession.maxLifeTime(20.seconds)
  .withSession.maxIdleTime(10.seconds)
  .newService("twitter.com")

The Expiration module takes two parameters:

  1. maxLifeTime - the maximum duration for which a session is considered alive
  2. maxIdleTime - the maximum duration for which a session is allowed to idle (not sending any requests)

See Expiration metrics for more details.