Jackson Integration

Finatra builds upon the TwitterUtil Jackson library for JSON which provides a c.t.util.jackson.ScalaObjectMapper. See the User Guide for details about the TwitterUtil Jackson library.

🚨 This documentation assumes some level of familiarity with Jackson JSON Processing library

Specifically Jackson databinding and the Jackson ObjectMapper.

There are several tutorials (and external documentation) which may be useful if you are unfamiliar with Jackson.

Additionally, you may want to familiarize yourself with the Jackson Annotations as they allow for finer-grain customization of Jackson databinding.

Case Classes

As mentioned, Jackson is a JSON processing library. We generally use Jackson for databinding, or more specifically:

  • object serialization: converting an object into a JSON String and

  • object deserialization: converting a JSON String into an object

The TwitterUtil Jackson integration is primarily centered around serializing and deserializing Scala case classes. This is because Scala case classes map well to the two JSON structures [reference]:

  • A collection of name/value pairs. Generally termed an object. Here, the name/value pairs are the case field name to field value but can also be an actual Scala Map[T, U] as well.

  • An ordered list of values. Typically an array, vector, list, or sequence, which for case classes can be represented by a Scala Iterable.

Basic Usage

Here we outline some simple usage patterns. More examples can be found in the TwitterUtil User Guide.

Let’s assume we have these two case classes:

case class Bar(d: String)
case class Foo(a: String, b: Int, c: Bar)

To serialize a case class into a JSON string, use

ScalaObjectMapper#writeValueAsString(any: Any): String

For example:

 Welcome to Scala 2.12.13 (JDK 64-Bit Server VM, Java 1.8.0_242).
 Type in expressions for evaluation. Or try :help.

 scala> import com.twitter.finatra.jackson.modules.ScalaObjectMapperModule
 import com.twitter.finatra.jackson.modules.ScalaObjectMapperModule

 scala> import com.twitter.util.jackson.ScalaObjectMapper
 import com.twitter.util.jackson.ScalaObjectMapper

 scala> val mapper: ScalaObjectMapper = (new ScalaObjectMapperModule).objectMapper
 mapper: com.twitter.util.jackson.ScalaObjectMapper = com.twitter.util.jackson.ScalaObjectMapper@5690c2a8

 scala> val foo = Foo("Hello, World", 42, Bar("Goodbye, World"))
 foo: Foo = Foo(Hello, World,42,Bar(Goodbye, World))

 scala> mapper.writeValueAsString(foo)
 res0: String = {"a":"Hello, World","b":42,"c":{"d":"Goodbye, World"}}

 scala> // or use the configured "pretty print mapper"

 scala> mapper.writePrettyString(foo)
 res1: String =
 {
   "a" : "Hello, World",
   "b" : 42,
   "c" : {
     "d" : "Goodbye, World"
   }
 }

 scala>

To deserialize a JSON string into a case class, use

ScalaObjectMapper#parse[T](s: String): T

For example, assuming the same Bar and Foo case classes defined above:

 Welcome to Scala 2.12.13 (JDK 64-Bit Server VM, Java 1.8.0_242).
 Type in expressions for evaluation. Or try :help.

 scala> import com.twitter.finatra.jackson.modules.ScalaObjectMapperModule
 import com.twitter.finatra.jackson.modules.ScalaObjectMapperModule

 scala> import com.twitter.util.jackson.ScalaObjectMapper
 import com.twitter.util.jackson.ScalaObjectMapper

 scala> val mapper: ScalaObjectMapper = (new ScalaObjectMapperModule).objectMapper
 mapper: com.twitter.util.jackson.ScalaObjectMapper = com.twitter.util.jackson.ScalaObjectMapper@2c1523cd

 scala> val s = """{"a": "Hello, World", "b": 42, "c": {"d": "Goodbye, World"}}"""
 s: String = {"a": "Hello, World", "b": 42, "c": {"d": "Goodbye, World"}}

 scala> val foo = mapper.parse[Foo](s)
 foo: Foo = Foo(Hello, World,42,Bar(Goodbye, World))

 scala>

You can find many examples of using the ScalaObjectMapperModule in the various framework tests:

As mentioned above, there is also a plethora of Jackson tutorials and HOW-TOs available online which provide more in-depth examples of how to use a Jackson ObjectMapper.

TwitterUtil ScalaObjectMapper

The TwitterUtil ScalaObjectMapper is a thin wrapper around a configured jackson-module-scala com.fasterxml.jackson.module.scala.ScalaObjectMapper. In Finatra, the recommended way to construct a ScalaObjectMapper is via the Finatra ScalaObjectMapperModule.

c.t.f.jackson.modules.ScalaObjectMapperModule

Finatra provides the ScalaObjectMapperModule, a c.t.inject.TwitterModule, which is the recommended way to configure and bind a ScalaObjectMapper to the object graph. This is similar to the jackson-module-guice ObjectMapperModule but uses Finatra’s TwitterModule.

The ScalaObjectMapperModule provides bound instances of:

Tip

Generally, you are encouraged to obtain a reference to the Singleton instance provided by the object graph over instantiating a new mapper. This is to ensure usage of a consistently configured mapper across your application.

The ScalaObjectMapperModule provides overridable methods which mirror the ScalaObjectMapper#Builder for configuring the bound mappers.

For example, to create a c.t.inject.TwitterModule which sets the PropertyNamingStrategy different than the default:

import com.fasterxml.jackson.databind.PropertyNamingStrategy
import com.twitter.finatra.jackson.modules.ScalaObjectMapperModule

object MyCustomObjectMapperModule extends ScalaObjectMapperModule {

    override val propertyNamingStrategy: PropertyNamingStrategy =
      new PropertyNamingStrategy.KebabCaseStrategy
}

Or to set additional modules or configuration:

import com.fasterxml.jackson.databind.{
  DeserializationFeature,
  Module,
  ObjectMapper,
  PropertyNamingStrategy
}
import com.twitter.finatra.jackson.modules.ScalaObjectMapperModule

object MyCustomObjectMapperModule extends ScalaObjectMapperModule {

    override val propertyNamingStrategy: PropertyNamingStrategy =
      new PropertyNamingStrategy.KebabCaseStrategy

    override val additionalJacksonModules: Seq[Module] =
      Seq(MySimpleJacksonModule)

    override def additionalMapperConfiguration(mapper: ObjectMapper): Unit = {
      mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, true)
    }
}

See the Modules Configuration in Servers or the HTTP Server Framework Modules for more information on how to make use of any custom ScalaObjectMapperModule.

c.t.f.jackson.modules.YamlScalaObjectMapperModule

Finatra also provides the YamlScalaObjectMapperModule which is an extension of the ScalaObjectMapperModule that can be used to configure and bind a ScalaObjectMapper using a Jackson YAMLFactory instead of a JsonFactory.

The YamlScalaObjectMapperModule provides bound instances of:

Since the YamlScalaObjectMapperModule is an extension of the ScalaObjectMapperModule is can be used in place of the ScalaObjectMapperModule where a YAML object mapper is desired.

Adding a Custom Serializer or Deserializer

To register a custom serializer or deserializer, configure any custom serializer or deserializer via the methods provided by the ScalaObjectMapperModule.

For example: first create the serializer or deserializer (a deserializer example is shown below)

import com.fasterxml.jackson.databind.JsonDeserializer
import com.fasterxml.jackson.databind.deser.Deserializers

// custom deserializer
class FooDeserializer extends JsonDeserializer[Foo] {
  override def deserialize(...)
}

// custom parameterized deserializer
class MapIntIntDeserializer extends JsonDeserializer[Map[Int, Int]] {
  override def deserialize(...)
}

// custom parameterized deserializer resolver
class MapIntIntDeserializerResolver extends Deserializers.Base {
  override def findBeanDeserializer(
    javaType: JavaType,
    config: DeserializationConfig,
    beanDesc: BeanDescription
  ): MapIntIntDeserializer = {
    if (javaType.isMapLikeType && javaType.hasGenericTypes && hasIntTypes(javaType)) {
      new MapIntIntDeserializer
    } else null
  }

  private[this] def hasIntTypes(javaType: JavaType): Boolean = {
    val k = javaType.containedType(0)
    val v = javaType.containedType(1)
    k.isPrimitive && k.getRawClass == classOf[Integer] &&
      v.isPrimitive && v.getRawClass == classOf[Integer]
  }
}

Then add via a Jackson SimpleModule or a jackson-module-scala JacksonModule:

import com.fasterxml.jackson.databind.module.SimpleModule
import com.fasterxml.jackson.module.scala.JacksonModule

// Jackson SimpleModule for custom deserializer
class FooDeserializerModule extends SimpleModule {
  addDeserializer(FooDeserializer)
}

// Jackson Module Scala JacksonModule for custom deserializer
class MapIntIntDeserializerModule extends JacksonModule {
  override def getModuleName: String = this.getClass.getName

  this += {
    _.addDeserializers(new MapIntIntDeserializerResolver)
  }
}

Note

It is also important to note that Jackson Modules are not Google Guice Modules but are instead interfaces for extensions that can be registered with a Jackson ObjectMapper in order to provide a well-defined set of extensions to default functionality. In this way, they are similar in concept to Google Guice Modules, but for configuring an ObjectMapper instead of an Injector.

Lastly, add the custom serializer or deserializer to your customized Finatra ScalaObjectMapperModule:

 import com.twitter.finatra.jackson.modules.ScalaObjectMapperModule

 object MyCustomObjectMapperModule extends ScalaObjectMapperModule {
   override val additionalJacksonModules = Seq(
     // added via a new anonymous SimpleModule
     new SimpleModule {
       addSerializer(LocalDateParser)
     },
     // added via a re-usable SimpleModule
     new FooDeserializerModule,
     // added via a re-usable JacksonModule
     new MapIntIntDeserializerModule
   )
 }

For more information see the Jackson documentation for Custom Serializers.

You would then use this module in your server. See the Modules Configuration in Servers or the HTTP Server Framework Modules for more information on how to make use of any custom ScalaObjectMapperModule.

Improved case class deserializer

The TwitterUtil ScalaObjectMapper provides a custom case class deserializer which overcomes some limitations in jackson-module-scala and it is worth understanding some its features:

The case class deserializer is added by default when constructing a new ScalaObjectMapper.

Tip

Note: with the TwitterUtil case class deserializer, non-option fields without default values are considered required.

If a required field is missing, a CaseClassMappingException is thrown.

Jackson InjectableValues Support

By default, the Finatra provides a ScalaObjectMapper via the ScalaObjectMapperModule configured to resolve Jackson InjectableValues via a given Google Guice Injector.

The default is very similar to the jackson-module-guice – GuiceInjectableValues.

Note

Jackson “InjectableValues” is not related to Dependency Injection or Google Guice. It is meant to convey the filling in of a value in a deserialized object from somewhere other than the incoming JSON. In Jackson parlance, this is “injection” of a value.

The Finatra c.t.finatra.jackson.caseclass.GuiceInjectableValues allows users to denote fields in the case class to fill with values that come from a configured Google Guice Injector such that you can do this:

import javax.inject.Inject

case class Foo(name: String, description: String, @Inject bar: Bar)

That is, annotate the field to inject with either:

and the framework will attempt to get an instance of the field type from the Injector with which the mapper was configured. In this case, the framework would attempt to obtain an instance of Bar from the object graph.

As mentioned, this is essentially the same functionality found in the jackson-module-guice GuiceInjectableValues with a key difference being that the Finatra version ensures we do not attempt lookup of a field from the Guice Injector unless there is a non-null Guice Injector configured and the field is specifically annotated with one of the supporting annotations.

Using the Foo case class above, you could then parse incoming JSON with the ScalaObjectMapper:

import com.twitter.finatra.jackson.modules.ScalaObjectMapperModule
import com.twitter.util.jackson.ScalaObjectMapper
import com.twitter.inject.Injector
import com.twitter.inject.app.TestInjector
import javax.inject.Inject

case class Foo(name: String, description: String, @Inject bar: Bar)

val json: String =
  """
    |{
    |  “name”: “FooItem”,
    |  “description”: “This is the description for FooItem”
    |}
  """.stripMargin

val injector: Injector = TestInjector(???).create
val mapper: ScalaObjectMapper = (new ScalaObjectMapper).objectMapper(injector.underlying)
val foo = mapper.parse[Foo](json)

When deserializing the JSON string into an instance of Foo, the mapper will attempt to locate an instance of type Bar from the given injector and use it in place of the bar field in the Foo case class.

Caution

It is an error to specify multiple field injection annotations on a field, and it is also an error to use a field injection annotation in conjunction with any JacksonAnnotation.

Both of these cases will result in error during deserialization of JSON into the case class when using the ScalaObjectMapper.

As mentioned, the Finatra HTTP integration provides extended Jackson InjectableValues support specifically for injecting values into a case class which can be obtained from different parts of an HTTP message. This uses the TwitterUtil @InjectableValues annotation on specific Finatra HTTP request related annotations.

See the HTTP Requests - Field Annotations documentation for more details on HTTP Message “injectable values”.

Important

It is important to note that Jackson currently only allows registration of a single InjectableValues implementation for an ObjectMapper.

Thus, if you configure your bound ScalaObjectMapper with a different implementation, the default behavior provided by the Finatra GuiceInjectableValues will be overridden.

Mix-in Annotations

The Jackson Mix-in Annotations provides a way to associate annotations to classes without needing to modify the target classes themselves. It is intended to help support 3rd party datatypes where the user cannot modify the sources to add annotations.

The TwitterUtil ScalaObjectMapper supports Jackson Mix-in Annotations for specifying field annotations during deserialization with the case class deserializer.

For example, to deserialize JSON into the following classes that are not yours to annotate:

case class Point(x: Int, y: Int) {
  def area: Int = x * y
}

case class Points(points: Seq[Point])

However, you want to enforce field constraints with validations during deserialization. You can define a Mix-in,

trait PointMixIn {
  @Min(0) @Max(100) def x: Int
  @Min(0) @Max(100) def y: Int
  @JsonIgnore def area: Int
}

Then register this Mix-in for the Point class type. There are several ways to do this. Generally, it is recommended to always prefer applying configuration in a custom ScalaObjectMapperModule to ensure usage of a consistently configured mapper across your application.

Implement via a Custom ScalaObjectMapperModule

For example, create a new Jackson SimpleModule:

import com.fasterxml.jackson.databind.module.SimpleModule

object PointMixInModule extends SimpleModule {
    setMixInAnnotation(classOf[Point], classOf[PointMixIn]);
}

Then add the SimpleModule to the list of additional Jackson modules in your custom ScalaObjectMapperModule:

import com.twitter.finatra.jackson.modules.ScalaObjectMapperModule

object MyCustomObjectMapperModule extends ScalaObjectMapperModule {
  override val additionalJacksonModules = Seq(PointMixInModule)
}

Deserializing this JSON would then error with failed validations:

{
  "points": [
    {"x": -1, "y": 120},
    {"x": 4, "y": 99}
  ]
}

As the first Point instance has an x-value less than the minimum of 0 and a y-value greater than the maximum of 100.