Augmenting Dropwizard with Swagger and Rollbar

Wednesday, January 6, 2016 :: Tagged under: engineering pablolife. ⏰ 10 minutes.

Hey! Thanks for reading! Just a reminder that I wrote this some years ago, and may have much more complicated feelings about this topic than I did when I wrote it. Happy to elaborate, feel free to reach out to me! 😄

The following is a cross-post with the Reonomy blog. It was made to soar and look good there, but figured I'd put it here since it's still my writing. 😛

These days, we do things a bit differently in the company, as do I personally. I use Kotlin with Dropwizard and a host of slightly different techniques than in the example project. Nonetheless, hope it's helpful to someone!

Hi Comrades! I'm Pablo 😄

I was responsible for writing the server layer for a new product under a tight deadline. I ended up writing it in Java with Dropwizard, and thought I'd share where I hit a pitfall or two in implementing the server to have certain features not specific to the domain logic of our app.

If you're considering implementing your next REST service with a powered-up deployment of Dropwizard, read on!

(follow along with the example project demonstrating these integrations!)

Requirements of the server

Beyond fulfilling the needs of the customers, my design goals from an engineering perspective were these:

  1. Work with stack with which I can move quickly enough to hit this deadline!

  2. Implement it such that other engineers who aren't me can maintain it with an acceptably low number of wtfs per minute.

  3. Have the server integrate with Swagger such that users of my server can easily see which endpoints are served, what their parameters are, and what the potential responses are without having to dig into the source. See an example of a Swagger-documented service in action with their petstore example.

  4. Have the server integrate well with other services we're using, like Rollbar.

  5. Enforce certain constraints that I believe make for better software, like minimization of null values or defaulting to immutability.

  6. Have reasonable performance characteristics, for some definition of "reasonable."

  7. Establish a groundwork for a product that is easy to measure, monitor, and deploy.

For 1 & 2, I chose Java

I know, Java is what boring kids use. Java was what the Rails crowd positioned themselves against to be awesome and make cool apps and things.

But I know it well, and am confident I can use it to build a server that fits the requirements above. It has solid performance characteristics compared to many "cooler" dynamic languages, and the realities of maintaining Java have made it a less unpleasant experience than many Cool stacks in my limited experience doing that. At a previous job I was tasked with a large-scale refactor on a ~280,000 line C++ project, and yet another previous experience had me refactor of a 6,000 line Python project. The C++ gig was much easier, mostly due to just being better engineered, but a significant amount is due to static types and good IDE support. If I'm thinking of my successors having to maintain this, I hope they get assistance in understanding the codebase and the program's semantics from this tool called a "computer."

(Not to knock too hard on up-and-coming languages: I actually completed this server while learning Elixir! Then I woke up, as I always do, a sad and broken human.)

And hey, Facebook and Etsy were built on PHP, so sweating too hard on language/stack choices eventually becomes a void proposition.

For 6 & 7, I chose Dropwizard

I came across Dropwizard through this marvelous series of blog posts, and when I tried it in pet projects, it impressed me. Built-in use of libraries that are "must-have" drop-ins like Joda-Time and Guava, out-of-the-box support for healthchecks and metrics, centralized configuration, while also still giving you the flexibility to do things your own way. This wasn't a do-everything-for-you framework, nor a we-serve-endpoints-and-nothing-else microframework.

For 3, 4, and 5…

And here's the meat! Out of the box, Dropwizard requires some configuration if you'd like to integrate Rollbar or Swagger. And those tools may not always play well if you're using certain Dropwizard features, like Auth-gated endpoints.

I'll go over our Swagger integration, our Rollbar integration, and making sure your JSON models are consistent if you're using Retrofit.

Swagger, or Love You Some Annotations

There's a lot to making your endpoints play well with Swagger. I'll go through the following here:

Hooking it up

You'll want to add the following to your build.gradle:

compile ('io.swagger:swagger-jersey2-jaxrs:1.5.3') {
    exclude group: 'org.glassfish.jersey.core', module: 'jersey-common'
    exclude group: 'org.glassfish.jersey.core', module: 'jersey-server'
    exclude group: 'org.glassfish.jersey.core', module: 'jersey-client'
    exclude group: 'org.glassfish.jersey.containers', module: 'jersey-container-servlet-core'
}
compile ('io.swagger:swagger-jaxrs:1.5.3') {
    exclude group: 'javax.ws.rs', module: 'jsr311-api'
}
compile ('io.swagger:swagger-annotations:1.5.3') {
   exclude group: 'io.swagger', module: 'swagger-parser'
}

That's a lot of exclusions! Turns out many of the Swagger libraries can cause version conflicts with many of Dropwizard's dependencies, and can give you ClassNotFoundExceptions or NoSuchMethodExceptions if the JVM loads the libraries in the wrong order at runtime. Java, amirite?

Once you have this, we can begin the heinous-looking but labor-saving Annotation-based programming of your Resources. For example, here is merely the method declaration for an endpoint:

@Timed
@PUT
@Path("/{id}")
@Consumes(MediaType.APPLICATION_JSON)
@ApiOperation(
    value = "Alters an existing X and returns the new, updated version.",
    response = Resource.class
)
@ApiResponses(value = {
    @ApiResponse(code = 404, message = "Resource ID not found"),
    @ApiResponse(code = 400, message = "Invalid resource spec")
})
public Response updateResource(
   @Auth AuthedUser user,
   @ApiParam(value = "Resource ID", required = true) @PathParam("id") UUID id,
   @ApiParam(value = "JSON object with the resource params", required = true) ResourceParameters newParams,
   @ApiParam(value = "Page number", required = false, defaultValue = "1") @QueryParam("page") Integer pageNumber,
   @ApiParam(value = "Page size", required = false, defaultValue = "20") @QueryParam("pageSize") Integer pageSize
) {

This looks mighty gross, so lets go over what you're getting here:

Data models are a fair bit easier:

@ApiModel(value = "MyModel")
public class MyModel {
    @ApiModelProperty(value = "The value stored by this model", required = false, name = "my_value")
    private Optional<String> myValue;

    // Repeat for the various fields…
}

Once your resources and models are hooked up, add the following to you toplevel run method:

env.jersey().register(ApiListingResourceJSON.class);
env.jersey().register(SwaggerSerializers.class);
ScannerFactory.setScanner(new DefaultJaxrsScanner());

Once this is set up, start up your server and navigate to <toplevel>/swagger.json. You should see a Swagger document with your values populated!

Working with @Auth

I later ran into a bug where the @Auth param was getting recorded in the Swagger documentation, suggesting to clients that my endpoints all required data they couldn't have a hope of providing. It turns out we need to instruct Swagger to ignore those parameters when scanning, but how?

Turns out we have to extend and register SwaggerSpecFilter. I worked off of this StackOverflow answer to build this class, which contains a lot of throat-clearing, but not a whole lot of functionality:

public class AuthParamFilter implements SwaggerSpecFilter {
    @Override
    public boolean isOperationAllowed(
            Operation operation,
            ApiDescription api,
            Map<String, List<String>> params,
            Map<String, String> cookies,
            Map<String, List<String>> headers) {
        return true;
    }

    @Override
    public boolean isParamAllowed(
            Parameter parameter,
            Operation operation,
            ApiDescription api,
            Map<String, List<String>> params,
            Map<String, String> cookies, Map<String,
            List<String>> headers) {
        String access = parameter.getAccess();
        if (access != null && access.equals("internal")) return false;
        return true;
    }

    @Override
    public boolean isPropertyAllowed(
            Model model,
            Property property,
            String propertyName,
            Map<String, List<String>> params,
            Map<String, String> cookies,
            Map<String, List<String>> headers) {
        return true;
    }
}

All it does is check the "access" value of a parameter annotation, and choose to ignore it if the value is "internal."

So if we go back to our resource definition, note the addition of access = internal:

@ApiResponses(value = {
    @ApiResponse(code = 404, message = "Resource ID not found"),
    @ApiResponse(code = 400, message = "Invalid resource spec")
})
public Response updateResource(
   @ApiParam(access = "internal") @Auth AuthedUser user,
   @ApiParam(value = "Resource ID", required = true) @PathParam("id") UUID id,
   @ApiParam(value = "JSON object with the resource params", required = true) ResourceParameters newParams,
  // ellided

Finally, we can register this in the run method, with our other Swagger registrations:

FilterFactory.setFilter(new AuthParamFilter());

Voila! The @Auth parameters (and anything else you mark as access = "internal") ceases to exist to the outside world!

Serving it up

This last bit is likely the most hackey: using Swagger UI to self-serve our docs. We'll likely move to a model where our docs are served by a dedicated instance reading specs from our services rather than have the services serve themselves, but until we get there, this is a decent way to make the docs available to any clients who need them.

To do this:

if (!isProduction()) {
    bootstrap.addBundle(new AssetsBundle("/swagger-ui", "/api-docs", "index.html"));
}

Now your clients can get pretty documentation and an API playground! Of course, now you have to maintain and update the local Swagger UI yourself, remember that bespoke change, and checking this in messes with your repo's statistics:

Now we're a JavaScript Project!

Now we're a JavaScript project!

But now it's done and you can improve it later. And what is engineering but a set of compromises that makes you cry?

Rollbar, a story of appenders and design choices

We use Rollbar to monitor errors in our company's applications, and wanted this product to fall in line with that. And they have Java integrations! Look at that! One of them works with Logback, one of the technologies Dropwizard ships with! This should be easy!

Alas, poor Yorick, it was a bit of a bear. The issue, succinctly:

Luckily, the linked threads, and this, illuminate a way forward: you need to write a custom wrapper class, then add a file to META-INF in src/main/resources as well as a few configuration variables.

So here's the build.gradle dependency addition:

compile 'com.tapstream:rollbar-logback:0.1'

Here is literally the entire class we wrote, based on the linked Gist:

@JsonTypeName("rollbar")
public class RollbarAppenderFactory extends AbstractAppenderFactory {

    @NotNull private String environment = "development";
    private String apiKey;

    @JsonProperty
    public void setEnvironment(String environment) {
        this.environment = environment;
    }

    @JsonProperty
    public void setApiKey(String apiKey) {
        this.apiKey = apiKey;
    }

    @JsonProperty public String getEnvironment() { return environment; }
    @JsonProperty public String getApiKey() { return apiKey; }

    @Override
    public Appender<ILoggingEvent> build(LoggerContext context, String applicationName, Layout<ILoggingEvent> layout) {
        final RollbarAppender appender = new RollbarAppender();
        appender.setApiKey(apiKey);
        appender.setEnvironment(environment);
        appender.setContext(context);
        addThresholdFilter(appender, threshold);
        appender.start();
        return wrapAsync(appender);
    }
}

You then have to include a specific file src/main/resources/META-INF/services/io.dropwizard.logging.AppenderFactory with this line: package.path.to.class.RollbarAppenderFactory for the JVM to register the class magically.

And finally, include the needed variables in your YAML config file:

logging:
<h1 id="-logging-settings-as-you-like"># logging settings as you like</h1>
    # logging settings as you like
    appenders:
        - type: rollbar
          apiKey: ${AQUIFER_ROLLBAR_TOKEN}
          environment: ${ENVIRONMENT}
          threshold: ERROR

JSON, and MORE

We use Retrofit to create HTTP clients for internal and external services that provide that API (as well as a client for our own services in integration tests). Retrofit 1.9.x used to rely on Gson by default for its JSON processing, though it looks like in the current 2.0 beta they make you explicitly plug in your provider.

Dropwizard uses Jackson, so make your life a lot easier by configuring Retrofit (or any other JSON consumer) to use Jackson when possible, and share logic for any ObjectMappers you use.

For example, in our app:

To enable this: add a few lines to build.gradle:

dependencies {
    compile 'com.squareup.retrofit:converter-jackson:1.9.0' 
    compile 'com.fasterxml.jackson.datatype:jackson-datatype-guava:2.6.2'
    compile 'com.fasterxml.jackson.datatype:jackson-datatype-joda:2.6.1'
}

Centralize your Object Mapper configuration logic. We use a static method, which can be fraught if abused but we're only doing it here, really:

public static ObjectMapper configureObjectMapper(ObjectMapper mapper) {
    mapper.registerModule(new GuavaModule())
        .configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false)
        .setSerializationInclusion(JsonInclude.Include.NON_ABSENT);
    return mapper;
}

Then in our toplevel app's run method:

public void run(AquiferConfiguration cfg, Environment env) throws ClassNotFoundException, IOException {
    ObjectMapper mapper = AppModule.configureObjectMapper(env.getObjectMapper());
    // Pass the ObjectMapper to whomever needs it…
}

Meanwhile, in your Retrofit RestAdapter.Builder, you can configure it to use Jackson + your Mapper like so:

// `mapper` is injected or configured per above
new RestAdapter.Builder()
    .setEndpoint("my.endpoint.com")
    .setConverter(new JacksonConverter(mapper))

Still to come!

There are a number of other integrations/properties that I've included or would like to include that didn't make it in time for this blog post:

(gifs taken by this Let's Play of Mega Man 4.)

Thanks for the read! Disagreed? Violent agreement!? Feel free to join my mailing list, drop me a line at , or leave a comment below! I'd love to hear from you 😄