Joey Bratton

Envoy currently does not produce a binary that runs out of the box on CentOS 6/7. The main build container is based on Ubuntu and it produces a binary that requires glibc >= 2.18. Unfortunately, CentOS 7 is on 2.17 and CentOS 6 is on 2.12.

Rather than taking on ownership of a custom build, we found a more creative way to get Envoy running. There's a tool called PatchELF that allows you to modify an ELF binary's interpreter path. On a Linux system, this is generally /lib/ld-linux.so.2, but with PatchELF we can change that to a relative path containing a custom build of glibc.

The process for creating our modified Envoy has 2 main steps:

  1. Fetch the Envoy docker image for the target version and extract the binary.
  2. Build glibc 2.18 and patch the Envoy binary to use it from a relative path.

Fetching the Envoy binary

We have an update script that plucks the binary from the docker image and saves some metadata about which version we currently have:

#!/usr/bin/env bash
#
# Usage
# ./fetch_envoy.sh <ENVOY_SHA>

set -eu

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
ENVOY_DIR="${DIR}/envoy"

ENVOY_SHA=$1

# Since envoy doesn't publish a standalone binary yet, we fetch their docker
# image and extract the binary from it.
ENVOY_CONTAINER_ID=$(docker create envoyproxy/envoy:${ENVOY_SHA})

docker cp ${ENVOY_CONTAINER_ID}:/usr/local/bin/envoy "${ENVOY_DIR}/envoy"
docker rm ${ENVOY_CONTAINER_ID}

echo "${ENVOY_SHA}" > "${ENVOY_DIR}/ENVOY_SHA"

Packaging the modified Envoy binary

A CentOS-based Dockerfile is then used to build glibc 2.18, prep Envoy to use it, and package it all together:

FROM centos:7

ARG glibc_version=2.18
ARG patchelf_version=0.9

RUN yum install -y autoconf automake gcc gcc-c++ git make wget

ADD glibc-${glibc_version}.tar.xz .
ADD patchelf-${patchelf_version}.tar.bz2 .

RUN mkdir glibc-${glibc_version}-build && \
    cd glibc-${glibc_version}-build && \
    ../glibc-${glibc_version}/configure --prefix=/opt/glibc-${glibc_version} && \
    make -j4 && \
    make install && \
    cd ..

RUN cd patchelf-${patchelf_version} && \
    ./configure && \
    make && \
    make install && \
    cd ..

WORKDIR /package
ADD envoy envoy

# Patch the envoy binary so that it uses our custom glibc build.
RUN cp -r /opt/glibc-${glibc_version}/lib . && patchelf --set-interpreter lib/ld-linux-x86-64.so.2 --set-rpath lib envoy

RUN tar cf envoy.tar *

# CMD was required because of the error & explanation here: https://github.com/hashicorp/vagrant/issues/4602
CMD ["/bin/bash"]

The Dockerfile assumes that glibc-2.18.tar.xz and patchelf-0.9.tar.bz2 exist along with it in the same directory as the raw Envoy binary.

You can run the build and extract the custom Envoy package with the following script:

#!/usr/bin/env bash
#
# Usage:
# ./package_centos_envoy.sh

set -eu

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
ENVOY_DIR="${DIR}/envoy"

# Use our envoy packaging image to do our custom glibc build and patch the envoy binary to use it.
docker build -f "${ENVOY_DIR}/Dockerfile-package-envoy" -t centos-envoy-package:build "${ENVOY_DIR}"

PACKAGE_CONTAINER_ID=$(docker create centos-envoy-package:build)

docker cp ${PACKAGE_CONTAINER_ID}:/package/envoy.tar "${ENVOY_DIR}/centos-envoy.tar"
docker rm ${PACKAGE_CONTAINER_ID}

That should leave you with a centos-envoy.tar package containing the modified Envoy binary and its custom glibc 2.18 build.

It feels like a pretty nasty hack, but we've been using it without issues for close to a year now. Over that time, creating and maintaining a custom CentOS-friendly build of the raw binary would have involved a lot more effort than this approach (which has essentially worked without changes since we started doing it).

Martin Fowler provides a good overview of Dependency Injection and Inversion of Control, so I'll skip the overview and assume that you are familiar with both concepts.

Most devs that I know who are familiar with DI tend to have strong feelings about it. Many of those feelings are negative because they've seen how easy it is to abuse DI. This is definitely a “great power, great responsibility” scenario, and using DI successfully requires a thorough understanding of the objects that you intend to inject. My goal here is to provide some guidelines for properly harnessing that power and avoiding common pitfalls.

I'll use Java in my examples, but the same principals should apply to any language or DI framework.

Prefer Constructor Injection Wherever Possible

Most DI frameworks provide a variety of methods for injecting objects (constructors, fields, methods, etc.). Constructor injection should be preferred because it leads to code that is less tightly coupled to your DI framework, and therefore easier to test in isolation and re-use outside the scope of your DI framework. For example, let's pretend we have a hypothetical WidgetManager interface:

public interface WidgetManager {
    void startWidget(Widget widget);
}

And 2 different implementations of that interface, one that uses constructor injection and one that uses field injection:

public class ConstructorInjectionWidgetManager implements WidgetManager {
    private final ThingService thingService;

    @Inject
    public ConstructorInjectionWidgetManager(ThingService thingService) {
        this.thingService = thingService;
    }

    public void startWidget(Widget widget) {
        // Use thingService to start the widget ...
    }
}
public class FieldInjectionWidgetManager implements WidgetManager {
    @Inject
    ThingService thingService;

    public void startWidget(Widget widget) {
        // Use thingService to start the widget ...
    }
}

At first glance, the field-based implementation definitely looks simpler. The issue is that it's not obvious from the public interface of that class what is required for it to function correctly. If you want to use it outside of the DI context (e.g. in a unit test), then you have to read through the internals of the class to know that you'll need to set that non-public thingService field.

The constructor-based approach uses the type system to force users of that class to provide any needed dependencies. The field-based approach can lead to subtle bugs that only show up at runtime if a dev erroneously manually constructs an instance and neglects to set a required dependency. With the constructor-based approach, the compiler would prevent that type of error from ever happening.

Avoid Depending On Lifetimes/Scopes

Most DI frameworks provide some way to specify a dependency instance's lifetime. The main examples are per-lookup (new instance on every injection), singleton (same, single instance for every injection), and per request (for web apps, same instance for every injection over the course of a single web request).

Injecting objects that function differently based on their scope can lead to runtime issues that are difficult to diagnose and may require major refactoring to fix. For example, let's use JPA's EntityManager. An EntityManager basically manages a single “unit of work” in the database and is used to fetch entities, track their changes, and commit those changes back to the database. In order to commit an entity instance to the database, it must be “tracked” by that instance of the EntityManager. Let's pretend we have two different services:

public class FetcherService {
    private final EntityManager entityManager;

    @Inject
    public FetcherService(EntityManager entityManager) {
        this.entityManager = entityManager;
    }

    public Widget getWidgetById(int id) {
        // Use the entityManager to fetch and return the requested widget
    }
}
public class UpdaterService {
    private final EntityManager entityManager;

    @Inject
    public UpdaterService(EntityManager entityManager) {
        this.entityManager = entityManager;
    }

    public void updateWidgetName(Widget widget, String name) {
        // Use the entityManager update the widget name and save it to the database
    }
}

If the EntityManager DI binding is scoped as something like singleton (which is a terrible idea since it keeps a ton of state) or per-request then you can fetch the widget in one service, pass it to the next service and update it without issue. On the other hand, if the EntityManager DI binding is scoped as per-lookup (new instance on every injection) then things get more complicated. Each service would therefore have a different instance of EntityManager, which means the UpdaterService would need to know that it has to attach that instance of the widget to its context in order to start tracking it before it can update it. With EntityManager you also run into a similar class of issues with transactions since each instance supports 1 (and only 1) concurrent transaction.

With classes like EntityManager that are so dependent on state and lifetime, you're probably better off manually managing their lifetime in order to be as explicit as possible about where instances are being shared and where they are not.

In my experience the best uses for scopes are as an optimization for objects that are time consuming to construct, and for setting hard bounds on the lifetime of a cache.