One of the projects I worked on when I was on the maps team at MapQuest was Avecado, a library for building Mapnik vector tiles, along with language bindings for Python and some utility programs. For the upcoming OpenStreetMap Carto hstore work I needed to benchmark database performance for rendering, without the overhead of generating the images. I had a number of requirements for parallelism and connection reuse to accurately simulate a real workload, and realized that avecado_server could meet my needs.

Avecado isn’t the easiest program to build, requiring very recent versions of Mapnik and Boost that are not packaged in most OS distributions. The precise versions needed are frequently changing, so as with any guide, be sure to read the documentation for any changes.

Since I was going to be benchmarking, I immediately turned to a server I use only for that. VMs are handy, but their performance is subject to too much variation, making them unsuited for the type of benchmarking I was going to be doing.

The overall plan for getting Avecado installed is to prepare the server, install pre-requisites, compile Boost, compile Mapnik, and finally compile Avecado.

This article assumes familiarity with Linux and compiling software.

Preparing the server

The first step is always to update the software on the server, which was running Ubuntu 14.04.

1
2

sudo apt-get update
sudo apt-get -y dist-upgrade

Installing pre-requisites

There’s a fair number of pre-requisites for a full Mapnik install with support for all the plugins and formats, but I just needed a minimal set that would let me do a basic build. Mapnik and Avecado have many dependencies in common.

1
2
3
4
5

sudo apt-get -y --no-install-recommends install git automake autoconf make g++ \
  libtool libcurl4-openssl-dev python-dev protobuf-compiler libprotobuf-dev \
  libicu-dev python-dev libxml2 libxml2-dev libfreetype6 libfreetype6-dev \
  libjpeg-dev libpng-dev libfreetype6 libfreetype6-dev libxml2 libxml2-dev \
  libpq-dev libharfbuzz-dev libproj-dev

Because the versions of Boost and Mapnik needed are not packaged, they need to be built from source alongside Avecado.

1
2
3
4
5
6

mkdir -p ~/src
cd ~/src
curl -L -o boost_1_57_0.tar.gz https://sourceforge.net/projects/boost/files/boost/1.57.0/boost_1_57_0.tar.gz/download &
git clone -q https://github.com/mapnik/mapnik.git &
git clone -q https://github.com/MapQuest/avecado.git &
wait

Building boost

Boost can take a long time to compile if you build everything, but fortunately we only need some of the libraries and can speed up the build by specifying them.

1
2
3
4
5
6
7
8

cd ~/src
tar -zxf boost_1_57_0.tar.gz
cd boost_1_57_0

mkdir -p $HOME/boost
./bootstrap.sh --prefix=$HOME/boost --with-libraries=program_options,thread,filesystem,python,regex,system,iostreams,date_time
./b2 -j8
./b2 install

If you want to install all of boost, --with-libraries can be omitted.

Building Mapnik

Mapnik makes heavy use of C++ templates and takes a lot of RAM and a long time to compile, sometimes an hour on slower machines.

Mapnik releases are very infrequent, with the last release almost two years ago, so a development version needs to be used, and specified by the git commit hash. This specific version is likely to change in the future, so be sure to check the documentation.

1
2
3
4
5
6
7
8

mkdir -p $HOME/mapnik
cd ~/src/mapnik
git checkout 7ee9745

./configure PREFIX=$HOME/mapnik BINDINGS=none \
BOOST_INCLUDES=$HOME/boost/include/boost BOOST_LIBS=$HOME/boost/lib

LD_LIBRARY_PATH=$HOME/boost/lib JOBS=8 make install

Building Avecado

We can now build Avecado, configuring it to use the previously installed Boost and Mapnik. It comes with a test-suite which can also be used to check everything works.

1
2
3
4
5
6
7
8
9

cd ~/src/avecado
mkdir -p $HOME/avecado
git submodule init
git submodule update
./autogen.sh
CPPFLAGS="-DBOOST_MPL_CFG_NO_PREPROCESSED_HEADERS -DBOOST_MPL_LIMIT_VECTOR_SIZE=30" ./configure --prefix=$HOME/avecado --with-boost=$HOME/boost --with-mapnik-config=$HOME/mapnik/bin/mapnik-config
make -j8
LD_LIBRARY_PATH="$HOME/boost/lib:$HOME/mapnik/lib" make check
make install

With the way Boost and Mapnik were installed, it’s necessary to specify LD_LIBRARY_PATH when running any of the Avecado command-line utilities.

The avecado utility can be run with LD_LIBRARY_PATH="$HOME/boost/lib:$HOME/mapnik/lib" ~/avecado/bin/avecado. This utility will allow rendering vector tiles, raster tiles, or bulk rendering vector tiles.

avecado_server is more useful for testing, and servers vector tiles over HTTP. It doesn’t do any caching, so even for previews it should be put behind a caching proxy like Varnish or Apache’s mod_cache. It can be run with LD_LIBRARY_PATH="$HOME/boost/lib:$HOME/mapnik/lib" ~/avecado/bin/avecado_server.

Next steps

The next step for my benchmarking is to load a database, point avecado_server at a suitable stylesheet and fetch vector tiles in parallel.