If you’ve ever used stat-tracking software like StatsD and Graphite, you may have already heard of InfluxDB. For those who have not, it is a time-series database, useful for tracking data to be aggregated over a period of time.

I’ve been using InfluxDB for some time to track a variety of data in my Rails applications. InfluxDB caught my eye because it not only tracked data, but metadata, too. Metadata is anything you might want to use to “scope” your query (that is not time-related). For me, this is the ID of a client in a multi-tenant application for background processing work.

To make it easier to use in my Rails applications I wrote a little library called Tremolo which has similar functionality to the Ruby StatsD library.

By way of introduction, I would like to show how I leveraged ActiveSupport::Notifications to send my data to InfluxDB using Tremolo. I’ll do this in three parts.

• Basics of Tremolo
• Setting up ActiveSupport::Notifications
• Tie the two together

As a bonus, I’ll also give you a peek at what the data looks like graphed on my dashboard, which uses Grafana.

Intro to Tremolo

Tremolo is a library to send data in InfluxDB’s wireline protocol over UDP, built using Celluloid. Using it is very straightforward. First, we make ourselves a tracker, pointing it at our InfluxDB instance.

Tremolo.supervised_tracker(:tracker, '0.0.0.0', 4444)

To send InfluxDB some stats, we use our new tracker.

tracker = Tremolo.fetch(:tracker)

# increment a stat by 1
tracker.increment('count.series-name')

Tremolo also supports decrement, timing with an integer for milliseconds, and most importantly for my examples: time with a block.

tracker = Tremolo.fetch(:tracker)

# increment a stat by 1
tracker.time('timing.series-name') do
  # do something that takes time, like making an API call
  Net::HTTP.get(URI('http://example.com/api/v1/things'))
end

Now that we’ve got the basics of Tremolo, let’s take a look at what Rails has to offer to make our tracking a bit easier.

ActiveSupport::Notifications

Rails has some very simple, but handy, functionality to allow us to instrument our code. Elsewhere, we can subscribe to get notifications of when our instrumented code is run. Rails uses this to instrument requests and calls to the database, while also enabling third-party libraries to hook into the system at both ends.

ActiveSupport::Notifications has some key features that make it especially well-suited to tie into a stats-tracking library like Tremolo.

1. instrument accepts a payload of extra information
2. instrument can handle timing a block of code
3. We can subscribe to a simple regex pattern to capture similar notifications

Let’s take a look at a simple example, making an API call: Note: I’m going to alias ActiveSupport::Notifications as ASN for brevity.

ASN.instrument('timing.external_requests', customer_id: customer.id) do
  ApiClient.get("/api/customers/#{customer_id}")
end

And to subscribe to this:

ASN.subscribe('timing.external_requests') do |*args|
  event = ASN::Event.new(*args)

  # event contains lots of data, including:
  event.name # the full name of the instrument, 'timing.external_requests'
  event.duration # timing for the block
  event.payload # extra hash we passed to `instrument`
end

Straightforward, and powerful. Let’s make use of this to build some flexible tracking in our application.

And Put ‘em Together

Subscriptions

Let’s start by setting up some broad subscriptions using regex patterns:

ASN.subscribe(/\Atiming/) do |*args|
  # fill in here
end

Now, let’s fill in some tracking with Tremolo:

ASN.subscribe(/\Atiming/) do |*args|
  event = ASN::Event.new(*args)
  tracker = Tremolo.fetch(:tracker)

  tracker.timing(event.name, event.duration, event.payload)
end

Inside our subscribe block, we make an event from the args, then get our tracker. Once we have those, we track the timing from our event, using the full name and duration, passing the payload as metadata last.

I also like to set up a count subscription, like so:

ASN.subscribe(/\Atiming/) do |*args|
  event = ASN::Event.new(*args)
  tracker = Tremolo.fetch(:tracker)

  data = event.payload
  value = data.delete(:value) || 1

  tracker.write_point(event.name, {value: value}, data)
end

There’s a little more to this one than our timing. First difference is we want to get a value for the count. If none is provided use 1 as a default. We use delete because we don’t want this value tracked into the metadata in InfluxDB.

Next we use write_point on our tracker because we’ve got our own value to track.

Instrument Our Code

Now that we have our subscriptions for both timing and count, we can start to instrument our code throughout. What’s great is we just use the code from earlier:

ASN.instrument('timing.external_requests', customer_id: customer.id) do
  ApiClient.get("/api/customers/#{customer_id}")
end

To track to a count series:

ASN.instrument('count.external_requests', {customer_id: customer.id})

What to Track

Here are some examples of the things I instrument in my own applications.

• timing of external API calls
• timing of background work
• counts of calls to deprecated methods
• business statistics like user logins/registrations
• cache misses

Though, if it takes time in code, or is of any importance to your business or the functioning of other code, track it!

Bonus Round: Grafana

InfluxDB is extremely easy to get installed and running. If you’re on a Mac brew install will get you running, and a simple configuration change will set up a UDP listener.

[udp]
  enabled = true # make sure this is true
  bind-address = ":4444" # pick your own port here, match to Tremolo's config
  database = "your-database-here"
  batch-size = 1000
  batch-timeout = "1s"

InfluxDB does not, however, have a UI for anything other than querying databases/series with a SQL-esque query language. This is where Grafana comes in.

Grafana has installation packages for most linux platforms, but for Mac OS X you’ll need to install from source. This was quick and easy on my system, and required no further (backend) configuration to get running. Just set your data source (influxdb) and go.

Wrap it Up

I’ve found ActiveSupport::Notifications and Tremolo to be a fantastic combo. I hope this article has made it easier for you to get into tracking important data in your own applications.

Read more here about ActiveSupport::Notifications and more about Tremolo.

If you’re wondering about the name Tremolo, I have a whole series of libraries for integrating stats in a variety of ways, all named for different musical terms. Legato pulls data from Google Analytics, and Staccato sends data to GA using the official tracking API. Tremolo is just the latest in this collection. Check ‘em all out!