Querying OpenTSDB with Java. Yes, it can be done!

We needed a Java client for OpenTSDB to store our time series data and so I'll share the result here.

Below, I'll be fleshing out two ways of storing data in OpenTSDB using the HTTP API with Java.  The first is using POST, where the payload is a JSON Object containing all of our query parameters.  The second way uses GET and all of the query parameters are in the request string.

Let's jump in.

We'll be using OpenTSDB version 2.0.1 today.

So for starters, let's do the POST approach.

First, open your HTTP connection in the usual way.  Make sure it's set to POST and you're setting DoOutput and DoInput.

URL url = new URL(urlString + "/api/query/");
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Accept", "application/json");
conn.setRequestProperty("Content-type", "application/json");
conn.setRequestMethod("POST");
conn.setDoOutput(true);
conn.setDoInput(true);

Now get yourself an OutputStreamWriter.

OutputStreamWriter writer = new OutputStreamWriter(httpConnection.getOutputStream());

At a minimum, OpenTSDB requires a start time, end time and metric for querying data.  The start and end times should be expressed in epoch time using a long.  It's also a good idea to specify the aggregator you want to use although that isn't required.

long startEpoch = 1420088400L;
long endEpoch = 1425501345L;
String metric = "warp.speed";
String aggregator = "sum";

I'll also add a couple of tags in order to show how to do it below.

So now, we create our main JSONObject.

JSONObject mainObject = new JSONObject();
mainObject.put("start", startEpoch);
mainObject.put("end", endEpoch);

So far so good.  Next, we need our query parameters.  This includes the aggregator and metric.  We will put these in a JSONObject which, in turn, will live inside a JSONArray.

JSONArray queryArray = new JSONArray();

JSONObject queryParams = new JSONObject();
queryParams.put("aggregator", aggregator);
queryParams.put("metric", metric);

And we put this into the array.

queryArray.put(queryParams);

Now, if you're using tags, add them next.

JSONObject queryTags = new JSONObject();
queryTags.put("starshipname", "Enterprise");
queryTags.put("captain", "JamesKirk");
queryTags.put("hullregistry", "NCC-1701");

And add that to the queryParams object.

queryParams.put("tags", queryTags);

Now add the queryArray to the main object.

mainObject.put("queries", queryArray);

You can now write this JSONObject as a String to your connection.

String queryString = mainObject.toString();

writer.write(queryString);
writer.flush();
writer.close();

Now, the server will respond with a response code.  If it's good (HTTP_OK) then you should be able to get the response data.

int HttpResult = httpConnection.getResponseCode();

if(HttpResult == HttpURLConnection.HTTP_OK){
     result = readHTTPConnection(httpConnection);
}

And here's the readHTTPConnection method:

public static String readHTTPConnection(HttpURLConnection conn){
StringBuilder sb = new StringBuilder();
BufferedReader br;
try {
br = new BufferedReader(new InputStreamReader(conn.getInputStream(),"utf-8"));
String line = null;
while ((line = br.readLine()) != null) {
sb.append(line + "\n");
}
br.close();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return sb.toString();
}

What you get back is a string that can be turned into a JSONArray.

return new JSONArray(result);

Now, that's the way I like to do it because working with a bunch of String appenders feels clunky to me, but if you're a fan of that approach, here's how you can run that same query using an HTTP GET request.

long startEpoch = 1420088400L;
long endEpoch = 1425501345L;
String metric = "warp.speed";
String aggregator = "sum";
String result = "";

So now we manually build our request string:

StringBuilder builder = new StringBuilder();

builder.append("?start=");
builder.append(startEpoch);
builder.append("&end=");
builder.append(endEpoch);
builder.append("&m=sum:");
builder.append(metric);

Now, if we have tags, they should be in pairs and the entire set is enclosed in curly braces and separated by commas.

builder.append("{starshipname=Enterprise,captain=JamesKirk,hullregistry=NCC-1701}");

URL url = new URL(urlString + "/api/query/" + builder.toString());
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Accept", "application/json");
conn.setRequestProperty("Content-type", "application/json");
conn.setRequestMethod("GET");

And we get the result data back exactly the same way we did above.

int HttpResult = httpConnection.getResponseCode();

if(HttpResult == HttpURLConnection.HTTP_OK){
     result = readHTTPConnection(httpConnection);
}

return new JSONArray(result);

So the GET approach is shorter but I personally prefer the POST method.  Either way will get you the same results.