UDP Module

Summary

blueflood-udp module is a demonstration for anyone who wants to learn how to implement new ingestion service for Blueflood.

Even though not needed for ingestion, you probably should still have blueflood running for query the data sent in.

When you build the jars, you will see two jar files also generated in the target under blueflood-udp/target

• blueflood-udp-1.0.4-SNAPSHOT.jar - this can be used to start UDP ingestion node
• blueflood-udp-1.0.4-SNAPSHOT-tests.jar - this can be used as the client

Start UDP Ingestor

You need the following jar files for the ingestion node to work

• blueflood-udp-1.0.4-SNAPSHOT.jar - the jar for the udp module
• blueflood-all-1.0.4-SNAPSHOT-jar-with-dependencies.jar - the uber jar
• netty-all-4.0.8.Final.jar - the library that the blueflood-udp module depends on

Update blueflood.conf

You need to update the blueflood.conf mentioned in the 10-minute Guide to have the following lines

# for UDP ingestion node
UDP_BIND_PORT=8888
UDP_BIND_HOST=127.0.0.1

Run start command

Here is a script that you can use to start the node

#!/bin/bash
/usr/bin/java \
	-Dlog4j.configuration=file:blueflood-log4j.properties \
	-Dblueflood.config=file:blueflood.conf \
	-Xms1G \
	-Xmx1G \
	-Dcom.sun.management.jmxremote.authenticate=false \
	-Dcom.sun.management.jmxremote.ssl=false \
	-Djava.rmi.server.hostname=localhost \
	-Dcom.sun.management.jmxremote.port=9181 \
	-classpath blueflood-udp-1.0.4-SNAPSHOT.jar:blueflood-all-1.0.4-SNAPSHOT-jar-with-dependencies.jar:netty-all-4.0.8.Final.jar com.rackspacecloud.blueflood.service.udp.MainIngestor 2>&1

Run the Client for UDP Ingestion

You need the following jars to broadcast the message that the UDP node can understand (test jar from udp module plus the jars for udp ingestion node)

• blueflood-udp-1.0.4-SNAPSHOT-tests.jar - the test jar from the blueflood-udp module
• blueflood-udp-1.0.4-SNAPSHOT.jar - the main jar from the blueflood-udp module
• blueflood-all-1.0.4-SNAPSHOT-jar-with-dependencies.jar - the uber jar
• netty-all-4.0.8.Final.jar - the library that the blueflood-udp module depends on

Here is the script to run the client

#!/bin/bash
/usr/bin/java \
	-DUDP_BIND_HOST=127.0.0.1 \
	-DUDP_BIND_PORT=8888 \
	-Dlog4j.configuration=file:blueflood-log4j.properties \
	-classpath blueflood-udp-1.0.4-SNAPSHOT-tests.jar:blueflood-udp-1.0.4-SNAPSHOT.jar:blueflood-all-1.0.4-SNAPSHOT-jar-with-dependencies.jar:netty-all-4.0.8.Final.jar com.rackspacecloud.blueflood.service.udp.Broadcaster 2>&1

You will see output like the following with information about the metric data generated

$ ./broadcast.sh
2013-09-25 05:10:22 INFO  MainIngestor        :51  - Sending metrics for tenant_1,udp_int_metric starting 1380085822875
2013-09-25 05:10:23 INFO  MainIngestor        :103 - Generating metrics for tenant_1,udp_int_metric until 1380085823013
2013-09-25 05:10:24 INFO  MainIngestor        :103 - Generating metrics for tenant_1,udp_int_metric until 1380085824011
2013-09-25 05:10:25 INFO  MainIngestor        :103 - Generating metrics for tenant_1,udp_int_metric until 1380085825011
2013-09-25 05:10:26 INFO  MainIngestor        :103 - Generating metrics for tenant_1,udp_int_metric until 1380085826011
2013-09-25 05:10:27 INFO  MainIngestor        :103 - Generating metrics for tenant_1,udp_int_metric until 1380085827011
2013-09-25 05:10:28 INFO  MainIngestor        :103 - Generating metrics for tenant_1,udp_int_metric until 1380085828012

In the above example, the tenant id is tenant_1, metric name is udp_int_metric, from and to are in the first line and last line, respectively. In this case, they are 1380085822875 and 1380085828012. So the following command will be able to retrieve the metric

curl -i -X GET 'http://localhost:20000/v1.0/tenant_1/experimental/views/metric_data/udp_int_metric?from=13805822875&to=1380085830013&points=10

which generates

HTTP/1.1 200 OK
Content-Length: 1149

{
  "values": [
    {
      "numPoints": 1,
      "timestamp": 1380085823012,
      "unit": "unknown",
      "average": 890
    },
    {
      "numPoints": 1,
      "timestamp": 1380085823013,
      "unit": "unknown",
      "average": 82
    },
    {
      "numPoints": 1,
      "timestamp": 1380085824011,
      "unit": "unknown",
      "average": 73
    },
    {
      "numPoints": 1,
      "timestamp": 1380085825011,
      "unit": "unknown",
      "average": 42
    },
    {
      "numPoints": 1,
      "timestamp": 1380085826011,
      "unit": "unknown",
      "average": 246
    },
    {
      "numPoints": 1,
      "timestamp": 1380085827011,
      "unit": "unknown",
      "average": 125
    },
    {
      "numPoints": 1,
      "timestamp": 1380085828012,
      "unit": "unknown",
      "average": 375
    },
    {
      "numPoints": 1,
      "timestamp": 1380085829013,
      "unit": "unknown",
      "average": 382
    },
    {
      "numPoints": 1,
      "timestamp": 1380085830013,
      "unit": "unknown",
      "average": 17
    }
  ],
  "metadata": {
    "limit": null,
    "next_href": null,
    "count": 9,
    "marker": null
  }
}