SenseiDB - Getting Started, Part Two

Getting Started Continued

So are you ready for more?

Build the Tweets extension

The Tweets example builds a special Gateway to stream from Twitter, we need to build it and plug it into Sensei.

Code

Read the source code for TwitterSampleGateway.

Build

Go to $SENSEI_HOME/example/tweets/conf/ext, the plugin artifact containing the Tweets custom gateway should be there.

Running the Tweets example

This example queries against the Twitter stream directly from Twitter via Twitter's Streaming (Sample) API

Step 1: Start Zookeeper

Zookeeper manages Sensei's network topology or cluster.

Start your Zookeeper instance: (You can start your own Zookeeper cluster by following instructions here). From $SENSEI_HOME

./bin/zookeeper-server-start.sh \ 
resources/zookeeper.properties

You can shut it down by running:

./bin/zookeeper-server-stop.sh

Step 2: Set Twitter credentials

You need your credentials to talk to Twitter to get data.

Edit the file:

$SENSEI_HOME/example/tweets/conf/sensei.properties

And fill out the properties:

sensei.gateway.custom.username = <YOUR TWITTER USERNAME>
sensei.gateway.custom.password = <YOUR TWITTER PASSWORD>

Step 3: Start Sensei Node

With Zookeeper running, we can start a Sensei node serving Tweets example listening on port 8080:

In $SENSEI_HOME, start your Sensei node by pointing to the Tweets example configuration directory:

./bin/start-sensei-node.sh example/tweets/conf

This starts a Sensei node with 2 partitions. (click here to learn more about Sensei Clusters)

Step 4: Run the Tweets example application

You can open the Tweets example application with your browser (no need to start any other servers.)

Open index.html file with your browser located at :

$SENSEI_HOME/example/tweets/webapp/index.html

and click the Refresh button and see data streaming!

Using the Sensei Shell

Of course, we can also query this Sensei node with BQL via the Sensei Shell:

./bin/sensei-shell.sh

Output:

$ ./bin/sensei-shell.sh localhost 8080 
Welcome to Sensei Shell
Url specified, host: localhost, port: 8080
> desc tweets
+------------+------------+---------+----------+-------------+---------+
| facet_name | facet_type | runtime | column   | column_type | depends |
+------------+------------+---------+----------+-------------+---------+
| time       | range      | false   | time     | long        | []      |
| hashtags   | multi      | false   | hashtags | string      | []      |
| tweeter    | simple     | false   | tweeter  | string      | []      |
+------------+------------+---------+----------+-------------+---------+
> select time,tweeter,hashtags browse by tweeter,hashtags limit 5
+------------------------------------------+-----------------+----------------+
| time                                     | tweeter         | hashtags       |
+------------------------------------------+-----------------+----------------+
| 0000000000000000000000000001323586320000 | _cupcakemafia   | likeagujjuboss |
| 0000000000000000000000000001323586320000 | Gorgeous_Biotch |                |
| 0000000000000000000000000001323586320000 | AeroNews        |                |
| 0000000000000000000000000001323586320000 | alaanjerezano   |                |
| 0000000000000000000000000001323586320000 | Wanji_Aswandi   |                |
+------------------------------------------+-----------------+----------------+
5 rows in set, 1082 hits, 1082 total docs (server: 70ms, total: 149ms)
+----------------+
| hashtags       |
+----------------+
| pickone    (3) |
| Aquarius   (2) |
| Gemini     (2) |
| nowplaying (2) |
| tellme     (2) |
| 2PM,JYP    (1) |
| Atletics   (1) |
| BMT        (1) |
| CBT        (1) |
| Cancer     (1) |
+----------------+
+---------------------+
| tweeter             |
+---------------------+
| CaptainShankAHo (2) |
| ahy_ch          (2) |
| alexiasantana_  (2) |
| kartalint       (2) |
| 000CashMoney000 (1) |
| 0Q27            (1) |
| 106andParkfans  (1) |
| 1205FZYJYJC     (1) |
| 14kakeru        (1) |
| 16956384        (1) |
+---------------------+
> exit

Using the Web-console

Same with the Car Example, we can use the web-console to interact with Sensei.

Simply point your favorite browser to: http://localhost:8080

Learn details of the Javascript API here.

Shutting down the Sensei node

In $SENSEI_HOME, do:

./bin/kill.sh

Screenshot

See the following screenshot of the tweets example:

Hook me up with Sensei

Now that you have the demos running, let's setup your app to work with Sensei by learning about the: Configuration »

Documentation

Developer

Project