Wednesday, November 02, 2011

Atmosphere with jquery tutorial to stream data to web browser

When I started to learn Atmosphere to do async server push to client I was overwhelmed with the available tutorials on the net! The blog at 6312 is the best but it had too much details. Not much patience to read, I just needed a really quick start and not a really long page with lot of details. So, Here is a short tutorial to the people of my kinds who just want to learn and implement server push fast and dirty. For more details and explanation see the blog I mentioned before.

I am using atmosphere 0.7.2, tomcat 6, and JDK 6.

Step 1: Download the sample jquery-pubsub-0.7.2.war from here . Extract the war to a known directory. Copy the jars from the WEB-INF/lib to your webapp lib which needs the atmosphere support. Make sure that your application lib contains only one version of the jar. You do not want to find yourself troubleshooting problems due to multiple version of same jar in the classpath of your webcontainer.

Step 2: Create a folder META-INF in the same level as WEB-INF in your application war. Create context.xml from the content shown below and put the same file in both META-INF and WEB-INF.

<?xml version="1.0" encoding="UTF-8"?>
<Loader delegate="true"/>

Step 3: Put the below snippet in web.xml between web-app xml tag. Make sure to note the “/account/number” and “/atm/*” path provided. We will use this path “/atm/account/number” to subscribe from the server as shown later.



Step 4: I would assume you know jquery. Create atm.js with the below code and add it to your collection of javascripts in war.

var topicSubs = 'all'
var urlLocation = 'http://localhost:8080/mywebapp/atm/account/';

var onData = function(data){
$('body').append("Message Received: " + data);
var appSubscriber = function() {
var callbackAdded = false;

function subscribe() {
function callback(response) {
if (response.transport != 'polling' && response.state != 'connected' && response.state != 'closed') {
if (response.status == 200) {
var data = response.responseBody;
if (data.length > 0 &&'-->')==-1) {

$.atmosphere.subscribe(urlLocation + topicSubs, !callbackAdded ? callback : null,$.atmosphere.request = { transport: 'websocket' });
callbackAdded = true;


Step 5: In your html page subscribe using the below shown code. Make sure to include the above script atm.js in the page. Here, I am overriding the variables declared in the atm.js above with required values. See the path in urlLocation, http”//localhost:8080/mywebapp” is your application URL, “/atm” is for atmosphere servlet URL pattern, “/account” is for jersey as you will see in server side code in sext step, and “/8832221” which is added to the URL by atm.js code is used by the server code to push data for that account number.

<script type="text/javascript">
topicSubs = '8832221';
urlLocation = 'http://localhost:8080/mywebapp/atm/account/';
onData = function(data){
$('#content').append("Message Received: " + data + "</br>");


Step 6: Now that we have the html UI ready, we can write the server side code. Note that the html page will get updates only those that are pushed using the below push method after the html page has loaded and subscribed using the JS above. If you need old updates, then you have to cache when the subscriber is not available for a account and if you need to make sure not some random person can access the data from others account number, then an auth token can be embedded in the subscription string with account number that can be validated. You can code these requirements in the EventsLogger class that has the callback hooks from the atmosphere when different events occur. Some of you may not get updates if you are using different applications talking to each other due to cross domain issue which is solved by adding a http header in the below shown code. If you look at the path annotation it maps the “/account/{topic}”, where the topic is the account number from the html page that is sent to subscribe. Push method also prints all the broadcasters available which are equal to the subscribers from the html and have the id as the name.

import org.atmosphere.cpr.*;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.util.*;
import com.sun.jersey.spi.resource.Singleton;

import org.atmosphere.annotation.Broadcast;
import org.atmosphere.annotation.Resume;
import org.atmosphere.annotation.Suspend;
import org.atmosphere.cpr.AtmosphereHandler;
import org.atmosphere.jersey.Broadcastable;
import org.atmosphere.jersey.JerseyBroadcaster;
import org.atmosphere.jersey.SuspendResponse;

public class OurAtmosphereHandler{
private @PathParam("topic")
Broadcaster topic;

public SuspendResponse<String> subscribe(@Context HttpServletResponse httpResponse) {
return new SuspendResponse.SuspendResponseBuilder<String>()
.addListener(new EventsLogger())


public static void push(String message, String topic){

Collection<Broadcaster> broadcasters = BroadcasterFactory.getDefault().lookupAll();

for(Broadcaster b : broadcasters){

System.out.println("Request to push- Message: " + message + ", Topic: " + topic);
Broadcaster b = null;
if(null != (b = BroadcasterFactory.getDefault().lookup(JerseyBroadcaster.class,topic))){
System.out.println("Request to push- Message: " + message + ", Topic: " + topic);
b.broadcast(message + "\n");

private class EventsLogger implements AtmosphereResourceEventListener {

public void onSuspend(AtmosphereResourceEvent<HttpServletRequest, HttpServletResponse> event) {
System.out.println("onSuspend(): " + event.getResource().getRequest().getRemoteAddr() + " : " + event.getResource().getRequest().getRemoteHost());

public void onResume(AtmosphereResourceEvent<HttpServletRequest, HttpServletResponse> event) {
System.out.println("onResume(): " + event.getResource().getRequest().getRemoteAddr() + " : " + event.getResource().getRequest().getRemoteHost());

public void onDisconnect(AtmosphereResourceEvent<HttpServletRequest, HttpServletResponse> event) {
System.out.println("onDisconnect(): " + event.getResource().getRequest().getRemoteAddr() + " : " + event.getResource().getRequest().getRemoteHost());

public void onBroadcast(AtmosphereResourceEvent<HttpServletRequest, HttpServletResponse> event) {
System.out.println("onBroadcast(): " + event.getMessage());

public void onThrowable(AtmosphereResourceEvent<HttpServletRequest, HttpServletResponse> event) {
System.out.println("onThrowable(): " + event);


So, have fun using streaming technology using atmosphere. I would suggest using chrome as it supports websockets and is most reliable. You can use polling if working with other browsers. Just change the transport in the atm.js with “polling”. Atmosphere is an awesome technology and I would suggest to visit the blog 6312 that I mentioned in the beginning for more details and many different options and customization.

Wednesday, September 21, 2011

Solution for: Spring JMS + ActiveMQ + Tomcat = Tomcat does not shutdown

This is one of the most frustrating problems. When you decide to use Tomcat with Spring JMS to connect and message with ActiveMQ, everything works fine till you try to shutdown Tomcat.  It keeps throwing exception and looks like the DefaultMessageListnerContainer thread lives on. As seen from the logs on tomcat console, the problem occurs because the tomcat shutsdown before the JMS Spring container “DefaultMessageListenerContainer” and you will see Nullpointer, Classnotfound exceptions, as tomcat container is down and the classloader doesn’t exist to give the necessary classes from the previously loaded jars. And I could not find a single solution on google. After, much googling I found a solution for a similar problem when using Quartz, tomcat and spring. So, this solution was derived from that one [Sorry, I don’t have that link anymore].

Create a ServletContextListener, and in contextDestroyed method get the jms container bean to manually call shutdown. The listener contextInitialized and contextDestroyed are called at start and end of Servlet lifecycle. The below code assumes that you have a class called SpringApplicationContextInstance that gives you the spring context using which you can access your beans defined in spring context.

import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;

* Created by Shreyas Purohit
public class JMSContainerShutDownHook implements ServletContextListener {
public void contextInitialized(ServletContextEvent servletContextEvent) {
LogManager.log(Level.INFO, "JMSContainerShutDownHook: Initialized called");

public void contextDestroyed(ServletContextEvent servletContextEvent) {
LogManager.log(Level.INFO, "JMSContainerShutDownHook: Destroyed called");
try {
LogManager.log(Level.INFO, "JMSContainerShutDownHook: Fetching JMS Container Bean from Application Context");
DefaultMessageListenerContainer container = SpringApplicationContextInstance.getInstance().getBean("jmsContainer");
LogManager.log(Level.INFO, "JMSContainerShutDownHook: Calling shutdown on DefaultMessageListenerContainer");
Thread.sleep(3000); //Wait for the container to shutdown
} catch (Exception e) {
LogManager.log(Level.ERROR, e);
LogManager.log(Level.INFO, "JMSContainerShutDownHook: Exiting");

Wednesday, September 14, 2011

ActiveMQ 5.5.0 performance test plugin

It not very simple to apply the perf test instructions given at I encountered many problems, and here is a way to make it run.

Required: Maven 2, SVN client like tortoise SVN


1. Checkout the source code for Active MQ 5.5.0 from SVN to directory AMQ

2. Checkout perf test code from SVN to directory AMQPerf

3. cd AMQ/activemq-tooling/maven-activemq-perf-plugin and edit pom.xml. Remove line <scope>test</scope> from org.slf4j dependency.

4. cd AMQPerf and edit pom.xml. Change <version> in <parent> to 5.5.0

5. cd AMQ/activemq-tooling and run “mvn clean install”

6. cd AMQPerf and run “mvn clean install”

7. Run Commands as provided in from the AMQPerf directory in separate command prompts/Consoles, that is-

Console1> mvn activemq-perf:consumer
Console2> mvn activemq-perf:producer

This works for me perfectly.

Monday, September 12, 2011

Ruby On Rails mysql2 error

I was having trouble installing mysql2 gem on my Win7 64 bit. I have copied these instructions from stackoverflow. And these really work.

For Win 7:

  1. Download a zip file with mysql server 5.1 NOT the msi one. Make sure it's 32-bit NOT 64-bit. (From here
  2. Since there is no installer file with this, create a folder c:\mysql-gem-install - you can remove it once you finish.
  3. Extract all the files from the zip file into the folder you just created.
  4. now run this command
    gem install mysql2 -- '--with-mysql-lib="c:\mysql-gem-install\lib\opt" --with-mysql-include="c:\mysql-gem-install\include"'
I just installed mysql2 gem v. 0.3.7
EDIT: One more thing: make sure you run the command in Command Prompt directly. As in not PowerShell or Consol2 - for some reason if you try that it will give you and error " invalid option" - has to do with the way -- is parsed.
--Answered by Nick Gorbikoff

For Linux:

sudo apt-get install libmysql-ruby libmysqlclient-dev

--Answered by spacemonkey

I successfully installed mysql 5.5, and gem mysql2-0.3.7

Wednesday, August 31, 2011

Install Ruby gems when behind a proxy

There are many posts out there on the net instructing you to do certain thing when behind a proxy. Different things work for different people, but none of that worked for me. So, try this one if you are facing trouble with proxy too, and keep your fingers crossed! I am using Ruby 1.9.2, with gems version 1.7.2 on Windows 7. The below command allowed me to use proxy and download and install any gem.

gem install gemname -p http://username:password@proxyaddress:port --platform=ruby

Thursday, August 25, 2011

Send SMS Using Ruby and Send Email using Ruby with Gmail SMTP Server

There was a HP Touchpad sale event where HP tablets where being sold out for $99 yesterday. Instead of manually monitoring the HP website, I decided to write a small Ruby script that will monitor and send SMS and Email to notify. I wanted to use Gmail SMTP server for mail, but that does work by default with Ruby SMTP support as it uses TLS. Finally, I found a ruby script on the internet which provides that support. I have copy pasted it from that website. There are many scripts out there on web, but none really work with gmail. Hence, it is important to spread this script. To send SMS I use the free service by making GET request using open-uri. The best thing about Ruby syntax is, its readable and need no explanation.

# Requests the HP page for 16GB Tablet URL:
# If status "Coming soon" is not displayed then alert using SMS and email
# SMS:
# Email: Uses gmail as sender smtp

require 'open-uri'
require 'net/smtp'
require_relative 'smtp_tls'

URL_16GB = ''
URL_32GB = ''
$carriers = {:auto => 0, :att => 3, :tmobile => 11, :metropcs => 12, :verizon => 16, :virgin => 21, :sprint => 9}
$numbers = {"XXXXXXXXXX" => :att}
$emails = [""]
$notInterestedStatuses = ["Coming soon", "not found", "outofstock"]
$urls = {[URL_16GB,"HP_Website_16GB",$notInterestedStatuses] => false, [URL_32GB,"HP_Website_32GB",$notInterestedStatuses] => false}
$running = true;
$sleepTimeInSec = 240

def send_email(to,subject,message)
username = 'xxxxxxxxx'
password = 'yyyyyyyyyy'
emailMessage = <<MESSAGE_END
From: Shreyas Purohit <>
To: #{to} <#{to}>
Subject: #{subject}


Net::SMTP.start( '' ,
'plain' ){ |smtp|
smtp.send_message( emailMessage,

def readWebsite(url)
uriContent = ""
open(url){|f| uriContent = }
return uriContent

def notifyObservers(customMessage, url)
# Send SMS using URL
$numbers.each do |key, value|

send_email($emails,"Alert- Website Update","Alert! Website Status Changed. Please Check Online (#{customMessage}) (#{url})")


def checkStatusOnWebsiteString(uriContent, statusNotOfInterestOnSite)
statusNotOfInterestOnSite.each do |status|
if (uriContent.include? status)
return false
return true

def logToFile(fileName, mode, content), mode) {|f| f.write(content) }

def process(urls)
checkedStatusOnce = false
statusChangedKeys = []

urls.each do |key, status|
url = key[0];
customMessage = key[1];
if !status
checkedStatusOnce = true
p "#{} : Trying HP Site #{customMessage} Now..."
uriContent = readWebsite(url);
statusChanged = checkStatusOnWebsiteString(uriContent,key[2])
logToFile("alter_#{customMessage}.log", "w+", "URI DATA AT " + + " for url #{url} \n" + uriContent)
if statusChanged
p "Status changed..(#{customMessage}) Notifying Observers.."
notifyObservers(customMessage, url)
statusChangedKeys << key
p "Status Not Changed... (#{customMessage})"

statusChangedKeys.each do |key|
urls[key] = true

if !checkedStatusOnce
$running = false

while $running do
p "#{} : Sleeping #{$sleepTimeInSec} sec(#{$sleepTimeInSec/60.0} min)..."

Also, The script smtp_tls.rb is below that is required for the above script to run.
require "openssl"
require "net/smtp"

Net::SMTP.class_eval do
def do_start(helodomain, user, secret, authtype)
raise IOError, 'SMTP session already started' if @started
check_auth_args user, secret, authtype if user or secret

sock = timeout(@open_timeout) {, @port) }
@socket =
@socket.read_timeout = 60 #@read_timeout
@socket.debug_output = STDERR #@debug_output

check_response(critical { recv_response() })

raise 'openssl library not installed' unless defined?(OpenSSL)
ssl =
ssl.sync_close = true
@socket =
@socket.read_timeout = 60 #@read_timeout
@socket.debug_output = STDERR #@debug_output

authenticate user, secret, authtype if user
@started = true
unless @started
# authentication failed, cancel connection.
@socket.close if not @started and @socket and not @socket.closed?
@socket = nil

def do_helo(helodomain)
if @esmtp
ehlo helodomain
helo helodomain
rescue Net::ProtocolError
if @esmtp
@esmtp = false
@error_occured = false

def starttls

So, that’s it! Have fun reusing code!!

Tuesday, July 26, 2011

Using JS caching on serverside data with jquery datatables plugin

First of all, I would like to say that I have been working on lots of stuff but have had no time to update this blog. Recently I was speaking with my friend who checks my blogs regularly, and was inspired to make some time and post stuff that is useful for many. So, here is one of the things that I worked on and pretty useful.

Pagination is one of the most important concepts that must be implemented correctly for better performance of the database and that of your application. It is common sense to say that caching all the data on the server session is bad, or contacting database for every page is also bad; esp when working on tables with millions of records. The better alternative is to load and cache a good set of data on UI, and as and when needed update cache with more data. This removes caching done on server side, say in session, and negative performance impact that you get when you query database often while pagination.

Many use jquery framework on javascript side. One of the plugins that I have worked with is datatables. Located at . There exist other frameworks like YUI for pagination and definitely other plugins for jquery framework for pagination. The datatables plugin works with server side data but does not provide good caching mechanism. Every new page is a request for getting data from the server to display that page. I wrote the below given javascript function that can be used with “fnServerData” option while initializing the datatables to provide caching.

var cache = new Object();
var rowsToCache = 2000; //Minimum Cache size >(must) 2 * Maximum Display Length * Number of Navigation Button (Avoids false hits while traversing distant pages), Default Assumption: 2000 > 2 * 100 * 5
//Intelligient to fetch only unfetched data. Maintains cache window to left(rowsToCache/2) and right(rowsToCache/2) of current position.

var fnJSManagedCacheServer =
function ( sSource, aoData, fnCallback ) {
//Get the required variable values
var _tableDispLength;
var _indexDisplayStart;
var _searchData;
var sEcho;
var _useCache = true;
var _bIsSearch = false;

for(var i in aoData){
var d = aoData[i];
if( == 'iDisplayLength') {
_tableDispLength = d.value;
if( == 'iDisplayStart') {
_indexDisplayStart = d.value;
if( == 'sSearch'){
_searchData = d.value;
if( == 'sEcho'){
sEcho = d.value;
//Process all conditions that may result in a true call and not access cache
//Find if the call is result of only pagination and not because of search
if(_searchData != undefined && _searchData != ""){
_useCache = false;
_bIsSearch = true;
//If cache doesnot exist
if(cache['aaDataServer'] == undefined){
_useCache = false;
//If _useCache is still true, try to get data from the cache, else allow call to server
//by setting _useCache to false
if(_useCache == true){
//Predict False hit
_useCache = false;

//Get data from cache and paint
//Does requested page end is lesser than or equal to cache endpage
if(_indexDisplayStart + _tableDispLength <= (cache.cached_indexDisplayStart + cache.realLength)
|| cache.realLength < cache.requestedLength){//Last page and no more data present at server
//Does requested _indexDisplayStart is greater than or equal to cache _indexDisplayStart
if(_indexDisplayStart >= (cache.cached_indexDisplayStart)){
//Data must exist in cache
var json = cache['jsonResult'];
var aaData = cache['aaDataServer'].slice(Math.abs(cache.cached_indexDisplayStart - _indexDisplayStart), Math.abs(cache.cached_indexDisplayStart - _indexDisplayStart) + _tableDispLength < cache['aaDataServer'].length ? Math.abs(cache.cached_indexDisplayStart - _indexDisplayStart) + _tableDispLength : cache['aaDataServer'].length);
json.aaData = aaData;
json.sEcho = sEcho;
//Cache hit
_useCache = true;

if(_useCache == false){

//Get rowsToCache/2 from existing cache and save
var _prevCacheData = [];
var _movingRight = true;
if(cache.cached_indexDisplayStart != undefined){
if(_indexDisplayStart > cache.cached_indexDisplayStart){
_movingRight = true;
_movingRight = false;
if(_indexDisplayStart > cache.cached_indexDisplayStart
&& _indexDisplayStart == (cache.cached_indexDisplayStart + cache['aaDataServer'].length)){
_prevCacheData = cache['aaDataServer'].slice(cache['aaDataServer'].length - rowsToCache/2);
}else if(_indexDisplayStart <= cache.cached_indexDisplayStart
&& _indexDisplayStart == (cache.cached_indexDisplayStart - _tableDispLength)){
_prevCacheData = cache['aaDataServer'].slice(0, cache['aaDataServer'].length == rowsToCache ? rowsToCache/2 : 0);
//Invalidate and re-init cache
cache = new Object();
cache._tableDispLength = _tableDispLength;
cache._indexDisplayStart = _indexDisplayStart;
cache['aaDataServer'] = _prevCacheData;
cache._movingRight = _movingRight;

if(_bIsSearch != true){
//Modify settings of datatables to fetch rowsToCache records
cache.requestedLength = 0;
var _indexDisplayStartIsZero = false;
for(var i in aoData){
var d = aoData[i];
if( == 'iDisplayStart') {
cache.requestedLength += d.value - (rowsToCache/2) >= 0 ? rowsToCache/2 : 0; //Order of these stmts important
if(cache._movingRight == false){
d.value = (d.value+_tableDispLength) - (rowsToCache/2) >= 0 ? (d.value+_tableDispLength) - (rowsToCache/2) : 0; //Shift _indexDisplayStart to left by half cache page size (adding _tableDispLength to get the _indexDisplayStart to original location before taking it forward, else we will miss a page)
cache.cached_indexDisplayStart = d.value;
cache.cached_indexDisplayStart = d.value - _prevCacheData.length;
for(var i in aoData){
var d = aoData[i];
if( == 'iDisplayLength') {
d.value = cache.cached_indexDisplayStart != 0 ? rowsToCache/2 : rowsToCache/2;
cache.requestedLength = d.value; //Adjust cache fetch requested
cache._bIsSearch = _bIsSearch;
//Use ajax to Call
$.ajax( {
"dataType": 'json',
"type": "POST",
"url": sSource,
"data": aoData,
"success": function(json) {
cache['jsonResult'] = json;
if(cache._movingRight == true){
cache['aaDataServer'] = cache['aaDataServer'].concat(json.aaData.slice(0));//append entire aaData to right of existing cached data
cache['aaDataServer'] = (json.aaData.slice(0)).concat(cache['aaDataServer']);//append entire aaData to left of existing cached data
cache.realLength = cache['aaDataServer'].length;
if(cache._bIsSearch != true){
//redraw a part of json, as requested between 0 and current pagesize(cache._tableDispLength)
json.aaData = cache['aaDataServer'].slice(Math.abs(cache.cached_indexDisplayStart - cache._indexDisplayStart), cache.realLength < Math.abs(cache.cached_indexDisplayStart - cache._indexDisplayStart) + cache._tableDispLength ? cache.realLength : Math.abs(cache.cached_indexDisplayStart - cache._indexDisplayStart) + cache._tableDispLength);
} );

To use this, just include the javascript, override “rowsToCache” if default is not appropriate for you, and set “fnServerData” : fnJSManagedCacheServer.

Do let me know if you find any bugs!

PS: No caching when searching.