Bash embedded with Java over HTTP

Prerequisite : Basic knowledge of Java,Bash and NAGIOS (including technical requirement)

My motive to write this blog is to describe how to merge different technologies to boost up the performance in distributed architecture environment. Here I am dealing with one hosting server for application (you can take as many you can), one DB server and one NAGIOS server. I have to compare latest token value present in cache of hosting server to latest token value present in database and report NAGIOS(monitoring server) whether the values are matching and if not, for which factors(stocks) in database there is a mismatch. Hopefully you got an idea that I am dealing with stocks and market. Somewhere this blog can help the naive bash developers in understanding bash-DB interaction. I wrote a bash script that was doing all the task alone. Here I am pasting the bash script:

#!/bin/bashemails=”mohit.malik.paxcel.net,1988malik@gmail.com”subject=”ERROR—Problem AREA ” body=”$BODY CONTENT .” #===============================================#parameter 1 which needs to be passes @ run timeserver=$1;fnAbout() { echo “The Purpose of this script is to demonstrate how we can make bash and java work together.”; } fnDataCheck(){ Array_cache_token=0.0;count=0;fault_count=0;START=$(date +%s);

#===============================================

#fetching data from database

data_res=$(mysql -uroot -hHOSTNAME -p’PASSWORD’ -D DBNAME-e “QUERY TO FETCH DATA FROM DB)”);

declare symbols=”symbol”;

declare declare -r tk_symbol=”p_KEY”; declare -i check=0;

declare server_val; declare faulty_KEY;

# ===============================================

#============COMAPRISON LOGIC IS HERE,FETCHING DATA FROM A SERVER AND COMPARING IT WITH DB DATA=================

# Beginning of the loop to traverse the data fetched from database.

for i in $data_res do

if ! [[ $i =~ ^[0-9]+([.][0-9]+)?$ ]] ; then if ! [[ “$i” == “p_key”|| “$i” == “token_value”]];then

symbols=$i; check=`expr $check + 1`;

#=================================================

#Getting the update From Cache #one by one

#================URL where to fetch Data===================

d_URL=”http://”ANY SERVER URL TO FETCH DATA FROM”/”$i;

server_val=`curl “$d_URL”`;

fi

fi

if [[ $i =~ ^[0-9]+([.][0-9]+)?$ ]] ; then

#===============================================

last_token=$(awk -vp=$i ‘BEGIN{printf “%.2f” ,p}’);

check=`expr $check + 1`;

#echo $check”:”$p_value;

fi

if [ $check -eq 2 ;

then DB_value=”{“‘”‘$key'”‘”:”$p_value”}”;

check=0; if [ $server_val != $DB_value ];

then touch failed.txt; fault_count=`expr $fault_count + 1`;

faulty_KEY=$faulty_KEY”,”$key; fi count=`expr $count + 1`;

fi

done

#================================================

echo “DATA FETCHED AND COMPARED FROM Database.TOTAL:=”$count” symbols”;

echo “SERVER NAME::”$server;

echo “FAULTY KEYS:=$fault_count;

echo “KEYS WITH PROBLEM:=$faulty_KEY;

END=$(date +%s); DIFF=$(( $END – $START ));

echo “TIME TAKEN TO COMPLETE THE PROCESS:=”$DIFF “seconds”; }

fnAbout;

flag=0;

fnDataCheck;

if [ -f fail_rep.txt ];

then mv fail_rep.txt fail_rep.txt 2.txt;

flag=1;

echo “MOM-N-108:MOM IREPORT `date`” | /MOHIT/folder1 -H ‘NAGIOS’s URL’ -c /move/folder1.cfg -d ‘:’ fi fnDATACheck;

if [ -f fail_rep.txt ]; then echo New Failure detected! mutt -s $subject $emails <$body; echo “MOM-N-108:MOM IREPORT `date`” | /MOHIT/folder1 -H ‘NAGIOS’s URL’ -c /move/folder1.cfg -d ‘:’

fi

exit 1;

Everything was in good shape but script was taking around 100 seconds.It is hell lot of time as far as response time is concerned.
I also tried while loop initially.Here is the code snippet.
#================================================================ #===============PROBLEM WHILE USING FOR LOOP====================
mysql -uroot -h’hostName’ -p’pass’ -D DBNAME -e “WHATEVER QUERY YOU WANT TO PUT TO FETCH DATA”| while p_key token_value…..;
do Array_key=$p_key; Array_token=$token_value;
#new_VAl_token=$(echo “scale=2;($token_value)” | bc);
new_VAl_token=$(awk -vp=$token_value ‘BEGIN{printf “%.2f” ,p}’);
DB_value=”{“‘”‘$p_key'”‘”:”$new_last_token”}”;
echo $DB_value; d_URL=”URL TO FETCH DATA FROM ANY SERVER”;
server_val=`curl “$d_URL”`;
#echo “SERVER CAHCE VALUE IS:= “$server_val; if [ $cahche_value != $DB_value ];
then echo “SERVER CAHCE VALUE IS:= “$server_val;
echo “DATABASE VALUE IS:= “$new_last_token;
echo “NEED UPDATE FOR SYMBOL ” $p_key;
count=`expr $count + 1`;
echo “COUNTER=”$count;
fi
done
echo “COUNTER=”$count;

PROBLEM IN USING WHILE LOOP:-Here I am also mentioning an interesting problem which developers who are new to bash face. When the count (in above mentioned code)variable is used to fetch the number of rows from the database it keeps on increasing as more rows are fetched. However when the same count variable is accessed outside the loop, it always sets back to its initially declared global value i.e. zero.

REASON:The reason for this weird behaviour, is that each sub shell (or pipe)  introduces a new variable context and environment. The while loop above is executed in a new sub shell so context of count has been set to new environment with a copy of count and  value is taken  from the parent shell which is zero. This copy keeps on changing. When the while loop is finished, the sub shell copy is discarded, and the original variable  count of the parent  is used in the echo command though parent was never altered, so it prints zero. With the while loop script was also taking 70+ seconds

SOLUTION TO SPEED UP: I merged Bash with Java using HTTP protocol. I created a new API and hosted it on the application server. This API would fetch data from database as well as from cache for each symbol and return a list of symbols for which the values are different in cache and database. So somehow I managed a separation of concerns. API did the filtering (calculation logic) and bash script did the parsing and notification part. After opting for this response time was reduced to, guess what 0-1 seconds.:) Here I am pasting generalized part of API and script.

 import javax.ws.rs.Path;import javax.servlet.http.HttpServletRequest;import javax.ws.rs.GET; import javax.ws.rs.Path;import javax.ws.rs.Produces;import javax.ws.rs.core.Context;import org.json.JSONArray;import org.json.JSONException;import org.json.JSONObject;//CAN BE ACCESSED VIA FOLLOWING URL//URL:http://hostname/appname/diectory/path1/methodPath1@Path(“/path1″) public class MOMRest {@GET @Path(“/getMethod1″) @Produces(“text/plain”)public String getSomeThing1(@Context HttpServletRequest request) throws Exception{JSONObject jsonObj=new JSONObject();

// ANY LOGIC TO PERFORM AND BIND THE OUTPUT TO JSON OBJECT

return jsonObj.toString();

}

//YOU CANT PUT ANY NUMBER OF METHODS AND JUST GIVE THEM RESPECTIVE MAPPED NAME

//SO THAT THEY CAN BE Accessible via HTTP

#!/bin/bashsubject=”ERROR : SOMETHING IS FISHY”body=”TESTING.”#===============================================#parameter 1 and 2 which needs to be passes @ run timeserver1=$1;server2=$2;fnAbout(){ echo “The Purpose of this script is to demonstrate how you can merge bash and java to boost up the performance.”; }fnDATACheck() { echo “FETCHING DATA FROM RESPECTIVE SERVER(s) .”;

count=0;

fault_count=0;

START=$(date +%s);

#===============================================

#ALL THE LOGIC PART WILL BE DONE BY JAVA APPLICATION HOSTED ON SERVER #FETCH CALCULATION RESULTS FROM URL d_URL_list=”http://”$server1″/MOMAppS/app2/path1/getMethod1″; d_URL_count=”http://”$server1″/MOMAppS/app2/path1/getMethod2″;

flag_token=”{“‘”‘”DEMONSTRATION$1″‘”‘”:[“”]}”;

list_NSSYM=`curl “$d_URL_list”`;

count_info=`curl “$d_URL_count”`;

echo ” FIRST SERVER NAME::”$server1;

echo “COUNT INFORMATION ABOUT SYMBOLS”$count_info;

echo “:”$list_NYM; EMAILMESSAGE=”DEMONSTRATION WAS SONE.HERE ARE THE RESULTS”; SUBJECT=”SUCCESS:”$server1″”;

#echo $list_NYM; #echo $flag_token;

if ! [ “$list_NYM” == “$flag_token” ];then

echo “PROBLEM:”; SUBJECT=”FAILURE:”$server1″NSS”;

touch fail_rep.txt;

fi

END=$(date +%s); DIFF=$(( $END – $START ));

echo “TIME TAKEN:=”$DIFF “seconds”;

EMAIL=”mohit.malik@paxcel.net”;

#=======

echo $EMAILMESSAGE | /bin/mail -s “$SUBJECT” $EMAIL; }

fnAbout; flag=0; if [ -f fail_rep.txt ]; then mv fail_rep.txt fail_rep.txt 2.txt; flag=1;

echo “MOM-N-108:MOM IREPORT `date`” | /MOHIT/folder1 -H ‘NAGIOS’s URL’ -c /move/folder1.cfg -d ‘:’ fi fnDATACheck;

if [ -f fail_rep.txt ]; then echo New Failure detected! mutt -s $subject $emails <$body; echo “MOM-N-108:MOM IREPORT `date`” | /MOHIT/folder1 -H ‘NAGIOS’s URL’ -c /move/folder1.cfg -d ‘:’ fi exit

1;

2 thoughts on “Bash embedded with Java over HTTP

Leave a Reply

Your email address will not be published. Required fields are marked *


five − 4 =

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>