Using camel FOP to convert text documents to PDF

Camel is awesome. No doubts about it. While working on camel I encountered a requirement where I had to convert text files to PDF. After looking into camel components I found out about "FOP" (http://camel.apache.org/fop.html). You just need to add dependency and create the required route. Seems, simple at the beginning, looking at the few snippets on the website. But, to convert the text file to PDF you need to generate the XSL-FO that contains both the formatting instructions (xslt) and the real data (xml). While this is good when you know the documents that you need to generate, it is not that obvious how to generate/convert any random document text file to PDF. But fear not, there is a way.
Create a route that reads text documents from a directory, send it through a processor that creates the XSL-FO around the document, set the file name, author etc attributes, forward to the FOP component, then to the file system. The code is shown below and is well documented where necessary.

package com.bitourea.camel.routes;
import com.bitourea.camel.util.AppUtil;
import org.apache.camel.Exchange;
import org.apache.camel.Processor;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.component.fop.FopConstants;

public class TextToPdf extends RouteBuilder{
    @Override
    public void configure() throws Exception {
        String readDir = "c:/Temp/camel/textToPdf";

        //read from directory, filter for text files
        from("file://"+readDir+"?noop=true&include=([a-zA-Z]|[0-9])*.(txt)")
           .routeId("textToPdf")
           .process(new Processor() {
               @Override
               public void process(Exchange exchange) throws Exception {
                   final String body = exchange.getIn().getBody(String.class);
                   final String fileNameWithoutExtension = AppUtil.getFileNameWithoutExtension(exchange);
                   final String convertToXSLFOBody = AppUtil.getFilledXSLFO(body);
                   exchange.getIn().setBody(convertToXSLFOBody);
                   exchange.getIn().setHeader(Exchange.FILE_NAME, fileNameWithoutExtension + ".pdf");
                   exchange.getIn().setHeader(FopConstants.CAMEL_FOP_RENDER + "author", "Shreyas Purohit");
               }
           })
           .to("fop:application/pdf")
           .to("file://" + readDir);
    }
}

The AppUtil used in the above code is shown below.

package com.bitourea.camel.util;
import org.apache.camel.Exchange;

public class AppUtil {
    public static final String EXT_DELIM = ".";
    private static final String fopMainTemplate = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n" +
                    "\n" +
                    "<fo:root xmlns:fo=\"http://www.w3.org/1999/XSL/Format\">\n" +
                    "\n" +
                    "<fo:layout-master-set>\n" +
                    "  <fo:simple-page-master master-name=\"A4\">\n" +
                    "    <fo:region-body margin=\"25pt\"/>\n" +
                    "  </fo:simple-page-master>\n" +
                    "</fo:layout-master-set>\n" +
                    "\n" +
                    "<fo:page-sequence master-reference=\"A4\">\n" +
                    "  <fo:flow flow-name=\"xsl-region-body\">\n" +
                        "#BLOCK_CONTENT" +
                    "  </fo:flow>\n" +
                    "</fo:page-sequence>\n" +
                    "\n" +
                    "</fo:root>";
    private static final String fopBlockTemplate = "    <fo:block font-family=\"Courier\" font-weight=\"normal\" " +
            "font-style=\"normal\" score-spaces=\"true\" white-space=\"pre\" linefeed-treatment=\"preserve\" " +
            "white-space-collapse=\"false\" white-space-treatment=\"preserve\" font-size=\"10pt\">#CONTENT</fo:block>\n";

    public static String getFileNameWithoutExtension(Exchange exchange){
        String fileName = (String) exchange.getIn().getHeader(Exchange.FILE_NAME);
        return fileName.substring(0, fileName.indexOf(EXT_DELIM));
    }

    public static String getFilledXSLFO(String content){
        return fopMainTemplate.replaceAll("#BLOCK_CONTENT", getXSLFOBlock(content));
    }

    private static String getXSLFOBlock(String line){
        return fopBlockTemplate.replaceAll("#CONTENT", line);
    }
}

If you want, you can externalize the template into xslt, and create required XML for the XSLT to generate the right XSL-FO. I find that overkill when all I want is to append the strings and forward to next route for all incoming documents. Also, note that I am using the monospace font Courier in the template since it maintains the formatting the best and I dont have to supply additional fonts(One of the 13 font family which are directly available with PDF readers- Helvetica, sans-serif, SansSerif, Times, Times Roman, Times-Roman, serif, any, Courier, monospace, Monospaced, Symbol and ZapfDingbats). The FOP component also support the XML configuration file that can be used to define/load the fonts from the file system if necessary that can be used instead. Change 'margin' in 'region-body' if you wish to increase or decrease the PDF document margin. For the text to be embedded as is in the PDF the following attributes are set, white-space=pre, linefeed-treatment=preserve, white-space-collapse=false, white-space-treatment=preserve.

At the moment of this writing the FOP component supported loading the configuration file from classpath, but, did not support custom UriResolvers to be plugged in or provided a classpath resolver. In effect, we could not load the fonts defined in the XML FOP configuration from classpath and they must be present on the filesystem.

This is one of the easiest way to convert text files to PDF documents.

Releasing chrome extension for UptimeRobot monitor

I have developed an opensource chrome browser extension to enable easy integration with Uptime Robot API's and monitor server/monitor statuses using chrome browser. This extension helps one to see server stats and get notification while the chrome browser is on.

Features

• Server up and down desktop notifications.
• Immediate visible server up and down notifications in the extension browser action icon.
• Group monitors to see them separated in the extension display.
• Options and data stored using chrome storage sync API allowing it to be sync'ed between chrome's if signed in.
• Uses jquery.plugin.uptimeRobotMonitor to provide a beautiful visualization of server status and statistics related to 1 day,7 day and all time server uptime percentages.
• Uses only Monitor API keys (not Account API keys) and hence trustable and secure.

Check for more new features and contribute to source at the project site. Download by clicking on the image below.

Releasing my first android app "Unlock Alert"

A project I started to learn android ended up in a full fledged app that can be used by many. I am releasing this app today on Google Play Store. 

Visit website to find out about the app and download it. Please remember to rate/comment it. The app has been in beta testing for some time now and is pretty stable. But since "things" happens, do let me know if it doesn't work or crashes on your device at google groups. I would like to get together and debug/solve the issue.

If you rather just download it, then here it is:

Review of Getting Started with SBT for Scala (Packt-Pub)

I got the opportunity to review a book in the scala domain published by Packt on SBT. I am very glad that so many books are coming up in this domain. SBT is one of the most important components in scala infra. Most, if not all, of the projects use SBT as the build tool in the scala projects. 

The book itself is very well written and has a good flow through out. The installation, running, and structure of SBT has also been explained. There is a very nice history of evolution of all the relevant build tools that are in the industry. I like chapter 1 esp for the way it quickly lets you get into code and start writing simple build scripts and to become aware of the minimum commands in the SBT that you ought to know. The chapter 1 provides intro to run, compile, test, scala REPL with real code examples that are easy to comprehend. One of the most important concepts of SBT has been clearly mentioned- Blank lines act as delimiter in a .sbt file. There are many operators in SBT like :=, +=, ++= etc which can be used in build definitions which has been covered clearly. There are nice tips which are present through out the book. Maven, Ivy has also been introduced. Ivy and its work lifecycle has been explained a bit in detail. I can live without that. Ivy itself is a big topic that can be read separately. The different types of scopes has also been covered. There is also a good list of commands/types in SBT which is presented.

The version's specified in the dependency need not be a single version as SBT is based on Ivy. There can be more constraints on it like 1.0.+ means select latest in 1.0.x in the dependency. This has been mentioned but no adequate examples has been given. I think this could have easily been added in one or two lines in a beginners book. This is a pretty useful feature. Next, scalaz ex in the full build definition is hardly understood. It should have been expanded a bit more. The full build definition chapter itself could have been explained a bit more. It is one of the key functionalities that allows a lot of flexibility in SBT. The multi project builds in that chapter has been explained well though.

Overall, I like the book. It gets you started in SBT as the title states pretty quickly. No book can replace your own research and work. This book can definitely serve as a quick reference to common commands, operators etc when in need after you have learned and worked with SBT.

The book is available on packt-pub here.

Apache Camel JMX with remote jconsole using host : port

By default when JMX is enabled in apache camel, it binds such that you will have to use a rather encrypted path in jconsole URI for connecting remotely. From there documentation-

service:jmx:rmi://localhost:<connectorPort>/jndi/rmi://localhost:<registryPort>/<serviceUrlPath>
or
service:jmx:rmi://localhost:2000/jndi/rmi://localhost:1099/jmxrmi/camel

You will not be able to connect using localhost:1099 in the jconsole. This was very important to me as I was not just using jconsole but zabbix to gather data from JMX. In case of zabbix the only option that is available was host:port. The reason this does not work is the serviceUrlPath which by default in camel is customized to /jmxrmi/camel instead of the more known /jmxrmi. The fix is easy as show below-

<camelContext id="camelAppContext" xmlns="http://camel.apache.org/schema/spring">
        <propertyPlaceholder id="properties"
                                     location="classpath:common.properties"/>
        <jmxAgent id="camelAppJmxAgent" registryPort="{{jmx.registry.port}}" connectorPort="{{jmx.connector.port}}"
                  createConnector="true" serviceUrlPath="/jmxrmi"/>
</camelContext>

Or by using the JMX from –D options.

-Dorg.apache.camel.jmx.serviceUrlPath=/jmxrmi

With this configuration you can connect using host:port (localhost:1099) to JMX using jconsole.

Liftweb 2.4 and Jamon 2.74 request monitoring

I like Jamon as a library for collecting performance statistics. It is well established and provides sufficient types of statistics. One of the favorites statistics of mine being the frequency distribution of response times. Using Jamon in simple but with Lift 2.4 it was little challenging. The Jamon MonitorFactory returns an instance of Timing Monitor that maintains the statistics. For the collection to occur the same instance must be accessed to call stop after method execution. i.e-

Monitor monitor = MonitorFactory.start("key1");
//Code whose performance needs to be measured
//
//
monitor.stop();

The problem with older version of Liftweb is there is no way of wrapping every request such that this monitor can be used. The S.addAround can be used with LoanWrapper but this gets executed multiple times as described here. As from the discussion from that thread you can see that a common LoanWrapper surrounding all requests would be (or has already been) added in Liftweb. But, if you can not upgrade the Liftweb to newer version then here is the solution. The idea is to use the LiftRules onBeginServicing and onEndServicing. But note that we need to use the same instance of Monitor created in onBeginServicing method. The request/response lifecycle in liftweb is described very well here. This will catch both Statefull GET and Ajax requests. The code below uses the setAttribute method of the original HHTPServletRequest to save and retrieve the Monitor between methods. Have fun with scala, liftweb and jamon!

//Boot.scala
def boot {
 val labelPrefix: String = "LIFT_WEB_HTTP."
 val monitorInReq: String = "monitorInReq"
 val extensionsNotToMonitor = List(".jpg", ".png", ".jpeg", ".ico", ".gif", ".bmp")

 def pathFilter = {
   req: Req =>
     req.path.wholePath.takeRight(1) match {
       case x :: _ if (extensionsNotToMonitor.contains(x.substring(x.lastIndexOf(".")))) => None
       case _ => Some(req.path.wholePath.mkString("/"))
     }
 }

 LiftRules.onBeginServicing.append((req: Req) => {
   val label: Option[String] = pathFilter(req)
   if (label.isDefined) {
     val start: Monitor = MonitorFactory.start(labelPrefix + label.get)
     S.containerRequest.map(r => (r.asInstanceOf[HTTPRequestServlet]).req.setAttribute(monitorInReq, start))
   }
 })

 LiftRules.onEndServicing.append((req, respBox) => {
   S.containerRequest.map(r => {
     val attribute: AnyRef = (r.asInstanceOf[HTTPRequestServlet]).req.getAttribute(monitorInReq)
     if (null != attribute) {
       attribute.asInstanceOf[Monitor].stop()
     }
   })
 })
}

SoapUI: Inserting an external XML node dynamically into an existing XML Response using XmlHolder

The Problem: I have a Soap Response, and I have to modify that xml such that I insert a new XML node into response. The new XML is a string that I can declare in a groovy script or I can retrieve from a property.

Few exceptions I encountered while trying to solve the problem: CData missing, can not add external document, Can not add groovy Node class to java Node class, viceversa and etc etc etc

Hints to solve the problem on internet: There are many websites that provides sample which helps to solve a particular case, but could not exactly find the one I present here. The most useful one was here "http://siking.wordpress.com/2012/01/06/dynamically-create-elements-in-a-soapui-request/" where in the coments he gives sample related to owner doc and importing into it.

Details:

Lets say you have a soap request and response in a test step of a test case. The body of the response is like below.

<Traveler xmlns="http://www.example.com/schemas">
   <Customer BirthDate="2001-01-01">
      <Person Language="EN-US">
         <Title>MR</Title>
         <FirstName>SHRE</FirstName>
         <MiddleName>NOWVERYMUCH</MiddleName>
         <LastName>WANTED</LastName>
      </Person>
      <Email Type="HOM" Address="notvalid@notvalid.com"/>
   </Customer>
</Traveler>

And I want add a new XML node Payment to Customer. Lets say his payment information like credit card number which is an XML fragment like below. Later I will add the card number to it.

def pay="""<Payment>
   <Card Type="INTERNATIONAL" Vendor="VISA" Number="">
      <Name>
         Huntsville Townhall
      </Name>
      <Issuer Name="Bank of Nowhere"/>
   </Card>
</Payment>"""

Create a new property transfer from Traveler xml in the Soap Response to copy the Customer node to a TestCase property "customer". You can do that by using the below configuration in a new Propperty transfer step after the Soap RQ/RS step.

Source: SoapRequestStepName Property: Response
declare namespace ns1='http://www.example.com/schemas';
//ns1:Traveler//ns1:Customer
Target: TestCaseName Property: nodeCustomer

Add a new test step with groovy script called updatePaymentInXML. First step is to create a XmlHolder for the source XML Traveler and extract Customer node out of it.

def getXmlHolder(){
   def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
   def nodeCustStr=testRunner.testCase.getProperty("nodeCustomer").getValue()
   log.info "Pre Update: " + nodeCustStr
   def holder = groovyUtils.getXmlHolder(nodeCustStr)
   holder.namespaces["ex"] ="http://www.example.com/schemas"   
   return holder
}

Next lets get the Customer node from it as a Node.

def holder = getXmlHolder()
def nodeCustomer = holder.getDomNode("//ex:Customer")

Next lets insert payment in the customer node.

def insertPaymentInCustomer(holder, nodeCustomer, pay){
   def ownerDoc=nodeCustomer.getOwnerDocument()
   def payHolder=new com.eviware.soapui.support.XmlHolder(pay)
   payHolder.namespaces["ex"] ="http://www.example.com/schemas"
   def nodePay = payHolder.getDomNode("//ex:Payment")

   def importedNodePay=ownerDoc.importNode(nodePay, true)
   nodeCustomer.appendChild(importedNodePay)   
   holder.updateProperty()
   log.info "Post Update: " + holder.getXml()
}
insertPaymentInCustomer(holder, nodeCustomer, pay)

Things to note in the above snippet is that we get the owner document and import the newly extracted Payment xml node from the xml string using a new XmlHolder instance. Lets add the card number to it.

def updateCCNumberInHolder(holderIn, ccNumberIn){
     holderIn["//ex:Card/@Number"]=ccNumberIn
     holderIn.updateProperty()
     testRunner.testCase.setPropertyValue("nodeCustomerUpdated",holderIn.getXml())
     log.info "Post Update: " + holderIn.getXml()
}
updateCCNumberInHolder(holder, testRunner.testCase.getPropertyValue("ccNumber"))

Now, the holder.getXml() gives you the latest XML with updated Payment and cc number in it. Test case property nodeCustomerUpdated has the all updated XML that you can use.