I worked on a Lift 2.4 web application that used lot of Surround and Embed tags in HTML files. Looking in the LiftSession we can see that lift repeats the merging process again for every request received. This is not required and significant performance can be gained when we cache a merged HTML template and process that instead of evaluating "surround" and "embed" tag on every request. If you enable TemplateCache, then this new cache below will act like a cache one level up in the hierarchy. Note that its a good idea to enable/use this cache only in production mode to not to hinder development activities.
There are multiple ways to implement this. The one I will explain here is by extending the LiftSession class and overriding the processTemplate method. But, if you have a custom content handler DispatchPF appended to lift dispatch, and you do some more stuff before calling the processTemplate yourself then you can add this caching in this class too. The code is very different when compared to the first approach and will try to provide at the end.
First thing to do is to define a synchronized cache that is used to cache the processed templates. The code below is self explanatory. Note that most of the code to follow is copied from Lift and modified to serve the desired purpose.
/**
* A synchronized cache that handles the caching of Processed Templates in production mode only. The strategy used
* is Least Recently Used caching.
*/
object ProcessedTemplateCache{
private val cache : LRU[String, NodeSeq] = new LRU(50)
def get(key: String): Box[NodeSeq] =
if(Props.productionMode)
cache.synchronized {
cache.get(key)
}
else Empty
def set(key: String, node: NodeSeq): NodeSeq =
if(Props.productionMode)
cache.synchronized {
cache(key) = node
node
}
else node
def has(key:String):Boolean =
if(Props.productionMode)
cache.synchronized {
cache.contains(key)
}
else false
def delete(key: String) {
if(Props.productionMode)
cache.synchronized(
cache.remove(key)
)
}
}
We need to split a NodeSeq to its components to use it in the CustomLiftSession to be defined later. This is done using pattern matching in scala.
/**
* A decompressor that expands a Node to its characteristics.
* Ther order being - element, kids, isLazy, attrs, snippetName
*/
private object CachedSnippetNode {
def unapply(baseNode: Node): Option[(Elem, NodeSeq, Boolean, MetaData, String)] =
baseNode match {
case elm: Elem if elm.prefix == "lift" || elm.prefix == "l" => {
Some((elm, elm.child,
elm.attributes.find {
case p: PrefixedAttribute => p.pre == "lift" && (p.key == "parallel")
case _ => false
}.isDefined,
elm.attributes, elm.label))
}
case _ => {
None
}
}
}
Now, lets extend the LiftSession class. Note that, many methods in LiftSession is private and hence not accessible. The only way around the problem is by pulling them in the new CustomLiftSession implementation. So, here the version is very important. The code below uses the LiftSession from 2.4 codebase. Lift 2.5 has modification/extensions to this. I understand this as a bad idea as the methods pulled in from the LiftSession class may be modified in future and HAS to be synched up manually when required, but there is not much we can do using this approach due to private modifiers in the LiftSession class. Note that all the code is written in package net.liftweb.http.
class CustomLiftSession(_contextPath: String, uniqueId: String,
httpSession: Box[HTTPSession]) extends LiftSession(_contextPath, uniqueId, httpSession){
//This val is copied as is from the super class, as it is private and not accessible from here.
private object overrideResponseCode extends TransientRequestVar[Box[Int]](Empty)
//This val is copied as is from the super class, as it is private and not accessible from here.
private val fullPageLoad = new ThreadGlobal[Boolean] {
def ? = this.box openOr false
}
/**
* This method is copied as is from the super class, as method is private and not accessible from here.
*
* @param path The path to parse
* @param session The session in which this is invoked
* @return
*/
private[http] def findVisibleTemplate(path: ParsePath, session: Req): Box[NodeSeq] = {
val tpath = path.partPath
val splits = tpath.toList.filter {
a => !a.startsWith("_") && !a.startsWith(".") && a.toLowerCase.indexOf("-hidden") == -1
} match {
case s@_ if (!s.isEmpty) => s
case _ => List("index")
}
Templates(splits, S.locale)
}
/**
* This method is responsible to do preProcessing cachedSurround and embed tags. Only these tags and any
* of the same tags in its child nodes are processed. No other snippets are evaluated. The reason to do this
* is to cache the merged pages one level up so that the merging "surround" and "merge" does not
* occur every time.
*
* @param page The page name if any.
* @param xhtml The page as an XML
* @return
*/
def preProcess(page:String, xhtml:NodeSeq): NodeSeq = {
def processSurroundAndInclude1(page: String, in: NodeSeq): NodeSeq = {
in.flatMap {
case Group(nodes) =>
Group(processSurroundAndInclude1(page, nodes))
//Only process cachedSurround and embed snippets
case CachedSnippetNode(element, kids, isLazy, attrs, snippetName)if(snippetName == "cachedSurround" || snippetName == "embed") =>
S.doSnippet(snippetName) {
S.withAttrs(attrs) {
processSurroundAndInclude1(page,
NamedPF((snippetName,
element, attrs,
kids,
page),
liftTagProcessing))
}
}
case v: Elem =>
Elem(v.prefix, v.label, v.attributes,
v.scope, processSurroundAndInclude1(page, v.child): _*)
case pcd: scala.xml.PCData => pcd
case text: Text => text
case unparsed: Unparsed => unparsed
case a: Atom[Any] if (a.getClass == classOf[Atom[Any]]) => new Text(a.data.toString)
case v => v
}
}
processSurroundAndInclude1(page, xhtml)
}
/**
* The custom process template method that tries to use the cached preprocessed lift template, if not then caching one for the first time.
* Every successive process template will use the merged template from the cache. This decreases one level of processing on
* every request.
*
* @param template The real unprocessed template
* @param request The request that wants the processed template
* @param path The path to search for the template
* @param code Override response code if any
* @return
*/
override def processTemplate(template: Box[NodeSeq], request: Req, path: ParsePath, code: Int): Box[LiftResponse] = {
overrideResponseCode.doWith(Empty) {
(template or findVisibleTemplate(path, request)).map {
xhtml =>
fullPageLoad.doWith(true) {
val path = S.request.get.path.wholePath.mkString("/")
//Phase 0: Preprocess cachedSurround and embed or get the preprocessed page from the cache
val preXML: NodeSeq = if(ProcessedTemplateCache.has(path))ProcessedTemplateCache.get(path).get
else ProcessedTemplateCache.set(path, preProcess(PageName get, xhtml) )
//
//---- Everything below here is as in the real Lift 2.4 codebase.
//
// allow parallel snippets
// Phase 1: snippets & templates processing
val rawXml: NodeSeq = processSurroundAndInclude(PageName get, preXML)
// Make sure that functions have the right owner. It is important for this to
// happen before the merge phase so that in merge to have a correct view of
// mapped functions and their owners.
updateFunctionMap(S.functionMap, RenderVersion get, millis)
// Clear the function map after copying it... but it
// might get some nifty new functions during the merge phase
S.clearFunctionMap
// Phase 2: Head & Tail merge, add additional elements to body & head
val xml = merge(rawXml, request)
// But we need to update the function map because there
// may be addition functions created during the JsToAppend processing
// See issue #983
updateFunctionMap(S.functionMap, RenderVersion get, millis)
notices = Nil
// Phase 3: Response conversion including fixHtml
LiftRules.convertResponse((xml, overrideResponseCode.is openOr code),
S.getHeaders(LiftRules.defaultHeaders((xml, request))),
S.responseCookies,
request)
}
}
}
}
}
Now, lets define a custom snippet dispatcher for a new tag "cachedSurround" that will be used instead of the "surround" tags in the HTML templates.
/**
* The snippet dispatcher that handles the "cachedSurround" lift tag.
*/
object CachedSurround extends DispatchSnippet {
def dispatch : DispatchIt = {
case _ => render _
}
def render(kids: NodeSeq) : NodeSeq =
(for {
ctx <- S.session ?~ ("FIX"+"ME: Invalid session")
req <- S.request ?~ ("FIX"+"ME: Invalid request")
} yield {
WithParamVar.doWith(Map()) {
val mainParam = (S.attr("at") openOr "main", ctx.asInstanceOf[CustomLiftSession].preProcess(PageName.get, kids)) //Do preprocess only, do not evaluate snippets right now
val paramsMap = WithParamVar.get + mainParam
ctx.findAndMerge(S.attr("with"), paramsMap)
}
}) match {
case Full(x) => x
case Empty => Comment("FIX"+ "ME: session or request are invalid")
case Failure(msg, _, _) => Comment(msg)
}
}
Next in Boot.scala you will add the following code that will allow lift to use CachedSurround dispatcher when it encounters the "cachedSurround" lift tag(<lift:cachedSurround with="default" at="pageContent">...</lift:cachedSurround>) and use our CustomLiftSession that does the preprocessing and caching of merged templates (with cachedSurround and embed tags only) in production mode.
LiftRules.snippetDispatch.append(
Map("cachedSurround" -> CachedSurround)
)
LiftRules.sessionCreator = {
case (httpSession, contextPath) => new CustomLiftSession(contextPath, httpSession.sessionId, Full(httpSession))
}
Now, many of this code can be avoided if you have a separate content handler dispatcher (this is added again in Boot.scala as LiftRules.dispatch.append(ContentHandler.dispatch)). Note that, this is the way if you want to do more before you process templates or load templates from some other place. I wont go into that detail, but if you do this then you will call the processTemplate on the session manually. So, this is what you can do in that case. Create the below code in net.liftweb.http package. When using below you do not need to extend LiftSession class nor create any new snippet dispatcher (whether you use surround or cachedSurround as in the comments below).
object Expand{
private lazy val logger = Logger(this.getClass)
def apply(template:NodeSeq) : NodeSeq = {
template.flatMap{
case Group(nodes) => apply(nodes)
case CachedSnippetNode(element, kids, isLazy, attrs, snippetName)if(snippetName == "cachedSurround") => //This part is modified from Surround.scala lift code. Also you can use "surround" directly here if you dont want to create a new lift tag.
(for (ctx <- S.session ?~ ("FIX ME: Invalid session")) yield {
val mainParam = Map(attrs.asAttrMap("at") -> kids)
ctx.findAndMerge(Full(attrs.asAttrMap("with")), mainParam)
}) match {
case Full(x) => apply(x)
case Empty => Comment("FIX ME: session or request are invalid")
case Failure(msg, _, _) => Comment(msg)
}
case CachedSnippetNode(element, kids, isLazy, attrs, snippetName)if(snippetName == "embed") =>//This part is modified from Embed.scala lift code
(for {
ctx <- S.session ?~ ("FIX ME: session is invalid")
templateOpt <- ctx.findTemplate(attrs.asAttrMap("what")) ?~ ("FIX ME the embed tag has what="+attrs.asAttrMap("what")+" which is not found")
} yield (attrs.get("what"),LiftSession.checkForContentId(templateOpt))) match {
case Full((what,template)) => {
val bindingMap : Map[String,NodeSeq] = Map(kids.flatMap({
case p : scala.xml.PCData => None // Discard whitespace and other non-tag junk
case t : scala.xml.Text => None // Discard whitespace and other non-tag junk
case e : Elem if e.prefix == "lift" && e.label == "bind-at" => {
e.attribute("name") match {
/* DCB: I was getting a type error if I just tried to use e.child
* here. I didn't feel like digging to find out why Seq[Node]
* wouldn't convert to NodeSeq, so I just do it with fromSeq. */
case Some(name) => Some(name.text -> NodeSeq.fromSeq(e.child))
case None => logger.warn("Found <lift:bind-at> tag without name while embedding \"%s\"".format(attrs.asAttrMap("what"))); None
}
}
case _ => None
}): _*)
BindHelpers.bind(bindingMap, template)
}
case Failure(msg, _, _) => throw new SnippetExecutionException(msg)
case _ => throw new SnippetExecutionException("session is invalid")
}
case v: Elem =>Elem(v.prefix, v.label, v.attributes,
v.scope, apply(v.child): _*)
case pcd: scala.xml.PCData => pcd
case text: Text => text
case unparsed: Unparsed => unparsed
case a: Atom[Any] if (a.getClass == classOf[Atom[Any]]) => new Text(a.data.toString)
case v => v
}
}
}And in your content handler dispatcher, you will use it like this.
val pathParams = S.request.get.path.wholePath
Templates(pathParams) match {
case Full(template) => {
val path = S.request.get.path.wholePath.mkString("/")
val preXML: NodeSeq = if(ProcessedTemplateCache.has(path))ProcessedTemplateCache.get(path).get
else ProcessedTemplateCache.set(path, Expand(template))
S.session.get.processTemplate(Full(preXML), S.request.get, S.request.get.path, 200)
}
case _@Empty => logger.error("Parsing empty content while loading %s template.".format(filePath))
Empty
case error: Failure => logger.error(error)
error
}
This will improve the performance significantly on long run in production mode as you already have cached templates that are expanded and merged from cachedSurround and embed tags. Note that this improvement is conditional upon the number of templates you use, and the amount of cachedSurround and embed tag in work. Just to make it simple, I am providing the package and the imports below.
package net.liftweb
package http
import xml._
import net.liftweb._
import net.liftweb.common._
import Box._
import util._
import Helpers._
import builtin.snippet._
import provider._
import xml.Group
import scala.Some
import xml.Text
Posts
No comments:
Post a Comment