Mindoo Blog Cutting edge technologies - About Java, Lotus Notes and iPhone http://www.mindoo.de/web/blog.nsf/ en-us Fri, 27 Apr 2012 19:00:09 +0200 XPages series #14: Using MongoDB’s geo-spatial indexing in XPages apps part 2 Fri, 27 Apr 2012 19:00:09 +0200 http://www.mindoo.com/web/blog.nsf/dx/27.04.2012190009KLEMXN.htm XPages Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/27.04.2012190009KLEMXN.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/27.04.2012190009KLEMXN.htm Click here for part 1.


Diving into the code: client side

The UI stuff is stored in an NSF database with basic page layout defined in an XPage and the client-side application logic code stored as Dojo class file in the Java perspective of Domino Designer.



The Dojo class location is defined and the class is loaded with the following code in the XPage "start.xsp":

<xp:this.resources>
        <xp:dojoModulePath prefix="mongo">
                <xp:this.url><![CDATA[#{javascript:'/dojo/mongo';}]]></xp:this.url>
        </xp:dojoModulePath>
                <xp:dojoModule name="mongo.test.PageHandler">
        </xp:dojoModule>
</this.resources>

After that, the class can be instantiated in a script block:

<xp:scriptBlock type="text/javascript">
        <xp:this.value><![CDATA[
        dojo.addOnLoad(function() {
                window.pageHandler=new mongo.test.PageHandler();
                window.pageHandler.init();
        });
        ]]></xp:this.value>
</xp:scriptBlock>

In the init() method, the page handler class registers event code for the form fields and buttons in the UI and creates the Dojo grid components (dojox.grid.LazyTreeGrid) to display the database content.

Now let's take a look at two UI operations in detail, to see how the UI communicates with the server-side code.

Adding data to the database

When places get added through the Administration tab, we first use the servlet "/mongogeo" to convert the addresses into coordinates:



What we get back is the JSON response of  the Google Geocoding API. It does not only contain the coordinates of an address, but other quite useful information like the name of the federal state, the country code and coordinates of a bounding box (for general addresses like city names that don't have exact coordinates).

Next, a POST operation to "/mongotest/addplaces" creates the actual place documents in the database followed by a reload operation of our Dojo grid to reflect the new data in the UI:



Query the database

The REST service "/mongotest/queryplaces" is used both for reading all places (on the Administration tab) and finding the nearest places for a certain position (on the Search tab). For the latter, we specify additional longitude/latitude arguments in the URL:

http://localhost/dev/ec12/nosql/mongodb/geo.nsf/mongotest/queryplaces?distance=3&longitude=8.3799444&latitude=49.009148&type=Shop&start=0&count=100

Dynamic sorting can be applied to the result with and without longitude/latitude arguments by specifying a sort parameter.
If no sort parameter and no position is specified, places are sorted by name as default sorting. For queries with longitude/latitude parameters, the distance between both points is used as default sorting.

http://localhost/dev/ec12/nosql/mongodb/geo.nsf/mongotest/queryplaces?distance=3&longitude=8.3799444&latitude=49.009148&type=Shop&start=0&count=100&sort=name

In the UI, data can be sorted by clicking one of the grid column headers.




Enough technical details, let's come to an end with this long blog article! :-)

If you would like to try out the sample in your own environment, here are the setup instructions:


Setup instructions - Prerequisites

I expect that you have downloaded and installed the latest version of MongoDB for your operating system. The sample was built for version 2.0.3 and I just found out that 2.0.4 is already available.

The server code is using localhost and port 27017 by default, but this can be changed by setting the environment variable NOSQL_MONGO_SERVER to something like "hostname1,hostname2:27123,hostname3", a comma separated list with hostname and optional port. The Mongo driver will use the list of servers for failover in case a server goes down.

Plugin installation

Download and extract the archive file from the specified download link.

Then follow the instructions in the Domino wiki article XPages Extension Library Deployment in Domino 8.5.3 and IBM XWork Server to create an NSF based update site database on the Domino server and import the update site from the download archive into the update site database, followed by a HTTP task restart (restart task http).

Database installation

Copy the sample database from the download archive to your Domino server and sign it with your Notes ID. Now open the database in the browser.

There is a section "Initialization" on the Administration tab which lets you import a set of default addresses (from Mindoo's home town Karlsruhe, Germany). This step is optional.

Download link

Click here to download the sample application



Phew... That was a long text. Thanks for reading until here!

And stay tuned for an article about my second demo from Entwicklercamp 2012:

Using Neo4J to solve the travelling salesman problem in XPages apps

]]> http://www.mindoo.com/web/blog.nsf/dxcomments/27.04.2012190009KLEMXN.htm http://www.mindoo.com/web/blog.nsf/dx/27.04.2012190009KLEMXN.htm?opendocument&comments XPages series #14: Using MongoDB’s geo-spatial indexing in XPages apps part 1 Fri, 27 Apr 2012 18:59:38 +0200 http://www.mindoo.com/web/blog.nsf/dx/27.04.2012185938KLEMXB.htm XPages Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/27.04.2012185938KLEMXB.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/27.04.2012185938KLEMXB.htm session about NoSQL databases at the German Entwicklercamp conference in March 2012. It demonstrates how the document-oriented NoSQL database MongoDB can be used in an XPages web application for the IBM Lotus Domino server.

The rise of location based services

Location based services have become quite popular in the past years: Most of the smartphones carry a GPS sensor, and there are a lot of popular apps out there (e.g. Foursquare, Yelp) that use cloud services to find people, restaurants and places nearby.

For a number of reasons (mostly scalability/performance related), the NSF database of Lotus Notes/Domino is not the best choice to implement these kind of applications/services. MongoDB is much more suited for this use case and the article shows how you can integrate MongoDB into Notes/Domino to combine both worlds and get the best from both of them.

The Dojo based web application that I am going to discuss in detail uses MongoDB's geospatial indexing feature to easily find the nearest points of interest for a mobile web user.

Here is how our application looks like:



The "Administration" tab provides a user interface to add/remove places to/from the MongoDB database. A place is stored as a document in a MongoDB document collection and consists of a name, a type (e.g. shop/gas station) and the position information as [longitude, latitude] value pair.

The database can be queried on the "Search" tab by entering an address, a distance in kilometers and an optional type. The address will automatically get converted to [longitude, latitude] coordinates by using the Google Geocoding API, which is then used to find places within the specified distance.

Our MongoDB document collection is indexed in a way so that we can quickly query the database for places that surround the current user's position and sort them by ascending distance between both points.



Since geospatial indexing is a built-in feature of MongoDB, the solution is very easy to implement.

Limitations of NSF

While it is not impossible to build this application in pure Lotus Notes/Domino technology with an NSF database, the missing geo index and limited scalability of NSF would make it difficult to keep response times low with large data sets, e.g. millions or even billions of places worldwide. MongoDB instead provides a sharding feature to distribute the data evenly to several servers and reduce server load.

Additional indexer like Apache Lucene in combination with the OSGi tasklet service plugin's extension hooks might help to improve Domino's limited indexing support (we already used a Lucene index with Domino data in a big customer project) , but using MongoDB for this use case is just so much easier.

Architecture

The application consists of client-side and server-side code:
Client-side code is written in JavaScript language as a Dojo class using the class declaration system of Dojo 1.6. This Dojo class handles all user interface operations.

Server-side code is written in Java as a servlet and is exposed to the client-side via REST API's (here: GET/POST requests sent via dojo.xhrGet / dojo.xhrPost).

We developed two servlets for this application. The first servlet handles init/add/remove/query operations in MongoDB. The second servlet simply acts as a proxy to access the Google Geocoding API from client-side Javascript in order to convert between addresses and geo coordinates (longitude/latitude).

Since the REST protocol is completely stateless, the server-side code is very easy to test. With a bit of refactoring, it's even possible to run the whole application in alternative servlet containers like Jetty, Tomcat or directly within the Eclipse IDE instead of the Domino server.

Personally, I really like this REST API based approach because it's clean, transparent and keeps technologies separated.
It enables you to replace the server environment, the database system and the client-side UI toolkit at any time. You can even build multiple UIs (e.g. based on Dojo 1.6, 1.7 and Sencha's ExtJS) or mobile clients that work with the same REST API.

Diving into the code: server side
Our sample comes as an Eclipse plugin "com.mindoo.mongo.test" to be run on Domino's OSGi framework (tested with Domino 8.5.3 GA without any fixpacks or XPages extension library):



As you can see above, I added the Mongo API classes to the plugin's classpath.
Adding the Mongo API classes to an NSF and using them directly from SSJS code might also be possible, but I prefer the plugin solution, because the MongoDB access is supposed to be a central service on the Domino server. In addition, by using a plugin we don't have any issues with Domino's restricted security manager that prevents operations to run properly in an XPages context (e.g. a lot of drivers are using Log4J for logging, which does not run well within an XPages application).

Our two servlets are defined in the plugin.xml file of the plugin:

<?xml version="1.0" encoding="UTF-8"?>
<?eclipse version="3.4"?>
<plugin>
        <extension point="org.eclipse.equinox.http.registry.servlets">
                <servlet alias="/mongotest" class="com.mindoo.mongo.test.servlet.MongoTestServlet"
                        load-on-startup="true">
                </servlet>
        </extension>
        <extension point="org.eclipse.equinox.http.registry.servlets">
                <servlet alias="/mongogeo" class="com.mindoo.mongo.test.servlet.GeoQueryServlet"
                        load-on-startup="true">
                </servlet>
        </extension>
</plugin>

That's all you need to declare your own servlet on Domino.
After that, the servlet can either be accessed on the URL base level (http://server/mongogeo) or in the context of a Domino database (http://server/path/to/db.nsf/mongotest).

If called in the context of a database, the Domino HTTP server first makes sure that the current user is allowed to access the database and if not, it displays a login prompt. A utility class (com.ibm.domino.osgi.core.context.ContextInfo) can be used to get session/database objects for the current HTTP request.

In our sample, we use the first format for our Geo API query, because it's an open service of the server without access restriction. Anyone can use it and the URL is simple to remember.

The second URL format (/path/to/db.nsf/mongotest) is used to access MongoDB data. This enables us to check the user against the Notes database ACL to see if he is allowed to read or write MongoDB data.

Here is the method of our MongoDB access servlet that handles POST requests:

protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        Session session=ContextInfo.getUserSession();
        Database dbContext=ContextInfo.getUserDatabase();
        if (dbContext==null) {
                ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN,
                        "Servlet must be run in the context of a Domino database", null);
                return;
        }

        try {
                String pathInfo=req.getPathInfo();
                if ("/deleteplaces".equals(pathInfo)) {
                        if (!isUserAuthenticated(MongoAccessMode.Write, session, dbContext)) {
                                ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN,
                                        "Current user is not allowed to write data", null);
                                return;
                        }
                        doDeleteGridData(session, dbContext, req, resp);
                }
                else if ("/addplaces".equals(pathInfo)) {
                        if (!isUserAuthenticated(MongoAccessMode.Write, session, dbContext)) {
                                ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN,
                                        "Current user is not allowed to write data", null);
                                return;
                        }
                        doAddGridData(session, dbContext, req, resp);
                }
                else if ("/queryplaces".equals(pathInfo)) {
                        if (!isUserAuthenticated(MongoAccessMode.Read, session, dbContext)) {
                                ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN,
                                        "Current user is not allowed to read data", null);
                                return;
                        }
                        doQueryPlaces(session, dbContext, req, resp);
                }
                else if ("/initdb".equals(pathInfo)) {
                        if (!isUserAuthenticated(MongoAccessMode.Write, session, dbContext)) {
                                ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN,
                                        "Current user is not allowed to write data", null);
                                return;
                        }
                        doInitDbWithValues(session, dbContext, req, resp);
                }
                else if ("/cleardb".equals(pathInfo)) {
                        if (!isUserAuthenticated(MongoAccessMode.Write, session, dbContext)) {
                                ServletUtils.sendHttpError(resp, HttpServletResponse.SC_FORBIDDEN,
                                        "Current user is not allowed to write data", null);
                                return;
                        }
                        doClearDb(session, dbContext, req, resp);
                }
                else {
                        ServletUtils.sendHttpError(resp, 500, "Unsupported command "+pathInfo, null);
                }
        }
        catch (Throwable e1) {
                e1.printStackTrace();
                //avoid posting the full stacktrace to the user (for security reasons)
                ServletUtils.sendHttpError(resp, 500, "An error occurred processing the request. The error has been logged. Please refer to the server log files/console for details", null);
                return;
        }
}

The code is pretty simple: We check if the user is allowed to call the specific service (read operations require the role [MongoDBRead], write operations the role [MongoDBWrite] ) and then forward the request to helper methods.

Note that I am not going into full detail in this blog article how we actually access data in MongoDB. You can find that in the provided download archive and there is a great tutorial at the MongoDB website about using the Java API.

Believe me, this stuff is really easy to use!


Click here for part 2 of this article! ]]> http://www.mindoo.com/web/blog.nsf/dxcomments/27.04.2012185938KLEMXB.htm http://www.mindoo.com/web/blog.nsf/dx/27.04.2012185938KLEMXB.htm?opendocument&comments English slides for my Entwicklercamp 2012 session about NoSQL databases Thu, 5 Apr 2012 00:07:41 +0200 http://www.mindoo.com/web/blog.nsf/dx/05.04.2012000741KLETXV.htm NoSQL Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/05.04.2012000741KLETXV.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/05.04.2012000741KLETXV.htm NoSQL session at this year's Entwicklercamp (developer camp) conference in Gelsenkirchen, Germany.

Image:English slides for my Entwicklercamp 2012 session about NoSQL databases

I am still searching for time to polish the two demos: MongoDb integration in XPages apps and NoSQL integration in XPages apps.
Next week is blocked for German holidays. Need to work on the demos afterwards.
]]> http://www.mindoo.com/web/blog.nsf/dxcomments/05.04.2012000741KLETXV.htm http://www.mindoo.com/web/blog.nsf/dx/05.04.2012000741KLETXV.htm?opendocument&comments Download link for my Entwicklercamp 2012 session slides about NoSQL databases (German) Tue, 27 Mar 2012 19:00:43 +0200 http://www.mindoo.com/web/blog.nsf/dx/27.03.2012190043KLEMY2.htm NoSQL Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/27.03.2012190043KLEMY2.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/27.03.2012190043KLEMY2.htm I plan to translate them to English, but this may take some time.

Image:Download link for my Entwicklercamp 2012 session slides about NoSQL databases (German)


I will also write blog entries about my two demos: leveraging the MongoDb geo index feature from an XPages application and embedding the Neo4J graph databases in XPages (travelling salesman algorithm to find the best route in a graph).

Update:
The English slides of the session are now online.
]]> http://www.mindoo.com/web/blog.nsf/dxcomments/27.03.2012190043KLEMY2.htm http://www.mindoo.com/web/blog.nsf/dx/27.03.2012190043KLEMY2.htm?opendocument&comments Speaking at Entwicklercamp 2012 about NoSQL databases Mon, 19 Mar 2012 09:08:18 +0200 http://www.mindoo.com/web/blog.nsf/dx/19.03.2012090818KLEBKA.htm NoSQL Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/19.03.2012090818KLEBKA.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/19.03.2012090818KLEBKA.htm Entwicklercamp in Gelsenkirchen, Germany. My session will provide an overview of the NoSQL market for the database types
  • key value stores
  • column oriented databases
  • document oriented databases
  • and graph database

It discusses Redis, Apache Cassandra, MongoDB and Neo4j in detail and I will be demo'ing, how to integrate MongoDB's geo-spatial indexing feature and Neo4j's graph search algorithms in XPages applications.

Finally, the session compares the feature sets of Lotus Notes with other NoSQL to see how compatitive Lotus Notes, one of the oldest NoSQL database systems, is today in this moving market.

Working on this topic was quite challenging, because there is A LOT of material to check out on the web, but it also was a lot of fun playing with the databases and building the demos.
]]> http://www.mindoo.com/web/blog.nsf/dxcomments/19.03.2012090818KLEBKA.htm http://www.mindoo.com/web/blog.nsf/dx/19.03.2012090818KLEBKA.htm?opendocument&comments Icon resources on the web for commercial and non-commercial use Wed, 8 Feb 2012 08:44:18 +0200 http://www.mindoo.com/web/blog.nsf/dx/08.02.2012084418KLEB4A.htm Notesdev Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/08.02.2012084418KLEB4A.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/08.02.2012084418KLEB4A.htm Since I found a new massive resource of free icons yesterday, I thought it might be a good idea so share some URLs.

For commercial applications, we bought the Icon Experience library a few years ago, which contains about 2500+ icons in 7 sizes at a very affordable price:
http://www.iconexperience.com

Free icons with varying licenses can be found here:

http://openiconlibrary.sourceforge.net
http://www.freeiconsweb.com
http://www.famfamfam.com
http://p.yusukekamiyamane.com

For the latter, licenses reach from GPL, LGPL, Creative Commons to Public Domain. I guess for pure inhouse development, this is not that important, but if you plan to sell your application or publish it on OpenNTF, you should doublecheck if the icon license is ok for your use case.

Feel free to add your own favorites by commenting on this article! ]]> http://www.mindoo.com/web/blog.nsf/dxcomments/08.02.2012084418KLEB4A.htm http://www.mindoo.com/web/blog.nsf/dx/08.02.2012084418KLEB4A.htm?opendocument&comments XPages series #13: XPiNC app development tool Thu, 2 Feb 2012 16:24:12 +0200 http://www.mindoo.com/web/blog.nsf/dx/02.02.2012162412KLEL3Q.htm XPages Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/02.02.2012162412KLEL3Q.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/02.02.2012162412KLEL3Q.htm Sencha's Ext JS for the UI and is completely based on our own web app development framework.
The framework does not use XPages design elements at all, but follows a simple REST API architecture with standard servlets to produce the data and user interface.

The benefit of this approach is that web applications can be developed, run and debugged from a pure Eclipse environment. They can even run on an different servlet engine than the Domino server's http task and - thanks to our data abstraction layer - they can even store the whole app data in a non-Domino database and mix/merge data between different database types.

Another benefit is that almost all the code has been written by ourselves. So it's not a kind of blackbox, made by IBM, where it's hard to work around occuring issues, but we are able to track issues down right until the service(HttpServletRequest req, HttpServletResponse response) call coming from the web container.

The downside is of course, that apps developed with the framework do not make use of IBM's XPages Extension Library, that contains various powerful UI controls. So we needed to create them ourselves.

And regarding XPiNC development, there is another downside: JavaScript errors do not get logged/displayed in the Notes Client, unless you register your own global error handler (providing a function for window.onerror).
Normal XPiNC applications do already contain such an error handler (somewhere within the XSP API object), which uses an internal bridge to post the error content to the Notes Client's status bar.

The development tool

To make development of XPiNC applications easier, I have created a small Eclipse plugin that displays three icons in the Client's toolbar:
Image:XPages series #13: XPiNC app development tool


The third icon lauches a piece of code that injects Firebug lite into the currently visible XPage:

        public void run(IAction action) {
                IWorkbenchPart part =
                  PlatformUI.getWorkbench().getActiveWorkbenchWindow().getPartService().getActivePart();

                if (part instanceof XspViewPart) {
                        XspViewPart xPart=(XspViewPart)part;
                        XspXulRunnerBrowser browser=xPart.getWebBrowser();
                       
                        HTMLDocument doc=browser.getDocument();
                        NodeList headNodes=doc.getElementsByTagName("head");
                        if (headNodes!=null && headNodes.getLength()>0
                           && headNodes.item(0) instanceof Element) {

                                //add this snippet to the HTML DOM tree:
                                //< script type="text/javascript"
                                // src="https://getfirebug.com/firebug-lite.js" > < /script >
                                String firebugUrl="https://getfirebug.com/firebug-lite-debug.js";
                                Element headNode=(Element) headNodes.item(0);
                                Element scriptNode=doc.createElement("script");
                                scriptNode.setAttribute("type", "text/javascript");
                                scriptNode.setAttribute("src", firebugUrl);
                                headNode.appendChild(scriptNode);
                        }
                }
                else {
                        MessageDialog.openError(Display.getDefault().getActiveShell(), "Error",
                          "This action does only work within an XPiNC application!");
                }
        }

The result looks like this:

Image:XPages series #13: XPiNC app development tool


Using Firebug Lite, you can easily inspect the HTML DOM tree and CSS attributes of the current page. Another very useful feature is the Console API, which you may know already from classic browser development:
it lets you write log/debug messages to the browser console (by calling console.log('...')) and has other nice features like stacktrace dumping of JavaScript calls.

To our surprise we found out that the pure Xulrunner engine, that is used to display XPages in the Notes Client, does not register the global "console" at all. So Firebug lite came to the rescue. You can find all your log messages in the Console tab.

The other two toolbar actions are even more powerful. One lets you define a custom script library URL, e.g. to a script library design element on a public or intranet web server. The other actions will then create a script tag in the current XPage that points to the library. That way, you can inject any code you like into the XPiNC application.

Download

Finally, here is the download link with the Eclipse plugin project, feature project and update site:

xpinc-firebuglite-helper.zip

Hope this helps!
]]> http://www.mindoo.com/web/blog.nsf/dxcomments/02.02.2012162412KLEL3Q.htm http://www.mindoo.com/web/blog.nsf/dx/02.02.2012162412KLEL3Q.htm?opendocument&comments Lotusphere 2012: Download links for 105 additional session slides Sun, 22 Jan 2012 22:59:10 +0200 http://www.mindoo.com/web/blog.nsf/dx/22.01.2012225910KLETSK.htm Lotusphere 2012 Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/22.01.2012225910KLETSK.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/22.01.2012225910KLETSK.htm Lotusphere 2012 website with my previously released list of session slide download links.
I found 105 new slide downloads:

ls12_20120122.csv
ls12_20120122.html

There are still a few missing slide decks, but it's a big step forward. ]]> http://www.mindoo.com/web/blog.nsf/dxcomments/22.01.2012225910KLETSK.htm http://www.mindoo.com/web/blog.nsf/dx/22.01.2012225910KLETSK.htm?opendocument&comments Lotusphere 2012: download links for session slides Tue, 17 Jan 2012 10:50:09 -0400 http://www.mindoo.com/web/blog.nsf/dx/17.01.2012105009KLELKX.htm Lotusphere 2012 Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/17.01.2012105009KLELKX.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/17.01.2012105009KLELKX.htm Lotusphere 2012 website. Since my Macbook Pro could not get a proper connection to the wireless network in Dolphin (network is quite bad both at Lotusphere and in the Yacht Club we are staying in), I had to do this on the iPhone, because it was the only device that got an IP address. That was fun. ;-)

So here are the links to the slides that are available so far, sorted by session ID. Unfortunately, many are still missing. From our last years experience, the speakers are not the one to blame here. All had to submit their slides back in December.

Hopefully, the remaining session slides will follow shortly.

ls12_20120117.csv
ls12_20120117.html

Use your preferred download utility (e.g. DownThemAll) for download. You need to be logged in on the LS12 website before downloading. ]]> http://www.mindoo.com/web/blog.nsf/dxcomments/17.01.2012105009KLELKX.htm http://www.mindoo.com/web/blog.nsf/dx/17.01.2012105009KLELKX.htm?opendocument&comments Status report / no session submission for LS12 Mon, 7 Nov 2011 10:27:35 +0200 http://www.mindoo.com/web/blog.nsf/dx/07.11.2011102731KLED4S.htm Lotusphere 2012 Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/07.11.2011102731KLED4S.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/07.11.2011102731KLED4S.htm To give you an impression, here are a few things that we've been working on:

Development of an OSGi based web application framework with Ext.js UI
Abstraction layer for web applications that unifies data access across document-oriented and relational database systems, allows for tracking of data object changes (old/new values, group multi-object changes as transactions), different kinds of data serialization like XML or JSON and visualizing data with an Ext.js based, dynamically created web UI (using Jamon template language) that reads and writes contents through REST APIs. The framework can run from pure Eclipse and also in Domino's OSGi container.

Development of an OSGi based XPages application framework with Dojo UI
XPages extensibility API add-on to develop the backend code of XPages directly in Java instead of writing SSJS code or Expression Language to bridge between UI element events/properties and application code.
The framework uses Java Annotations and Java Reflection APIs for UI/backend code weaving.

Development of an IBM Websphere portal server applications
Custom development of market-analysis application for consumer goods industry.

Development of dynamic web application based on Glassfish and JPA, with jQuery and Highcharts
Data analysis tool for the buying department of a German automotive company.

Plugin development for the Lotus Notes Client
Several productivity enhancement plugins for the Lotus Notes Client

Plugin deployment support
Autodetection routine for XPages applications in Client/Server to automatically check for required Eclipse features/plugins when an application is opened (implemented in Lotusscript) so that the XPages runtime does not display those confusing error messages about missing XPages extensions. Instead, errors are properly handled without user interaction and missing features are automatically deployed on the machine.

XPiNC development framework
Spent time to work on enhancements for the XPages2Eclipse toolkit that provides Eclipse APIs to XPages applications in the Lotus Notes Client.

Classic Domino web application development
Improved round tripping quality of richtext editing between web browser and Notes Client including paste optimization from MS Word. Solution uses a combination of CKeditor add-ons and HTML/DXL conversions developed in Lotussript code for R7.  Customer: print industry

Classic Notes Client development
Built a Notes application to automatically deploy and update a list of mail folders to iOS users on Traveler.


A word about Lotusphere 2012
Since time is so limited, we decided not to submit a session proposal for Lotusphere 2012.

We will however participate with full conference pass and I just booked hotel and flight two days ago. In contrast to previous years, we will arrive one day earlier, on Friday, 13th of January 2012 (hopefully not a bad day for travel, at least it's a direct flight :-)).
Are there any plans yet for a blogger meeting for a beer or two on Saturday 14th like in previous years?

Another reason not to submit a session is that things get more complicated year after year. To learn about OSGi development or XPages extensibility APIs in a hands-on session for a day or two does make much more sense than watching a one hour presentation.

We could have submitted our plugin development session from last year another time (overview of plugin development for Notes Client and DDE with 14 demos), because slides and demos are already done. But that session (which got very good feedback at LS11) has now been presented four times at two conferences (Lotusphere and German Entwicklercamp 2010/2011). It's hard to get motivated for the 5th time :-).


So, to sum up this long blog entry, I am really looking forward to seeing many of you guys at LS12 - and to having more time to work on concepts and product prototypes in the next weeks. ]]> http://www.mindoo.com/web/blog.nsf/dxcomments/07.11.2011102731KLED4S.htm http://www.mindoo.com/web/blog.nsf/dx/07.11.2011102731KLED4S.htm?opendocument&comments XPages series #12: XAgents and performance bottlenecks Sun, 17 Jul 2011 10:18:55 +0200 http://www.mindoo.com/web/blog.nsf/dx/17.07.2011101855KLEBRW.htm XPages Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/17.07.2011101855KLEBRW.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/17.07.2011101855KLEBRW.htm XAgent is a term that describes the equivalent of a classic Notes web agent in XPages technology: an XPage is called via URL and produces any kind of data (e.g. HTML, dynamic images, data in JSON format or ODF documents) by sending Strings or even raw bytes directly to the browser.
 
Chris Toohey and Stephan Wissel have already blogged about this topic a few years ago and discussed some use cases:

IBM Lotus Notes Domino REST Web Services via XPage XAgents
Web Agents XPages style

How to write an XAgent
To write an XAgent, you basically need to do two things: first you add the attribute rendered="false" to the xp:view tag of an XPage to prevent the XPages engine from rendering any output. Second, you need to write the code that produces the data and add it to the beforeRenderResponse event of an XPage.

Here are two simple example XAgents. The first one writes XML content to the writer object of the servlet response (which is used to return character data):

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" rendered="false">

        <xp:this.beforeRenderResponse>><![CDATA[#{javascript:try {
        var uName = session.createName(session.getEffectiveUserName());
        var exCon = facesContext.getExternalContext();
        var response = exCon.getResponse();
        var writer = response.getWriter();
        response.setContentType("text/xml");
        response.setHeader("Cache-Control", "no-cache");
        writer.write("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n");
        writer.write("<test>Hello "+uName.getAbbreviated()+"</test>");
        facesContext.responseComplete();
}
catch (e) {
        _dump(e);
}}]]></xp:this.beforeRenderResponse></xp:view>

The second example demonstrates how you can load a file resource from the database design and write it to the output stream object of the servlet response:

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" rendered="false">

        <xp:this.beforeRenderResponse><![CDATA[#{javascript:try {
        var exCon = facesContext.getExternalContext();
        var response = exCon.getResponse();
        var outStream = response.getOutputStream();
       
        //load a file resource from db design and write it to the output stream
        var cl=com.ibm.domino.xsp.module.nsf.NotesContext.getCurrent().getModule().getModuleClassLoader();
        var buf=new byte[16384];
        var len;
        var filePath="/test/document.docx";
        var inStream=cl.getResourceAsStream(filePath);
        if (inStream==null) {
                var errMsg="File "+filePath+" not found";
                _dump(errMsg);
                response.sendError(404, errMsg);
        }
        else {
                //set mime type according to file type
                response.setContentType("application/vnd.openxmlformats-officedocument.wordprocessingml.document");
                response.setHeader("Cache-Control", "no-cache");
                //send http header with filename (needed so that the browser does not use the XPage's name as filename)
                response.setHeader("Content-Disposition", 'attachment; filename="document.docx"');
               
                //read bytes from inStream into the buffer
                while ((len=inStream.read(buf))>-1) {
                        //and write the data to the output stream
                        outStream.write(buf, 0, len);
                }
                inStream.close();
                outStream.close();
        }
        facesContext.responseComplete();
}
catch (e) {
        _dump(e);
}}]]></xp:this.beforeRenderResponse></xp:view>

The loaded file resource must be part of the NSF project's build path to make the XPage code find it (that's where the module classloader is looking for files).
Please refer to part two of this XPages series to get more information about how to add a folder to the build path.


Image:XPages series #12: XAgents and performance bottlenecks

If you just want to offer a file for download, developing an XAgent to stream the data would probably be the wrong solution.

In that case, you should add your files to the database design in the Java Perspective of Domino Designer instead and reference them directly in a URL. This is something that is used in the OpenNTF project XPages Mobile Controls to add a newer Dojo version (e.g. 1.6 with advanced mobile device support) to a Domino server than the default one (e.g. version 1.4.3 for Lotus Domino 8.5.2).

But the second code example from above could of course be modified to do something more useful, for example to replace placeholders in the file, create a zip file from multiple files on the fly using Java's ZipOutputStream or render a captcha image for spam protection.

Sounds great?
Well, before you start thinking about building large applications based on this approach, you should read on to get to know about it's limitations.

The limitations
What I'm going to show you is not only a limitation of XAgents, but of XPages technology in general. For XPages producing the user interface of an application, this limitation may not be critical, but it can become a show stopper for using them the XAgent way.

I've created a small test application with three XPages: Start.xsp, Frame1.xsp and Frame2.xsp (download link). The first one (Start.xsp) contains two iframes and a button. By clicking on the button, the other two XPages are loaded as iframe contents:

var now=new Date();
var nowTime=now.getTime();
var iframe1=document.getElementById("iframe1");
var iframe2=document.getElementById("iframe2");
iframe1.src=document.location.pathname+"/Frame1.xsp?startTime="+nowTime;
iframe2.src=document.location.pathname+"/Frame2.xsp?startTime="+nowTime;

Frame1.xsp contains beforeRenderResponse event code that delays the rendering for 5000 milliseconds (java.lang.Thread.sleep(5000)), while Frame2.xsp waits for 2500 milliseconds.

You would expect that both XPages are opened and rendered concurrently. Because of the delay, Frame2.xsp should be visible first, then Frame1.xsp.

Unfortunately, the output looks like this instead (click on the image to open the test application):

Image:XPages series #12: XAgents and performance bottlenecks

Frame1.xsp is displayed first, followed by Frame2.xsp. Frame2.xsp takes about 7600 milliseconds to be rendered, which is the sum of both delays, because the XPages engine does not render multiple XPages in an application at the same time for a single user.

It was a real surprise when we noticed this effect in a customer project. Our UI was based on the Ext JS toolkit which is heavily using REST services to read the UI component data. Our plan was to use XAgents for those REST services, but we quickly noticed that all UI elements only received data one at a time, which was a no-go for the application UI performance. Nothing was loading in parallel.

There are technical reasons for such a restriction, as Philippe Riand, XPages Chief Architect at IBM, explained to me in April 2011.
As you may know, the XPages runtime is based on JSF technology, which uses a tree of components on the server side that represents the structure of a web page and its current state. The tree is created/restored when a HTTP request comes in and saved/discarded after it is completed.
Unfortunately, this tree does not support being accessed by two concurrent threads, which would break many things like data sources and component states.

To prevent this to happen, XPages synchronizes on the user session object (HttpSession), so that only one thread can access the tree at a time. But by synchronizing on the user session (and not just on the component tree instance), the XPages dev crew also prevented any other XPage in an application from running (the Domino server seems to assign session objects on a per database basis).

The consequence is that you should think twice about your XPages application architecture, if you have many concurrent HTTP requests or if some of them take a lot of time to be processed. An XAgent may be the easiest solution to deploy, but may not produce the best user experience in all cases.

Workarounds and solution
There is some hope that the situation will improve in Domino 8.5.3.

As Philippe said, the next XPages engine will be smarter in synchronization and synchronize less than previously.
In addition, there will be a new database property called xsp.session.transient. This flag means that unique session objects will be created per request to the server and discarded right after the request ended. This is a first attempt to provide session less mode.
If you use this option, then you can create one database with all the services and no synchronization will happen, as each request will have its own session object.

I have requested a more granular option so that you can activate session less mode for single XPages in an application, but I'm not sure if this will make 8.5.3.

For Domino 8.5.2, I can see three things that you can do if you can't live with this performance issue:

1. Move XPages that run for a long time to separate databases
This does not help a lot, but if you have an XPages application with an XPage that needs some time to process, you could move it to different database. In that case, the rest of the application would still be responsive, although your long running XPage for example produces a 100 MB log output in XML format that you want to download.

2.  Write your own servlet in Java
By writing your own servlet as an Eclipse plugin running in Domino 8.5.2's OSGi framework, your code can run without any performance bottleneck caused by synchronization. See this article in the OpenNTF blog for implementation details.

3. There is an app for that :-)
As you may know, we have developed XPages2Eclipse, an extensive toolkit for XPages development, which is currently in beta-testing (June 2011). The articles Building servlets in JavaScript and Building servlets in Java demonstrate how you can write your own servlet implementation in JavaScript or Java language.
This solution reads all your code from the database design, so that you don't have to deploy every servlet individually. Just install the toolkit on a Domino server once, then you can create any number of servlets by writing code in Domino Designer and replicate them to the server.

]]> http://www.mindoo.com/web/blog.nsf/dxcomments/17.07.2011101855KLEBRW.htm http://www.mindoo.com/web/blog.nsf/dx/17.07.2011101855KLEBRW.htm?opendocument&comments XPages2Eclipse: New Beta-Build available Mon, 11 Jul 2011 21:48:16 +0200 http://www.mindoo.com/web/blog.nsf/dx/11.07.2011214816KLER8Q.htm XPages Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/11.07.2011214816KLER8Q.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/11.07.2011214816KLER8Q.htm
The issue should now be fixed, I just uploaded a new version of the update site archive, which you can download via the download link on the product website :

Image:XPages2Eclipse: New Beta-Build available

]]> http://www.mindoo.com/web/blog.nsf/dxcomments/11.07.2011214816KLER8Q.htm http://www.mindoo.com/web/blog.nsf/dx/11.07.2011214816KLER8Q.htm?opendocument&comments Beta version of XPages2Eclipse available: Development toolkit for XPages in the Notes Client Wed, 29 Jun 2011 09:10:37 +0200 http://www.mindoo.com/web/blog.nsf/dx/29.06.2011091037KLEAF7.htm XPages Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/29.06.2011091037KLEAF7.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/29.06.2011091037KLEAF7.htm

Today we are releasing the first public beta version of a new XPages development toolkit to the public!

It's called XPages2Eclipse, a language extension for XPages development in the Lotus Notes Client 8.5.2 and higher.

With XPages2Eclipse, XPages applications become first class citizens in the Lotus Notes Client.

The toolkit contains a variety of APIs that developers can leverage to connect their XPages applications to the Eclipse-based Notes Client UI, classic Lotus Notes applications and even communicate with IBM Lotus Symphony R3, all from server-side JavaScript (SSJS) code.

Here are only a few features that the toolkit has to offer:
  • Embed existing Notes applications: fill in Notes forms and Emails with data from XPages applications, access documents selected in classical Notes views, run existing LotusScript code
  • Import or export of data from IBM Lotus Symphony, supporting documents, spreadsheets and presentations
  • Execute document attachments with associated desktop-applications (for Windows, Linux and Mac OS)
  • Access the system clipboard to store HTML, text, images or files
  • Execute long-running operations in the background, display their progress and let the user cancel the operation if necessary
  • Convenient features like file selection, including multi-selection, and folder selection
  • Launch JavaScript code on Notes Client startup, when a toolbar button is clicked and on specific UI events like focus changes in the Client
  • Develop servlets using Javascript and Java with code stored in NSF
To get more information, download a beta version and find a lot of documentation with demo applications please visit the XPages2Eclipse product area on our website:


http://xpages2eclipse.mindoo.com - for our English audience


http://xpages2eclipse.mindoo.de - for our German audience

]]> http://www.mindoo.com/web/blog.nsf/dxcomments/29.06.2011091037KLEAF7.htm http://www.mindoo.com/web/blog.nsf/dx/29.06.2011091037KLEAF7.htm?opendocument&comments XPages series #11: Log data changes using beans and the DataObject interface Fri, 18 Mar 2011 10:47:26 +0200 http://www.mindoo.com/web/blog.nsf/dx/18.03.2011104725KLEDH8.htm Lotusphere 2011 Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/18.03.2011104725KLEDH8.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/18.03.2011104725KLEDH8.htm last week, this blog article demonstrates how managed beans can be used to create and edit Notes documents, transparently log data changes and even support alternative storage systems. This is my third and final sample from the Lotusphere 2011 session BP212.

If you followed this blog series from the beginning and take a look at the slides for the session BP212, you already know most of the technical details this sample is about:
  • declare managed beans
  • bind XPages UI fields with bean properties
  • use managed properties to declaratively configure managed beans

So there is not much new content, but it's a nice sample that shows how all the parts fit together in an application.


The download for this article contains a simple Notes database with four XPages:

Start.xsp
This XPage is the starting point when you open the database. It contains three buttons to create new company documents, view a list of existing documents and a button to view a change log that tracks data changes in the database:

Image:XPages series #11: Log data changes using beans and the DataObject interface


Company.xsp
The Company XPage let's you create and edit company documents with fields for the company name and its address:

Image:XPages series #11: Log data changes using beans and the DataObject interface


Here is the source code for the XPage. The important parts are marked in red:

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core">
        <xp:span style="font-size:18pt;text-decoration:underline">Company information</xp:span>
        <xp:br style="font-size:18pt"></xp:br>
        <xp:br></xp:br>
        <xp:span style="font-size:18pt">Company name:</xp:span>
        <xp:br></xp:br>
        <xp:inputText id="inputText1" style="width:295.0px;font-size:18pt"
                value="#{data.companyname}"></xp:inputText>
        <xp:br></xp:br>
        <xp:br></xp:br>
        <xp:span style="font-size:18pt">Address 1</xp:span>
        <xp:br style="font-size:18pt"></xp:br>
        <xp:inputText id="inputText2" style="width:295.0px;font-size:18pt"
                value="#{data.address1}"></xp:inputText>
        <xp:br></xp:br>
        <xp:br></xp:br>
        <xp:span style="font-size:18pt">Address 2
</xp:span>
        <xp:br></xp:br>
        <xp:inputText id="inputText3" style="width:295.0px;font-size:18pt"
                value="#{data.address2}"></xp:inputText>
        <xp:br></xp:br>
        <xp:br></xp:br>
        <xp:span style="font-size:18pt">
                City
        </xp:span>
        <xp:br style="font-size:18pt"></xp:br>
        <xp:inputText id="inputText4" style="width:295.0px;font-size:18pt"
                value="#{data.city}"></xp:inputText>
        <xp:br></xp:br>
        <xp:br></xp:br>
        <xp:button value="Save" id="saveButton" style="font-size:18pt">
                <xp:eventHandler event="onclick" submit="true"
                        refreshMode="complete">
                        <xp:this.action><![CDATA[#{javascript:actions["save"].execute();
context.redirectToPage("CompanyList");}]]></xp:this.action>
                </xp:eventHandler>
        </xp:button>
        <xp:button value="Cancel" id="cancelButton" style="font-size:18pt">
                <xp:eventHandler event="onclick" submit="true"
                        refreshMode="complete">
                        <xp:this.action><![CDATA[#{javascript:context.redirectToPage("Start");}]]></xp:this.action>
                </xp:eventHandler>
        </xp:button>
</xp:view>

We do not directly bind UI fields to document items (there is no document datasource declaration), but use a managed bean called "data" instead, hence the EL strings like #{data.companyname}.

Actually, the bean implementation does not handle the load/store operation itself, it just redirects the calls to another class that handles them. This redirection is used to be able to change the storage system later on, e.g. from NSF to SQL, without modifying anything in the XPages UI.

All we need to do is change a single managed property in the faces-config.xml file, which can be found in the WebContent/WEB-INF folder in the Java perspective of DDE:

<?xml version="1.0" encoding="UTF-8"?>
<faces-config>
        <managed-bean>
                <managed-bean-name>data</managed-bean-name>
                <managed-bean-class>com.ls11.uibackend.PageDataBean
                </managed-bean-class>
                <managed-bean-scope>request</managed-bean-scope>
               
                <managed-property>
                        <property-name>dataProviderClass</property-name>
                        <!-- Dummy implementation to simulate loaded data -->
                        <!--
                        <value>com.ls11.uibackend.dummy.DummyPageDataProvider</value>
                        -->

                        <!-- Implementation that loads/stores data in a Notes database -->
                        <!--
                        <value>com.ls11.uibackend.nsf.NSFPageDataProvider</value>
                         -->
                         
                        <value>com.ls11.uibackend.dummy.DummyPageDataProvider</value>
                </managed-property>
        </managed-bean>
        <managed-bean>
                <managed-bean-name>actions>
                <managed-bean-class>com.ls11.uibackend.PageActionsBean
                </managed-bean-class>
                <managed-bean-scope>request>
        </managed-bean>
  <!--AUTOGEN-START-BUILDER: Automatically generated by IBM Lotus Domino Designer. Do not modify.-->
  <!--AUTOGEN-END-BUILDER: End of automatically generated section-->
</faces-config>

As you can see, the database contains two implementations to load/store data:
The NSFPageDataProvider  reads the documentId query string parameter and uses the corresponding Notes document to get the data. DummyPageDataProvider is just a dummy implementation that creates placeholder text, which might be useful for the UI designer to already test the UI even though the database developer is still working on the NSF/SQL storage code.

Image:XPages series #11: Log data changes using beans and the DataObject interface
Company XPage powered by the DummyPageDataProvider



CompanyList.xsp
The CompanyList XPage contains a view control and displays the company documents in the database:

Image:XPages series #11: Log data changes using beans and the DataObject interface


ChangeLog.xsp
The main benefit of using managed beans for the UI bindings instead of directly binding fields to document datasource properties is that you can track, log, transform and prevent data changes in your code. During the page request, JSF calls the method setValue in our Java class with the item names and the new item values.

We can then compare the old and new values and create a log record if the value has been changed:

        public void setValue(Object id, Object newValue) {
                Object oldValue=getValue(id);
               
                boolean changed=(newValue==null & oldValue!=null) || (newValue!=null && !newValue.equals(oldValue));
                if (changed) {
                        m_changedValues.put(id.toString(), newValue==null ? null : newValue.toString());
                        logChangedData(id.toString(), oldValue, newValue);
                }
        }

        /**
         * Create a new change log entry for the modified field
         *
         * @param id field name
         * @param oldValue old value
         * @param newValue new value
         */
        private void logChangedData(String id, Object oldValue, Object newValue) {
                Session session=NotesContext.getCurrent().getCurrentSession();

                try {
                        DataChangeLogEntry newEntry=new DataChangeLogEntry(m_documentId==null || "".equals(m_documentId) ? "-New-" : m_documentId, session.getUserName(), new Date(), id, oldValue==null ? null : oldValue.toString(), newValue==null ? null : newValue.toString());
                        m_pendingLogEntries.add(newEntry);
                } catch (NotesException e) {
                        throw new FacesException(e);
                }
        }

When the Company XPage is saved, all pending log records are written to a permanent log list which is displayed in a data table on the ChangeLog XPage:

Image:XPages series #11: Log data changes using beans and the DataObject interface


A word about com.ibm.xsp.model.DataObject
If you take a deeper look at the code in the sample database, you might notice that the data bean implements the interface com.ibm.xsp.model.DataObject.
This interface is part of the XPages runtime (not part of the JSF standard!) and simplifies both writing beans and the EL syntax to access bean properties.

You will also find this interface in the session slides for BP212. Here is an excerpt from the slides:

       package com.acme.demo.persondata;

        import com.ibm.xsp.model.DataObject;

        public class PersonData implements DataObject {
                public Class getType(Object id) {
                        // Return the type of class that id resolves to.
                        // Complex case could return Employee or Customer
                        return Person.class;
                }

                public Object getValue(Object id) {
                        // Retrieve a record from some store, based on id
                        return null;
                }

                public boolean isReadOnly(Object id) {
                        // You are free to implement your own, or rely on your underlying data store
                        return false;
                }

                public void setValue(Object id, Object value) {
                        // Store value in your data store using id
                }
        }

Implementing the DataObject interface makes JSF call the setValue/getValue methods in your bean for any string behind the bean name in the EL string: #{data.companyname], #{data.address1}, #{data.address2}, #{data.whatever}.

You don't have to write getter and setter methods for every property!

Please note that this could already be done without the DataObject interface by implementing the java.util.Map interface (JSF then calls the method Map.get(Object) instead of DataObject.getValue(Object)), but this XPages runtime class makes the implementation much easier and cleaner.

That's it for today! Here is the download archive with the sample database:
ls11_uibackendsep.zip

]]> http://www.mindoo.com/web/blog.nsf/dxcomments/18.03.2011104725KLEDH8.htm http://www.mindoo.com/web/blog.nsf/dx/18.03.2011104725KLEDH8.htm?opendocument&comments The View article: XPath - A Flexible Solution for Filtering, Visualizing, and Binding XML Data in Your XPages Applications Thu, 3 Mar 2011 22:52:42 +0200 http://www.mindoo.com/web/blog.nsf/dx/03.03.2011225242KLETNJ.htm The View Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/03.03.2011225242KLETNJ.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/03.03.2011225242KLETNJ.htm The View has been published on their website. This article describes a way to use the XPath query language in XPages Expression Language, something I demonstrated in the Lotusphere session BP212.

Here is the public abstract:

XPath - A Flexible Solution for Filtering, Visualizing, and Binding XML Data in Your XPages Applications

By using the XPath query language, you can query XML data in an XPages application and then seamlessly bind the results to the application’s fields and repeater controls. This enables you to easily and dynamically build interfaces using resources in your application or display XML data in a data table. Learn the syntaxes to use and get a demonstration of how to implement them in this flexible, easy-to-maintain solution.
XPath, the XML path query language, is a powerful tool for selecting nodes and attributes from an XML document. Using XPath statements in XPages expression language, you can build dynamic XPages content based on an XML configuration document. No third-party library is required for this task. All the functionality you need is already available in IBM Lotus Notes and Domino 8.5.2, and all the information you need is here.

Image:The View article: XPath - A Flexible Solution for Filtering, Visualizing, and Binding XML Data in Your XPages Applications

Download link of sample application:
theview_xpath-el.zip

Full article (subscription required):
Article on eview.com ]]> http://www.mindoo.com/web/blog.nsf/dxcomments/03.03.2011225242KLETNJ.htm http://www.mindoo.com/web/blog.nsf/dx/03.03.2011225242KLETNJ.htm?opendocument&comments Slides from my two sessions at Entwicklercamp 2011 (German content) Mon, 28 Feb 2011 09:25:21 +0200 http://www.mindoo.com/web/blog.nsf/dx/28.02.2011092521KLEBVX.htm Entwicklercamp 2011 Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/28.02.2011092521KLEBVX.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/28.02.2011092521KLEBVX.htm two sessions at Entwickercamp 2011, Gelsenkirchen/Germany. As always, it was an interesting and very well organized event.
Image:Slides from my two sessions at Entwicklercamp 2011 (German content)
Session 1: Enhance XPages applications in the client with desktop features


Image:Slides from my two sessions at Entwicklercamp 2011 (German content)
Session 2: Leveraging the new Java APIs in IBM Lotus Notes 8.5.1/8.5.2
]]> http://www.mindoo.com/web/blog.nsf/dxcomments/28.02.2011092521KLEBVX.htm http://www.mindoo.com/web/blog.nsf/dx/28.02.2011092521KLEBVX.htm?opendocument&comments Speaking at Entwicklercamp conference 2011 with two sessions Mon, 7 Feb 2011 09:49:38 +0200 http://www.mindoo.com/web/blog.nsf/dx/07.02.2011094938KLECD3.htm Entwicklercamp 2011 Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/07.02.2011094938KLECD3.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/07.02.2011094938KLECD3.htm Entwicklercamp 2011 in Gelsenkirchen, Germany from 21st-23rd of February.

I will contribute two sessions to this great event with three days packed full with sessions for Notes/Domino developers. The first one is a rerun of an Entwicklercamp 2010 session with updated content, the second one is a completely new session.

German session descriptions:

Add-ons für Client/Designer auf Basis neuer Java APIs von Lotus Notes 8.5.1 und höher

Wir präsentieren in der Session, wie sich Lotus Notes Client und Domino Designer (DDE) durch die Verwendung neuer Java-Programmierschnittstellen von Lotus Notes 8.5.1 und höher erweitern lassen.
Nutzen Sie die neuen Java UI-Klassen, um Lösungen zu entwickeln, die mit bestehenden Domino-Anwendungen interagieren und diese erweitern, ohne dass das Design der Anwendungen geändert werden muss. Wir demonstrieren die Entwicklung solcher Erweiterungen in der Programmiersprache Java, gehen jedoch auch auf Wege ein, wie sich herkömmlicher Lotusscript-Code einbinden lässt.
Der Domino Designer wird durch Eclipse-Plugins erweitert, um eigene Designänderungen an ausgewählten Designelementen durchzuführen.
Die Session geht auf die verfügbaren APIs ein und gibt Tips&Tricks für Einsteiger. Eine Reihe von kleinen Beispielanwendungen steht im Abschluss an die Session zum Download zur Verfügung.

XPages-Anwendungen mit Desktop-Features ausstatten

Der IBM Lotus Notes Client stellt eine Fülle von Services und Features bereit, die in XPages-Anwendungen genutzt werden können, um mehr zu sein als nur lokale Web-Applikationen.
In dieser Session zeigen wir, wie XPages-Anwendungen durch lokale Desktop-Features wie Menü-Integration, Office-Automation, Interaktion mit klassischen NSF-Inhalten, Eclipse API-Aufrufe und mehr erweitert werden können, ohne jedoch die gesamte Anwendung plattformabhängig zu machen.
Die Session enthält eine Vielzahl an Demos und Tipps&Tricks, um die Benutzerakzeptanz Ihrer eigenen Anwendung zu erhöhen!

English session descriptions:

Leveraging the new Java APIs in IBM Lotus Notes 8.5.1/8.5.2

The session demonstrates how IBM Lotus Notes and Domino Designer on Eclipse (DDE) clients can be enhanced by using the Java programming interfaces of IBM Lotus Notes 8.5.1 and above. Leverage new features such as new Java UI classes to build solutions that interact with and enrich existing Lotus Notes client applications – without actually changing the application's design. We'll show you how to develop usable extensions in Java, and introduce ways to reuse existing IBM LotusScript code! DDE can also easily be enhanced with Eclipse plug-ins that do exciting things such as add custom design functionality. You'll learn through many code examples, and take away best practices for developers new to Java.

Enhance XPages applications in the client with desktop features

The IBM Lotus Notes client offers a rich set of services and features that XPages applications can leverage to be more than just local web applications. During this session, we'll demonstrate how local desktop features like menu integration, Office automation, interaction with legacy NSF contents, Eclipse API calls and more can greatly enhance Xpages applications without actually making the whole application platform dependent. Take away lots of demos and best practices to improve the usability of your own applications!
]]> http://www.mindoo.com/web/blog.nsf/dxcomments/07.02.2011094938KLECD3.htm http://www.mindoo.com/web/blog.nsf/dx/07.02.2011094938KLECD3.htm?opendocument&comments XPages series #10: Running JUnit tests on XPages code Mon, 7 Feb 2011 09:03:00 +0200 http://www.mindoo.com/web/blog.nsf/dx/05.02.2011131456KLEGDD.htm Lotusphere 2011 Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/05.02.2011131456KLEGDD.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/05.02.2011131456KLEGDD.htm BP212 - Deep Dive into IBM XPage Expression Language Syntax:
It shows how you can develop and test most of your XPages code outside of DDE.

So why should you development code outside of DDE?
1. Workaround for classloader issues
Well, you can experience a very nasty caching behaviour when you develop Java code in the Java perspective of DDE and test it live in an XPages application on the client. This might result in ClassCastExceptions like "com.company.packagename.MyClass cannot be cast to com.company.packagename.MyClass", which is caused by the JSF servlet: It still has a cached instance of the first bean in memory, created with one classloader while it tries to run the XPages application after code changes with a new classloader which is incompatible with the first one used.

This behaviour is discussed in an article in the Lotus Notes and Domino Application Development Wiki and IBM recommends to restart the http task in this case (only a viable option if you develop on a local Domino server), because this shuts down and restarts the JVM used to load the classes. Unfortunately this is not an option for development in the Notes Client, since you really don't want to restart your Notes Client/DDE every time you change code.
In addition to classloader issues, the JSF runtime also seems to cache Java classes and XPages intermittently, a problem that I faced just recently when I worked on the demos for BP212 and tested them in the client.

2. Version control for source code
Another reason why you should develop code outside of DDE is that you can use version control systems like Subversion or CVS. With 8.5.2 and an additional source control plugin from OpenNTF this aspect is less important than before, but I still prefer to have my code outside of a virtual file system that is stored in an NSF. I just don't trust that stuff and don't want to lose my code, e.g. if DDE does a rebuild on two developer machines and then they replicate with the same db instance on a server.

3. Use code in other Non-XPages projects
We do not only develop XPages apps, we develop libraries to be used in agents, standalone Eclipse RCP/Swing applications, server-side OSGi plugins and for many other areas. That's why we want to share most of the code between multiple development projects.

4. Run unit tests on the code for quality checks
To ensure that our code still produces the expected results after a code change, we use JUnit test cases on a manual or scheduled basis.

And that is exactly what this sample is about.

Workspace content
The download archive for this blog article contains two directories: a sample XPages application and an exported Eclipse workspace.

The XPages application consists of a simple XPage:

Image:XPages series #10: Running JUnit tests on XPages code



Here is the source code:

<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core">
       <xp:span style="font-size:20pt">Current User Information</xp:span>
       <xp:br></xp:br><xp:br></xp:br>
       <xp:span style="font-size:16pt">
               Common Name:
               <xp:br></xp:br>
               <xp:text escape="true" id="commonName" value="#{currentUser.commonName}"
                       style="color:rgb(0,128,255)"></xp:text>
               <xp:br></xp:br>
               <xp:br></xp:br>
               <xp:span style="font-size:16pt">Abbreviated Username:</xp:span>
               <xp:br></xp:br>
               <xp:text escape="true" id="abbrName" value="#{currentUser.abbreviatedName}"
                       style="color:rgb(0,128,255)"></xp:text>
       </xp:span>
</xp:view>

All it does is read the name information of the current user from a managed bean "currentUser" and display it on screen.
The managed bean class is not directly part of the NSF, but has been added in a "lib" folder below the WebContent/WEB-INF folder as a JAR file and then added to the classpath of the NSF project in DDE (after that, it's not visible in the "lib" folder anymore, but below "Referenced Libraries").

After you import the projects of the "workspace" directory into your personal Eclipse workspace, it should look like this:

Image:XPages series #10: Running JUnit tests on XPages code


Please note that you might get compile errors and need to change the path of the referenced "Notes.jar" file in the com.ls11.dominohelper project to fit your Notes configuration.

There are four projects in the workspace:

com.ls11.dominohelper
  • The project contains helper classes to execute a piece of code in the context of a Notes session (see below for details)
com.ls11.externalcodesample
  • This project contains the bean implementation to read the current username
com.ls11.externalcodesample.build
  • We use the ANT script of this project to create the JAR file with the necessary content to work in an XPages app
com.ls11.externalcodesample.test
  • Finally this project contains a JUnit test case to check that our bean is working correctly

Running the same code standalone and from an XPages request
Let's dive into the class CurrentUserInfo.java. The method .getCommonName() looks like this:

/**
* Returns the common name of the current user
*
* @return common name
*/
public String getCommonName() {
       if (m_commonName==null) {
               IDominoCallable<String> callable=new IDominoCallable<String>() {

                       @Override
                       public String call(Session session) throws NotesException {
                               Name currName=session.createName(session.getUserName());
                               String commonName=currName.getCommon();
                               currName.recycle();
                                       
                               return commonName;
                       }
               };
                       
               try {
                       m_commonName=DominoExecution.run(callable).get();
               } catch (InterruptedException e) {
                       throw new RuntimeException("Could not read username info", e);
               } catch (ExecutionException e) {
                       throw new RuntimeException("Could not read username info", e);
               }        
       }
       return m_commonName;
}

Now that looks interesting: We are using an IDominoCallable implementation to retrieve the common name part of the username.
We do this in order to run the same code from an XPages environment and from a standalone application. In the DominoExecution there's some magic that detects whether it gets executed from XPages code or not.

In standalone mode, DominoExecution currently launches a new NotesThread for every call, but that could easily be changed to have a permanent thread and just feed it with the IDominoCallable's one after another.

The run method of DominoExecution returns a Future object of Java's concurrency framework. Calling its .get() method let's the executing code wait for the result to be computed.
In XPages mode, the IDominoCallable is executed synchronously, because the code is already running in a Domino enabled thread.

Running JUnit in Eclipse
The project com.ls11.externalcodesample.test contains a JUnit test case that checks whether the methods of the bean are working correctly: For example it ensures that abbreviated name and common name are both not null and compares the first part of the abbreviated name with the common name.

/**
* Check that common and abbreviated name
*/
public void testCommonNameAndAbbrNameConsistent() {
       String commonName=m_currUserInfo.getCommonName();
       String abbrName=m_currUserInfo.getAbbreviatedName();
       
       //the values cannot be null
       assertNotNull(commonName);
       assertNotNull(abbrName);
               
       int iPos=abbrName.indexOf("/");
       assertTrue((iPos ! = - 1) && (iPos ! = 0) );
               
       String firstAbbrNamePart=abbrName.substring(0, iPos);
       assertEquals(commonName, firstAbbrNamePart);
}

You can launch the test case by first creating a JUnit run configuration:

Image:XPages series #10: Running JUnit tests on XPages code


On the Environment tab, please add a variable "PATH" that contains the path to your Lotus Notes program directory:

Image:XPages series #10: Running JUnit tests on XPages code


Now you can execute the test case and get the following result:

Image:XPages series #10: Running JUnit tests on XPages code


The method testCommonNameAndAbbrNameConsistent could be executed without errors, but calling testOrganisation led to an error.
It checks that CurrentUserInfoTest.getOrganisation() is not null, but unfortunately, it seems like we forgot the method's implementation:

public String getOrganisation() {
       // TODO Auto-generated method stub
       return null;
}

After fixing that, the JUnit succeeds:

Image:XPages series #10: Running JUnit tests on XPages code



Now that we know that the bean works, we can package everything by right clicking on the ANT script of project com.ls11.externalcodesample.build, choose "Run As/Ant Build", then do a refresh on the "lib" folder and replace the JAR in the XPages application database with the new one.

That's it!

Here is the download link for the archive:
LS11-BP212-JUnit.zip ]]> http://www.mindoo.com/web/blog.nsf/dxcomments/05.02.2011131456KLEGDD.htm http://www.mindoo.com/web/blog.nsf/dx/05.02.2011131456KLEGDD.htm?opendocument&comments LS11: Slides for session BP203 - Leverage the New Java APIs in IBM Lotus Notes 8.5.1 and 8.5.2! Sun, 6 Feb 2011 17:17:25 +0200 http://www.mindoo.com/web/blog.nsf/dx/06.02.2011171725KLEM4Y.htm Lotusphere 2011 Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/06.02.2011171725KLEM4Y.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/06.02.2011171725KLEM4Y.htm
Image:LS11: Slides for session BP203 - Leverage the New Java APIs in IBM Lotus Notes 8.5.1 and 8.5.2!


Compared to the slides posted on the Lotusphere website, we changed the order of the slides (DDE part in the middle, not at the end) and changed a few demos. ]]> http://www.mindoo.com/web/blog.nsf/dxcomments/06.02.2011171725KLEM4Y.htm http://www.mindoo.com/web/blog.nsf/dx/06.02.2011171725KLEM4Y.htm?opendocument&comments LS11: Slides for session BP212 - Deep Dive into IBM XPage Expression Language Syntax Sat, 5 Feb 2011 10:59:17 +0200 http://www.mindoo.com/web/blog.nsf/dx/05.02.2011105917KLEDQM.htm Lotusphere 2011 Karsten Lehmann http://www.mindoo.com/web/blog.nsf/dx/05.02.2011105917KLEDQM.htm?opendocument&comments http://www.mindoo.com/web/blog.nsf/dx/05.02.2011105917KLEDQM.htm
Image:LS11: Slides for session BP212 - Deep Dive into IBM XPage Expression Language Syntax


I will provide downloads for the three demos that I did at the end of the session and write blog articles about them as part of the XPages blog series.

I'm trying to do this as soon as possible, but I'm already preparing slides for the next conference:

Entwicklercamp 2011 in Gelsenkirchen, Germany
]]> http://www.mindoo.com/web/blog.nsf/dxcomments/05.02.2011105917KLEDQM.htm http://www.mindoo.com/web/blog.nsf/dx/05.02.2011105917KLEDQM.htm?opendocument&comments