XPages series #15: Free FTP server on top of Domino’s OSGi framework
Karsten Lehmann 3 November 2012 14:14:02
Last week we had a conference call with IBM. They provide free VMware images of several IBM products for business partners to be used for demo and development purpose, including IBM Connections 4, already set up for the well known Renovations company.Those images very pretty large, with several GB's of data and the fastest and easiest way to get our hands on the images was to set up an FTP server and have the IBM'ers upload the files.
That was a good occasion to build a small proof-of-concept that I had already thought of 1-2 years ago:
An FTP server running on top of Domino
The implementation is actually quite easy: We simply take an existing FTP server and integrate it into the OSGi framework of the Domino server as a plug-in.
I selected the Apache FtpServer for this purpose, because it's available as open source with Apache License and easy to set up.
The first task was to find all necessary libraries to run the server. The FtpServer project is available via Apache's Maven repositories. Maven is a powerful build system that takes care of all code dependencies.
Although Maven can often be a real PITA, creating a build means basically just to download the source code and use the command
mvn clean install or
mvn clean install -DskipTests The option "-DskipTests" skips the automatic test case execution during the build process and reduces build time.
As tweeted a few months ago, there is a way to tweak Maven's pom.xml file to let it generate all required Jar files and copy them to a target directory.
You just need to add this snippet to the pom.xml file:
<
plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<phase>install</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/lib</outputDirectory>
</configuration>
</execution>
</executions>
</plugin> I then copied all generated Jar files into a new plug-in:
and exported the relevant packages in the plug-in's manifest file:
Next, I created a plug-in "com.mindoo.ftp" that uses the code from "org.apache.ftpserver" and creates the actual FTP server instance.
It reads the server configuration and user account data from a small Notes database and writes log documents into another one.
For now, the FTP server reads and writes its files from disk and not from document attachments in NSF databases, although that would be technically possible and might be an option for the future.
I decided not to use NSF for the files, because a) this is just a proof-of-concept and b) the FTP server should be able to resume uploaded and downloaded files. The standard Notes Java API does not provide any real streaming functionality (EmbeddedObject.getInputStream() internally just writes the file to disk first, which would kill performance).
Streaming of attachments out of NSF is possible via NSFNoteExtractWithCallback in the C API (we used this in the MindPlan application to let the user cancel attachment extraction) and there may be a similar method to import attachments via stream.
Server Configuration
Our configuration database looks like this - nothing fancy yet:
I added fields for the most relevant server settings to the configuration form, e.g. the server's TCP/IP port, a blacklist to block IP addresses and subnets as well as parameters to use the server behind a NAT router (I tried running it from inside a VMware image).
For users, we can configure their home directory, set a password (we only save a one-way hash generated by @Password) and set a few options to limit the user's access to the server, e.g. define an access period and remove write permissions.
Running the server
The FTP server plugins should be deployed into the Domino server's HTTP task OSGi framework and start when the HTTP server gets started.
Unfortunately, in 8.5.3 this is not that easy. There is no such extension point that can be used to automatically start a plug-in on HTTP task startup, like org.eclipse.ui.startup for RCP applications (we have an open feature request for 8.5.4). Everything is started lazily when it is used for the first time. Bad for a server that needs to listen for incoming connections on a TCP/IP port.
However, the XPages runtime provides one extension point that we can use:
<plugin>
<extension
point="com.ibm.commons.Extension">
<service
class="com.mindoo.ftp.startup.StartupHandler"
type="com.ibm.designer.runtime.extensions.RuntimeInitializationEvent">
</service>
</extension>
</plugin> This way, our plugin gets started the first time an XPage is called. That's a first step, but we need to automate this XPage invocation somehow, to be sure that the server is running after the HTTP task or the whole server is restarted.
Fortunately, tweeting about this issue helped. Thanks to René Winkelmeyer, I could come up with a solution that leverages the XPages preload feature of Domino 8.5.3 to automatically start the FTP server plugin.
All you have to do is add the following lines to the server's Notes.ini:
XPagesPreload=1
XPagesPreloadDB=mindoo-ftp-config.nsf/ftplauncher.xsp "ftplauncher.xsp" is a small dummy XPage in our configuration database. You can use any other XPage instead, since the purpose is just to start the OSGi framework.
DOTS
There is an alternative OSGi implementation that can run as part of the Domino server installation: DOTS, also known as Domino Tasklet Service for IBM Lotus Domino.
In constrast to the HTTP OSGi framework, DOTS contains an extension point to launch plug-ins on framework startup.
I did not choose DOTS to run the FTP server, because as of 8.5.3, DOTS is not part of the Domino server and needs to be deployed manually. In addition, by using the HTTP OSGi framework instead of DOTS, we can directly call plug-in methods from XPages applications, which might be useful for some applications.
Server console commands
I added three server console commands to control the FTP server from the server console:
tell http osgi mftp restart - stops and starts the server
tell http osgi mftp stop - stops the server
tell http osgi mftp start - starts the server And of course, the FTP server gets restarted on
restart task http which restarts the whole HTTP task.
Adding and modifying users does not require a server restart. The lists of blocked IP's and subnets in the server configurations does also get automatically re-read when somebody tries to log into the server.
All other options currently require a restart of the FTP server.
Free as in beer
This proof-of-concept is made available for free. I am sure there are other cool use cases for OSGi based background tasks in Domino. In particular, a scheduler plug-in might be a good thing.
Setup instructions
The download archive contains an update site with the plugin code that needs to be deployed onto the Domino server just like the XPages Extension Library : create an NSF update site database, import the plug-ins and features, configure the Notes.ini variable OSGI_HTTP_DYNAMIC_BUNDLES and restart the HTTP task.
The default paths for the configuration database is "mindoo-ftp-config.nsf" and "mindoo-ftp-config-log.nsf" for the log database.
An alternative configuration database path can be configured via the Notes.ini variable "$mndFTPConfig=configdb.nsf". For the log database, the variable is called "$mndFTPLog=logdb.nsf".
And finally, the base directory of the FTP server can be set via "$mndFTPBaseDir=c:\temp\mftp".
Hope this tool is useful!
Here is the download link:
mftp.zip
- Comments [11]