How to create patches for OSGi bundles

The standard practice that was followed with respect to providing patches (which contain bug fixes) for WSO2 Java products was to create a JAR file containing the fixes, which contains only the classes that have been changed, and placing this patch file, say foo-patch.jar, in the $PRODUCT_HOME/lib/patches directory. When the classpath is constructed, the $PRODUCT_HOME/lib/patches directory is placed in front of the $PRODUCT_HOME/lib/ directory, hence the patched classes take precedence. However, this mechanism does not work on the newer generation of Java products, which are based on OSGi. Hence we had to figure out how to patch OSGi bundles. The obvious answer one can think of is, 'Ship a new version of the bundle'. However, this is not sometimes feasible due to several reasons.

1. The patch may be large (it is the entire bundle that is shipped), even if only a single class changed.
2. Some other bundles may depend on a fixed version of the packages exported by the bundle that is being patched. Hence, we will have to ship new versions of the affected bundles as well, which can lead to a chain reaction. In the worst case, all the bundles may have to be shipped due to a change to a single class in a core bundle!
3. If the dependent bundles import packages in a range of versions, such as [1.0,2.0), the importer will be wired to the bundle which exports the package with the highest version. In this case, shipping a new version of the bundle may work, provided that backwards compatibility is satisfied. However, this may not be always possible.

We can provide a simple solution to this problem using 2 OSGi constructs; require-bundle & fragments. Figure 1 shows that the required bundles take precedence over the bundle's classpath. Hence we can use required bundles to override classes in the bundle.

Figure 1: Class loading flowchart. Source: OSGi R4 spec

However, at the time of shipping a bundle, we cannot know what patch bundles will be required. Hence we cannot put a Require-Bundle manifest header pointing to the patch, in the original bundle. This is where fragments come into play. The entries in the fragments' manifest are merged into that of the fragment host. Hence, we can create a fragment which will require the patch bundle. In effect we have managed to dynamically add a Require-Bundle header to the original host bundle. However, we should ensure that the original bundle is the primary exporter of the patched packages. This can be done by specifying a mandatory attribute. We need to add this to the Export-Package header of the fragment;
e.g. Export-Package:*;partial=true;mandatory:=partial
Once the fragment is attached to the host & the host is refreshed, we will be able to see the functionality in the patch taking effect.

If you ever require to revert the patch, it is just a matter of removing the fragment & patch bundles, and refreshing the main bundle.

I have created a PoC project which demonstrates this concept. You can download the code and try it out. The main bundle is the bundle that needs to be patched. The patch bundle is the required bundle & the fragment bundle attaches to its host, the main bundle. The Knopflerfish desktop tool can be used to test this out.

NOTE: Don't confuse this with bytecode level patching at runtime. That is a different aspect. I've written another post on how that can be done.

Converting legacy code into OSGi bundles

Any person who starts to do something serious with OSGi soon hits a wall; issues with legacy code such as;

1. Calls to System.exit()
2. Starting up using main(String[]) method
3. Using the Thread Context Classloader (TCCL)

Due to such issues, it seems like it is next to impossible to make use of legacy code without code changes & recompiling. However, this is impractical since

1. The source may not be available
2. Modification of source code may violate copyright
3. It may be practically impossible to find & fix all instances of violations

This is one of the major issues we faced while making the functionality in the WSO2 Java products into OSGi compatible Carbon components.

The Knopflerfish OSGi framework has come up with a solution to address these concerns; bytecode level patching at runtime. The violations are located & patched at runtime. There is a very good presentation by OSGi veterans Gunnar Ekolin & Erik Wistrand from Makewave titled 'Everything Can be a Bundle', which is a MUST READ for any OSGi newbie & even for people who have been working with OSGi for sometime.

Debug OSGi wiring issues with Structure101

If you have worked with OSGi, you would know that debugging problems related to bundle wiring can be a tedious task. The next generations of the Java family of products from WSO2 focuses on improved modularity. OSGi was the inevitable choice for us. The next WSO2 Data Services release is completely based on OSGi. During the development, we came across OSGi wiring related issues from time to time, and Structure101 seems to be a useful tool for debugging such issues. Structure101's rich visual environment exposes your software structure which helps you to understand structure, control complexity and define architecture. The latest version of Structure101 allows users to view OSGi bundle wiring.

Here's what you need to do.

1. Download Structure101g & the example flavors from http://www.headwaysoftware.com/downloads/structure101/g.php

2. Extract structure101_gen_134.tar.gz (or zip) & structure101g-example-flavors.zip & copy the "flavors" directory into the "structure101/s101g" directory.

3. Request for an evaluation license. This will be emailed to you.

4. Copy the license file into the "structure101-3/s101g" directory

5. Start structure101 by running the "structure101g" script

6. Select "Load on Demand" and select com.headway.examples.osgi from the drop-down list.

7. Provide the bundles directory. This is the directory in which your OSGi bundles are located. In the case of the WSO2 Data Services project on my machine, the location is /home/azeez/projects/wso2/trunk/solutions/data-services/java/modules/distribution/target/wso2-dataservices-1.0-beta10/webapps/ROOT/WEB-INF/plugins. Make sure that you click on the "Include Eclipse extensions" cell even if you do not intend to change the corresponding parameter value. Otherwise, the value you entered for the "Bundles directory" will not be taken. This is due to an inherent problem in JTables in Java Swing MVC. Click on "Finish"

8. You will now be able to view the OSGi bundle wiring. The diagram below shows the OSGi bundle wiring for WSO2 Data Services components. You can navigate to various bundles & wirings and view their respective properties.

To learn more about Structure101, see this demo

Axis2 services & modules within OSGi Containers

Saminda has written a comprehensive document on deploying Axis2 Web services & Axis2 modules within an OSGi container. The article describes how to deploye Axis2 services & modules within the popular Eclipse Equinox OSGi container. This is a must read for anyone intending to deploy Axis2 modules & services within an OSGi environment.

See Axis2 Services & Modules in OSGi containers

Server-side OSGi in WSO2 Products

Here is a good presentation about server-side OSGi on InfoQ. At WSO2, we are in the process of migrating our products to OSGi. We are building a base platform called Carbon, on top of which, all other products and solutions will be built. The following diagram nicely demonstrates how Carbon is positioned in our product & solutions stack.

The features that can be shared across products and solutions are built as Carbon components. The real power of OSGi comes from the fact that these components, and even multiple versions of these components and other libraries, can be be combined in different ways to build a desired set of functionalities in order to satisfy varying user requirements.

Just heard about OSGi, and no idea what it is? Then you should read OSGi for Beginners