Minerva Database Pools 0.99

This is the documentation for the Minerva pools. It is aimed at both developers and users, so be selective. I suggest everyone read the section on JDBC vs the JDBC Standard Extension, since a lot of the rest may be pretty muddled without it. You don't need a very thorough understanding to use the JDBC Standard Extension pools - all the tricky stuff is done under the covers. Just be aware of the advantages and limitations.



Introduction to Minerva

The Minerva library is a multi-layered set of classes that can manage pools of Java objects. At the lowest level, this can be any type of Java object. At a higher level, it provides implementations for database connections. It is part of the jBoss project (an open-source J2EE server), but does not depend on jBoss - it can be used independently in any Java 2 environment. You will need the JDBC 2 Standard Extension, JNDI, and JTA to take full advantage of the J2EE JDBC features, but you can ignore all that if you just want a simple object or database pool (you may still need the jdbc 2 standard extension and JNDI jars at runtime, but you won't have to use them directly).

The rest of this document is devoted to the different uses of the Minerva pools - J2EE connections, plain JDBC connections, objects, etc. There's a section on integrating the Minerva pools with jBoss, which you should read if you're using jBoss, and skip if you're not.



JDBC 1/2 vs the JDBC Standard Extension

Concepts

JDBC 1 was part of Java 1, and JDBC 2 is part of Java 2. So most of you probably have a pretty clear understanding of JDBC. However, the JDBC 2 Standard Extension brings a whole new set of concepts to the table.

In "normal" JDBC, there's a very tight coupling between transactions and connections. Every transaction is attached to exactly one connection (even if auto-commit is on - whereupon they're just very short transactions). Though there may be many transactions over the life of one connection, the reverse is not true - there cannot be more than one connection associated with a single transaction. As soon as you create a new connection, you've created a new transaction too. Additionally, you cannot start a new transaction on a connection until the last one is complete. There is no concept of more than one set of "pending work" for a connection. Once you start doing things, the only way to get a new transaction on the same connection is to commit or rollback, and at that time a new transaction is automatically started.

To quote a little green guy, "you must un-learn what you have learned" in order to work with the JDBC 2 Standard Extension. Here, connections and transactions are completely independent. You may use many connections in the scope of a single transaction, and you may do work on many uncommitted transactions with a single connection. A connection is just a way for you to communicate with the database - it's up to you and the database to decide what you're talking about (which transaction, etc).

The primary advantage of this is that you can do work across many databases, or indeed many data sources, and commit it all together or roll it back all together. One way to do this is known as "two-phase commit." In the first phase, you ask all your data sources whether they can commit, and then in the second phase, based on the responses, you tell everything to either commit or rollback. And it may be that some of these data sources are legacy systems, message queues, or something else altogether. Another advantage is that it makes it easy and efficient to pool database connections - the connection can be returned to the pool immediately, even if it will be a while before the work is committed.

So a J2EE server needs to support the JDBC standard extension, diverse data sources, and transaction management. If you see me refer to a "transactional" or "XA" connection, I'm talking about a JDBC 2 Standard Extension connection, which can participate in JTA transactions. JDBC 1 or JDBC 2 connections can't, since they don't support two-phase commit, and they can't separate a connection from a transaction.

Reality

Unfortunately, most database vendors do not yet support the JDBC Standard Extension. And those that do generally don't have very robust support yet. This presents a problem, since most J2EE users want to use databases.

The solution to this is a set of wrapper classes for plain JDBC 1 or JDBC 2 connections. This allows them to participate in JTA transactions, though in a somewhat limited fashion. First, you can only use one connection per data source per transaction. Second, if the connection cannot commit, it won't tell you during the first inquiry phase when it's supposed to - it will fail during the commit phase, which may mean some data sources get committed and others rolled back. That's clearly undesirable, which is why we want all vendors to provide good JDBC 2 Standard Extension implementation. Take this moment you bother your vendor's support line. Third, a connection cannot be closed or returned to a connection pool untle the pending work is either committed or rolled back - you wouldn't want someone else adding work to the transaction before you commit it!



Using Minerva JDBC 1/2 Pools

In order to create normal JDBC pools, you should interact with the class org.jboss.minerva.datasource.JDBCPoolDataSource. It's pretty straightforward - create it, set all the JDBC properties (URL, user, password, etc.) and pool properties (name, min size, max size, whether it blocks, shrinks, etc.) that you need, and then initialize() it. It implements the DataSource interface from the JDBC 2 Standard Extension, so you can just call getConnection() to get a connection. When you're done with the connection, close() it, and it will get returned to the pool. When you're done with the pool, call close() on the data source and it will close all the connections in the pool and shut everything down.

If you want to make your pools more accessible, you can do that in two ways. The first is to call setJNDIName to register the pool in JNDI. The second is to use the JDBC driver org.jboss.minerva.datasource.PoolDriver and a URL of the form "jdbc:minerva:PoolName". In this way, you get pooled database connections in the same way you'd normally get connections - DriverManager.getConnection(url). There's no need to supply a user, password, or any properties - all that is set by the pool itself.

Finally, if you want to interact with the pool directly rather than using the DataSource or DriverManager interfaces, you can use the Object pools with the object factory org.jboss.minerva.factories.JDBCConnectionFactory. Take a look at the source code for JDBCPoolDataSource if it's not clear.



Using Minerva JDBC 2 Standard Extension Pools

In order to create JDBC 2 Standard Extension pools, you should interact with the class org.jboss.minerva.datasource.XAPoolDataSource. This also implements the DataSource interface. All the transactional-ness is handled under the covers. You'll need to set some additional properties, but you will still get a normal java.sql.Connection, and close it when you're done to return it to the pool. You don't have to know anything about the inner mechanics of transactions or It's pretty straightforward - create it, set all the JDBC properties (URL, user, password, etc.) and pool properties (name, min size, max size, whether it blocks, shrinks, etc.) that you need, and then initialize() it. It implements the DataSource interface from the JDBC 2 Standard Extension, so you can just call getConnection() to get a connection. When you're done with the connection, close() it, and it will get returned to the pool. When you're done with the pool, call close() on the data source and it will close all the connections in the pool and shut everything down.

If you want to make your pools more accessible, you can do that in two ways. The first is to call setJNDIName to register the pool in JNDI. The second is to use the JDBC driver org.jboss.minerva.datasource.PoolDriver and a URL of the form "jdbc:minerva:PoolName". In this way, you get pooled database connections in the same way you'd normally get connections - DriverManager.getConnection(url). There's no need to supply a user, password, or any properties - all that is set by the pool itself.

Finally, if you want to interact with the pool directly rather than using the DataSource or DriverManager interfaces, you can use the Object pools with the object factory org.jboss.minerva.factories.XAConnectionFactory. Take a look at the source code for XAPoolDataSource if it's not clear.



Using Minerva JDBC Pools with jBoss 2.0

There are several steps required to use Minerva pools with jBoss 2.0:

  1. Add your database driver to jboss.properties
    Add or update a line with the driver class names (comma-separated) like:
    jdbc.drivers=oracle.jdbc.driver.OracleDriver
  2. Add a Minerva pool entry to jboss.conf
    See the example below for Standard Extension drivers or JDBC 1/2 drivers.
  3. Add the pool's specific configuration to jboss.jcml
    See the example below for Standard Extension drivers or JDBC 1/2 drivers.
  4. Add a resource reference to your EJB's ejb-jar.xml (not jBoss specific)
    See the example below.
  5. Add a resource manager to your EJB's jboss.xml
    See the example below.

You can use more than one Minerva pool at the same time, and even from the same bean. You'd have to repeat all the steps for each one, unless they use the same drivers (only add it once to jboss.properties). Once you have a pool set up, you can refer to it from additional beans by repeating the last 2 steps for each bean.



Using Minerva JDBC Pools with Other J2EE Containers

The only real requirement here is that the TransactionManager for the container must be in JNDI, so the pools can find it and register connections as they are used. You'll need to find out how to start a service for the container, and probably provide a skeletal class that lets the user set all the configuration parameters. You can use the JMX classes to load Minerva pools in jBoss as a reference (see org.jboss.jdbc.*). There would be a different way to link an EJB DataSource reference to the actual DataSource implementation, but the container vendor probably has a GUI tool for deployment like EJX.



Using Minerva Object pools

If you want to pool something other than JDBC connections, you can do that with the classes org.jboss.minerva.pools.ObjectPool and org.jboss.minerva.pool.PoolObjectFactory. ObjectPool has all the logic, so you just need to implement a custom subclass of PoolObjectFactory. The only method you need to implement is createObject(), though you can customize the object handling more than that. You can manipulate each object as it is created, destroyed, given out by the pool, returned to the pool, etc. You can even make the pool give out objects other than the actual instances in the pool, as long as there's a one-to-one mapping (some kind of wrapper, etc.).

You can use any kind of Object for a custom pool. However, you may choose to use objects that implement org.minerva.jboss.pools.PooledObject. If you do this, the pool will handle close and error events generated by your objects, so you would not need to explicitly return objects to the pool. It will also handle last-used events to automatically track the usage of pooled object for garbage-collection purposes.

There are several example below of Object Pool usage, but I encourage you to visit the JavaDoc for ObjectPool, PoolObjectFactory, and PooledObject.



Minerva Architecture

All the Minerva classes have JavaDoc comments, and you can see the JavaDoc output online on the jBoss JAWS page.

Overview

There are five packages under minerva:

poolsPool implementation
jdbcSupport classes for JDBC pools
factoriesDefault object factories for JDBC 1/2 and XA connections
datasourceClient interface for JDBC 1/2 and XA pools
xaWrapper classes for JDBC 1/2 drivers in an XA environment

So, you'll want to look in a different place depending on what you want to do. If you want to change the way to pools work, look in pools (specifically, ObjectPool). If you want to create an implementation to pool something other than database connections, add a PoolObjectFactory to factories. If you want to change the timestamping or other characteristics of the connections that are given out from the pool, look in jdbc. The logic around XA connections is split between xa (for the JDBC 1/2 wrappers) and the XA factory in factories (which registers with the current transaction, etc.).

Object Pools

At heart, Minerva is a pooling scheme for any type of objects. The problem was, JDBC 1/2 pools need to pool java.sql.Connections, while XA pools need to pool javax.sql.XAConnections. Even though in the end, you get a Connection either way. (See the JDBC 2 Standard Extension document section 7.3.1)

The easiest way to implement this is as a generic Object pool. So there's a class that can pool anything, and an abstract base class for an Object Factory that creates (and destroys) the object for the pool. The factory turns out to be fairly complex if you override all the methods -- you don't have to actually return the exact object in the pool, you can process things going in and out, etc. But for a basic implementation, all you have to do is override createObject and return whatever kind of objects you want to pool.

There's also an interface (PooledObject) that an object in the pool can implement to activate some additional functionality. For example, when a PooledObject sends a close or error event, the pool will automatically take the correct action. Otherwise, you have to return things to the pool by hand, manually update last-used times, etc. You can do the same things either way, it's just a matter of what works best for the specific objects you're pooling.

So to create a pool, you need the class of the instances you're going to pool, and a factory to create those instances. Plug the factory in to the ObjectPool, set some parameters (size, etc.), and voila! You have a pool.

Pool Parameters

I was about to rewrite descriptions of all the pool parameters, but it's already in the JavaDoc for ObjectPool. Just look at all the setters.

JDBC Helpers

The JDBC class has wrapper classes that manage things like: returning a connection to the pool when it's closed, updating the last-used timestamp when a connection or child object is used, sending notifications when a SQLException crops up, etc. It has been suggested that these could be proxies instead of full-blown classes, but there are a fair number of methods overridden...

JDBC 1/2 Wrappers for XAConnections

The xa package holds the things that make a JDBC 1/2 connection look like an XAConnection. There are implementations of all the XA classes, hooks into the JDBC wrappers mentioned above, and so on. However, note that some of the logic dealing directly with pools lives in the pooled object factory for XAConnections.

Basically, each java.sql.Connection gets wrapped by an XAConnection and XAResource, which stay with it for as long as it's in the pool. They make one-off java.sql.Connection instances each time the client requests one, which die when they are closed by the client (so the client can't hook into a connection after closing it by just stashing the reference somewhere). The factory registers the XAResource with the TransactionManager when it's given out from the pool, and deregisters it when it's returned. The XAResource explodes if you try to use it for a new transaction before the old one has been committed or rolled back (since the underlying connections don't support that).



Minerva Examples

Creating a JDBC Pool

JDBCPoolDataSource source = new JDBCPoolDataSource();
source.setPoolName(poolName);
source.setJDBCURL(url);
source.setJDBCUser(username);
source.setJDBCPassword(password);
source.setMinSize(minSize);
source.setMaxSize(maxSize);
source.initialize();
    

Getting a JDBC Pool connection directly

JDBCPoolDataSource source = ...
Connection con = source.getConnection();
    

Getting a JDBC Pool connection from JNDI

JDBCPoolDataSource source = ...
source.setJNDIName("jdbc/pool-"+source.getPoolName());
source.initialize();
...
DataSource ds = (DataSource)context.lookup("jdbc/pool-PoolName");
Connection con = ds.getConnection();
    

Getting a JDBC Pool connection from the DriverManager

JDBCPoolDataSource source = ...
Connection con = DriverManager.getConnection("jdbc:minerva:PoolName");
    

Creating a JDBC Standard Extension (S.E.) Pool

XADataSource vendorSource = ... // Could use org.jboss.minerva.xa.XADataSourceImpl
XAPoolDataSource = new XAPoolDataSource();
source.setPoolName(poolName);
source.setDataSource(vendorSource);
source.setJDBCUser(username);
source.setJDBCPassword(password);
source.setMinSize(minSize.intValue());
source.setMaxSize(maxSize.intValue());
source.setTransactionManagerJNDIName("TransactionManager");
source.initialize();                 //  ^^^  Name depends on your container
    

Getting a JDBC S.E. Pool connection directly

XAPoolDataSource = ...
Connection con = source.getConnection();
    

Getting a JDBC S.E. Pool connection from JNDI

XAPoolDataSource = ...
source.setJNDIName("jdbc/xapool-"+source.getPoolName());
source.initialize();
...
DataSource ds = (DataSource)context.lookup("jdbc/xapool-PoolName");
Connection con = ds.getConnection();
    

Getting a JDBC S.E. Pool connection from the DriverManager

XAPoolDataSource = ...
Connection con = DriverManager.getConnection("jdbc:minerva:PoolName");
    

Configuring a JDBC S.E. Pool with jBoss and a JDBC S.E. Driver

This will result in the pool's DataSource living in JNDI under the name PoolName.

In jboss.conf:

<MLET CODE="org.jboss.jdbc.XADataSourceLoader" ARCHIVE="jboss.jar,vendor.jar" CODEBASE="../lib/ext/">
    <ARG TYPE="java.lang.String" VALUE="PoolName">
    <ARG TYPE="java.lang.String" VALUE="VendorXADataSourceClassName">
</MLET>
    

In jboss.jcml:

<mbean name="DefaultDomain:service=XADataSource,name=PoolName">
    <attribute name="URL">jdbc:VendorURL</attribute>
    <attribute name="JDBCUser">user</attribute>
    <attribute name="Password">password</attribute>
</mbean>
    

You'll notice that after you run jBoss with a jboss.jcml like the one above, it will update your jboss.jcml to list all the available properties for the XADataSource, with their default values. If the vendor uses properties other than URL, username, and password to configure their XADataSource, you'll have to use the "properties" attribute to list them (in the form "name=value;name=value" etc.).

Configuring a JDBC S.E. Pool with jBoss and a JDBC 1/2 Driver

This will result in the pool's DataSource living in JNDI under the name PoolName.

In jboss.conf:

<MLET CODE="org.jboss.jdbc.XADataSourceLoader" ARCHIVE="jboss.jar,vendor.jar" CODEBASE="../lib/ext/">
    <ARG TYPE="java.lang.String" VALUE="PoolName">
    <ARG TYPE="java.lang.String" VALUE="org.jboss.minerva.xa.XADataSourceImpl">
</MLET>
    

In jboss.jcml:

<mbean name="DefaultDomain:service=XADataSource,name=PoolName">
    <attribute name="URL">jdbc:JDBC1/2URL</attribute>
    <attribute name="JDBCUser">user</attribute>
    <attribute name="Password">password</attribute>
</mbean>
    

You'll notice that after you run jBoss with a jboss.jcml like the one above, it will update your jboss.jcml to list all the available properties for the XADataSource, with their default values. If the JDBC 1/2 driver uses properties other than URL, username, and password to connect, you'll have to use the "properties" attribute to list them (in the form "name=value;name=value" etc.).

Configuring an Enterprise Java Bean with a DataSource

You need to add a resource reference to your ejb-jar.xml file. The requirement and XML syntax is standard across all containers - but each container has a different GUI for configuring this if you don't do it manually. The key parts here are that you specify a resource reference of type javax.sql.DataSource, and you give it the name you expect to see in JNDI. So, in the example below, if you call it TestDB, it will be in JNDI under java:comp/env/TestDB. However, you'll need to do something container-specific to match this up with a specific database pool.

<resource-ref>
    <description>Test Database</description>
    <res-ref-name>TestDB</res-ref-name>
    <res-type>javax.sql.DataSource</res-type>
    <res-auth>Container</res-auth>
</resource-ref>
    

Linking the EJB DataSource to a Minerva Pool with jBoss

The easiest way to do this is to use EJX. You need to add a resource manager that links the EJB ref-name (see example above) to the database pool JNDI name (see example above). Since the database pool JNDI name is PoolName that should be your resource jndi name, and the name in ejb-jar.xml should be the resource name.

If you don't use EJX, you will need to add the reference manually in two places - first at the top level of jboss.xml, and then again at the bean level for each bean that uses the resource. Technically you can give the resource manager a different name that the bean expects, and use the bean-specific name with the resource manager name in the bean-level entry. This is only recommended if you have multiple beans in the JAR that have different resource names but can all be served by the same database pool.

<resource-managers>
    <resource-manager res-class="org.jboss.ejb.deployment.JDBCResource">
        <res-name>TestDB</res-name>
        <res-jndi-name>TestPool</res-jndi-name>
    </resource-manager>
</resource-managers>

<enterprise-beans>
    <session>
        <resource-ref>
            <res-ref-name>TestDB</res-ref-name>
            <resource-name>TestDB</resource-name>
        </resource-ref>
    ...
    <session>
        <resource-ref>
            <res-ref-name>OtherDB</res-ref-name>
            <resource-name>TestDB</resource-name>
        </resource-ref>
    ...
    

Getting a connection in an EJB implementation

Context naming = new InitialContext();
DataSource ds = (DataSource)naming.lookup("java:comp/env/"+poolName);
Connection con = ds.getConnection();
    

Creating a pool for a generic Object type

PoolObjectFactory myFactory = new MyPoolObjectFactory();
... // Configure Factory
ObjectPool pool = new ObjectPool(myFactory, "MyPoolName");
... // Configure Pool
pool.initialize();
    

Getting and returning Objects

ObjectPool pool = ...
MyObject o = (MyObject)pool.getObject();
...
pool.releaseObject(o);
    

Getting and returning PooledObjects

ObjectPool pool = ...
MyObject o = (MyObject)pool.getObject();
...
o.close();  // Assumes that a PoolEvent(o, PoolEvent.OBJECT_CLOSED) is sent
    


Outstanding Features