<?xml version="1.0" encoding="ISO-8859-1"?>
<document url="http://castor.org/jdo-examples.xml">
  <properties>
    <title>Castor JDO - Best practice</title>
    <abstract>
       Castor JDO - Best practice
    </abstract>
    <status>Draft</status>
  </properties>
  <body>
    <title>Castor JDO - Best practice</title>
    <header>
    </header>

    <section title="Introduction">

    <p>There's many users of Castor JDO out there, who (want to) use Castor JDO in 
       in high-volume applications. To fine-tune Castor for such environment, it is 
       necessary to understand many of the product features in detail and to be able to
       balance their use according to the application needs. Even though many of these
       features are detailed in various places, people have frequently been asking 
       for a 'best practise' document, a document that brings together these technical 
       topics (in one place) and presents them as a set of easy-to-use recipes.</p>
       
    <p>Please be aware that this document is <i>under construction</i>, but still we 
       believe that - even when in its conception phase - it provides valuable
       information to users of Castor JDO.</p>

    </section>
    
    <section title="General suggestions">

        <p>Let's start with some general suggestions that you should have a look at. Please don't 
           feel upset if some are really primitive but there may be users reading this document
		   that are not aware of them.</p>
          
        <ol>
            <li><p>Switch to version 0.9.9 of Castor as we have fixed some 100+ bugs that may cause
                some of your problems.</p>

                <p>Sidenote: Performance has, generally, improved recently. If you're not seeing 
                   performance improvements, then it's worth spending some time thinking about why.</p></li>

            <li><p>Initialize your JDOManager instance once and reuse it all over your application.
				Don't reuse the Database instances. Creating them is inexpensive, and JDBC rules
				state that one thread &lt;-> one JDBC connection is the rule. Do not multithread inside 
                of a Database instance; as a corrolary, do not multithread on a single JDBC 
                connection.</p></li>            
                
            <li><p>Use a Datasource instead of a Driver configuration as they enable connection 
                pooling which gives you a great performance improvement.</p>

                <p>We highly suggest DBCP, here, with the beneficial use of prepared statement caching.</p>

                <p>Should you be running on a system where read performance is critical, feel free 
                   to take the SQL code generated by castor, and dumped to logs during the DB 
                   mapping load in debug output, and turn those into stored procedures that you 
                   then invoke via SQL CALL to perform those loads; however, I find personally that 
                   stored procedures would be a minimal improvement over the DBCP prepared statement 
                   cache; your mileage may vary. db.load() has performance benefits that are 
                   worth keeping, IMO, and the pleasure of having pretty stored procedures in your 
                   database is far outweighed by the nightmare of change management.</p>

                <p>Have a look at 
                   <a href="jdo-pooling.html#Jakarta-Commons-DBCP---BasicDataSource">the HTML docs</a>
                   for Jakarta DBCP, which has details about how to use and configure DBCP 
                   with Castor and Tomcat.</p>
                   
                <p><b>Note:</b> 'prepared statement caches' refer to the fact that DBCP is a JDBC 
                   3.0-compliant product, and as such has to support caching of prepared
                   statements. This basically allows the JDBC driver to maintain a pool 
                   of prepared statements across all connections, a feature that has been added
                   to the JDBC specification with release 3.0 only.</p>

                <p>DBCP setup is generally outside of the scope of this list, but basically, here's 
                   my two cent description:</p>

                <p><ul>
                    <li>Use tomcat 5.5, because mucking about in server.xml sucks. For those of you 
                        working with Tomcat 4.1.x, there's no need to muck about in server.xml, either. 
                        Afaik, a web app can be deployed using a web app descriptor copied into 
                        $TOMCAT_HOME/webapps, which is the place top define anything specific
                        to a web app context. Details can vary, of course.</li>
                    <li>Create a META-INF directory in your WAR deploy scripts, and put a 
                        context.xml in it.</li>
                    <li>In that context.xml, describe all of the things you want to be made available 
                        via JNDI to your application. These include things like UserTransaction and 
                        TransactionManager (for those of us using JOTM), all your database connection 
                        pools as datasources, etc. You can also add your JDO factory here, should you 
                        choose to do so.</li>
                     <li>Configure Castor to load those JNDI names to retrieve connections.</li>
                    </ul></p>
                    
                 <p>Hit the deploy button, and bob's your uncle.</p>
            </li>
            
            <li><p>Always commit or rollback your transactions and close your Database instances 
                properly; also in fail situations.</p>
			    <p><b>Note:</b>Just the obvious general rule on Java objects that hold resources: Don't
				   wait for the VM to finalize to have something happen to your objects when you 
                   could have released critical resources at the appropriate point in the codebase.</p></li>
            <li><p>Keep your transactions as short as possible. If you have an open 
                transaction that holds a write lock on an object no other transaction can get 
                a write lock on the same object which will lead to a LockNotGrantedException.</p>

            <code-panel>
execute() {
   Database db = jdo.getDatabase();
   db.begin();
   // query objects from database with read only
   db.commit();
   db.close();

   // do some time consuming processing with the data

   Database db = jdo.getDatabase();
   db.begin();
   // use db.load() to load the objects you need to change again
   // create, update or delete some objects
   db.commit();
   db.close();
}</code-panel>

                <p>It doesn't make sense to make a own transaction for every change you want to
                   do to an object as this will slow down your application. On the other hand
                   if you have transactions with lots of objects involved taking an valuable
                   amonth of time you may consider to split this transactions to reduce the
                   time an object is locked.</p>

                <p>Also keep in mind that folks using lockmode of DBLocked do FOR UPDATE calls 
                   on things they read while the transaction is open; if you're using dblocked 
                   mode, be aware of how your application does things. If you're in one of the 
                   other modes, locks happen inside castor, and it's your responsibility to 
                   always use the right access mode when accessing content.</p>

                <p>If you can, for example, decide at the API layer whether or not an operation 
                   is going to ever need to modify an object, and know that you will only ever 
                   use an instance in read only mode, load objects with access mode read only, 
                   and not shared.</p>

                <p>Limit use of read-write objects to situations in which it is likely you will 
                   need to perform updates.</p>

                <p>Imagine, for a moment, that these transactions were in DBLocked mode - 
                   transactions which translate directly into locks on the database.</p>

                <p>If you're opening something up for modification on the DB - marking it as 
                   select FOR UPDATE - then that row will be locked until you commit. The database 
                   would prevent any other transaction that wants to touch that row from doing 
                   anything to it, and it would block on your transaction - deadlock at the 
                   SQL level.</p>

                <p>Castor does the same things internally for its own access modes - Shared and 
                   Exclusive. Each has different locking semantics; having good performance 
                   means understanding those locking semantics.</p>

                <p>For example - read only transactions (should be) cheap. So there's no issue
				   with holding those transactions open a long time; because they only translate,
				   for an instant, into a lock. The lock is released the moment the load is completed
				   and the object is dropped into read-only state within your transaction; read only
				   operations therefore operate, pretty much, without locking.</p>

                <p>The lock is of course acquired because you might also have it in SHARED or 
                   EXCLUSIVE mode on another thread - and that read-only operation isn't safe 
                   until those transactions close.</p>

                <p>Once the lock is released, you're lock-free again, so the transaction basically 
                   has nothing in it that needs anything doing.</p>

                <p>That's not to say that holding transactions open is good practice - but 
                   transactions should always be thought of as cheap to create and destroy and 
                   expensive to hold on to - never do heavy computation inside of one, unless 
                   you're willing to live with the consequences that arise from holding 
                   transactions on object sets that others might need to access.</p></li>
                   
           <li><p>Query or load your objects <b>read only</b> whenever possible. Even if castor creates 
                  a lock on them this does not prevent other threads from reading or writing them. 
                  Read only queries are also about 7 times faster compared with default shared mode.</p>

                <p>for queries:</p>

                <code-panel>
String oql = "select o from FooBar o";
Query query = db.getOQLQuery(oql);
QueryResults results = query.execute(Database.ReadOnly);</code-panel>

                <p>to load an object by its identity:</p>

                <code-panel>
Integer id = new Integer(7);
Foo foo = (Foo) db.load(Foo.class, id, Database.ReadOnly);</code-panel>

                <p>Default accessmode is evaluated as follows:</p>
                
                <p>
                    <ul>
                        <li>if specified castor uses access mode from db.load() or query.execute(),</li>
                        <li>if this is not available it takes access mode specified in class mapping,</li>
                        <li>if nothing is specified in mapping it defaults to shared.</li>
                    </ul>
                </p>

                <p>One cannot stress how important this is: If 99% of your application never writes 
                   an object, and you as a programmer know it won't, then do something about it. 
                   If you're in a situation where you want the object to be read-only most of 
                   the time, and only want a writable every now and then, do so just-in-time 
                   by performing a load-modify-store operation in a single transaction for the 
                   shareable you want.</p>

                <p>In other words: Don't use read-write objects unless you know you're likely 
                   to want to write them.</p></li>
        
           <li><p>If there is a possibility you should prefer <tt>Database.load(Class, object)</tt> over 
                  <tt>Query.execute(String)</tt>. I suggest that as <tt>load()</tt> first tries to load the 
                  requested object from cache and only retrieves it from database when it is 
                  not available there. When executing queries with <tt>Query.execute()</tt> the object 
                  will always be loaded from database without looking at the cache. You may 
                  gain a improvement by a factor of 10 and more when changing from 
                  <tt>Query.execute()</tt> to <tt>Database.load()</tt>.</p></li>
        </ol> 
        
    </section>

    <section title="Further optimization">

        <p>We hope above suggestions help you to resolve the problems you have. If you still 
           need more performance there are areas of improvement that are more difficult to 
           resolve. For further ideas to improve your applications performance you should
		   take a loock at out performance test suite (PTF) which you can find in Castor's
		   source distribution under: src/tests/ptf/jdo.</p>

        <p>Now, there's lots left to do - there is still the issue, for example, of dependent 
           objects being slightly sub-optimal in performance both in terms of the SQL that 
           gets generated and the way it gets managed - but there will be improvements over 
           time to the way that this and other operations are performed.</p>

        <p>But performance should be good right now. If it isn't, you'll need to think about 
           whether you are using the optimal set of operations. No environment can predict 
           your requirements - hinting to the system when objects can be safely assumed to 
           be read-only is vital to a high-performance implementation.</p>

    </section>

    </body>
</document>

