Saturday, March 27, 2010

Blog moved

this blog has been moved to http://andresteingress.wordpress.com

Contract-Oriented Programming with Groovy

Introduction

One of the first programming-related books i've read thoroughly has been "Object Oriented Software Construction" by Bertrand Meyer [0]. Although the second release (!) has been released back in good-old 1997 it still contains a huge amount of object-oriented principles not seen in nowadays main-stream programming languages like Java or .Net C#.

One of the concepts I liked best is contract-oriented programming aka "design by contract". The basic idea is pretty genius: applying the mathematical principle of Hoare triples to classes and objects.

A Hoare triple is always of the following form:

{P} A {Q}

A concrete triple says: before some operation A executes the state P must be true. After execution of operation A, state Q must hold. It's as simple as that.

In an object-oriented context, application of Hoare triples result in so-called assertion types:

Preconditions:  specifies the state the object must hold before a method is executed.

Postconditions: specifies the state the object must hold after a method is executed.

Class-Invariant: specifies the state an object must hold after it has been constructed, before and after each method call on that object.

Unfortunately the principle of contract-oriented programming is not natively supported in most programming languages (except Eiffel). With the advance of aspect-oriented programming there have been some attempts in the Java world to add contract support, but none of them really seemed appropriate to me.

Contracts for Groovy - gcontracts

When I first read Peter Niederwiesers blog-post on closure annotations - contract-oriented programming came immediately to my mind. Through the introduction of AST transformations in Groovy 1.6 it is possible to modify a program's abstract syntax tree and add custom byte-code or modify it - i've already written a short introduction on local and global AST transformations.

With AST transformations and closure annotations in mind a started to implement a first prototype that already supports the previously mentioned assertion types by providing Java annotations (@Invariant, @Requires, @Ensures). Before we take a look at the annotations and on the corresponding AST transformation stuff, I want to show you the outcome and how those contract annotations can be applied on Groovy classes:

e.g. a Person class

@Invariant({ firstName != null && lastName != null })
class Person {
  String firstName
  String lastName
  String name


  public Person() {}


  public Person(final String firstName, final String lastName) {
    this.firstName = firstName
    this.lastName = lastName
  }


  @Override
  @Requires({ firstName.size() > 0 && lastName.size() > 0 })
  def String toString() {
    return "${firstName} und ${lastName}"
  }


  @Requires({ delimiter in ['.', ',', ':'] })
  @Ensures({ name.size() > 1 } )
  def initCompleteName(String delimiter) {
    name = "${firstName}${delimiter}${lastName}"
  }
}

I know, the good-old stack example is a better choice but I needed a quick-and-dirty example on how to apply gcontracts annotations ;-) Let us quickly skip through all annotations and their meanings in that particular context:

1. the @Invariant annotation just says that during the entire life-time of a person it's firstName and lastName property must never be set to null.
2. whenever toString() is executed the object's firstName and lastName size() must be greater than zero
3. whenever initCompleteName is called, parameter delimiter must be in a certain range.
4. whenever initCompleteName is done, the name must be initialized to something bigger than 1.

Note that e.g. the @Requires closure annotation has access to all method parameters and instance variables, so checking the delimiter being in a certain Groovy range just works. In addition the AST transformation process injects the class invariant wherever feasible, e.g. 


def p = new Person('Max', 'Mustermann')
p.setFirstName(null)

causes an AssertionError since after the call to setFirstName the class invariant is not satisfied anymore.

I hope you already see and feel that applying gcontracts can greatly improve readability and maintainability of your source since interface contracts are specified explicitly and are proved (if activated) at runtime. Of course, such mechanisms are ment to be used to determine programming errors and not business validation errors - but i guess i am going to take a look at invariants and their role in the software development process in another blog-post.

Let's take a quick look at the implementation of gcontracts done so far. gcontracts by now provides three annotations:

@Target(ElementType.TYPE)
public @interface Invariant {
    Class value();
}

is used on type-level to specify the class invariant.

@Target(ElementType.METHOD)
public @interface Requires {
    Class value();
}

is used on method-level to specify preconditions.

@Target(ElementType.METHOD)
public @interface Ensures {
    public abstract Class value();
}

is used on method-level to specify postconditions.

As you can image, the real work of gcontracts is done in the AST transformation implementation which is done in the ContractValidationASTTransformation class.

In a nutshell, the transformation visits all code parts which are annotated with the annotations above and adds Java assertions at the appropriate places. Although there is no explicit support for contract-oriented programming, Java provides the assertion statement: whenever assertion-checking is enabled and an assertion is found, e.g.

assert count > 0 : "count must be greater than zero";

that boolean expression is evaluated and if false a ava.lang.AssertionError will be thrown. AST transformation uses that statement to inject pre-, postcondition and class-invariant checks (loop invariants are supposed to be added next) at appropriate code places. That includes that disabling checking of these constraints is as simple as deactivating Java assertions [1].

gcontracts on Github

I pushed the current source code to a github repository: http://github.com/andresteingress/gcontracts

Please keep in mind that the project's code has neither been thoroughly tested nor is being ready to be used in production code - but i am going to work on it, in order to release the first version as soon as possible.

[0] Object-Oriented Software Construction (2nd Edition), Bertrand Meyer, 1997, Prentice Hall
[1] Programming with Assertions, Sun

Tuesday, March 23, 2010

Getting the Persistence Context Picture (Part I)

This article series deals with Hibernate‘s basic APIs and how Hibernate is used in Grails applications. The first part of this series is meant to be seen as overall introduction to objects, persisting objects and Hibernate as a persistence framework.
I have been in a lot of projects where Hibernate and the persistence layer was handled as the application‘s holy grail: whenever an error was thrown, programmers did not try to understand the concrete problem, but consumed their time by finding work-arounds. From my experience, that behavior was simply caused by a lack of knowledge about basic persistence context patterns. In this article i will try to explain the most fundamental patterns and concepts which should already help to gain knowledge on how persistence frameworks core data-structures work.

Objects, objects everywhere...

Let's start with object-orientation. Implementing a persistence framework mainly involves the question on how to map objects from an object-oriented domain into a relational data model, which is found in most databases we're dealing today. In order to understand persistence mechanisms from bottom-up, we should revise the basic concepts on objects and classes. The basic definition of objects is:




Whenever we are talking about objects in an object-oriented context, we speak of runtime representatives of classes, whereas the classes can be seen as construction plans to be used when running the program and constructing new instances. A class consists of attributes and operations on that attributes. Objects at runtime represent the attribute‘s values which are tightly connected with the class‘s operations on them. If seen from a logical view, an object is represented by it‘s state and operations.



Attributes might be of any datatype available in the programming environment. Most programming languages decide between simple datatypes and custom class data types, whereas custom class data-types contain custom as well as API classes. At runtime therefore attribute values either contain scalar values (e.g. a number, a string, a boolean value, etc.) or references to other objects. A reference‘s value might either reference another object or is void.

Every object created during the execution of an object-oriented program has an object identity. Depending on its context object identity has two meanings:

1. By reference: an object A denotes as being equal to another object B if their references are equal.




2. By state: an object A denotes as being equal to another object B if their attribute values are equal.



In object-orientational theory the first one is named „object identity“ and the latter one „object equality“, thus being identical is not as being the same.

Unfortunately in the Java environment being an identical object means the same as being equal to another object, since java.lang.Object.equals() implements reference comparison with the == operator by default.

Let‘s change our view to a completely relational mapping model. In a first naive approach our tables would correspond to classes, whereas columns represent the class‘s attributes.



At runtime an object instance would be represented by a single database row filled with values for each of the available columns. In fact, this is how it is done most of the time when we are using persistence frameworks like Hibernate.

The examples above show very simple structured object's classes. In practice persistence frameworks also need a way to map relationships (1:n, m:n, m:1, 1:1) between objects and database tables. Usually this is done using foreign keys in the relational model, but imagine a collection with a lot of referring objects - the persistence frameworks APIs need to provide mechanisms for batch loading, cursor support etc.

We have already seen object identity and its two characteristics - with database persistency a third identity comes into play: the object's primary key. But the problem is that objects don't necessarily know about their primary key until a key is explicitly requested from the database. It gets even trickier if you think of relationships between objects - how can a programmer ensure that relationships are mapped in the correct order depending on foreign key constraints between the objects database tables.

Since programmers really should not deal with issues like object relational mapping, object identity, batch loading relationships for relationship traversal, etc. various persistence or object-relational mapping (ORM) frameworks have prospered. They all have in common that they provide functionality that persists objects of an object-oriented programming environment into some persistent store, thus persisted objects are called persistent objects.


The Persistence Context

For persistent objects there needs to be some explicitly defined context in which creation, modification and retrieval of persistent objects can happen. This context is known as Persistence Context (or Persistent Closure). In fact, most of the persistence frameworks provide APIs that provides access to a persistence context, even though the persistence context‘s functionality is often split into several APIs.

Let's take a look at the basic definition of persistence contexts ([0] "Persistence Context" pattern):



Notice that the term „business transaction“ does not refer to database transactions. A business transaction is a logical transaction that might span several operations e.g. ordering a pizza from the customer‘s view is a single business transaction, but it might be the case that this single business transaction involves several technical transactions to complete the request.

To lookup the current persistence context during execution classes might use a Registry which provides access to the current persistence context.

The persistence context deals with few problems caused by the object-orientation/relational mapping mismatch. It ensures that all operations on objects are tracked for the persistence context‘s life time, to keep possible db transactions as short as possible. It handles the problem of object identity and ensures that there will never be multiple object instances with the same database primary key. It tracks associations and resolves them in order to satisfy foreign key constraints. It implements a cache mechanism to automatically gain a certain transaction isolation level to solve repeatable read problems and to lower the number of executed SQL statements. Overall, a persistence provider already has gained a lot of knowledge about database systems and ORM so I would consider decisions for custom implementation of persistence contexts as highly risky.

Hibernate‘s Persistence Context APIs

Let‘s take a look at how Hibernate implements the Persistence Context and related patterns. Overall, Hibernate‘s persistence context API mainly consists of the following classes:

org.hibernate.SessionFactory

The SessionFactory resembles the persistence context managing component in Hibernate - the registry. This is the central place for global configuration and set-up. During runtime  there will only be a single session factory instance in your application‘s environment (except for multiple data-sources, e.g. legacy databases, ldap providers etc.) which acts as factory for retrieving persistence context objects.

org.hibernate.Session

A Session represents a component which tracks (de-)attachment, modification and retrieval of persistent objects. This is Hibernate‘s persistence context (finally). Whenever you need a persistence context in your application you have to look up a SessionFactory and create a new Session using the openSession() method. Be aware that Hibernate is not restricting programmers in how you handle sessions. If you decide to implement a long-running business transaction (aka conversation) with a single session instance, you are free to do so.

org.hibernate.Transaction

A Transaction actually is used to implement a Unit of Work within the application. A single session might span multiple transactions and it is recommended that there is at least a single uncommitted transaction when working in a session. Note that the actual implementation of how the transaction is handled on database-side is hidden by Hibernate‘s implementation of that interface.

Still if you use Hibernate without an explicit call to session.beginTransaction() Hibernate will operate in auto-commit mode (be sure to specify connection.autocommit=„true“ in your configuration‘s xml).

Lifecycle Management

So far we have heard from persistent objects as being the type of objects which have been persisted by the persistence context. But that persistent state is just a single station in the life-cycle of objects managed by Hibernate as a persistence provider.

In fact, I assume that in most applications the domain model entities will have to be kept in some persistent store. Therefore an entity object‘s instance will run through several states between instance creation and being actually stored in the persistent store of your choice:

  1. (De-) Attaching Instances
  2. Saving/Updating Instances
  3. Removing Instances

(De-) Attaching Instances



Imagine your first application‘s bootstrap. Chances are good that you might have to create some persistent objects on startup to get the system working. Whenever creating object instances which have never been persisted we are talking of transient objects. These objects have in no way been connected to a persistence context.

The process of letting the persistence context know of the existing of a transient object is called „attaching“. Therefore, newly attached objects are called either attached or persistent objects.

On the other way around the process of disconnecting persistent objects from the current persistence context is known as „dettaching“ or „evicting“.

Saving or Updating Instances



Save or update operations can only be applied to persistent objects. If you need to save a transient or dettached object, you as an application developer have to attach that object instance to the current persistence context. Hibernate eases these two steps (attaching/save) since it provides update/save methods which automatically attach transient or detached objects.

Removing Instances



Removing can only be applied on persistent objects. As it is the case with save/update operations transient or dettached objects first needs to be attached to the current persistence context to get removed. Whenever a persistent object is removed it actually is just marked as being in state „removed“ by the underlying persistence mechanisms.

Summary

The gained knowledge about the Persistence Context pattern and Life-Cycle of persistent objects already equips us with a lot of basic knowledge on how persistence frameworks like Hibernate operate. In the next part we will take a look at using Hibernate APIs in applications and how Grails utilizes Hibernate.

[0] Patterns of Enterprise Application Architecture, Martin Fowler


Friday, March 19, 2010

Creating Custom Constraints

Now and than there is the requirement to create custom constraints. As you might already know, there are plenty of constraints that already are shipped with Grails. Those can be applied in the constraints closure of your domain or command classes:

class User  {
   String firstName
   String surName
   Short  age
   static constraints = {
      firstName(blank: false, maxSize: 150)
      surName(blank: false, maxSize: 150)
      age(min: 18 as Short, max: 99 as Short)
   }
}

In order to implement custom constraints like i.e. an age constraint, which encapsulates the constraint's implementation you could extend from Grail's AbstractConstraint class. The problem with this approach is, that you than need to register your custom constraint at bootstrap, to use it in every possible scenario (going from testing to running the app).

Fortunately there is a plugin that helps with creating custom constraints: the constraints plugin.

grails install-plugin constraints

Installs the plugin and adds a dependency entry in your application or plugin settings. If installation was successful your grails help command lists the create-constraint command, that from now can be used to create custom constraints.


Creating an application-specific age constraint would be as simple as:

grails create-constraint Age

which creates grails-app/utils/AgeConstraint which i modified to be

class AgeConstraint {
    def validate = { propertyValue ->
        return propertyValue >= 18 && propertyValue <= 99
    }
}


After declaring the constraint it can be used in domain or command classes:

class User  {
   String firstName
   String surName
   Short  age


   static constraints = {
      firstName(blank: false, maxSize: 150)
      surName(blank: false, maxSize: 150)
      age(age: true)
   }
}

If you take a look at the plugin's documentation [0] there are several to use default message codes for i18n of error messages and so on. But this plugin works perfectly for adding custom constraints to your Grails environment.


[0] Constraints Plugin - http://github.com/geofflane/grails-constraints

Structuring Grails Projects

A Plain Project

Creating a Grails project is damn easy. Just run

grails create-app

and Grails will create the initial project structure with the grails-app folder. The grails-app folder indeed is not just a simple folder but resembles the "convention over configuration" approach - meaning that each folder within it has a predefined meaning to the grails web application framework.


So far so good. But there are times when you are working on applications that go beyond the simple CRUD application style - some call them "enterprise applications". I guess there is no unique definition of what it really means for an application to be an "enterprise application", but there is at least one thing you as a software developer will have to come up with: structuring your application into several components.


Grails indeed has a neat mechanism that supports structuring your applications into several components: plugins.


The Plugin Concept


You might already know that there are hundreds of Grails plugins available - just take a look at http://grails.org/plugins. Even we know plugins of being an integration gateway to other frameworks such as Compass, Spring WebFlow, etc. they can be seen as a great mechanism to create independent components within your application too. 


Let's assume i wanted a component which encapsulates and holds my application's business domain. In my project's folder I already created the main project by running 


grails create-app GrailsEnterpriseApplication


In order to create a separate component for my domain model, i would create a new plugin with


grails create-plugin GrailsEnterpriseDomain


After executing these steps, the overall project's structure will look like this (i.e. in IntelliJ)






Hint: just ignore the two tomcat plugin projects, Grails 1.2.1 defines dependencies on it when creating apps and plugins. 


Since every Grails plugin in fact is a Grails application we can use all the creation and generation commands available in Grails from within that project. For example, running


grails create-domain-class org.ast.User


on the GrailsEnterpriseDomain project will create a new domain class User and the according unit tests in our application's domain plugin. This means that it's possible for another group of developers to work independently on that component, without any restrictions on testing, creating artifacts and so forth.


So far we have not defined a reference between the main project and the plugin. In Grails 1.2.1 this can be done with an entry in the application.properties file in the main project:


plugins.grails-enterprise-domain=0.1


This line in fact means that our GrailsEnterpriseApplication depends on version 0.1 (the initial version) of our GrailsEnterpriseDomain plugin.


The last step for being able to reference our custom plugin at runtime is the execution of the 


grails package-plugin


command on the GrailsEnterpriseDomain project. This command create a zip archive of your plugin but more importantely a plugin.xml file which later on is used by the main application on bootstrap, for loading and resolving possible dependencies of our domain plugin. So if our domain plugin would depend on other plugins, Grails would resolve those dependencies on application startup.


There are several commands and configuration settings to define plugin repositories and push plugins to them, but as it is the case here we just want to use our domain plugin without packaging and deploying it, as this is the case when the initial development work is done. Luckily there is a configuration setting in Grails 1.2.1's BuildConfig.groovy which allows the specification of local unpackaged plugins:


grails.plugin.location.'grails-enterprise-domain' = "../GrailsEnterpriseDomain"


When adding this line to the GrailsEnterpriseApplication's BuildConfig.groovy, Grails on startup instead of using plugin repositories makes a local file system lookup to load the grails-enterprise-domain plugin.


As a consequence running i.e. 


grails generate-controller org.ast.User


in the GrailsEnterpriseApplication project (!) automatically will reference the User class in your domain plugin. Meaning that separating the business model from the MVC part of your application can be done easily.


Summary


Of course, the example just showed a very simple use-case but I think you got the idea. Through Grail's plugin mechanism it is possible to create structured applications with explicit dependencies/automatic dependency resolution and even with revision management included.


[0] Grails Plugin Portal - http://grails.org/plugins
[1] Grails Plugins Introduction - http://grails.org/doc/latest/guide/12.%20Plug-ins.html



Tuesday, March 16, 2010

Implementing a @Serializable annotation with Groovy AST transformations

Groovy 1.6 provides hooks to intercept the compilation process and modify the generated AST (abstract syntax tree).

There are two categories of AST transformations:
  • global transformations 
  • local transformations
This blog-post deals with local transformations. A local AST transformation is usually done in three steps:

  1. you have to implement an annotation that marks a code fragement for AST transformation. the chosen code fragement depends on you transformation's functionality. In my case, i just wanted a @Serializable annotation instead of using Java's java.io.Serializable interface, so that @Serializable annotation is ment to be applied at ElementType.TYPE level. If your use-case is a different one, of course you could choose other element-type levels.
  2. you actually have to implement the AST transformation. this is done by implementing the org.codehaus.groovy.transform.ASTTransformation interface.
  3. you have to wire your custom annotation with your AST transformation implementation of step 2. this is done by using the @GroovyASTTransformationClass annotation (a meta-annotation) with the complete path to your ast-transformation class.

In the end, applying the @Serializable annotation is as simple as specifying the annotation on some Groovy class

@Serializable
class User {
  // ...
}

The implementation of step 1) is just a simple annotation declaration

@Retention(RetentionPolicy.SOURCE)
@Target(ElementType.TYPE)
@GroovyASTTransformationClass("org.nlr.annotations.SerializableASTTransformation")
public @interface Serializable {
}

and the transformation code is as simple as

@GroovyASTTransformation(phase= CompilePhase.SEMANTIC_ANALYSIS)
public class SerializableASTTransformation implements ASTTransformation {

    public void visit(ASTNode[] nodes, SourceUnit source) {
        if (!(nodes[0] instanceof AnnotationNode) || !(nodes[1] instanceof AnnotatedNode)) {
            throw new RuntimeException("Internal error: wrong types: $node.class / $parent.class");
        }

        AnnotatedNode parent = (AnnotatedNode) nodes[1];
        AnnotationNode node = (AnnotationNode) nodes[0];

        if (parent instanceof ClassNode) {
            ClassNode classNode = (ClassNode) parent;

            classNode.addInterface(ClassHelper.make(java.io.Serializable.class));
        }
    }
}

Notice, that the specified compile phase is "semantic analysis" - a property of local transformations is that they can only be applied in "semantic analysis" compilation phase or above.

If you take a look at the generated (disassembled) byte-code you'll see that the User class's byte-code now implements java.io.Serializable.

public class org.nlr.domain.User extends java.lang.Object implements java.io.Serializable,groovy.lang.GroovyObject{
    public static final java.lang.Class $ownClass;
    java.lang.Long id;
    java.lang.Long version;
    java.util.Set registrations;
    ...