Unit testing Spring Controllers

I found this DevX blog by Edmon Begoli really useful for writing unit tests for Spring controllers using MockHttpRequest and MockHttpResponse. However, I think the ideas in it can be extended slightly to help those using Spring Security and Hibernate, particularly the Open Session In View pattern, perhaps aided by Spring's OpenSessionInViewFilter.

My first enhancement would be to load the Controller instances from Spring rather than assembling them by hand. The key to doing this with Maven is to move the "spring-servlet.xml" file that configures the controllers from the src/main/webapp/WEB-INF folder in to src/main/resources. This doesn't cause any problems as you can load the spring-servlet.xml from anywhere on the classpath.

  spring
    org.springframework.web.servlet.DispatcherServlet
    
       contextConfigLocation
       classpath:spring-servlet.xml  
     
  1

Having done this, you can then load the spring-servlet.xml application context in your Controller unit tests.

The setUp() method below shows how to do this, in addition to obtaining a reference to the Spring security AuthenticationManager and starting a Hibernate Session. The latter allows us to replicate the effects of the OpenSessionInViewFilter while the former gives us the means to log in to Spring Security before executing a test.

protected void setUp() throws Exception {
  super.setUp();
  
  // Set up Spring application context
  String[] paths = { "application-context-datasource-test.xml",
       "application-context-dao.xml",
       "application-context-facade.xml",
       "application-context-mail.xml",
       "spring-servlet.xml" };
  ctx = new ClassPathXmlApplicationContext(paths);
      
  // Get Spring AuthenticationManager
  am = (AuthenticationManager)ctx.getBean("authenticationManager");  
       
  // Create mock request and response
  request = new MockHttpServletRequest();
  response = new MockHttpServletResponse();
  
  // Start session (simulate open session in view)
  factory = (SessionFactory)ctx.getBean("sessionFactory");
  session = sessionFactory.openSession();
  TransactionSynchronizationManager.bindResource(factory, new SessionHolder(session));
}

A simple "log in" method can be provided for use in the tests.

protected void login(String username, String password) {
  Authentication auth = am.authenticate(new UsernamePasswordAuthenticationToken(username, password));
  SecurityContextHolder.getContext().setAuthentication(auth);
}

A sample test method might look like this.

public void testStuff() throws Exception {
  // Get the controller
  MyController ctrlr = (MyController ) ctx.getBean("myController");

  // Login as manager
  login("bill.business-manager", "secret");

  // Controller renders form on GET method
  request.setMethod(AbstractController.METHOD_GET);

  // Execute the request
  ModelAndView mav = ctrlr.handleRequest(request, response);
  
  // Test stuff
  assertNotNull(mav);
}

Using database views in Hibernate (or, anything view can do, HQL can do better)

It's very easy to map database views using Hibernate, but there are a few considerations you might want to er... consider before you do it. I suspect the main reason people do it is for performance, but I think that a lot of the gains are not that great and might stem from lack of knowledge about what HQL can do.
Certainly, loading an entity and then iterating through child entities to calculate aggregate values is not going to perform as well as the equivalent view. However, as I will explain below you may be able to achieve the same result using the equivalent features in HQL.
Finally, I do admit that some databases enable indexing of views and that might be faster than the equivalent operations in Hibernate as a result.
First, I'll just look at how you go about mapping a view and then discuss some of the details a bit later.
You can map a view exactly as if it was a table. Just put the view name in the table attribute. Optionally, you can specify that the class is immutable as in most cases you won't be able to update a view-backed entity.

 
 ...
 

The first consideration with this is portability. Not all databases support views so you would be limited to supporting those that do.
However, you can use the subselect element to largely the same effect, although obviously this wouldn't benefit from indexing in the same way that a "proper" view might.

 
     
       select p.Name, sum(s.Bits) from Project p inner join Stuff s etc
     
 ...
 

However, at this point you may need to consider how your views will interact with the 2nd-level cache. The good news is that you can enable this as you would a normal entity and you can cache HQL queries on this view. I tend to do this in the Hibernate configuration file. Note again that it's worth specifying that the usage is "read-only".

The bad news is that you may get some unexpected results on your cached views when you update the tables that are joined by the view. Basically, the view queries won't reflect the current contents of the database immediately as the level 2 cache will hold stale data. Why is this data not evicted from the cache? Because Hibernate can't really guess which entities are related to your view unless you tell it.
You can let Hibernate know which other tables are linked to your view as below:

 
     <synchronize table="dbo.Project" />
     <synchronize table="dbo.Stuff" />
     <synchronize table="dbo.Things" />
     ...
 

One thing to watch for is that Hibernate is very fussy about how you specify the table attribute. It is case sensitive, and in the case of Microsoft SQL Server 2005 also requires the name to be qualified with the schema. If you get it wrong, Hibernate doesn't complain - but it also won't automatically evict your views from level 2 cache. You just need to fail your way to success, unfortunately.
By now you have got your views working with Hibernate and it is playing nice with the level 2 cache.
But did you need to do it in the first place?
I can think of two scenarios where a view might be used. One would be to conveniently load a "flattened" representation of an object graph in one efficient join query, thereby avoiding the famous "N+1 query" problem of loading object graphs in Hibernate. Two would be to calculate aggregate values on an object graph (such as a sum of a field in a child object), again in a single efficient query.
However, both of these things can be achieved in HQL. The first can be achieved using a "join fetch" query, that causes an object graph to be loaded using a single join query. The second can be achieved using the support for aggregate functions in HQL that can return scalar values in a result set.
Here is how to "eagerly" join fetch a child object using HQL, which is done in a single query.

select o from Timesheet o left join fetch o.activities where o.employee.id=? and o.date=? 

Aggregates can be easily calculated on child objects using a HQL query like this:

select timesheet, sum(activity.hours) from Timesheet timesheet join timesheet.activities activity where timesheet.timesheetId = 1 group by timesheet.timesheetId, timesheet.date, timesheet.employee.employeeId

This query needs slightly different handling in code as it returns a List of Object[] arrays, rather than a List of entities.

Query query = session.createQuery("select timesheet, sum(activity.hours) from Timesheet timesheet join timesheet.activities activity where timesheet.timesheetId = 1 group by timesheet.timesheetId, timesheet.date, timesheet.employee.employeeId");
List<Object[]> results = query.list();
for (Object[] result : results) {
  Timesheet timesheet = (TimesheetData)result[0];
  BigDecimal hours = (BigDecimal)result[1];
}

So my conclusion - views can be used in Hibernate and can be made to work well with the level 2 cache. But I would strongly advise not doing it if you can avoid it, as it "clutter" your domain model with redundant psuedo-entities. It's like the object-oriented version of denormalised data!

Automated Integration Testing with Maven, Selenium, Failsafe, Jetty and DBUnit

Most Maven users are aware that unit tests can be executed as part of the automated build process. Maven can also automate integration tests, which are run as part of the “integration-test” and “verify” phases of the Maven build process. This article shows how to automate the integration tests using a number of useful Maven plug-ins. Integration testing tests groups of software modules (as opposed to unit testing, which tests individual units). It is intended to catch problems that arise when modules developed in isolation are assembled and used together, which is often the case when an application is being built by a team of developers. It also encompasses testing components within a container, especially if that container has been “mocked” in the unit tests. For example, a Spring controller might be unit tested with the MockHttpServletRequest and MockHttpServletResponse within the spring-test module. Business logic such as Spring service classes might have been mocked as well. An integration test in this scenario would involve firing real HTTP requests at the Spring controller running inside an application server such as Jetty or Tomcat.

The first consideration is how to create integration tests. Depending on how you define them, they could be standard NUnit tests or they could be Cactus or HTTPUnit type tests which call code executing from within the application server. However, Selenium offers a simple alternative to these approaches that can be easily integrated with Maven. Selenium consists of an IDE (a Firefox add-on) that can be used to record and automate actions (such as clicking links and selecting options in a select box) made within the browser. These can be then exported in a number of formats, including as a JUnit test (in fact, a SeleneseTestCase which extends TestCase). I’m aware that this approach to integration testing verges on functional testing. However, I think the boundaries between unit, integration and functional testing are blurry If it makes you feel any better, pretend this blog is called "Automated Functional Testing with Maven, Selenium, Failsafe, Jetty and DBUnit".

Here’s quick overview of how it all works. There are plenty of good tutorials about Selenium and using the IDE to write tests, so I won’t go into it here. After you have recorded your test case in the IDE, you can export it in Java format.

public class MyITCase extends SeleneseTestCase {

public void setUp() throws Exception {
 setUp("http://localhost:8080/", "*firefox");
}

public void testMy() throws Exception {
 selenium.open("/myapp/login.htm");
 selenium.type("j_username", "percy.project-manager");
 selenium.type("j_password", "password");
 selenium.click("//button[@type='submit']");
 selenium.waitForPageToLoad("30000");
 selenium.click("link=Client list");
 selenium.waitForPageToLoad("30000");
 selenium.click("link=View");
 selenium.waitForPageToLoad("30000");

}

}

In order to play-back the SeleneseTestCase, the Selenium Maven plug-in is needed. This is a “server” which runs an instance of the browser to be remote controlled by the SeleneseTestCase. The plug-in starts the Selenium server in readiness for test cases during the pre-integration-test phase and stops it afterwards (in post-integration-test).

org.codehaus.mojo
selenium-maven-plugin
1.0

 
  start-pre-integration
  pre-integration-test
  
   start-server
  
  
   true
  
 
 
  stop
  post-integration-test
  
   stop-server
  
 

In addition to needing the Selenium server, you will need a method to fire off the integration tests during the “integration-test” phase of the Maven build. Obviously, you could use the Surefire plug-in for this. However, the configuration of the plug-in gets a bit more tricky as you need to exclude the unit tests from the integration test and vice verse – especially the integration tests during unit testing as the Selenium server (and Jetty) won’t be running during the unit test phase.

There is a way to do this by using inclusion and exclusion filters within the configuration of the Surefire plugin and I have seen several explanations of how to do this. However, there is a fork of the Surefire plugin called Failsafe which is designed for integration testing. By default, it only runs tests with that contain IT and ITCase in the name. The main benefit is that the Failsafe plugin will continue to run after a test failure or error. That may seem to be a bit strange, but it is intentional. It is to ensure that the post-integration-test phase is executed, which might perform vital clean-up and tear-down operations that should run after the tests are finished. Additionally, it seems to have an enhanced class loader architecture compared to Surefire to handle some more exotic test scenarios – but that’s not relevant to this discussion.

Here is the necessary configuration for the Failsafe plugin.

org.codehaus.mojo
failsafe-maven-plugin
2.4.3-alpha-1

 
  integration-test
  
   integration-test
  
 
 
  verify
  
   verify
  
 

Now we have created our integration tests, and have a means of playing them back during the integration-test phase of the Maven build. However, we also need to fire up an application server to run the application itself during the integration-test phase. For this, the Jetty plug-in is ideal. It can be configured to start during the pre-integration-test phase and stop during the post-integration-test phase.

org.mortbay.jetty
maven-jetty-plugin
6.1.16

 10
 8005
 STOP


 
  start-jetty
  pre-integration-test
  
   run
  
  
   0
   true
  
 
 
  stop-jetty
  post-integration-test
  
   stop
  
 

I recommend using the jetty:run goal rather than jetty:run-war or jetty:run-exploded (although both of these work too). That is because the latter two goals cause the unit tests to run a 2nd time when the application folders are assembled in the target/ directory. However, if you don’t mind sitting through a 2nd run through of your unit tests then this doesn’t matter.

At this point you might be happy to stop, but there is one final step you might wish to take which is to initialise your database prior to the integration-test phase. This can be done with the SQL Maven plug-in (to create the schema) and the DBUnit plug-in (to insert test data). These can both be configured to run during the pre-integration-test phase. The reason for doing this is that if you are using assertions in your Selenium case (i.e. assertTrue( isElementPresent(//div[@id=’content’]/table[@id=’results’]/tbody/tr[3]/td[2])); then you need to be certain that the data in the database is going to be correct with respect to these assertions.

org.codehaus.mojo
sql-maven-plugin
1.2

 
  com.microsoft
  sqljdbc
  1.2.0
 


 com.microsoft.sqlserver.jdbc.SQLServerDriver
 jdbc:sqlserver://mssql2005server:1433;databaseName=MyDatabase;SelectMethod=cursor
 secret
 squirrel


 
  create-schema
  pre-integration-test
  
   execute
  
  
   true
   
    src/main/database/schema-mssql.sql
   
  
 

DBUnit configuration example.

org.codehaus.mojo
dbunit-maven-plugin

 
  com.microsoft
  sqljdbc
  1.2.0
 


 com.microsoft.sqlserver.jdbc.SQLServerDriver
 jdbc:sqlserver://mssql2005server:1433;databaseName=MyDatabase;SelectMethod=cursor
 secret
 squirrel
 src/test/resources/web-test-data.xml
 MSSQL_CLEAN_INSERT
 flat


 
  pre-integration-test
  
   operation
  
 

Hope it helps!

Create a default access rule for URLs using Spring Security

Using Spring Security, you can specify access rules for URLs in a declarative manner in a central configuration file. However, it's easy to forget a controller or two, which could have tragic, or at least embarassing, consequences!

One way to avoid that is to create a catch-all rule that will ensure that all URLs are secured. This will be fired if you access a forgotten URL, both prompting you to fix but also preventing anyone exploiting the mistake.

The key to it is realising that the first matching rule from the top is applied to each URL - therefore, defining a catch-all at the bottom will ensure that any unmatched URL is matched by the catch all. The pattern matching is based on Ant patterns, so /** will do the trick.

This is demonsrated below.

Note that the login and logout URLs have been set to ROLE_ANONYMOUS. Otherwise, attempts to access them will result in an infinite loop as the catch all will fire. ROLE_ANONYMOUS is the default role given to all unauthenticated requests by Spring Security - if you have overridden it, use the overridden name instead.

Using Spring Security with Tuckey URLRewriteFilter

If you are developing a servlets, JSF or Spring MVC application, it’s very likely you will want to implement declarative security to restrict access to functions based on URLs. It's also likely that you want to provide intuitive, SEO friendly and RESTFul URLs for your app.

Luckily, both of these requirements can be met by existing, high-quality open source frameworks in a way that is not intrusive to your application code. I’m referring to the Spring Security framework (what used to be Acegi Security) and the Tuckey URLRewriteFilter.

I won’t go in to the details of setting up both of these frameworks, as these are well covered in other excellent blogs:

http://sziebert.net/posts/restful-urls-with-spring-mvc-and-urlrewritefilter/
http://www.mularien.com/blog/2008/07/07/5-minute-guide-to-spring-security/

However, as I discovered after some false starts, getting both of these components to work together needs the correct configuration, which might not be apparent unless you really understand how Servlet Filters work.

Both frameworks rely on servlet filters to apply rules to requests – the Spring Security framework’s filter chain applies access decision rules to incoming requests and the URLRewriteFilter intercepts requests and forwards them to the re-written URL. The order in which these things occur is obviously quite important.

For example, consider this simplified Spring Security configuration (which goes in your application context).

This rule restricts the URL /clientlist.htm (which maps to a Spring MVC controller but could be a servlet or JSP) to users who are in the ROLE_MANAGER role. Note that the pattern matching is based on Ant pattern matching, and this is described in the Spring Security documentation.

However, we want this to be accessed on the more friendly URL of /clients. To do that we use a URL rewriting rule in the URLRewriter filters configuration file:

^/clients$
  /clientlist.htm?status=1

Of course, we would also need a corresponding outbound rule:

  /clientlist.htm\?status=1$
/clients

However, that’s not really important for the purposes of security. The important thing to note is that the URL Rewriter will take a request for /clients and forward it to /clientlist.htm?status=1. At the same time, the Spring Security filter applies access decisions to requests for /clientlist.htm.

Clearly, the order that the filters process the request in is important. For this to work, we would want the URL rewriting to happen first (translating /clients to /clientlist.htm) before the security rules are applied. Otherwise, Spring Security would need to be configured to match the re-written URLs rather than the “real” URLs. While this is technically feasible it might cause maintenance difficulties. Imagine how much head-scratching would be involved for a maintainer trying to understand why the Spring Security config doesn’t match up with the expected URLs for the Spring Controller definitions!

We all know (because we memorised the Servlets specification) that Filters are applied in the order that they are defined in the web.xml. So easy, we just put the URL rewriter in front of the Spring Security filter in the web.xml! Job’s a good ‘un, now what’s for lunch?

Not so fast... Just defining the filters in the right order is not enough on its own, especially if using the standard configuration examples provided in the documentation for both products.

The symptoms of an incorrect configuration would be that URL rewriting works but the Spring Security filter does not process the request, resulting in either an error (if your code expects an authenticated user) or in security not being applied.

That is because the URL rewriter forwards the request to the rewritten URL. Having memorised the servlet spec, we know that filters do not automatically process forwarded requests. By default, they only apply to direct requests. Luckily, this can be easily changed using the <dispatcher /> element in the filter-mapping configuration.

A working configuration of Tuckey URLRewriteFilter and the Spring Security filter including the additional <dispatcher>FORWARD</dispatcher> configuration is included below:

    UrlRewriteFilter
    
       org.tuckey.web.filters.urlrewrite.UrlRewriteFilter
    
    
       logLevel
       WARN
    
    
       confReloadCheckInterval
       
    




   UrlRewriteFilter
   /*

 


    springSecurityFilterChain
    org.springframework.web.filter.DelegatingFilterProxy  
  



    springSecurityFilterChain
    /*
    REQUEST
    FORWARD
    INCLUDE
    ERROR

Adding the <dispatcher /> elements within the filter-mapping element for the Spring Security filter will cause it to process the forwarded request. It means you can configure security based on the “real” URLs rather than the rewritten URLs. Note you should also make the filter apply to direct requests, otherwise you won't be able to access Spring Security URLs like j_spring_security_check (unless you use the URL rewriter to create a forward to this URL, which seems unnecessary).

As a final remark; note that any other filters that you want to apply to requests will need to be configured as above as well. For example, the Spring Hibernate “open session in view” filter that is needed for lazy initialisation and loading in the web tier:

    hibernateFilter
    
org.springframework.orm.hibernate3.support.OpenSessionInViewFilter
   
   
       sessionFactoryBeanName
       sessionFactory
    




    hibernateFilter
    /*
    REQUEST
    FORWARD
    INCLUDE
    ERROR

Hope this is useful to someone.

Service Method Caching and Hibernate LazyInitializationExceptions

I ran into a slightly baffling problem using a combination of the Caching Spring Module (0.9) with Ehcache, Spring 2.5.6 and Hibernate 3.3.1 in a servlet application. After adding a Spring @Cacheable attribute to a method in a service object that returned a Hibernate data object I started to get the dreaded LazyInitializationException when iterating through a child collection in the JSP view whenever I hit the page a second time – and I was already using the OpenSessionInViewFilter!

@Cacheable(modelId=projectCachingModel)
@Transactional(readOnly = true)
public Project getProject(Integer projectId) {
return projectDao.load(projectId);
}

What gives? It seems that if the Project object is returned from the cache rather than from the DAO (which, incidentally, extends Spring HibernateDaoSupport) then the lazy loaded collections within the object can’t bind on to the Hibernate Session opened in the view, hence the LazyInitializationException. This sort of thing can be hard to test in a normal unit test. Luckily, it's possible to simulate what's going on in the view in a unit test using a few useful utilities in the Spring framework:

org.springframework.transaction.support.TransactionSynchronizationManager;

Now you can open and close sessions in much the same way that they would within the view as you navigate between pages.

// Get Hibernate session
SessionFactory sessionFactory = (SessionFactory)ctx.getBean("sessionFactory");
Session s = sessionFactory.openSession();

// Bind session to transaction synchroniser
TransactionSynchronizationManager.bindResource(sessionFactory, new SessionHolder(s));

// Get Project (loads from database)
Project project = projectFacade.getProject(3);
  
// Close session
SessionHolder holder = (SessionHolder) TransactionSynchronizationManager.getResource(sessionFactory);
s = holder.getSession(); 
s.flush();
TransactionSynchronizationManager.unbindResource(sessionFactory);
SessionFactoryUtils.closeSession(s);
  
// Start new session
Session s2 = sessionFactory.openSession();
TransactionSynchronizationManager.bindResource(sessionFactory, new SessionHolder(s2));

// Reload Project (from cache)
Project reload = projectFacade.findProject(3);
// Test lazy loading/initialisation
  
// Close session
holder = (SessionHolder)TransactionSynchronizationManager.getResource(sessionFactory);
s2 = holder.getSession(); 
s2.flush();
TransactionSynchronizationManager.unbindResource(sessionFactory);
SessionFactoryUtils.closeSession(s2);