Using Data Mapper for Persistence Layer & In Memory Persistence for Testing

* I tried to improve usability by highlighting important words through the sentences. Please do give me feedback if it works.

Need for the separation of a persistence layer

The 3 layered approach to software development, consists of presentation, business and persistence layers.

Designing the architecture, the layers should be loosely coupled in order to provide easy refactoring, testing, reusability and change. What's more is the presentation layer should not be reaching the persistence layer directly ,and not the persistence layer nor the business layer should depend on the presentation layer, neither should have any knowledge about the presentation.

The business layer, is the layer that knows how the business is done, should contain the flow, the representation of the real world problem domain into the object oriented development model. The purchase of some items, the discount mechanism, checking user’s authorization to Access some functionality, the next step in the workflow, the algorithm to check for spelling etc. The real world objects does not care about how they are persisted or kept, they just know what they are doing and need to do. Keeping the knowledge of persistence in the objects, eg. User object knowing how to call itself from database or how to insert, does not suit the separation of concerns. This object knows too much, and is a point of risk in any change requirement.

The objects’ data in database in tables, and the design its implemented, has nothing to do with the domain object. There are just rows and columns in this RDBMS and these are subject to change due to performance issues or change of vendor, orm framework etc. And these changes should not be causing any changes on the domain model, whose responsibility is to know the business logic.

The Project may be using mssql server at the beginning but may in future need to be run on oracle or mysql. Using ansisql may seem like a solution, but there may be some transition to object databases (ODBMS) . Using an orm framework may seem like sufficient, but some need may arise to change the framework decided upon at the beginning later on, a Project may be using Hibernate for orm but may be needed to use another framework later on, for example. It is a very big risk to implement an orm framework without any possibility to convert it to some another, especially if it is newly being used by the team.

This matters are the concern of the persistence layer, and should not affect the business layer or some mechanism spread all over the code, the persistence logic should be kept in one place and be defined by interfaces, making it interchangable. In real world, tightly coupling this functionality may result in failed projects, due to not meeting the changing requirements.

One of the driving forces in the project I supplied some examples from was the presence of such an orm mapper. The framework is llblgen, used for dotnet, and the implementation needed to handle the rdbms by this framework. Using such a framework, without prior investigation, the team will surely have some questions about performance, usage, bugs, usability, coding speed. It is too risky to implement it without abstracting it from the rest of the system, resulting in a disasterous rewrite of the whole persistence layer afterwards, if possible.

The Data Mapper Pattern

Studying on the subject, the best match I came across is the Data Mapper Pattern, in Martin Fowler’s Patterns of Enterprise Application Architecture.

A Data Mapper is an object which knows how to persist an object or how to interact with the mechanism persisting the object.

Persisting a person object for example, there is a person object, which knows nothing about persistence, just the stuff the business needs. It need not to implement any interface or something. Just a plain object.

This object is sent to a mapper, and this mapper, knows how to persist a Person object on a specific persistence medium, nothing more. Another mapper class is needed for another class, such as, PersonTitleMapper for PersonTitle class. If the object is to be persisted on Oracle and no orm framework is used, the PersonMapperOracle will do the stuff for oracle, another mapper class PersonMapperMssql will do the stuff for sql server. What PersonMapperOracle and PersonMapperMssql should have in common is the PersonMapper interface, which knows the required functionality for all PersonMappers.

The persistence layer should know how to persist (fit) the domain objects to the rdbms, so passing the object onto the persistence layer should do the job. The layer responsible for organizing or keeping the object together (the glue), the controller of mvc (smalltalk) in my case, passes the domain object to the persistence object and orders persistence object to select,insert, update,delete it from the database. The persistence layer knows which mapper to use for the object and uses the layer accordingly. Each domain object has a mapper, and each mapper only knows how to interact with the database for the object it is mapping only.

There can be multiple mappers for an object and each mapper could be referring a different orm system, rdbms vendor or persistence mechanism. That way it is possible to make the persistence interchangable, using either xml, odbms, xml, oracle, sql server etc, one just has to write the mappers for each persistence mechanism and the domain object.

It may seem like that it is overkill for a project using an orm tool, but it is not, as the project being very much dependent on the orm mechanism, being tightly coupled on it, may result in headaches. Whether you are using Hibernate or ado.net or anything, the persistence layer should pack the stuff together, allowing it to be expandable and changable, without making the business model complex.

Database Design And Testing

Database design and testing is another issue. In the project mentioned about, being a team consisting of developers specialized in different areas, one person doing the orm, other doing the database structure, other one testing and another one implementing the business layer, I had to come up with a solution that everybody can work at the same time without waiting for the other one, working on the interfaces. What's more, the orm framework that is to be used was completely a stranger to the team,it had to be learnt during the project development process, putting everybody to work on it was just waste, and everybody waiting it to be finished was a bigger waste. I had to separate the concerns.

The solution I came upon was building a Dummy Persistence Layer that could be changed to a RDBMS layer on runtime or compile time, so that the team would not be waiting for the database to be ready for coding and testing.

Using a singleton object on application server (web server that is) I persisted the objects on the memory, using memory mappers for each object, like mock objects. So the business layer kept being developed, the orm mechanism was being developed, the interfaces of the domain objects were being made up, as at the same time another developer was designing the rdbms.

I inserted dummy objects into the code that will be present when the software runs, and any insert update or select is run on the mappers that map to the objects kept in the memory. The data was discarded on a restart of the computer or the web server, but the data coded into the mappers were kept, so that the tests would run as expected.

Implementation

Code below is C#, but could be used for any language.

First, an interface or a base object was needed to gather the base functionality of the mappers. Thus the IMapper interface.

public interface IMapper { object getObject(object domainObject); object insertObject(object domainObject); bool updateObject(object domainObject); bool deleteObject(object domainObject); ToolBox.Collection.Collection getAllCollection(); }

All mappers implements this interface. Also, interfaces regarding the domain objects themselves are needed, so that the interfaces can be sam efor all persistence mechanisms. The implementation will vary depending on the mechanism, but the interface functions shall not.

The comment mapper, responsible for handling the interaction of the Comment domain object with the persistence.

public interface ICommentMapper:IMapper { Galleon.ModelObjects.UserInput.CommentCollection GetCommentsOfCommentable(Galleon.ModelObjects.UserInput.ICommentable commentable); Galleon.ModelObjects.UserInput.CommentCollection GetCommentsFromCommentCollection(Galleon.ModelObjects.UserInput.CommentCollection CommentCollectionWithIds); void InsertRelatedObjects(object comment,object commentOwner); }

The party mapper, responsible for handling the interaction of the Party domain object with the persistence.

public interface IPartyMapper : IMapper { ToolBox.Organization.Party ConstructParentParty (ToolBox.Organization.Party _party ); ToolBox.Organization.PartyCollection GetFirstHierarchyLevelParties(); ToolBox.Organization.PartyCollection GetFirstChildrenOfParty(ToolBox.Organization.Party _party); ToolBox.Collection.Collection FillDomainCollectionWithEntities(ToolBox.Collection.Collection col,object ec); void ConstructGrantedParties(ToolBox.Authorization.Role role); PartyCollection GetLeadersOfParty(Party party); Galleon.ModelObjects.Organization.GalleonWallet InsertWallet(Party party); bool DeleteWallet(Party party); bool InsertLeadersOfParty(Party party); bool DeleteLeadersOfParty(Party party); ToolBox.Organization.PartyCollection GetGrantedParties(int roleId); }

I used a façade (GOF) layer to hide the details of combining the mapper and the domain object, implementing functions such as InsertPerson (Person objectname) which know how to retrieve the suitable mapper and order it to execute some action.

All Mappers inherit the IMapper interface. Best practice is to have as many mappers as persistent objects are. Mappers implements the general methods such as insert, update, delete etc., needed by the IMapper Mappers also implements the appropriate interfaces specialized for their types. The interfaces that mappers should implement are at least the IMapper and the interface that is unique for their type (eg. Comment - Party) By this approach, it is possible to build other mappers that can be used instead of another persistence medium's mappers, as they share the same interface.

The Bonding Mechanism - Persistence Factory

A mechanism for selecting the current persistence medium was needed, directing the persistence orders to the right mechanism and the mappers, and an abstract factory pattern (GOF) was the suitable pattern for this job. Implementing this pattern, defining a base factory which defines the abstract methods that a persistence factory in this architecture should implement.

Another simple factory is returning the persistence factory that needs to be executed to return the appropriate mappers. It is a good practice to implement the mappers and factories as singletons (GOF) , as data per instance of this objects is not needed.

Abstract PersistenceFactory

public abstract class AbstractPersistenceFactory { public AbstractPersistenceFactory(){} public abstract ICommentMapper ConstructCommentMapper(); public abstract IPartyMapper ConstructPartyMapper(); }

PersistenceFactory

public class PersistenceFactory { public PersistenceFactory() {} public static AbstractPersistenceFactory getPersistenceFactory(PersistenceType pt) { switch (pt) { case PersistenceType.LLBLGenPersistenceFactory_Prod: LLBLGenFactory.ConnectionString=System.Configuration.ConfigurationSettings.AppSettings.Get("Main.ConnectionString"); return LLBLGenFactory.getInstance(); case PersistenceType.LLBLGenPersistenceFactory_Test: LLBLGenFactory.ConnectionString=System.Configuration.ConfigurationSettings.AppSettings.Get("Test.ConnectionString"); return LLBLGenFactory.getInstance(); case PersistenceType.DummyPersistenceFactory: return DummyFactory.getInstance(); default: return LLBLGenFactory.getInstance(); } } }

PersistenceType

public enum PersistenceType {LLBLGenPersistenceFactory_Prod,LLBLGenPersistenceFactory_Test,DummyPersistenceFactory}

DummyFactory

public class DummyFactory : AbstractPersistenceFactory { protected DummyFactory(){} protected static DummyFactory instance; public static DummyFactory getInstance() { if (instance==null) {instance=new DummyFactory();} return instance; } public override ICommentMapper ConstructCommentMapper() {return CommentMapper.GetInstance();} public override IPartyMapper ConstructPartyMapper() { return Galleon.Persistence.Mappers.Dummy.PartyMapper.getInstance(); } }

The calling sequence of the functions is;

1. Façade / Controller or any other object calls the getPersistenceFactory from PersistenceFactory (simple factory pattern), supplying PersistenceType object.


2. PersistenceFactory creates the appropriate factory object, in this case, dummyfactory. (Factories are singleton, so getInstance is called, which returns the factory instance.)


3. PersistenceFactory returns the factory.


4. Façade / Controller or any other object, wants the commentmapper from dummyfactory.


5. DummyFactory returns the suitable comment mapper instance for dummy persistence (mappers are singleton, so getInstance is called)


6. Façade / Controller or any other object wants the Comment object, supplying a comment object whose Id is set.


7. Comment Mapper provides a filled object, using the Object Id.