Microservices: Splitting relational data model conundrum!
Introduction
Microservices has been a popular design pattern for the last few years as an alternative design philosophy to break the rigidity of monolithic design. Over the years, applications have been built as monolithic design with relational data model persisted using common application schema to the underlying data stores such Oracle, MySQL, Postgres, MongoDB etc. Since databases store the data on the disk and make it available on demand, they have been a popular way to integrate two functions. One function leaves the entries in the database and the other picks it from there. Since both functions share the data schema, while reengineering the monolithic application into microservices, one the most dreadful conundrum is to split the data model by the microservices.
Shared data model
Microservices advocate loose coupling & strong cohesion as core principles. To make an ideal microservice, the service logic along with data interaction needs to define a bounded context. ‘Bounded context’ is specific responsibility enforced by explicit boundaries. This means we need to find boundaries in our relational databases, so that we can split them out cleanly. But as Sam Newman quotes in his book “Building Microservices” — Databases, however, are tricky beasts. So what makes databases a tricky beast?
- Most relational databases have data entities inter-linked via foreign key constraints. This results in one entity being used by multiple modulus as a shared data model.
- Unlike code logic which can be replaced, database records can’t be replaced but have to be migrated.
For instance, say an order processing module tracks payments made by users for their purchases, and also tracks refunds given on return of items. Meanwhile, the inventory management module updates inventory records to show that items for users have been dispatched or received. All of this data is displayed at one place in the app so that users can see their account holistically.
To keep things simple, we have stored all this information in a fairly generic user, inventory & order tables but in the real world it would ideally be multiple related tables joined to each other for each of the three entities.
Shared model as Service
One of the most ideal solutions to deal with the shared data model between modules is to identify the bounded context for the shared model and expose it as a service. Say the service is created to expose User model and now Inventory Management service and Order Processing service both use the User service.
Breaking the monolithic system into smaller microservices as Inventory Management service, Order Processing service and User service does come with two trade offs:
Performance challenge: Inventory Management service or Order Processing service have to make two calls first one to user service for user data and then to inventory table or order table to get their data before the business logic is applied. Would this lead to a performance challenge? Performance needs to be benchmarked to decide the trade off — Would making one thing slower in exchange for other things is the right thing to do, especially if slower is still perfectly acceptable?
Dropping referential integrity: Referential integrity between the order, inventory and user table are no longer maintained at database schema level. These integrities have to be enforced programmatically in the respective modules. For instance, Order Processing service needs to make sure the order is associated with a valid user before calling the user service.
Conclusion
Though splitting the data model by the bounded context and exposing it as a domain specific service is an ideal solution, it is easy said, difficult done. Often the extent of code and database refactoring it calls for enforces the product engineering teams to leave the databases as monolithic schema even if code logic is broken into smaller services (Restful APIs in many cases). Delivery pressures often leave this technical debt to be addressed for the future. However dismantling shared data model, one shared entity at time allows product engineering teams to address this problem in staggered manner over multiple milestones. Once addressed, it aligns the microservice architecture principle of loose couple & strong cohesion.
References
- Building Microservices — Sam Newman (O’Reilly Publication)
- Monolith to Microservices — Sam Newman (O’Reilly Publication)
