You can find useful information about service-oriented architecture (SOA) and microservice architecture (MSA) here. Especially you may get interested in telling the difference between hypes and reality of microservices. The conclusion is SOA is a must, but MSA is an option for SOA. As Sam Macklin of CA Technology said, “In the future, every successful company will be a software company and every successful software company will be an API provider.”
Definition of Microservice
There is no formal definition of microservice. Microservice is a collection of emerging patterns of service-oriented architecture (SOA) that have been proven effective in a number of companies including Amazon, Netflix, Gilt, REA Group, Google, eBay, Twitter, LinkedIn, SoundCloud, Uber, Airbnb, Hailo, etc. Microservice emerged out of the business needs of those companies to stay at the top of competition in their respective markets.
See what other people said about microservices. Adrian Cockcroft, VP of Cloud Architecture Strategy at Amazon, described microservices as loosely-coupled service-oriented architecture with bounded context. Here bounded context is a term defined by Eric Evans in his book on Domain-Driven Design. It means an explicitly defined context for each service within which a domain model expressing a ubiquitous language exists. Refer to the Business Analysis page in this web site for the definition of domain model.
James Lewis, Principal Consultant at ThoughtWorks, expressed his view on microservices as developing an application as a suite of small services, each running in its own process and communicating with lightweight … API. … built around business capabilities and independently deployable by fully automated deployment machinery.
Let's look into more details about what characterizes microservices, next.
Microservice Benefits and Drawbacks
Figure 1. Netflix' Fine-Grained Service-Oriented Architecture
Understanding the business needs for microservices of the above listed companies which have pioneered microservices will be helpful in understanding why microservice came about and what it tries to achieve. The companies listed above have some common characteristics and common business needs. They are probably the most advanced digital companies in the world. Their businesses are software-intensive, deliver web-based services on a global scale, run entirely on the cloud, and are continually innovated setting examples for other companies in their respective industries. What their businesses need in common is to change their business models and processes fast enough so as to fend off imitators, and to change their applications fast enough to deliver incremental values to customers frequently.
What they had as impediments in their way was monolithic applications hard and slow to change. Their applications were already layered, modular and service-oriented, but packaged and deployed as a monolith. As a result, application is too large and complex to fully understand and make changes fast and correctly; the sheer size of application can slow down the start-up time; they must redeploy the entire application on each update; impact of a change is usually not very well understood which leads to extensive manual testing; application can be difficult to scale when different modules have conflicting resource requirements; bug in any module can bring down the entire process; it is difficult and extremely expensive to adopt new technologies since changes in frameworks or languages will affect an entire application. (https://articles.microservices.com/monolithic-vs-microservices-architecture-5c4848858f59)
To move faster they had to devise a way to make changes simpler. The solution was what are now regarded microservice architecture (MSA) patterns. They divided the monolith into a large number of small SOA services; made these services as independent as possible; minimized inter-service dependencies by having each service physically contain and exclusively access its own database; and had a single team of small size with deep knowledge of the bounded context both develop and operate each service.
The benefits of MSA include: it tackles the problem of complexity by decomposing application into a set of manageable services which are much faster to develop, and much easier to understand and maintain; it enables each service to be developed independently by a team that is focused on that service; it reduces barrier of adopting new technologies since the developers are free to choose whatever technologies making sense for their service and not bounded to the choices made at the start of the project; it enables each service to be deployed independently, thus making continuous deployment (CD) possible for complex applications; it enables each service to be scaled independently.
Netflix, a company serving movies and TV shows to more than 75 million global subscribers, began converting its monolithic, datacenter-based Java application to cloud-based Java microservices and migrated those services to AWS in 2008. (https://medium.com/netflix-techblog/how-we-build-code-at-netflix-c5d9bd727f15) Netflix' software engineering went from a few teams checking code into a large monolithic application running on tens of servers to having tens of engineering teams developing hundreds of component services that run on thousands of servers. Figure 1 shows hundreds of fine-grained SOA services in Netflix that communicate with each other using a light-weight REST based protocol. (G. Oliffe, You Are Not Netflix: How and When to Use Microservices in the Enterprise, Gartnet Catalyst Conference, 2016.)
However, there are serious drawbacks of MSA if your company is not ready with the skills, methods, tools, frameworks, platforms and infrastructure required to build and operate MSA applications properly: MSA is essentially an extremely distributed computing system which requires a complex web of inter-process communication based on either messaging or RPC; it requires writing code to handle partial failures and other fallacies of distributed computing; it has the partitioned database architecture with each service physically containing its own database, and therefore, transaction management to preserve ACID properties is more challenging; it is more complex to test services because they typically depend on many other services; it requires each instance of each service to be configured, deployed, scaled and monitored, and this level of complexity requires a high level of automation.
Martin Fowler advised that if you don't have certain baseline competencies, you shouldn't consider using the microservice style. He listed rapid provisioning, basic monitoring, rapid application deployment, and DevOps culture as required competencies. (https://martinfowler.com/bliki/MicroservicePrerequisites.html)
Microservice Architecture Adoption Strategy
The decision of whether to adopt microservice architecture (MSA) in an enterprise is not an all-or-nothing proposition. Depending on the nature of business, the maturity level of IT, the competency level of software engineers, all of the MSA patterns found in the pioneering companies may not be necessary or feasible.
To select appropriate MSA patterns to adopt in an enterprise, the company should think about the business goals to achieve and/or the urgent business or IT problems to solve by adopting MSA. Business drivers for adopting MSA of early adopters have been:
Fast Release Cycle: Pushing each feature in one cycle using IaaS+PaaS and DevOps automation.
Global Scalability: Automated scaling based on IaaS+PaaS.
Technological Diversity: Allowing multiple languages, development frameworks, data storage technologies, infrastructure types within an application.
Large Development Organization with Small Independent Teams: Small autonomous teams (1~10) comprised of fully empowered, competent developers independently producing microservices.
In some companies, application users or product customers don't need or want fast upgrades. Not many companies need extreme agility, global-scale scalability and technology diversity that drove the MSA patterns found in those pioneering companies. If your company doesn't really have any of the above drivers, you should step back and now think about the benefits of service-oriented architecture (SOA) in general. MSA evolved as a new style of SOA exploiting new advances in computing technologies such as IaaS, PaaS, containers, container management tools, CI/CD automation tools, API gateways, and backing services such as persistent services and messaging services, and telemetry including monitoring, logging, etc. Even if your company can't use all these new technologies enabling the emerging MSA patterns, you can still pursue benefits of SOA by selecting those SOA patterns and some of the new MSA patterns that fit your business purpose and can be handled with your current IT capabilities.
In fact, Gartner predicted that by 2019, more than 90% of organizations that investigate MSA will find the paradigm too disruptive and adopt miniservices instead. Here miniservices mean SOA services that have relaxed architectural constraints than state-of-art microservices; for example, sharing a database instead of physically owning one; using RPC or orchestration instead of choreography based on the publish-and-subscribe, event-driven messaging, etc. (Gartner, Innovation Insight for Miniservices, 2017.)
Why should you consider SOA while there are more advanced architectures like MSA, even-driven architecture, etc? SOA remains the foundation of modern application architecture. Understanding principles of SOA, its benefits, design patterns, and engineering methods to build SOA applications are all still essential to understand and implement MSA correctly. Refer to: Gartner, Where to Start (or Restart) with Service-Oriented Architecture, 2016. SOA contributes to many patterns of software business: service-oriented architecture pattern and mass customization pattern for software product business, business analysis expertise pattern and asset-based service pattern for professional IT service business, self-service pattern for cloud service business, continuous deployment pattern and partner ecosystem pattern for SaaS business, IoT reference architecture pattern, pace-layered application architecture pattern and system of systems pattern for digital business. SOA is also a critical factor in business analysis as mentioned in domain & engineering expertise pattern, business requirements specification pattern, business service model pattern, business process model pattern, use case-object model mapping pattern, software requirements metamodel pattern for business analysis.
If your company aspire any of the SOA benefits discussed below, but has not institutionalized SOA yet, you should start the journey of adopting SOA. MSA patterns can be adopted selectively along the way, but you will have less risk after you have built a solid foundation in SOA.
SOA Benefits
Figure 2. Service-Oriented Architecture using Process Orchestration to Compose Services
The benefits of (i.e., drivers for, goals of, or values of) SOA, which are also benefits of MSA, include:
Business-IT Alignment: SOA services are designed to encapsulate business process activities and the information used by the activities so that the business model is faithfully represented by the collection of services. New business requirements can be addressed by establishing a service layer capturing the new business model and wrapping useful functionalities in legacy applications, with the effect of increasing the speed and reducing the cost of new application development. The business requirements specification pattern for business analysis recommends identifying business services in the business analysis phase of application development, which will be detailed and implemented as SOA services in later phases. The business service model pattern for business analysis recommends defining SOA services from business viewpoint when designing the enterprise architecture, and making each service to encapsulate closely related business processes and business objects. The business process model pattern for business analysis recommends a combined use of SOA and BPM (business process management) to enable dynamic process orchestration which facilitates in-flight process reengineering in response to fast changing business environments. Figure 2 shows an SOA where services are composed dynamically using process orchestration.
Service-Oriented Enterprise Architecture: SOA services are united while maintaining their individual autonomy. Business capabilities are expressed by service APIs in a standardized manner so that they can be easily composed into various applications across the enterprise. J. Ross, et al. showed, based on a survey of more than 200 companies, that companies that were most mature in enterprise architecture (EA) had achieved componentization of business capabilities with standard interfaces for business modularity (J. Ross, et al., Enterprise Architecture as Strategy, Harvard Business Review Press, 2006.) SOA has become a de facto standard for EA among global companies by around late 2000s. Figure 3 shows the metamodel for service-oriented enterprise architecture recommended in TOGAF 9. Refer to: The Open Group, Using TOGAF for Enterprise SOA (http://www.opengroup.org/soa/source-book/togaf/p4.htm). The domain & engineering expertise pattern for business analysis (discussed in Business Analysis page of this web site) recommends SOA to be an essential competency for business analysts (BAs) as well as enterprise architects today.
Figure 3. TOGAF 9 Metamodel for Service-Oriented Enterprise Architecture
Universal Interoperability: SOA services are natively interoperable using APIs for inter-service communication. Applications in SOA can be easily integrated via APIs, with the effect of far reducing the efforts and costs required for application-to-application (A2A) integration as well as business-to-business (B2B) integration. Using standard API technologies such as SOAP and REST web services, any piece of software wrapped in published API can be called by any other software through the Internet. While the Internet enabled a computer to be connected to any other computer in the world if it has an IP address, the Internet-based web services enabled a piece of software to be connected to any other software in the world if it has a published API using standard protocols.
As was discussed in Software Business page of this web site, cloud services are SOA services using standard SOAP or REST web services. Amazon, which pioneered IaaS and branded it as Amazon Web Services (AWS), adopted SOA enterprise-wide in 2002, published web service API of its product data to third-party affiliates—Web sites that advertised Amazon products using the Amazon’s API and received a portion of Amazon’s resulting sales--in the same year, and expanded the web services to sell its storage, computing and other technology services to software developers the next year 2003. Digital businesses connect things, people and businesses together, and they are all connected via API. Even endpoint devices expose REST API. A quote on this point by Sam Macklin of CA Technology: “In the future, every successful company will be a software company and every successful software company will be an API provider.”
The layered cloud architecture pattern and self-service pattern for cloud service business, partner ecosystem pattern for SaaS business, and IoT reference architecture pattern, product servitization pattern, and API business pattern for digital business are all based on this universal interoperability pattern of SOA.
Implementation Independence: Service APIs are decoupled from the service implementation so that implementation technologies can be changed without disrupting the federated application architecture, with the effect of prolonging the lifespan and ROI of investments in applications. The use case-object model mapping pattern for business analysis recommends that use cases and classes that have a high degree of affinity are encapsulated into an SOA service. This allows service APIs to be described in business terminologies in an implementation agnostic way.
Asset with Long-Term ROI: Services encapsulating common functionalities and data can be reused across different applications saving time and cost. Reusable SOA services are an important asset in any enterprise that can generate recurring returns over a long term. This benefit of SOA was mentioned earlier in the asset-based service pattern for professional IT service business.
Business Agility: Enterprises with a service inventory comprised of more reusable SOA services take less effort and time to change business processes and applications. This benefit of SOA was mentioned earlier in the pace-layered application architecture.
SOA and MSA Principles
Kent Beck suggests a useful distinction between values, principles and practices in software engineering (K. Beck, Extreme Programming Explained, 2nd ed., 2005). Practices are things you actually do repeatedly--such as a clustering analysis on an affinity matrix to generate a first-cut design of business services (discussed in Business Analysis page of this web site). Values are purposes of practices. The above listed drivers (i.e., goals, benefits) of SOA and MSA are values generally sought after by enterprises when investing in SOA or MSA. If your company doesn't place a high value on business agility, it may not choose certain MSA patterns such as continuous deployment. Practices should be chosen based on the values you company thinks are important. The values pursued by different companies can be different.
Principles are guidelines that bridge the gap between values and practices. For example, the Standardized Service Contract principle discussed below leads you to adopt standard protocols such as SOAP or REST web services across your enterprise in practice, so that the Universal Interoperability value can be realized. SOA principles are discussed next. It is followed by discussions on SOA and MSA patterns. Principles listed below are adopted from Thomas Erl, SOA: Principles of Service Design, Prentice-Hall, 2007. (http://serviceorientation.com/) These principles apply to both SOA and MSA.
Standardized Service Contract: Services within the same service inventory should be in compliance with the same contract design standards. A service inventory is an independently standardized and governed collection of complementary services within a boundary that represents an enterprise or a meaningful segment of an enterprise. This principle is applied when designing a service's public technical interface and determining the content to be published as part of a service's official contract. REST uses uniform contract consisting of resource identifier (e.g., URI), media type (e.g., XML) and method (e.g., HTTP) SOAP web services have several areas affected by the standardization including the WSDL definition of service metadata, XSD schema and WS-policy.
Service Loose Coupling: A service contract should impose low coupling to its service consumers, its implementation and surrounding environment. This principle promotes the independent design and evolution of a service's logic and implementation while still guaranteeing baseline interoperability with consumers relying on the service's capabilities. Figure 4 shows various types of potential coupling that should be minimized: coupling between service consumer and service API, coupling between service API and service logic, coupling between service logic and service implementation, coupling between service logic and backing services such as shared databases, coupling between service logic and other services, coupling between business process and service API or logic. Many SOA design patterns are applied to attain loose coupling.
Figure 4. Service Coupling (Adopted from http://serviceorientation.com/)
Figure 5. Autonomous Services with Dedicated Databases
Service Abstraction: Service contracts should only contain information limited to what is published in service contracts. Service abstraction hides as much of the underlying details of a service as possible to maximize loose coupling.
Service Reusability: Reusable services with functionalities and data common to many applications should be maximally identified.
Service Autonomy: Services should exercise a high level of control over their underlying runtime environment. Service autonomy can be compromised by overlapping (i.e., non-normalized) service contracts, shared logic, shared database, and shared physical resources (e.g., servers). Several microservice patterns aim at improving autonomy of services (i.e., independence among services) beyond conventional SOA services--such as keeping persistent data dedicated to a service, using event-based choreography instead of process orchestration, etc. Dedicated databases also facilitate polyglot persistence, allowing a more autonomous choice of data storage model for each service, as illustrated in Figure 5.
Service Statelessness: Services should minimize resource consumption by deferring the management of state information when necessary. The management of excessive state information for serving different clients and tasks can compromise the scalability and availability of a service. Services are therefore ideally designed to remain stateful only when required. A set of dedicated database tables can be used for temporary relocation of state information (viz., state deferral).
Service Discoverability: Services should be supplemented with communicative metadata and registered in service registry so that they can be effectively discovered and interpreted.
Service Composability: Services should be effective composition participants. A composition controller service can be developed to compose other services. Alternatively, an orchestration platform (e.g., BPMN or BPEL engine) can be used to coordinate services to carry out a business process. Microservices prefer choreography based on pub/sub messaging.
Intel is exemplary in honoring SOA principles. The company established a guideline regarding web services that requires that: All software products and infrastructure (including commercial on-premise software, public SaaS, internally developed applications, private SaaS, private IaaS, public IaaS) should include a consumable API; APIs should be logically layered to provide useful abstraction; APIs should be discoverable at runtime; Functionality provided through CLI or GUI must also be available through an API, and furthermore implemented as wrappers for APIs; APIs should support role-based access with proper authentication and authorization; REST is preferred to SOAP; JSON is preferred; There should be loose association between clients and API; Services should be stateless and all state data be returned to the client; Asynchronous messaging is preferred; Services should be fault-tolerant. Refer to: Intel IT, Maximizing Cloud Advantages through Cloud-Aware Applications, 2013. (https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/maximizing-cloud-advantages-through-cloud-aware-applications-paper.pdf)
SOA and MSA Service Design Patterns
A pattern for SOA or MSA is a general, reusable solution for, or a general practice to solve a commonly occurring problem in software architecture. Certain practices are needed to realize a pattern. Each pattern observes certain principles and provides certain values. Values and principles of SOA and MSA are discussed above in the MSA Benefits and Drawbacks, SOA Benefits, SOA and MSA Principles sections.
SOA and MSA design patterns can be classified into service design patterns at the individual service level, service composition patterns at the business process or application level, and service inventory patterns at the enterprise level. A service composition is an aggregate of services collectively composed to automate a particular task or business process. To qualify as a composition, at least two participating services plus one composition initiator need to be present. A service inventory is an independently standardized and governed collection of complementary services within a boundary that represents an enterprise or a meaningful segment of an enterprise. Services for multiple solutions can be designed for delivery within a standardized, enterprise-wide inventory architecture wherein they can be freely and repeatedly recomposed. Refer to: Thomas Erl, SOA Design Patterns, Prentice Hall, 2009. (http://soapatterns.org/)
Let's start with service design patterns.
Model-Driven Design: The identification of services in a business domain requires grouping into a service a set of business tasks (or use cases) and business objects that are closely related and together provide a discrete business capability. Service analysis activities, therefore, typically involve modeling of business processes, business semantics, business use cases, etc., and deriving a domain model based on those conceptual-level models. This domain model is increasingly used to define SOA and MSA services as discussed later in the bounded context pattern for specification-level SOA service design. This pattern is essential for supporting the service abstraction, composability, reusability and loose coupling principles. The Business Analysis page in this web site explains a systematic engineering process to derive the domain model from business models.
The domain model should be tightly related to the code giving the code meaning and making the model relevant. Model-driven design puts analysis and design in a single iterative loop so that a model faithfully express key concepts of the domain and at the same time practical for implementation. Development becomes an iterative process of refining the model, the design, and the code as a single activity. Refer to: E. Evans, Domain-Driven Design, 2003.
Object-oriented analysis, design and programming with help of tools can serve this purpose. Java and many other tools allow the creation of objects and relationships directly analogous to business requirement models as shown in the Business Analysis page. With a model-driven design, a portion of the code is an expression in the model; changing that code changes the model. Programmers are modelers whether anyone likes it or not. So it is better to set up the project so that the programmers do good modeling work. Refer to the hands-on modeler pattern for business analysis in the Business Analysis page.
Bounded Context: The concept of bounded context is defined in Eric Evans, Domain-Driven Design, 2003 [DDD]. Each service should have a explicitly defined context, called bounded context, within which a domain model expressing a ubiquitous language exists. Figure 6 shows an example of defining bounded contexts on a domain model. The domain model expressed in UML class diagram is partitioned into three packages, each representing a bounded context. The domain model is explained in the Software Requirements Specification section in the Business Analysis page, and in the Software Requirements Metamodel section in the VOLF (value-obsessed lean framework) page in this web site.
Latest comments
21.03 | 17:18
Took me time to read all the comments, but I really enjoyed the article. It proved to be Very helpful to me and I am sure to all the commenters here! It’s always nice when you can not only be informed
05.02 | 01:55
Very good job of discussing both MSA as well as SOA and providing adoption considerations etc.
10.09 | 15:53
Thank you for covering Mendix in your blog so comprehensively. The Mendix community would welcome your participation should you wish to join?
Share this page
Figure 19 shows a context map where the role of each service in the relationship between a pair of services is specified as either upstram or downstream, and the pattern of service composition to be applied to the pair is specified. Refer to: Dino Esposito and Andrea Saltarello, Microsoft .NET: Architecting Applications for the Enterprise (2nd ed.), Microsoft Press, 2014. (https://www.microsoftpressstore.com/articles/article.aspx?p=2248811&seqNum=3)
Conformist: When two development teams have an upstream/downstream relationship in which the upstream has no motivation to provide for the downstream team's needs, the downstream team is helpless. Eliminate the complexity of translation between bounded contexts by slavishly adhering to the model of the upstream team. Also, you will share a ubiquitous language with your supplier team. If these trade-offs are not acceptable, but the upstream dependency is indispensable, the second option still remains: Insulate yourself as much as possible by creating an anticorruption layer, an aggressive approach to implementing a translation map.
Separate Ways: Integration is always expensive. Sometimes the benefit is small. Just because features are related in a use case does not mean they must be integrated. Declare a bounded context to have no connection to the others at all. The features can still be organized in middleware or the UI layer.
Open Host Service: When a subsystem has to be integrated with many others, customizing a translator for each can bog down the team. There is more and more to maintain, and more and more to worry about when changes are made.
Define a protocol that gives access to your subsystem as a set of services. Open the protocol so that all who need to integrate with you can use it. Other teams are forced to learn the particular dialect used by the host team. In some situations, using a well-known published language as the interchange model can reduce coupling and ease understanding.
Figure 20. Service Composotion Patterns from Domain-Driven Design (Adopted from E. Evans, DDD, 2003)
Messaging Metadata: Because messaging does not rely on a persistent connection between service and consumer, it is challenging for a service to gain access to the state data associated with an overall runtime activity. Message contents can be supplemented with activity-specific metadata that can be interpreted and processed separately at runtime. (See Figure 10.) This pattern requires a messaging framework that supports message headers or properties. This pattern is related with the standardized service contract principle. Refer to: T. Erl, SOA Design Patterns, 2009.
Figure 21. Messaging Metadata in SOA (Adopted from T. Erl, SOA Design Patterns, 2009)
Service Agent: Service agents (a.k.a. filters, listeners, interceptors, handlers) is an event-driven program capable of transparently intercepting and processing messages sent to or from services. Common utility functions (e.g., encryption, authentication, validation, logging, routing) can be deferred to event-driven programs that don't require explicit invocation, thereby reducing the size of service compositions. Service agents can be designed to automatically respond to predefined conditions without invocation via a published contract. Reliance on service agents, however, can further tie an inventory architecture to proprietary vendor technology. Governance can also become an issue in that service agents need to be maintained with understanding of inventory-wide impacts of their changes. Refer to: Ibid.
Asynchronous Service Messaging: Various asynchronous messaging mechanisms can be applied to exchange messages among services. Queuing allows services to exchange messages via an intermediary buffer so that the services process messages independently by remaining temporally decoupled. Message paths can be dynamically determined by service agents for various types of intermediate routing logic such as content-based routing, load balancing, etc. Message delivery can be guaranteed by having service agents track messages, manage the issuance of acknowledgements, and persist messages during failure conditions, at the cost of added processing overhead and performance degradation. When building services as SOAP web services, this pattern is commonly applied by implementing a combination of the WS-Reliable Messaging standard and guaranteed delivery extensions, such as a persistent repository.
Publish-and-Subcribe Messaging: Events that occur within the functional boundary encapsulated by a service may be of relevance to other services, but without resorting to inefficient polling-based interaction, the other services have no way of learning about these events. The related other services establish themselves as subscribers of the service. The service, in turn, automatically publishes notifications of relevant events to its subscribers. A messaging framework is implemented capable of supporting publish-and-subscribe message exchange and associated complex event processing.
Enterprise Service Bus: An enterprise service bus represents an environment designed to foster sophisticated interconnectivity between services. It establishes an intermediate layer of processing that can help overcome common problems associated with reliability, scalability, and communications disparity. Enterprise service bus (ESB) is comprised of the co-existent application of many service design and composition patterns such as service adapter, service agent, asynchronous service messaging, publish-and-subscribe messaging, policy centralization, and rules centralization patterns. Refer to: Ibid.
Service Orchestration: Service orchestration integrates multiple services together to automate a process, creating a composite application. Orchestration environments such as Business Process Management Suite (BPMS) products supporting BPMN 2.0 modeling, BPMN 2.0 execution, and WS-BPEL execution support sophisticated and complex service composition logic that can result in a long-running runtime process. The orchestration pattern is a composite pattern comprised of several other service design, composition and inventory patterns including process abstraction, process centralization, rules centralization, service adapter, service agent, atomic service transaction, compensating service transaction, and state repository patterns, and often is used together with the ESB pattern. Refer to: Ibid.
Service Choreography: Service choreography is another way to create a composite application in service-oriented architectures. A service choreography model works without a central orchestrator. In service choreography, the participating services each know the business logic and sequence and timing of message exchanges, while in service otchestration, the participating services don’t know that they are being orchestrated as part of a higher-level service; only the central controller knows the business logic and messaging sequence. In service choreography, the logic of the message-based interactions among the participants is specified from a global perspective, while in service orchestration, the logic is specified from the local point of view of one single participant, called the orchestrator.
We have to deal with the problem of managing business processes that stretch across the boundaries of individual services. With microservices we hit this limit sooner than usual. While with orchestration we rely on a central coordinator like the conductor in an orchestra, with choreography each service work out the details like dancers in a ballet. A service publishes an event in an asynchronous manner, and other services subscribe to the event and react accordingly.
This approach based on the publish-and-subscribe messaging pattern is more decoupled because some other services needing to react to the event can just subscribe to it. However, a service always needs to know which events published by other services it should subscribe to. In this aspect, choreography makes services tightly coupled.
BPMN 2.0 includes diagrams to model service choreographies. W3C has specified Web Service Choreography Description Language (WS-CDL) and Web Service Choreography Interface (WSCI) for modeling choreographies.
Figure 22.A. BPMN Orchestration Diagram
Figure 22.B. BPMN Choreography Diagram
Atomic Transaction: When runtime activities that span multiple services fail, the parent business task is incomplete and actions performed and changes made up to that point may compromise the integrity of the underlying solution and architecture. Runtime services can be wrapped in a transaction with rollback feature (such as two-phase commit) that resets all actions and changes if the parent business task cannot be successfully completed. A transaction manager (inside an ESB or a BPMS--explained in the enterprise service bus pattern and service orchestration pattern, respectively) is used for rollback of short-lived transactions. As explained in the service data separation pattern, if a database is split across many services, you can attain only BASE (Basically Available, Soft state, Eventual consistency) semantics in contrast to ACID guarantees provided in a monolithic database, according to the CAP (Consistency, Availability, Partition Tolerance) theorem. This pattern is supported by WS-Coordination and WS-Atomic Transaction standards. Refer to: Ibid.
Transaction Compensation: Whereas uncontrolled runtime exceptions can jeopardize a service composition, wrapping the composition in an atomic transaction can tie up too many resources, negatively affecting performance and scalability, especially for long-lived transactions. Compensation routines are introduced, allowing runtime exceptions to be resolved with the opportunity for reduced resource locking and memory consumption. Compensation logic is pre-defined and implemented as part of the parent composition controller logic such as process orchestration (explaoned in the service orchestration pattern). Both WS-BPEL and WS-Business Activity standards provide functionality in support of this pattern. This try-later approach promises that the system will get itself back into a consistent state eventually, hence called eventual consistency. This violates the ACID properties of transactions, but the violation is unavoidable according to the CAP theorem, as explained in the service data separation pattern. Refer to: Ibid.
API Composition for UI: Think of user interfaces as composition layers—place where we weave together the various strands of the capabilities we offer. Have your user interfaces directly make API calls and map them to UI controls (e.g., web-based UI using JavaScript to get JSON via HTTP). This approach could require fairly chatty communication, and could drift back into old layered architecture where UIs and APIs are tightly coupled. Refer to: Sam Newman, Building Microservices: Designing Fine-Grained Systems, 2015.
UI Fragment Composition: Have your services provide parts of the UI directly and then just pull these fragments in to create a UI. These coarse-grained fragments are served up from server-side apps that are in turn making the appropriate API calls. You still need some sort of assembly layer to pull these parts together. This could be some server-side templating or some smart URI routing. Drawbacks of this approach are that some techniques such as style guides need to be added to ensure consistency of UX, and that it cannot easily serve many different types of UI such as mobile devices, native apps, thick clients, dynamic UI, etc. (Ibid.)
API Gateway for UI: A common solution to the problem of chatty interfaces with backend services, or the need to vary content for different types of devices is to have a server-side aggregation endpoint, or API gateway.
This can marshal multiple backend calls, vary and aggregate content if needed and serve it up. The gateway could end up too thick with too much behavior infiltrating into the domain logic, and start losing isolation of various user interfaces limiting your ability to release them independently. (Ibid.)
Backends for Frontends: Restrict the use of the backends to one specific user interface or application. This makes each service more self-contained and independent with its own UI and own backend. Just beware of the backends infiltrating into the domain logic which must stay with the services themselves. The backends should only contain behavior specific to delivering a particular user experience. (Ibid.)
Figure 23.A. API Composition for UI
Figure 23.C. API Gateway for UI
Figure 23.B. UI Fragment Composition
Figure 23.D. Backends for Frontends
SOA Service Analysis & Design Example: Public Software Training Application
In the Business Analysis page in this website, we took a public software training business as an example to illustrate work products produced in business analysis (BA). The illustrated BA work products included business motivation model, business architecture model, business service model, business process model, business semantic model, business use case model and use case scenarios. Work products from the subsequent object design activities were also illustrated using the same example, including a domain model and a sequence diagram. Figure 24 shows bounded contexts identified on the domain model of the Public Software Training business (see Figure 11 in the Business Analysis page in this website). Figure 24 therefore is a context map (in the Domain-Driven Design terminology) showing SOA services in that business domain.
Figure 24. Bouinded Contexts Identified on the Domain Model
Figure 25 shows a service interface diagram for the Course Scheduling Service expressed in SoaML. (See OMG, Service Oriented Architecture Modeling Language Specification Version 1.0.1: https://www.omg.org/spec/SoaML/1.0.1/) The Course Scheduling Service here corresponds to the Course Scheduling Service identified in the business architecture design shown in Figure 5.D in the Business Analysis page. The operations listed in the Course Scheduling Capability include all the operations belonging to the classes that are included in the bounded context corresponding to the service. Message types of the service are also defined using classes and attributes defined in the domain model.
Figures 6-12 in the Business Analysis page and the subsequent Figures 24-25 in this page represents a bottom-up approach to SOA design where the domain model is developed first at a detail level and then bounded contexts, i.e., SOA services, are identified as groups of classes in the domain model. On the other hand, Figures 5.A to 5.D showing business modeling, business architecure design and business service identification in the Business Analysis page represents a top-down approach to SOA analysis. You may apply both top-down and bottom-up approaches in an evolutionary manner. When your project is a brownfield engineering project and tries to reuse proven functionalities in the legacy applications, the bottom-up approach is unavoidable. Even in this case, the top-down approach is helpful to ensure that your business goals, strategies and to-be business models are realized as planned.
Figure 25. Service Interface Diagram in SoaML
Figure 26 shows a service architecture diagram in SoaML that describes the service-oriented architecture of the entire business domain. Three services particpate in the SOA landscape, which are orchestrated by the Public Software Training Process. There are inter-service interactions (of course via APIs) between Course Development Service and Course Scheduling Service, and between Course Scheduling Service and Class Booking & Operation Service. Figure 27 shows a sequence diagram for the Schedule Courses process activity (shown in Figure 6.A in the Business Analysis page) which corresponds to the Make a Course Schedule use case (shown in Figure 8 in the Business Analysis page). The sequence diagram describes how SOA services are orchestrated by the business process, or collaborate to realize the use case. Refer to: Figures 5 to 12 in the Business Analysis page on this website to understand the context of the Public Software Training business.
Figure 26. Service Architecture Diagram in SoaML
Figure 27. Service Interactions for the Schedule Courses Process Activity Expressed in Sequence Diagram
SOA and MSA Service Inventory Patterns
Service-Oriented Enterprise Application Architecture: A service-oriented enterprise application architecture (SO-EAA) encompass service inventories. Establishing an single enterprise service inventory may be unmanageable, and attempts to do so may jeopardize the success of an SOA adoption. Services can be grouped into manageable, domain-specific service inventories, each of which can be independently standardized, governed, and owned. However, standardization disparity between domain service inventories imposes transformation requirements and reduces the overall benefit potential of the SOA adoption. A service inventory has runtime platforms and middleware, and represents a concrete boundary for a standardized architecture implementation. Services are delivered into a service inventory from which service compositions are drawn. Services are composed solely based on service contracts into fully functional applications.
Figure 28. Service-Oriented Enterprise Application Architecture (Adopted from T. Erl, SOA: Principles of Service Design, 2007)
Service Normalization: When delivering services as part of a service inventory, there is a constant risk that services will be created with overlapping functional boundaries. The service inventory needs to be designed with an emphasis on service boundary alignment. Ensuring that service boundaries are and remain well-aligned introduces an extra up-front formal service modeling, and on-going governance effort. Refer to: T. Erl, SOA Design Patterns, 2009.
Logic Centralization: If agnostic services are not consistently reused, redundant functionality can be delivered in other services, resulting in problems associated with inventory denormalization and service ownership and governance. Access to reusable functionality is limited to official agnostic services. Service consumers should be required to reuse functionality provided by a single designated agnostic service. Organizational issues reminiscent of past reuse projects can raise obstacles to applying this pattern. Refer to: Ibid.
Rules Centralization: The same business rules may apply across different business services, leading to redundancy and governance challenges. The storage and management of business rules are positioned within a dedicated architectural extension from where they can be centrally accessed and maintained. The use of a business rules management system(BRMS), which is often part of an ESB, is employed and accessed via system agents or a dedicated service. Refer to: Ibid.
Schema Centralization: Different service contracts often need to express capabilities that process similar business documents or data sets, resulting in redundant schema content that is difficult to govern. Select schemas that exist as physically separate parts of the service contract are shared across multiple contracts. Up-front service analysis effort is required to establish a schema layer independent of and in support of the service layer. Refer to: Ibid.
Policy Centralization: Policies that apply to multiple services can introduce redundancy and inconsistency within service logic and contracts. Global or domain-specific policies can be isolated and applied to multiple services. Up-front analysis specific to defining reusable policies is recommended, and an appropriate policy enforcement framework (often using ESB) is required. Also required is a governance structure with policy custodian roles and processes to control changes to shared policies and avoid conflicts and incompatibilities (due to vendor lock-in) between different policies. Refer to: Ibid.
Process Abstraction: Grouping task-centric logic together with task-agnostic logic hinders the governance of the task-specific logic and the reuse of the agnostic logic. A dedicated parent business process service layer is established as a separate inventory layer to support governance independence and the positioning of non-agnostic task services as potential enterprise resources. Business process logic is typically filtered out after utility and entity services have been defined, allowing for the definition of task services that comprise this layer. Abstracting parent business process logic allows carrying out that logic via the composition of other services. Refer to: Ibid.
Process Centralization: When business process logic is distributed across independent service implementations, it can be problematic to extend and evolve. Logic representing numerous business processes can be deployed and governed from a central location. Middleware platforms generally provide the necessary orchestration technologies to apply this pattern. Significant infrastructure and architectural changes are imposed when the required middleware is introduced. Refer to: Ibid.
API Centralization: APIs enable A2A and B2B integration, multichannel applications, service-oriented architecture, cloud services and the reuse of legacy applications and data—viz., “pervasive integration” of mainframe, databases, big data, cloud services, mobile apps, social networks, connected devices, etc. Adopt API mediator to virtualize, enable, control, protect and monitor APIs. The API mediation layer encapsulates a service's native API (the inner API) and exposes an abstracted and managed API (the outer API) to all potential consumers.
API mediator may be built using a variety of technologies, such as API gateways, cloud access security brokers (CASB), mobile back-end services (MBaaS), integration platform as a service (iPaaS), and enterprise service buses (ESBs). API management tools can be used to implement the API mediator, which are now deemed essential for digital business. Full lifecycle of API management is about planning, design, implementation, publication, operation, consumption, maintenance and retirements of APIs. Refer to: Gartner, Mediated APIs: An Essential Application Architecture for Digital Business, 2016; Gartner, Use Mediated APIs to Connect Your Legacy and Packaged Systems With Modern Applications, 2016; Gartner, Magic Quadrant for Full Life Cycle API Management, 2016.
Figure 29. API Concentration Pattern (Adopted from Gartner, Use Mediated APIs, 2016)
Layered SO-EAA: To achieve enterprise-wide loose coupling, collections of services representing corporate business logic and those of technology-specific application logic can be physically separated. Business services can be further classified into task-centric business services (a.k.a. task services) and entity-centric business services (a.k.a. entity services). Entity services are agnostic business services reusable across many business processes that base their functional context on existing business entities. Task services encapsulate non-agnostic business logic specific to a business process. Task services can be centralized as part of an orchestration layer in which case they are called orchestrated task services (or process tasks). Orchestration combines business process models with SOA models and enables workflow automation. (Thomas Erl, Service-Oriented Architecture: Concepts, Technology and Design, Prentice Hall, 2005.)
Figure 30.A. Layered Service-Oriented Enterprise Architecture (Adopted from T. Erl, SOA, 2005)
Members of each layer are aware of and are able to use the services of the layers "below“ (any lower layer, not just one immediately below), but unaware of and independent of the layers "above.“ (Buschmann et al. Pattern-Oriented Software Architecture, 1996) Refactor the model across bounded contexts so that the responsibilities of each domain object, aggregate and module fit neatly within the responsibility of one layer. Figure 30.B shows an example of layered, enterprise service-oriented architecture for a factory automation system adopted from E. Evans, DDD, 2003.
Figure 30.B. Layered Service-Oriented Enterprise Architecture (Adopted from E. Evans, DDD, 2003)
Core Domain: A large complex domain, such as a business domain in an enterprise, requires model distillation—a process of separating the components of a mixture to extract the essence in a form that makes it more valuable and useful. Find the core domain and distinguish it from the mass of supporting model and code. Bring the most valuable and specialized concepts into the core domain, but make it small. Refer to: E. Evance, DDD, 2003.
Figure 31 shows that for a shipping company, a set of classes used for cargo delivery operations comprise the core domain, while those needed for customer relationship management, general transportation and billing are left out (as generic subdomains explained next). The core domain can be partitioned into coherent packages of their own (called segregated core). You may factor the core domain into distinct abstract classes or interfaces (called abstract core).
Figure 31. Core Domain in Doman-Driven Design (Adopted from E. Evans, DDD, 2003)
Generic Subdomain: You should identify cohesive subdomains that can be horizontally used across multiple applications or across different verticals. Factor out generic models of these subdomains and place them in separate modules. Consider off-the-shelf solutions (such as open source code, public SaaS, commercial products, frameworks), published models (such as industry reference models, analysis patterns) or outsourced implementation for these generic subdomains. In Figure 32, core domain is shown in bold. Further, core domain and generic subdomains are clarified by the responsibility layers. Refer to: E. Evance, DDD, 2003.
Figure 32. Core Domain and Generic Subdomains in a Layered SO-EAA (Adopted from E. Evans, DDD, 2003)
Pluggable Component Framework: Distill an abstract core of interfaces and interactions and create a framework that allows diverse implementations of those interfaces to be freely substituted. Allow any application to use it strictly through its interfaces. Refer to: Ibid.
COTS/SaaS Integration: Build if the business capability is unique to what you company does and is considered a strategic asset; buy if it is not that special and can benefit from global best common practices. If you decided to buy a COTS product or subscribe to a SaaS but the particular capabilities it provides aren’t specialized for your needs, it might make sense to change how your company should work rather than embark on complex customization.
You need to select COTS and SaaS which are easy to integrate with and customize through APIs. But technical decisions have been made for you by the vendors, and the key is to move things back on to your own terms. Refer to: Sam Newman, Building Microservices: Designing Fine-Grained Systems, 2015.
Evolving Order: Let the large-scale structure of enterprise application architecture across bounded contexts evolve with the application, possibly changing to a completely different type of structure along the way. Don't overconstrain the detailed design and leave freedom for development teams in distinct contexts to vary the model in ways that address their local needs. Refer to: E. Evans, DDD, 2003.
Knowledge Level (a.k.a. Meta Level): Create a distinct set of objects that can be used to describe and constrain the structure and behavior of the basic model. Keep these concerns separate as two "levels," one very concrete, the other reflecting rules and knowledge that a user is able to customize. Figure 33 shows a domain model of EMR system where the bounded context Clinical Pathway constitutes a knowledge-level service. Refer to: Ibid.
Figure 33. Knowledge Level: Example in an EMR system
Comments
Very good job of discussing both MSA as well as SOA and providing adoption considerations etc.
vinod k dadhra
05.02.2020 01:55
A ubiquitous language is a versatile, shared team language using the domain model as its backbone. Developers, domain experts and customers should use the same language in diagrams, code, writing and speech. The vocabulary of a ubiquitous language includes classes, prominent operations, rules, patterns, etc. You should relentlessly exercise the ubiquitous language to hammer out a shared view of the model as the concepts evolve in different people's heads.
The bounded context pattern can be applied only if the service provider has adopted the model-driven design pattern. It also supports the service abstraction, composability, reusability and loose coupling principles. Bounded contexts have a deeper infulence on MSA, setting explicitly set boundaries in terms of team organization, and physical manifestations such as code bases and databases.
A bounded context may have multiple modules (or subdomains). In Figure 7, subdomains are in the problem domain, while bounded contexts are in the solution domain. The e-Commerce System might have been further decomposed into four bounded contexts (i.e., services). However, there are costs incurred by decomposing a service into finer-granule services. Integration across the boundaries of bounded contexts necessarily will involve some translation, and more messages will be exchanged among the services.
Figure 7. Subdomains and Bounded Contexts in a Domain Model (Adopted from E. Evans, DDD, 2003.)
DDD recommends instituting a process of frequently merging all code and other implementation artifacts within a single bounded context, with automated tests to flag fragmentation quickly. Extreme programming (Kent Beck, Extreme Programming Explained, 1999; 2004) or called XP is recommended for maintaining model integrity within a single bounded context while the design is being constantly changed by many people. Figure 8 shows the practices recommended in XP. The Incremental Design, Test-First Programming, Ten-Minute Build, Continuous Integration and Daily Deployment practices are all very important to develop and maintain high-quality applications in SOA and MSA.
Figure 8. XP Practices (Adopted from Kent Beck, Extreme Programming Explained, 2004)
Figure 9. Designing a DDD-Oriented Microservice at Microsoft
Figure 9 shows a DDD-based approach to designing MSA developed by Microsoft. (Microsoft, Designing a DDD-Oriented Microservice: https://docs.microsoft.com/en-us/dotnet/standard/microservices-architecture/microservice-ddd-cqrs-patterns/ddd-oriented-microservice) In the domain model layer, the domain entity model corresponds to a conceptual-level class diagram, and the domain entities with data + behavior corresponds to a specification-level class diagram explained in the Software Requirements Specification section in the Business Analysis page in this web site. Various object design patterns and DDD patterns are applied in this approach including the Command pattern (one of the 23 Gang of Four design patterns), the Command-Query Separation (CQS) pattern developed by Bertrand Meyer, and Aggregates and Repositories patterns in DDD.
Intention-Revealing Interface: Specify intention-revealing interfaces by naming classes and operations to describe their effect and purpose, not the means, with the effect of relieving the client developer of the need to understand the internals of services. Refer to: E. Evans, Domain-Driven Design, 2003. DDD recommends writing a test for a behavior before creating it, to force your thinking into client developer mode. This pattern follows the service abstraction principle of SOA.
Contract Centralization: Access to service logic is limited to the service contract, forcing consumers to avoid implementation coupling, and thus supporting the service loose coupling principle. Refer to: Thomas Erl, SOA Design Patterns, Prentice Hall, 2009. (http://soapatterns.org/)
Decoupled Contract: For a service to be positioned as an effective enterprise resource, it must be equipped with a technical contract that exists independently from its implementation yet still in alignment with other services. The service contract is physically decoupled from its implementation, and service functionality is limited to the feature-set of the decoupled contract medium, supporting the service loose coupling principle.(Ibid.) This is reminiscent of the Dependency Inversion Principle, in Bob Martin's SOLID principles of class design (http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod).
Concurrent Contracts: A service's contract may not be suitable for or applicable to all potential service consumers. Multiple contracts can be created for a single service, each targeted at a specific type of consumer. (Ibid.)
This is also reminiscent of one of the SOLID principles called the Interface Segregation Principle.
Figure 10. Service Façade (Adopted from T. Erl, SOA Design Patterns, 2009)
Service Façade: The façade pattern is a software design pattern commonly used with object-oriented programming. It is one of the 23 well-known GoF design patterns (Eric Gamma, Richard Helm, Ralph Johnson and John Vissides, Design Patterns: Elements of Reusable Object-Oriented Software, Addison Wesley, 1994: https://en.wikipedia.org/wiki/Design_Patterns). The service façade pattern supports the service loose coupling principle, and is used to realize the decoupled contract pattern as well as the concurrent contracts pattern. A service façade component can be added for the abstraction of core service logic from the contract. This allows multiple façades for the same service. The addition of the façade component introduces performance overhead.
Service Adapter: The adapter (a.k.a. wrapper, service broker) pattern is also one of the 23 GoF design patterns that converts the interface of an existing class into another interface that another class requires without modifying their source code. While a service façade provides a more abstract interface for a service, a service adapter is an intermediary between two services that converts the interface of one to that of the other, and vice versa, so that incompatibility between the interfaces of two services is resolved.
Incompatibility between two services may lie in incompatible semantic models that may contain synoyms, homonyms and other various semantic mappings between the two services. In this case the service adapter should convert data based on different data models at runtime. A metadata management is helpful to correctly transform data by the service adapters. Service adapters help maintain a federated database system where the constituent database systems are interconnected, and yet remain autonomous. Other sources of incompatibility include disparity in data format (e.g., CSV, XML, WSDL, REST) and that in communication protocols (e.g., XMK-RPC, JSON-WSP, SOAP). Service adapters also supports the service loose coupling principle, and is used to realize the decoupled contract pattern as well as the concurrent contracts pattern.
Anti-Corruption Layer (ACL): Wrapper services required to encapsulate legacy logic often introduce a non-standard service contract requiring high technology coupling throughout all service consumer programs. The non-standard wrapper service can be replaced by or further wrapped with a standardized service contract that extracts, encapsulates, and possibly eliminates legacy technical details from the contract.
ACL (a.k.a. legacy wrapper) is the isolating layer consisting of the standardized service contract and required service logic. ACL needs to be developed (or provided by software product vendors) as a service façade or a service adapter to abstract the proprietary legacy interface. The layer talks to the legacy system through its existing interface, requiring little or no modification to the legacy system. Internally, the layer translates in both directions as necessary between the two domain models. This pattern is a special case of the service façade pattern and the service adapter pattern. Refer to: Ibid. and Eric Evans, DDD, 2003.
Figure 11. Legacy Wrapper (Adopted from T. Erl, SOA Design Patterns, 2009)
Hexagonal Architecture: In many software applications business logic infiltrated into the user interface code. The problem (a.k.a. smart UI anti-pattern) occurs often because part of the logic needing to be tested is dependent on oft-changing visual details. The problem makes it difficult to shift from a human-driven use of the system to a batch-run system or to a program to be driven by another program. A bounded context can have multiple technical services realized through RESTful resources, SOAP interfaces, message types, DB accesses, etc.
Create your service to be independent of either UI or database and to provide adapters for different input/output sources such as GUI, DB, test harness, RESTful resource, etc. Implement the publish-and-subscribe messaging pattern.
As events arrive at a port, an adapter (a.k.a. Service Agent pattern) converts it into a procedure call or message and passes it to the application. When the application has something to send out, it sends it out through a port to an adapter, which creates the appropriate signals needed by the receiver. This pattern realizes both the decoupled contract pattern and the concurrent contracts pattern. Refer to: Alistair Cockburn, Hexagonal Architecture (http://alistair.cockburn.us/Hexagonal+architecture)
Figure 12. Hexagonal Architecture
HATEOAS: Use cases the client needs are often very likely different from the pure domain model. In this case, the API model should be decoupled from the core domain to maintain the service loose coupling principle of SOA. The API can be a set of REST resources reflecting the use cases the client needs. Yet each resource is built from classes belonging to the core domain.
REST API compliant with the HATEOAS pattern is preferred in MSA to further remove coupling between clients and services. HATEOAS (Hypertext As The Engine Of Application State) is the highest maturity level of REST in the REST maturity model developed by Leonard Richardson. HATEOAS is a pattern of REST application architecture where a client enters a REST application through a specific URL, and all future actions the client may take are discovered within resource representations returned from the server. In this way, REST interaction is driven by hypermedia with self-contained discoverability. (https://spring.io/projects/spring-hateoas)
This is different from the WSDL-driven interface of conventional SOA applications which is a fixed specification staged somewhere in a service registry. One obvious benefit of HATEOAS is that the service provider can change the service's capabilities without breaking clients. The service provider can freely add and juggle all URIs other than the initial entry points. (https://martinfowler.com/articles/richardsonMaturityModel.html) This pattern realizes the decoupled contract pattern.
Service Data Replication: Service logic can be deployed in isolation to increase service autonomy, but services continue to lose autonomy when requiring access to shared data sources. Services can have their own dedicated, materialized views with replication to shared data sources without requiring exclusive ownership over the data. An additional database needs to be provided for the service and one or more replication channels need to be enabled between it and the shared data sources.
Service Data Seperation: Sharing an integrated database among services have a few drawbacks: First, you can’t change the database schema without affecting the services. Second, you can’t change the database technology (e.g., from relational to NoSQL) for different services. Third, the same sorts of logic to process data can be spread among multiple services.
Once you split the database and allocate the partitions to individual services, services now owning their databases are better decoupled from other services. Accesses to the database owned by another service is done via API calls. Join queries involving tables owned by different services now need to be processed in memory (see Figure 16). You also face the problem of maintaining integrity across related tables owned by different services. You have to implement cross-table constraints such as referential integrity, assertions and triggers in your service code.
When a database is spit across many services, a transaction involving write operations on multiple separate databases requires complex processing to achieve ACID properties. Nonetheless you can attain only BASE (Basically Available, Soft state, Eventual consistency) semantics in contrast to ACID guarantees provided in a monolithic database, according to the CAP (Consistency, Availability, Partition Tolerance) theorem. Refer to related patterns in the SOA and MSA Service Composition Patters section: atomic transaction pattern, transaction compensation pattern, choreography pattern, and publish-and-subscribe messaging pattern. Refer to: Sam Newman, Building Microservices: Designing Fine-Grained Systems, 2015.
Figure 14. Service Data Separation
Command and Query Responsibility Segregation (CQRS): Operations can be broadly divided into two categories—commands (i.e., updates) and queries. Updates may leave side effects (i.e., any unintentional effect on the state of the system) and may better be segregated from queries. Place as much logic as possible into functions (i.e., operations that return results without producing side effects) and make them separate services. Make all operations (except initializers) of value objects to be functions. Strictly segregate commands into very simple operations that do not return domain information. Either the service data replication pattern or the service data separation pattern can be used for realizing CQRS.
Martin Fowler warned, "CQRS is a significant mental leap for all concerned, so shouldn't be tackled unless the benefit is worth the jump. While I have come across successful uses of CQRS, so far the majority of cases I've run into have not been so good, with CQRS seen as a significant force for getting a software system into serious difficulties. In particular CQRS should only be used on specific portions of a system (i.e., bounded contexts) and not the system as a whole. In this way of thinking, each bounded context needs its own decisions on how it should be modeled." Refer to: Greg Young, CQRS, Task Based UIs, Event Sourcing agh!, 2010 (http://codebetter.com/gregyoung/2010/02/16/cqrs-task-based-uis-event-sourcing-agh/); Martin Fowler, CQRS, 2011 (https://martinfowler.com/bliki/CQRS.html).
Figure 15. Command and Query Responsibility Segregation (CQRS)
SOA and MSA Service Composition Patterns
Most enterprise systems are tightly integrated systems spanning the entire business
The challenge is to accomplish modularity without losing the benefits of integration, allowing different parts of the system to interoperate to support the coordination of various business processes.
Context Map: People on other teams won't be very aware of the context boundaries and will unknowingly make changes that blur the edges or complicate the interconnections. Identify and name each model on the project and define its bounded context. Determine which modules belong in which context. Map the existing terrain.
Describe the points of contact between the models, outlining explicit translation for any communication and highlighting any sharing. You need to determine which composition patterns to apply in integrating services. Refer to: Eric Evans, DDD, 2003.
Figure 16. Context Map (Adopted from Eric Evans, DDD, 2003)
Service-Responsibility-Collaboration: Recall that CRC(Class-Responsibility-Collaboration) card game is a popular object design practice for allocating operations needed in use cases to classes and also determining what classes should collaborate to realize each operation. Refer to the Class Responsibility Assignment pattern in the Business Analysis page on this site. The same card format as the CRC card can be used to show the interface of a service and the other services that each operation in the interface need to be collaborate with. A stack of SRC cards will correspond to a context map. Figure 17 shows a SRC card for an Insurance Policy Management service whose Calculate Premium method need to collaborate with three other services.
Figure 17. Service-Responsobility-Collaboration Card
Shared Kernel: Uncoordinated teams working on closely related applications can go racing forward for a while, but what they produce may not fit together. They can end up spending more on translation layers and retrofitting than they would have on continuous integration in the first place, meanwhile duplicating effort and losing the benefits of a common ubiquitous language. Designate some subset of the domain model that the two teams agree to share. Of course this includes, along with this subset of the model, the subset of code or of the database design associated with that part of the model. This explicitly shared stuff has special status, and shouldn't be changed without consultation with the other team. Integrate a functional system frequently, but somewhat less often than the pace of continuous integration within the teams. At these integrations, run the tests of both teams. Refer to: Ibid.
Customer/Supplier: The freewheeling development of the upstream team can be cramped if the downstream team has veto power over changes, or if procedures for requesting changes are too cumbersome. The up-stream team may even be inhibited, worried about breaking the downstream system. Meanwhile, the downstream team can be helpless, at the mercy of upstream priorities.
Make the downstream team play the customer role to the upstream team. On an XP project, there already is a mechanism in place for doing just that: the iteration planning process. All we have to do is define the relationship between the two teams in terms of the planning process. Jointly develop automated acceptance tests that will validate the interface expected. Add these tests to the upstream team's test suite, to be run as part of its continuous integration. This testing will free the upstream team to make changes without fear of side effects downstream.
Figure 18. Shared Kernel Pattern for Service Composition (Adopted from Eric Evans, DDD, 2003)
Figure 19. Customer/Supplier Pattern for Service Composition (Adopted from Esposito and Saltarello (2014))
Figure 6. Bounded Contexts Defined on a Domain Model (Adopted from E. Evans, DDD, 2003)