Business Architecture in a closed-loop process lifecycle

The desire to have adaptive business processes that can be refined and optimized to respond to changing business environment, government regulations and competitive pressures has resulted in the need for closed loop Business Process Management(BPM) system. The following Closed-loop lifecycle diagram of BPM implementation is well known to most of us –

The greatest benefit of the closed-loop process lifecycle is “Continues Process Improvement”. Majority of organizations follows this methodology, however, they start with executable BPMN process modelling. Recently an increasing number of organizations are aligning their Enterprise Architecture discipline to include the entire gamut of Business process Management by adopting an extended-closed-loop lifecycle. Conceptually the extended-closed-loop lifecycle can be depicted as follows–

Business Architecture defines the blue print and high level construct of the organizational processes in terms of Enterprise Map, Value Chain and Strategy Model that describes the business areas, business goals, objectives, strategies and detailed business process steps to achieve the same. A business process step in a value chain is further decomposed into executable business process which not only delivers the business value it meant to; but is also can be monitored and is measurable.

Note: You may refer to my previous post where the convergence of Enterprise Architecture and Business Process Management has been discussed in detail.

In the classic closed-loop BPM lifecycle operational processes are modelled, simulated, deployed and process performance is monitored by defining KPI on the business indicators. To illustrate, a typical “Order Fulfilment” operational process can be modelled to capture business indicators like Sales Volume as a Measure and Zone as a dimension.  These two indicators can be used to define the KPI – “Zone wise Sales Volume for a specific period of time”.

Sometimes Business Architecture is considered as a mare modelling exercise with the goal of creating a repository of process maps. But when the Business Architecture is considered as part of closed-loop process lifecycle, it brings both high level business process and executable business process in the same process lifecycle. It enables the KPIs from executable business processes to roll up to the high level business processes so that the performance of the high level business processes can be monitored and optimized. Ability to measure high level business processes performance is critical for the business owners to refine the business goal, objective and strategy.

In a separate post, I will detail out the steps to derive executable business processes from high level processes and rollup the KPI from your executable process to high level business process using Oracle BPM 12C. 

Supporting Canonical Message across multiple composites

In any integration projects multiple SOA/BPM projects or composite are created that use a set of common xml schemas. The Business Components are created based on these  xml schemas to be used to define the process input/output. An effective mechanism needs to be institutionalized to store these common  xml schemas in a central place and referred by all SOA/BPM composites.

This can be achieved using Metadata service (MDS) in Oracle SOA Suite.  At design time XSDs are defined in JDeveloper and pushed to local MDS, also knows as DesignTimeRepository, which is file based MDS and stored on developer's local filesystem. The XML schemas stored in DesignTimeRepository are accessible to any project within the BPM Application and even across the Applications while developing in JDeveloper. By default this DesignTimeRepository is located in the <userhome>\jdeveloper\soamds folder. The DesignTimeRepository folder should be configured under version control system, so that all common artifacts are version controlled and synchronized across every developers' local DesignTimeRepository. During design time these common schemas are used in different SOA composites, but before deployment to SOA server, the components in DesignTimeRepository should be imported into runtime MDS of the SOA server. 

The example below illustrates the steps to save XML schemas in DesignTimeRepository, refer that in multiple BPM projects (using Oracle BPM 12.2.1) and deploy them in MDS on SOA server.

First, create a BPM Application.

In the "Create BPM Application" wizard, create a BPM project.

Next, create a XML schema claim.xsd in the BPM project.

Now, we need to store the claim.xsd into the DesignTimeRepository. Right click on the claim.xsd and select "Share using SOA - Design Time MDS Repository" option on the menu. This will move the claim.xsd from the project to the  DesignTimeRepository.

You can view the components stored in DesignTimeRepository from the component window.  All the components you store in the DesignTimeRepository, is stored under "apps" folder.

Once the claim.xsd is stored in the DesignTimeRepository, you can refer to that from any project.  Create another BPM Project. Then create a Business Object based on External Schema. Select the claim.xsd from the MDS.

That's it. You can do the same for all projects and shared artifacts.

Now, it is time to deploy the projects on SOA server. If you try to deploy, the  deployment will fail as the claim.xsd is not available on SOA Server MDS. 

To save the components from local DesignTimeRepository to SOA Server MDS, login to EM and navigate to soa-infra > Administration > MDS Configuration.

Enter the folder location of DesignTimeRepository, the default location is <userhome>\jdeveloper\soamds and click import. That's it. Now you can deploy the projects which refer the claim.xsd.

If you dont have access to EM, you can also deploy the shared components using WLST or the SOA bundle deployment.  Refer this blog to understand how to use WLST to add/delete shared components on SOA MDS. 

Build and Deployment using Maven

In most of client organizations I see Maven is the official tool for deployment as part of their standard build and deployment tool. Oracle BPM supports maven based build and deployment. In fact it provides out-of-box capability to add application level and project level pom in Jdeveloper.

Before adding pom for the BPM application and BPM projects in JDeveloper, the local maven repository should be configured. Below the steps to configure the local maven repository.

Setting up Maven local repository

Step 1 : you can use the maven binary distributed with Oracle BPM 12C (12.2.1) or can download apache maven. Set the path variable to include maven bin folder-

In mac os following is the command to set the path of variable -

export PATH=$PATH:/Users/ctsuser/Oracle/Middleware12c/Oracle_Home/oracle_common/modules/org.apache.maven_3.2.5/bin

Step 2: Before running maven:install to setup the local repository, you may need to configure the proxy so that maven can download the dependent libraries from central repository. The global maven configuration file is located at -

/Users/ctsuser/Oracle/Middleware12c/Oracle_Home/oracle_common/modules/org.apache.maven_3.2.5/conf/settings.xml

Step 3: to execute the install and crawl, you need to use JDK 1.7, it does not work with JDK 1.8.

In mac os, following is the command to set the JAVA_HOME environment variable. 

export JAVA_HOME='/Library/Java/JavaVirtualMachines/jdk1.7.0_49.jdk/Contents/Home’ 

After the variable is set, make sure your jdk is pointing to desired version 

java - version

Step 4: You need to install soa plugins. 

Goto 

/Users/ctsuser/Oracle/Middleware12c/Oracle_Home/oracle_common/plugins/maven/com/oracle/maven/oracle-maven-sync/12.2.1

Execute 

mvn install:install-file -DpomFile=oracle-maven-sync-12.2.1.pom -Dfile=oracle-maven-sync-12.2.1.jar 

Step 5: Next you need to run the ‘Push’ Command to upload all the Oracle Maven artifacts to local repository.

mvn com.oracle.maven:oracle-maven-sync:push -DoracleHome=/Users/ctsuser/Oracle/Middleware12c/Oracle_Home -X -DpushDuplicates=true

Step 6: To ensure that all artifacts are in order, execute the crawl

mvn archetype:crawl -Dcatalog=/Users/ctsuser/.m2/archetype-catalog.xml

/Users/ctsuser/.m2/ is the local repository in my case.

Ok. by now you have the local maven repository setup with Oracle soa artifacts.

Adding Maven support to BPM Application in JDeveloper

Next we will add the pom in our existing BPM application and BPM projects in JDeveloper. Steps as follows - 

Step 1: Select the application and choose Add New From Gallery > Select Application POM (Maven )

Enter the GroupId, Artifact ID and click OK.

For each projects select Add New From Gallery and choose Maven POM for Project(Maven)

Enter the GroupId, Artifact ID and click OK.

Edit the Application and Project level POM (pom.xml)

Running Maven for Deployment

Make sure you have maven in the path

export PATH=$PATH:/Users/ctsuser/Oracle/Middleware12c/Oracle_Home/oracle_common/modules/org.apache.maven_3.2.5/bin

Make sure ORACLE_HOME is set

export ORACLE_HOME='/Users/ctsuser/Oracle/Middleware12c/Oracle_Home/‘

Make sure JAVA_HOME points to jdk 1.8

export JAVA_HOME='/Library/Java/JavaVirtualMachines/jdk1.8.0_65.jdk/Contents/Home’

Change the server connection details in pom

<serverUrl>

      http://localhost:7001

    </serverUrl>

<user>

      weblogic

</user>

<password>

      password

</password>

Execute the command 

mvn install -X -Dmaven.test.failure.ignore=true

Note: It is always good to use configuration plan for deployment of composite to different environments ( DEV, SIT, QA, Prod env). You can generate the deployment plan by right clicking over the composite file and clicking on the menu option "Generate Configuration Plan". To include the configuration plan set the <configplan> property in the pom as follows -

<configplan>${scac.input.dir}/configplan.xml</configplan>

Anatomy of Conversation and Correlation

During my interaction with Oracle BPM developers I frequently notice that the concept of "Conversation" is not clear to most. When inquired about "Conversation", most of them used to state that "Conversation" is used when inter-process communication is required from within a multi-instance subprocess but they fails to explain why ? "Correlation" is the popular method used in inter process communication and can even be used to carryout inter process communication from multi-instance subprocess, when why we use "Conversation" ? What are those scenarios which "Correlation" cannot handle and "Conversation" is the way to go ?

Before explaining the scenarios,  let me spend couple of lines to explain the similarities and differences between "Correlation" and "Conversation".

"Correlation" is a technique to attach a business attribute to the instance of a process to uniquely identify that process instance while communicating with that process instance from outside (outside of the process). For an Order Fulfillment process you may have a step which invokes (through Send Task or Throw Message) Order Shipment process asynchronously and wait (in Receive Task or Catch Message) for the response. In this case, to enable  Order Shipment process instance to send the response back to that specific instance of Order Fulfillment process correlation can be used. Correlation is initiated in the Send Task and used in the Receive Task of the Order Fulfillment process. OrderId could be used as correlation property if uniquely identify a Order Fulfillment process instance. The Send Task of Order Shipment process should pass OrderId as argument to Receive Task of Order Fulfillment process, where the same is used in correlation to indetify the instance associate with OrderId.

The above mentioned inter process communication can also be implemented without Correlation, using Conversation.

Unlike Correlation, Conversation does not require unique attribute association. System automatically keeps track of instance that has initiated the conversation and when receiving the response backed from called processes it can automatically identify the specific instance of the calling process

To implement the above mentioned process using Conversation, we will do little bit of twicking. Remove the correlation used in SendTask and ReceiveTask of Fulfillment process and create a Conversation in the structure pane of the Fulfillment process.

Next we need to the implementation of Send Task and Receive Task of Fulfillment process to use the Conversation we just created instead of calling Shipment process directly.

In the earlier approach you might have called the ReceiveTask of Fulfillment process directly from SendTask of Shipment process, however when we implement the communication using Conversation we must implement the SendTask of Shipment process as asynchronous call back as follows -

So we observed that in the above mentioned scenario either Correlation or Conversation can be used, be the communication initiated from the SendTesk on the main process or from within multi-instance subprocess. However Correlation cannot be used when you cannot attach a unique business identifier to a process instance, generally which is the case for instances in a multi-instance subprocess. In those cases Conversation is only way to go.

Apparently  though "Advanced Conversation" looks superior as compared to "Correlation" or messaged based conversation, the major shortcoming of "Advanced Conversation" is - not all client has the capability to initiate a conversation. Consider a scenario where your order fulfillment process is waiting at a Message Catch activity for an external j2ee system to notify the process instance about the order shipment status. In this case conversation cannot be used as the external j2ee system would not be able to participate in the conversation. "Correlation" should be used in this case.

Note: Please note that "Correlation" is a form of "Conversation", it is message based conversation. And by "Conversation" we generally refer to "Advanced Conversation".

Exposing a Process endpoint as Rest Service with Basic Authentication

The momentum of REST popularity and adoption is rapidly increasing. I recently came across 80% of cases where the process service consumer asked for REST endpoints for process services. You must be aware that Oracle BPM 12C offers its extensive process API as REST (https://docs.oracle.com/middleware/1221/bpm/bpm-rest-api-ref/toc.htm). In addition is also provides the capability to expose process endpoints as REST along with SOAP.

To illustrate the mechanism, consider the following BPMN process.

by default Oracle BPM expose the message start activity through SOAP end point which is evident from the corresponding composite diagram as follows -

To Expose a REST endpoint for the same process, drag the Rest Binding adapter from the component panel and drop in the "Exposed Services" lane.

Select the checkbox "Service will invoke components using WSDL interface"; that the trick. In the next step wizard would allow you to create a Rest endpoint for an existing SOAP endpoint. While you have manually configure it, I would use the "Shortcut" option - 

This would enable you to select the component, select the BPMN process you would like to expose

In the next step, configure the HTTP Verb and click Finish. You are all set.

Select the operation binding entry, and click the edit icon. On the next screen "Generate Sample payload" button. Use the Sample Type radio options to generate the payload in your desired format. Copy the sample payload. We will use it later.

On the same screen in the "URI Parameters" section, select the "Generate Sample URL for operation" button to generate the url parameter. This is the URL to be shared with service consumer. In my case it is - http://<host>:<port>/soa-infra/resources/<partition-name>/OrderManagementBpmProject/RestService/

Post deployment, we will use this url (with server and port name substituted) from SOAPUI to test the service endpoint.

Now, to secure the Rest endpoint with basic token based authentication, right click on Rest Reference and click "Configure SOA WS Policies"; on the policy binding section add wss_http_token_service_policy.

Deploy the application.

Before we test our application, we need to configure Weblogic domain credentials. Right click on the weblogic soa domain and go to Security > Credentials. Create a Map with name oracle.wsm.security and add key named basic.credentials within the map. Provide the username and password you want the service client to pass.

In the SOAPUI create a Rest Project and "create an new REST Service from URI". Configure the username and password in the Request Property, select the proper method, configure the payload.

Convergence of Enterprise Architecture (EA) and Business Process Management (BPM) through Hierarchical Process Modelling

A large number of BPM implementations are undertaken outside Enterprise Architecture (EA) purview, which often fails short to achieve the benefits EA as a discipline delivers to the organization. A majority of stakeholders in a BPM implementation holds the perception that EA and BPM are separate discipline. In contrary, BPM is part of EA. BPM delivers the tools and methods to address the Business Architecture, which is an integral component of EA.

This article illustrates the synergy and relations between BPM and EA, explains the methods and tools to translate and decompose business processes into executable artifacts.

One of the key pursuits of most organizations is to maximize their shareholder value over time and to achieve this objective organizations are engaged in assessing ways in which their processes can be improved. To optimize the shareholder value through process automation, optimization and harmonization organizations adopt Business Process Management (BPM) as a holistic approach.

The adoption process generally starts with the awareness that BPM can improve shareholder value and their desire to adopt BPM to improve their shareholder value results into initial a set of individual BPM project implementations that proves the benefits of BPM. In the next step, organizations apprehend their BPM projects in a more strategic centralized BPM program which is also known as BPM portfolio management. Under BPM Portfolio Management, organizations look for a way to structure and prioritize their BPM projects to relate their BPM activities in a BPM road map, provide a consolidated view of the complete business process landscape of the organization emphasizing the enterprise value perspective than an IT solution.

The effectiveness of BPM implementation is measured by the tangible business value it delivers over the period of time, and to achieve that, an efficient convergence of business and IT is indispensable. BPM and Enterprise Architecture, both the disciplines offer great synergy in terms of principles, methodology, technique, and tools. BPM provides the business context, understanding, and metrics, and Enterprise Architecture provides the discipline to translate business vision and strategy into architectural constituents. This synergy between BPM and EA enables not only convergence at the principles, technique and methodology but also structured management and harvesting of EA artefacts.

Gartner illustrated and showed that BPM project implementation organizations are not isolated, but part of the EA organization.

The EA component that BPCC team deals with is Business Architecture. The Business Architecture captures the Goal, Strategy, Objectives, People and Processes and lays the foundation of the IT solution. Hierarchical modelling is used to decompose the business processes to comprehend the business goals, strategies, objectives and their relationship easily.

Concept of Hierarchical Process Modelling

An Enterprise is a collection of “bubbles” of goals, strategies, objectives, KPIs, value propositions, role, people, procedures and tasks.  Holistically an Enterprise is a complex network of “bubbles” and their relations represented as links. Process decomposition is the method of organizing the “bubbles” in different baskets of focus area based on their inter-relation and then further decomposed to granularity till a “bubble” becomes the atomic task which no further decomposable.

Hierarchy offers the efficient management of complexity to allow one to understand the value streams. Enterprise Map, Value Chain Map, Strategy Map, Process Model is used to defines the Business Process Architecture with hierarchical process definitions with process decomposition at contextual, conceptual, logical and physical or actionable processes. Each level is same, but narrates in more detail.

Thumb rules (illustrated by Dr. Mike Yearworth, University of Bristol in Process of Model (holon) building) to define an effective (grouping of related process, balance of level and detailing) process hierarchy -  

• Challenging process/system boundary by asking why?
• Eliciting process through repeated question of how?
• Layering as required in order to provide grouping of processes that conveys meaningful and relevant information to target stakeholders
• Proceed until there is no longer a process exists to be questioned how

Tools

While the concept of hierarchical process modelling remains same across various products (like ARIS, SCOR, eTOM, Oracle BPM etc), the tooling and some nomenclature differs. Oracle BPM 12C product suite has been considered in this paper to illustrate the hierarchical process modelling.

-        Enterprise Map

Enterprise Map is a contextual Model that defines the scope and boundaries of business processes in a single view. It is targeted to provide insights to Executives, Strategists, Business Process Architects and Business Analysts.

The Enterprise Map contains Lanes and Process Areas. Process Areas provide the boundaries and scope for the lower levels of modelling. Lane offers grouping of related Process Areas based on their characteristics. The processes of typical manufacturing company at a minimum can be clarified as Core, Management and Support, each being represented as lane in Enterprise Map. Product Sale, Product Servicing and Product Manufacturing may be considered as three Core Process Areas. Process Areas for Management and Support processes are represented in their corresponding Lane, as depicted in the Enterprise Map below -

-        Value Chain Model

Value Chain Model is used to define both Conceptual Model and Logical model. Value Chain Conceptual Model decomposes the Process Areas into series of steps that transform the business inputs into business outputs to create a business value.  The audience for Conceptual Value Chain models are Business Owners, Business Analysts, and Business Architects. The value chain begins with a start step, which represents the inputs, and then each following chain step represents a transformation process, until the end step which represents the outputs.  One Process area can have multiple value chains. Each Step may have one or multiple KPI assigned to measure the performance. “Product Sale” process area can be composed of following Conceptual Model Value Chains

Value Chain Logical Model decomposes the steps in Conceptual Value Chain Model into further detailed Value Chain Model. The steps are decomposed into sub Value Chain till the process can be model as executable physical model i.e., BPMN model.  Each step in the value chain takes a business input, adds some business value and produces a business output. The value a step targets to produce are documented along with KPI.

A step in a value chain is either decomposed into a sub value chain model or a physical BPMN model.

-        Strategy Model

Each process area is a black box container of goals, strategies, objectives, KPIs, value propositions, role, people, procedures and tasks. Value chain and physical model (BPMN) caters with value propositions, role, people, procedures and tasks. The Strategy map ties the Value Chain Model with business goals, strategies and objectives.

Goal defines the end result the organization is trying to achieve (there can be multiple goals within each process areas). To achieve the goal, one or more objectives are defined and Strategy is the plan to achieve the objectives. The strategy can be linked to a value chain.

The below mentioned Strategy Model, ties some of the Goals, Objectives, Strategy and Value Chain Models that falls within “Product Sale” process area. The “Acquire 60% market share” goal is achieved by attaining three objectives – “Fulfil Order”, “Promote Product” and “Increase Dealer”. Each of these objectives must have some KPI defined to evaluate the extent the objective has been achieved. To achieve an objective, one or multiple strategy or plan is used, and each strategy uses one or multiple Value Chain Model to device the strategy.

-        Physical Process Model

The Physical process Models represents the complete business and system requirements without reference to a specific implementation. Physical model is represented with BPMN notations. The audiences for this level of details in a process model are the Subject Matter Experts, Business Analyst, and End Users. The “Process Return Request” step in the value chain can be decomposed into following BPMN Model -

-        Windup

The hierarchical process modelling provides stake holder at various level the right set of process knowledge and sets the stage for further process discovery, optimization and improvement. This is effective tool for managing large BPM portfolio by means of project scoping, prioritizing and road mapping.

Either Top-down or bottom-up, any approach can be adopted to define the Business Architecture based on the organization’s SOA and BPM maturity. It is often observed organizations adopting an iterative approach to define the Enterprise Business Architecture.