Key value for OSB and SOA

most of the applications use key-value pair and lookup frequently.

1. In OSB project can be create OSB project to have xquery containing key-value pair and can be accessed as an OSB service
2. In SOA we can use the concept of DVM and create a simple project accessing these value from MDS

Since the frequency is high, we can have result caching in OSB service calls.

Cannonical data model implementation for Oracle SOA and OSB(Oracle Service Bus)

if you are going to use OSB and OSAO, two suggestions:

1. Reuse of Schema accross OSB and SOA
2. Establishing Cannonical Data Model for reuse

The two possibility :

1. OSB XSD definetions used by both OSB and SOA
2. MDS for maintaining schemas

I Would prefer earlier one and it can be shared widely due to access over HTTP

Scalability

Three sides of scalability:

Business scalability
System scalability
Client scalability 

Patterns consideration for SOA

Service discovery pattern : example UDDI,
Security pattern: Message, transport, service
Asynchronous message pattern
Conversation pattern: multiple system communicating using messages
Service interaction pattern
Workflow pattern
Orchestration pattern
Pattern language

SOA Programming Pattern

SOA has changed the world to think more about programming models instead of procedural and OOPS:

Composition: example SCA, few vendors are providing tool to program like Oracle SOA, open source Tuscany 

Process models: BPM tools

Declarative programming: XSLT, Rule engines etc

Event-driven programming: All tools facilitating EDA, including Oracle SOA with EDN


http://www.eaipatterns.com/docs/SoaPatterns.pdf

OSB links

Configuring SMTP:
https://blogs.oracle.com/christomkins/entry/sending_an_email_from_oracle_s

URLs of OSB resources

WSDL :

http://host:port/sbresource?WSDL/project_path/wsdlname




http://host:port/proxy_service_endpoint_URI?WSDL (for WSDL-based HTTP proxy services)

Proxy Service:

http://host:port/sbresource?PROXY/project_path/proxyname




http://host:port/sbresource?ORAPROXY/project_path/proxyname




Business Service:

http://host:port/sbresource?BIZ/project_path/bizname




http://host:port/sbresource?ORABIZ/project_path/bizname




WSS:

http://host:port/sbresource?POLICY/project_path/policyname

MFL:

http://host:port/sbresource?MFL/project_path/mflname

SCHEMA:

http://host:port/sbresource?SCHEMA/project_path/schemaname

SOA Service design lifecycle

Identifying business processes: short, medium, long

Service design and modeling

Service Bus pattern

Basic battern

Protocol switch
Modifier: transformation, enrichment
Router, based on rule
Discovery
Distribute
Monitor
Correlate : complex event processing
Service registry

Complex pattern

Canonical adapter: protocol switch+Modifier
Transform+log+route: Modifier+Monitor+Router
Proxy or gateway

Enterprise patterns

Global ESB+one registry
ESB Gateway: security and firewall
Federated ESB:
Brokered ESB

http://www.ibm.com/developerworks/websphere/library/techarticles/0712_grund/0712_grund.html#N105AF

Additional point of view for patterns, use cases for enterprise level services:

Common Integration Patterns:

1. Continuous Pipeline Processing Pattern: from sequential batch processing to continuous event processing mode. Advantage: 1)Reducing cycle times, 2)Improving data consistency.

2. Architectural flexibility: 1) able to reduce inter-operability issues, 2) agility in adding and removing new channels

3. Remote Information Access Pattern: 1) address multiple data sources 2) dynamic processing of data for portals, 3) decoupling data and integration logic from presentation, 4) variant of model is used for native and Cloud data sources(SaaS, WS, SOA).

4. Remote Data Distribution: 1) Data distribution for real time business events.

5. Response to Real-time Events

http://www.progress.com/docs/whitepapers/playbook-for-esb-ebizq-progress.pdf

Message exchange patterns (MEP)

There are four basic type of MEP between service provider and service consumer:

1. Notification
2. Solicit response
3. Request- response
4. One way: fire and forget(Broadcast, publish/subscribe, point-to-point messaging)

1 & 2 are not supported by WS-I Basic profile

SOA life cycle

SOA Governance lifecycle
SOA Service design lifecycle
SOA Service Run lifecycle

SOA Governance lifecycle

Plan: Establish the governance needed
Define: Design the governance approach
Enable: Put the governance model into action
Measure: Monitor and measure governance process
Maintenance: Review and adopt.

Schema and test data repository

Enterprise applications based on SOA, Webservices or EAI,  there are lot of data routed, enriched, transformed. In some projects documentation of schema and test data(XML and native data) is part of project document repository.

Issues:

lack of real time schema availability.

less possibility of data validation

Oftenly vendor for development and maintenance is different 

reuse of schema

WS-I* compliance

Why?
Enterprise vision for data modeling
Enterprise wide adoption for common data model

Layers:

Data
Data access

Service layers
Schema access services - create, update,delete,versioning
Test data access services - create, update delete, versioning
Schema validation services - availability, validation, namespace compliance, standard compliance

Technology layer:

NoSQL for Schema and test data storage
Rest services to operate on schema and test data

Message Library:  EDIs, standard formats

Trouble shooting DLQ on IBM Websphere MQ

Determine why a message is on the DLQ (dead letter queue): 

1. Browse the message on the dead letter queue.
2. Locate the dead letter header. The dead letter header starts with the characters '"DLH". Example: 444C 4820 0000 0001 0000 010A 4D59 2E51 'DLH........MY.Q'
3. Find the dead letter Reason (the reason the message arrived on the DLQ) in the third word in the dead letter header. In this example the dead letter Reason is: x'0000 010A'.
4. Convert hex x'0000 010A' to decimal 266.
5. Use the dead letter reason code to determine why the message was placed on the dead-letter queue. The deal letter reason code is a Reason Code (MQRC_*) ,or a Feedback Code (MQFB_*). Example: MQFB_TM_ERROR 266 X'0000010A'

For a list of reason codes, see Reason codes.

For a list of feedback codes, see Feedback codes.

http://www-01.ibm.com/support/docview.wss?uid=swg21176930

Business Requirement Analysis

1. Identify stakeholders(more importantly Killer SH): Compliance, Customer, Resource, Owner/Manager
2. Identify drivers: Problems, Opportunities, Constraints
3. Identify objectives: Should be SMART
4. Change Management:

Context diagram and scope dimension:
1. Locations
2. Processes
3. Applications
4. Organization units
5. Technology
6. Any other
7. Out of scope

Requirements for change:

1. Scope requirement
2. Functional requirement; Priority: Must,Should,Could,Would
3. Detailed requirement

Processes are made around


1. run at Locations
2. Implemented on Applications
3. used by Organization units
4. use Technology
5. execute Business Rules
6. manipulate Data
7. implement Functional requirements

Next step:

Data model
Business process model

Data specification facets:
1. Non-Functional requirement: Access, archive, availability
2. Data content roles
3. Data relationship rules

Actions:

prepare data models(ER/UML)
document data content rules
document data non-functional requirements


Data Model Concepts: Entity Relation Diagram(ERD can be normalized)/Class diagram.

Basic components or ERD:
1. Entity
2. Relationship
        relationship cardinality: Crow Foot/UML class cardinility
3. Attribute

Building a data model

There is a process to go through which will be described in more detail in the next section:
1. identify candidate entities

2. select a central candidate entity

3. define the primary key

4. work through the rest of the candidate entities:

• consider whether it is in scope
• define the primary key
• define the relationship(s) between the candidate entity and all other candidate entities on the diagram
• add description entities as needed
• fully express the cardinality of the relationship(s)
• name the relationship(s)

5. for each entity on the diagram

• define the remaining attributes
• define the non-functional requirements

6. review and refine
7. Advanced data modelling concepts

• Super-types and sub-types
• Hierarchies
• Exclusive relationships
• 3rd normal form

Migration case study 1

My application is now moving to different platform, data model will be same however application which contains the business logic, data access, UI is going to migrate, what are my options for implementing SOA. How can I decouple and increase reuse?

1. Loose couple entity objects and use REST to provide linkage between entity and business service.
2. Webservice for business logic
3. ESB for end point and service virtulization

Advantage:
1. Loose coupling between entity and data access layer
2. Loose coupling between data access and business service
3. Service and endpoint virtulization
4. Divide governance for process level SLA and data level SLA
5. Additionally with canonical model and schema repository reuse of data objects
         

IBM MQ best practices from IBM

Design

Use short names for Qmgr and MQ Objects
Always assign a DLQ
Use standards like JMS where ever possible
Build with performance in mind
Avoid fixed queue names in programs
When feasible, cluster your MQ server
Few Qmgr with more queues

Build

Capture completion and reason code for all MQI calls
Code application to continually process messages
Close and disconnect cxn properly
Be aware of the different features of various MQ clients

Run

Use security best practices
Restrict remote adminintrative authority using SYSTEM.ADMIN.SVRCONN channel of production-level boxes
Do not use sample get and put utilities
Use runmqlsr in lieu of inetd listener
Use SupportPacs to extend MQ functionality

Maintain

When not using third party tool, automate configuration with runmqsc scripts
Use appropriate logging mechanism
Schedule and apply fix packs on a regular basis

http://www.ibm.com/developerworks/websphere/library/techarticles/0807_hsieh/0807_hsieh.html

Hermes configuration for OpenMQ or SUN MQ

Start SOAPUI

Goto Tools menu select HermesJMS

In Sessions tab there will be jms-> sessions

Right click ->new->new session

Name the session as SUNMQTEST

Goto provider tab

Right click-> add group and name it SUN MQ

Right click -> add jars and add following jars in list

fscontext.jar

imq.jar

imqadmin.jar

imqbroker.jar

imqjmx.jar

imqutil.jar

imqxm.jar

jaxm-api.jar

jms.jar

jhall.jar

imqservlet.jar

hermes-imq.jar

Press apply and go to Sessions tab

In the loader select SUN MQ will be available as loader

Now add class as below

And in the property below add as below

You will see sessions tab.

Right click on newly created session SUNMQTEST-> discover

You get pop saying how many destinations are discovered, click yes to pop up

And you are there!

Best practices for XML

working with WS

Organizing service layers:

1. Applicaiton Service layer: ERP, mainframe, Custom application, RDBMS, WS
2. Virtual service: decouple application services from consumers, Service description and end point virtualisation
3. Business service: Service provide common schema and encapsulated business logic.Functional, Entity, task based
4. Business processes:
5. UI layer


Service design considerations:

Granularity with performance
Service consumption
Security
Management: SLA
Support
Change Management
Validation

What you need to ensure while designing service:

Service granularity: performance is key here when you are breaking almost everything into pieces
Service re-use
Service Lifecycle
Service Security
Service Management

Design a service:

Schema definitions
WSDL definitions
WS-Policy definitions

Schema guidelines
Naming standard
Namespace considerations
Do not use default name space
Prefer qualified namespaces for elements
unqualified name spaces for attributes
http://<domain, com/in/blah blah..>/<sub-domain>/<namespace-type>/<subject_area>
Any simple element should be defined as global type element, specifically those which are used as input/output parameters of any service. And same can be used to define any schema.

Common issues

Weblogic log file could not rotate:

At the top of all reasons your restart failures is the log is locked and that mean jvm is running. Find out and +kill the jvm process running( in windows check task ctrl+alt+del task manager)

and last thing is go to the location "..\Oracle\Middleware_3\user_projects\domains\<you domain>\servers\<managed server name>\tmp". There must be a file with <managed server name >.lck. remove the lock and restart your server.

Log files are piling up quickly and disk full issue?

Create a script for compressing log file and one script for cleanup of log and compressed log files.

<BEA-000388> <JVM called WLS shutdown hook. The server will force shutdown now>

The admin stopped due to the reason that JVM in Weblogic, monitors and catches OS signals(CTRL+C, CTRL+D) and alerts WLS to shutdown. It will not throw any error in such conditions however allows Weblogic to clean and destroy the resources and shutdown.

Two possible fixes for Weblogic server: http://unni-at-work.blogspot.no/2009/12/bea-000388-jvm-called-wls-shutdown-hook.html

Event Driven Architecture(EDA) and Event Driven SOA

Event: An event is a notable thing that happens inside or outside your business.

An event (business or system) may signify
1. a problem
2. impending problem
3. an opportunity
4. a threshold or
5. a deviation.

Specification and Occurrence: Event refers to both specification(definition) and individual occurrence(instance).

Event in business terms: An event to be meaningful down stream subscribers(human or automated) it is imperative that event(name and body) is specified in business terms not data or application.

Event info should contain:

Header:
 specification ID
 event type
 event name
 event timestamp
 event occurence number
 event creator

Body: should specify what happened

EDA: Notable thing happening in or out of business/system, which goes immediately to interested parties. Interested parties can evaluate and optionally take action(invoke a service, trigger a business process).

1. provide agility and flexibility to business.
2. customer and business works in real time
3. passive request/reply to proactive publish/subscribe
4. active info sharing


EDA principles:

1. decoupled
2. reusability
3. Real time notifications
4. publish/subscribe
5. immediate response
6. freedom to act
7. statelessness


Extreme loose coupling

Event processing styles:

1. Simple event processing: Simple event processing is commonly used to drive the real-time flow of
work—taking lag time and cost out of a business.

2. Stream event processing: Stream event processing is commonly used to drive the real-time flow of information in and around the enterprise––enabling in time decision making.

3. Complex event processing: CEP is commonly used to detect and respond to business anomalies, threats, and opportunities.

Event flow Layers:
1.   Event generators
2.   Event channel
3.   Event processing
4.   Downstream event driven activity


Event-driven architecture implementation components:
1. Event metadata: Event metadata includes event specifications and event processing rules. Event specifications must be made available to event generators, event format     transformers, event processing engines, and subscribers.
2. Event processing: The cores of event processing are the engine and the event occurrence data.simple events engines are usually in house developed. CEP is used for trend analysis and audit.
3. Enterprise Integration: Some of the service are required from integration services are: event pre-processing (filters, routes,and transformations), event channel transport,service invocation, business process invocation,publication and subscription, and enterprise information access.
4. Sources and Targets:
5. Event Tooling: required for event specifications and processing rules, and to manage subscriptions. Event management provides admin and monitoring.



Ref:

https://gupea.ub.gu.se/bitstream/2077/23491/1/gupea_2077_23491_1.pdf

Loose coupling

Three phases of composites:
Modelling phase: semantic coupling: Strong(Service components), Weak(QoA, Governance Aspects), Non-Predominant
composition phase: Syntactic coupling: String and weak
execution phase: Physical coupling

Level of Loose coupling:

Loose coupling the implementation: Standard based, inter-operable specifications or protocol

Loose coupling the service contract: what happens when the service contract itself changes?difficult proposition, but there are plenty of best practices, service contract change management: Active management and service intermediaries and SOA governance.

Loose coupling the service policy: A policy is a form of metadata, as are contracts. changes to policy should be handled in the same way they handle changes to service contract: late-binding, service intermediary-enabled, registry-based, and governed. 

Loose coupling the process: Once you have a business process that composes a bunch of services together, we have another area of potential tight coupling. What happens when the process changes?The movement in the SOA and BPM markets has been towards separating the process definition layer from the underlying implementation.

Loose coupling the data schema: 

Loose coupling the infrastructure



Contextually coupling in EAI depends on:
Number of input data items
number of output data items
Dependency of service on other service
Services dependent on the service
Use of shared data
time dependency
Protocol uses


http://searchsoa.techtarget.com/tip/The-seven-levels-of-loose-coupling
http://www.zapthink.com/2006/10/03/the-role-of-transformation-services-in-soa/

Service, Oriented, Arhictecture

Service:
         Encapsulation
         Interface
         Contract or service level agreements

Orientation:
         Granularity
         Collaboration

         Universality


Architecture:
         Consistency
         Reliability

         Extensible
         Scalable

Consistent architecture with standards(open and SOA) gives inter-operability.


Service orientation is an architectural concept that refers to the loose coupling of a service (an abstract resource with a defined job) and its provider (the physical asset(s) that perform the job tasks).

The two primary styles of SOA used in business solution development are
1. composite application development and flow.
2. a service may generate an event.

SOA principles:

Loose coupling
Inter-operability
Reusability

Standardization
Modularity
Composibility
Discoverability
Abstraction
Coarse-grained interfaces
Statelessness

SOA Implementation components:
Business Services
Application services: user access, business function, common services, user interaction, business process choreography, and information management.
Common Services: utilities
Information Management service: provide uniform way of representing, accessing, maintaining, managing, analyzing, and integrating data and content across heterogeneous information sources.
ESB: Service and metadata(e.g., endpoint) virtulization


What SOA gives to a business and technology?

It provides a common term of understanding for both business and technical org/ppl.
Interoperability: technology players agreed to be interoperate with SOA standards
Extensions and evolution: recognizes the ever changing world
Reuse:

Jargons:

Service component architecture
Component: a piece of business logic, process logic, BPEL process, mediator
Service
Reference
wire
composite.xml

OSB: service abstraction layer, simple orchestration

Oracle SOA: service composition

BPEL: complex orchestration

BPEL PM: system centric process engine

Business PM: human centric process engine

Business rule: business decision making, for decision point.

Secutiry is characteristic of service.

Centralize policy repository and distributed enforcement at boundaries

Enterprise Manager: is policy manager

B2B: EDIFACT, ANSIX12, Rossetanet, ebXML

CEP: time or series based analysis of data.

CEP(complex event processing) : requires to monitor several of them to spot and respond to trends
vs
MP(Message processing): requires some action on their own with little or minimal context.

EDN: need more citation

Application Server: WLS, CEP does not requires AS, OSB and SOA Suite components run in separate containers.

Database: to maintain runtime info and configuration info.
Oracle SOA designed to run on any sql DB. Oracle DB is best for production.


SOA Suite Layers:

Service Layer: BAM,B2B,BPEL, Mediator, Rules, CEP, HumanWorkflow
Infrastructure Layer: Transformation, Policy, SCA, Registry
Binding Layer: SOAP, JCA
Portability Layer: interface between SOA Suite and AS/J2EE platform

OSB VS Mediator:

1. Weblogic vs Multi platform(weblogic, jboss, websphere)
2. (M)validation, routing and transformation within composite
3. (M)low cost for service virtualisation comes with SOA
4. (O)

OSB:

Binding: provides abstract interface no info of physical end point and other configuration
Port: provides physical end points

Proxy service defines the message flow.

Service end point virtualisation: move the service to anywhere

"Test Console" service is not running. Contact administrator to start this service.

http://anuj-dwivedi.blogspot.no/2010/07/osb-11g-11113-test-console-not-working.html

Adaption path for SOA

Business architecture: as:is -> to be-> gap analysis-> process design-> governance requirements

SOA Application architecture: (as is)application communication protocol-> to be-> governance framework

Data Architecture: data models(source and target) and repository, database design

Service design

Governance reporting

JCA with Oracle SOA

JCA 1.5 compliant resource adapters.

Oracle JCA Adapters include:

• Oracle Technology Adapters
  
  These adapters include:
  
  • Oracle JCA Adapter for Files/FTP
  • Oracle JCA Adapter for Sockets
  • Oracle JCA Adapter for Oracle Streams Advanced Queuing (Oracle JCA Adapter for AQ)
  • Oracle JCA Adapter for JMS
  • Oracle JCA Adapter for Database
  • Oracle JCA Adapter for MQ Series
• Legacy Adapters
  
  These adapters include:
  
  • OracleAS Adapter for Tuxedo
  • OracleAS Adapter for CICS
  • OracleAS Adapter for VSAM
  • OracleAS Adapter for IMS/TM
  • OracleAS Adapter for IMS/DB
    
    Legacy adapters include the following components in the architecture
    
    • Oracle Connect: Oracle Connect is a component that resides on the legacy and mainframe platforms. It consists of native adapters for communicating with the mainframe application and data stores.
    • Oracle Studio
    • J2CA Adapter
• Packaged-Application Adapters
  
  These adapters include:
  
  • OracleAS Adapter for PeopleSoft
  • OracleAS Adapter for SAP R/3
  • OracleAS Adapter for Siebel
  • OracleAS Adapter for J.D. Edwards
    packaged-application adapters consists of OracleAS Adapter Application Explorer (Application Explorer), J2CA 1.5 resource adapter, and Business Services Engine (BSE).
    
    Application Explorer is a Java swing-based design-time tool for configuring packaged-application adapters. Using Application Explorer, you can configure the back-end application connection, browse the back-end application schemas, and expose these schemas as adapter services. Application Explorer is shipped with packaged application-specific plug-ins for browsing the back-end application-specific metadata.
    
    Application Explorer works in conjunction with the BSE, which is deployed in the Oracle WebLogic Server container of the Oracle Application Server. BSE uses SOAP as a protocol for accepting requests from clients, interacting with the back-end application, and sending responses from the back-end application back to clients.
    
    
• Oracle Adapter for Oracle Applications
  Adapter for Oracle Applications is also informally known as Oracle E-Business Suite Adapter
  
  
  Adapters provide the following types of services to facilitate communication between applications:
  
  • Request-Response (Outbound Interaction) Service
  • Event Notification (Inbound Interaction) Service
  • Metadata Service
  
  
  
  The request-response service, also known as J2CA outbound interaction,
  and the event-notification service, also known as J2CA inbound interaction,
  are described in J2CA binding configuration files.
  The run-time connection parameters are specified in the weblogic-ra.xml file and linked to a Java Naming and Directory Interface (JNDI) name, which is specified during design time.
  
   Oracle technology adapters are deployed in the J2CA container of the Oracle WebLogic Server. Oracle Fusion Middleware integrates with these J2CA 1.5 adapters through the JCA Binding Component, which converts Web service messages into J2CA interactions and back.