API Led Connectivity 101 - Amazon Simple Storage Service (S3)
API Led Connectivity 101
Author: Brett Roeder Principal Client Architect
API Led Connectivity Three Layer Architecture
Table of Contents API Led Connectivity 101 Table of Contents ALC Three Layer Architecture Review Layer Definitions System API Layer Definition Example Process Layer Definition Example Experience Layer Definition Example ALC Use Case 1999 App Suites Cure Integration! 1999-2006 Leveraging the System APIs 2006 SaaS Apps Begin Infiltration of IT 2008 M&A Madness! 2009 Defining Core Enterprise Processes 2011 Here comes Mobile! 2012 Gotta Open Up to Partners 2015 Bye Bye Old Apps Summary
MuleSoft Proprietary & Confidential
2/19
API Led Connectivity Three Layer Architecture
ALC Three Layer Architecture Review MuleSoft’s API-led connectivity approach provides a framework for ordering and structuring the building blocks of a composable enterprise to optimize the agility and flexibility benefits of using the Anypoint Platform. This comes in a form a multi-tier architecture with three distinct API layers. This document describes these 3 layers in principle and in practice, and illustrates the types of use-cases defined by each layer.
Layer Definitions MuleSoft’s proposed reference architecture for API Led Connectivity is based on three layers: System, Process and Experience. Let us first try and define the different layers.
System API Layer
Definition
The System API (sAPI) layer is the foundational layer of the three tier architecture. System APIs are typically defined for the various domains of the organization (ex. Customer, Contact, Order, Product, Notification, Employee, Position, …) A System API defines the contract for how to interact/communicate with each domain. It contains the resource(s) (ex. Customer), methods (GET, POST, PUT, …), input/output schemas (json, xml, …), response codes (200, 201, 400, 404, …), etc.
Example
The diagram above depicts three different System APIs (Product, Customer and Employee). You can see from the diagram that each API actually performs CRUD operations against multiple systems/applications. For the purposes of thoroughly understanding System APIs let’s look in more detail at the Customer sAPI.
MuleSoft Proprietary & Confidential
3/19
API Led Connectivity Three Layer Architecture The RAML spec below specifies the Customer API definition/contract. For more information on RAML, see raml.org.
#%RAML 0.8 title: Customers API version: v1 #baseUri: http://0.0.0.0:8081 baseUri: http://mythicalcustomers.cloudhub.io traits: !include ./traits/traitcollelction.yaml /customer: displayName: Customers description: | Working with customers: * Retrieving all customers * Creating customers get: description: | Retrieve all the Customers. is: [ collection ] responses: 200: body: application/json: example: !include ./examples/customer_collection_example.json post: description: | Create a new Customer body: application/json: schema: !include ./schemas/customer_schema.json example: !include ./examples/customer_example.json responses: 200: description: Created body: application/json: example: !include ./examples/customer_post_example.json /{name}: get: description: | * Retrieve all the customers by name is: [ collection ] responses: 200: body: application/json: example: !include ./examples/customer_collection_example.json 400: body: application/json: example: | { MuleSoft Proprietary & Confidential
4/19
API Led Connectivity Three Layer Architecture "message": "Bad message" } 404: body: application/json: example: | { "message": "Resource not found" } /{id}: displayName: Customer ID description: Get Customer by ID uriParameters: id: type: integer minimum: 0 example: 451 description: Represents the sales order unique id get: responses: 200: body: application/json: schema: !include ./schemas/customer_schema.json example: !include ./examples/customer_example.json 400: body: application/json: example: | { "message": "Bad message" } 404: body: application/json: example: | { "message": "Resource not found" } put: body: application/json: schema: !include ./schemas/customer_schema.json example: !include ./examples/customer_example.json responses: 204: delete: responses: 204:
MuleSoft Proprietary & Confidential
5/19
API Led Connectivity Three Layer Architecture Focusing in on the Customer sAPI we see there are five systems that contain Customer data: ● Sugar CRM ● Salesforce ● SAP ● ORACLE DB ● Marketo But of course, the customer records in each application are not the same. They aren’t even called Customer in all of them! Sugar CRM = Account Salesforce = Account SAP = Customer Database = Client Marketo = Account Obviously the attribute names and types in the various systems are different as well. So you end up with a flow like the one to the right. A search request comes in. It is scattered to all the systems to retrieve matching customers. The records from each system is transformed to the Customer schema defined in the API. The aggregated result set is then returned to the requesting app. The key point here is the client application does not know whether the data is coming from one system or all five. They simply perform a search based on name and they get a response. That is the insulation layer the System APIs provide to all consumers.
MuleSoft Proprietary & Confidential
6/19
API Led Connectivity Three Layer Architecture
Process Layer Definition
Process APIs (pAPI) fulfill many different purposes. They orchestrate and choreograph the invocation across multiple System APIs. The orchestrations may perform data aggregation (split-join), conditional routing (if Region=APAC then do this…), filtering (Only show available orders), and more. The focus in designing a process API is to define the business logic for a given process. Examples are Employee Onboarding and Order Fulfillment. The System APIs implemented the heavy lifting of how to orchestrate creates, updates and deletes and aggregate query results into a common format across multiple systems. With the System APIs defined, defining the business process is significantly simplified since the designer uses a more a standardized approach to access the various heterogeneous systems using simpler protocols like HTTP and more generic data types.
Example
The example above shows how the Order Fulfillment pAPI leverages the Product, Customer and Order sAPIs to fulfill and order. Without System APIs in place the Order Fulfillment Process API would look something like the following. It’s very complex due to the many systems containing customer data.
MuleSoft Proprietary & Confidential
7/19
API Led Connectivity Three Layer Architecture
It’s complex because each developer must orchestrate the interaction with the core systems. If a new system is added, one retired, or one is modified (maybe due to an upgrade to a new version), the process must be updated. With System APIs in place defining the Order Fulfillment process becomes a little bit easier :-)
You can see from the above two Process API examples the power of leveraging System APIs. You don’t know, and don’t care what systems are involved.
MuleSoft Proprietary & Confidential
8/19
API Led Connectivity Three Layer Architecture
If you just had one process that needed to leverage Product, Customer and Order data you wouldn’t be saving much time. But you’re not going to have just one process that needs Product, Customer and Order data, you’re going to have many! If you had to embed the complex logic in each it would be a nightmare to manage and maintain. But… leveraging System APIs hides the complexity in one place the System API - and makes creating the process APIs easy. When the orchestration related to managing the data domain changes, it is the System API that is changed, no processes are impacted.
Experience Layer Definition
System APIs are now exposing the core data domains of the organization through a common API contract and the Process APIs have now exposed our core processes via a common API. Almost all APIs today are designed to expose data via JSON. But some applications, partners, devices, etc. need don’t understand JSON and need a different format (ex. XML, CSV, …) To deliver the data in the appropriate format you would create an Experience API (xAPI) and map the JSON to/from the desired format. But there are other reasons to implement an xAPI. ● Provide data format translation ● Aggregate multiple APIs into a single xAPI ● Remove extraneous/unneeded methods to simplify the API for the designated consumer ● Apply specific policies to the xAPI that are different for other sAPIs, pAPIs and xAPIs Typical xAPIs are just performing data format translation or providing an alternate management option so they are typically very lightweight. The biggest benefit customers see from leveraging xAPIs is the speed at which the API contract with the consumer can be defined. The key to that speed is the web-based API Designer and it’s Mocking Service. The desired xAPI can be quickly defined, the project generated and the the mapping defined between the xAPI and the Process and System APIs it may leverage.
MuleSoft Proprietary & Confidential
9/19
API Led Connectivity Three Layer Architecture
Example
The Mobile Storefront xAPI was designed from a Mock Up of a Mobile app in collaboration with the mobile app designers. The xAPI contains all resources (Product, Order and Customer) and methods needed to build the Mobile Storefront app. Another xAPI example would be you have a partner that wants to send you xml but your pAPI is expecting JSON. Putting an xAPI in front of the pAPI allows you to quickly perform a data transformation from XML to JSON and back.
MuleSoft Proprietary & Confidential
10/19
API Led Connectivity Three Layer Architecture
As your API Led Connectivity environment evolves you will end up with many apis across the different layers (as shown below).
The power of the API Led Connectivity Three Layer Architecture is the flexibility it provides to the organization. Let’s review the benefits.
MuleSoft Proprietary & Confidential
11/19
API Led Connectivity Three Layer Architecture System APIs
● ●
Retire old applications and on-board new systems without impacting the Process APIs, Experience APIs, and applications that utilize a System API. Provide a common vocabulary that is used across all applications/systems involved in a domain. Ex. Mythical calls their customers customers. Sugar CRM, Salesforce, and Marketo refer to Customer as Account. SAP = Customer and your proprietary database = Client. This extends down to the attributes and methods as well.
Process APIs
●
Process APIs allow you to define a common process which the organization can share. Rather than Order Fulfillment being implemented multiple times across the organization, it is defined once and published for all to utilize.
Experience APIs
●
Experience APIs allow you to adapt the Domain and Process APIs for particular application, partner or other use. Rapidly transform data formats for a partner or app Apply unique policies for a particular use case
● ●
MuleSoft Proprietary & Confidential
12/19
API Led Connectivity Three Layer Architecture
ALC Use Case Now that we have defined API Led Connectivity and the Three Layer Architecture let’s look at the impact it would have on Mythical Corporation.
1999 App Suites Cure Integration!
In 1999 Mythical Corp implemented the entire SAP stack. The CIO had the foresight to have her Chief Architect define System APIs (contracts) for Product, Customer, Order, Employee, Location, … which defined how Mythical wanted to expose their data domains vs. how they were defined by SAP. This included defining the domains, changing attribute names, refining/changing data types, defining enumerations, breaking apart complex entities and more. The focus of the domain definition was to ensure the contracts would stand the test of time and not be tied to any specific application. After the domain contracts were defined, CRUD services were created based on the core domain definitions and map the SAP capabilities were mapped to the System API.
1999-2006 Leveraging the System APIs
From 1999 to 2006, 273 different applications and integrations were implemented that leveraged the System APIs. These implementations consisted of internal integrations, web apps, shopping cart, point of sale, partner integration, and more.
MuleSoft Proprietary & Confidential
13/19
API Led Connectivity Three Layer Architecture
2006 SaaS Apps Begin Infiltration of IT
Fast-forward to 2006. The EVP of HR for Mythical Corp went to a conference and got excited about this new Workday thing. After an evening of drinking in Las Vegas he signs a contract. He comes back, walks into the CIO’s office, and tells her she now has to yank out SAP HR and switch everything to Workday.
Because the CIO had the foresight to have the applications interact with System APIs instead of SAP directly, her IT organization’s job of migrating from SAP HR was easier. Other organizations who would have directly connected to SAP would have had to change the 273 applications to integrate with Workday. It would have been a multi-year implementation that would have cost a significant amount of money. But because Mythical’s IT organization hid SAP behind System APIs they only had to change the System APIs (ex. Employee, Position, Location, …) from SAP to Workday. The 273 applications and integrations required no modification.
2008 M&A Madness!
In 2008 Mythical Corp acquired Fictitious Fixtures. In order to minimize disruption to the business, the decision was made to leverage Mythical Corp’s web site and customer facing systems but allow Fictitious Fixtures to continue to run their core applications: ORACLE E-Business Suite for their ERP, Salesforce for CRM, and PeopleSoft for HR.
MuleSoft Proprietary & Confidential
14/19
API Led Connectivity Three Layer Architecture
Integration into the Mythical infrastructure was easy since none of the 273 applications needed to be modified in order to work with Fictitious Fixtures enterprise applications. All that needed to be done was to add Fictitious FIxtures applications (ORACLE e-Biz, SFDC, PeopleSoft, ...) into the existing System APIs and adjust the orchestration within the System API accordingly.
2009 Defining Core Enterprise Processes
Over the years the CIO and Chief Architect had realized tremendous savings in time and money from their System API implementation. In 2009 began implementing Process APIs to define and expose the core business processes of the organization (ex. Employee On-Boarding, Order Fulfillment, …) to provide a similar level of abstraction.
Implementing the process APIs was quicker and easier than expected. They realized since they had exposed all their applications via the System APIs they did not have to worry about how to connect to any of the source systems (ex. SAP, PeopleSoft, …) and figure out how to transform the data (ex IDOCs). They could simply leverage the System APIs which all exposed the data as JSON and were accessible via REST calls.
MuleSoft Proprietary & Confidential
15/19
API Led Connectivity Three Layer Architecture
2011 Here comes Mobile!
In 2011 Mythical saw the opportunity to open new channels of revenue. One of those channels was to introduce a mobile app. While the Order Fulfillment Process API exposes the necessary functionality to place an order, the mobile app also needed to provide product search and the ability to create and manage an account (customer). It was easy to create a Mobile Order Experience API that exposed the necessary functionality to the mobile app.
In addition to exposing the methods and data to the mobile app, the Mobile xAPI also allows Mythical Corp to apply policies to the xAPI directly related to managing applications that access the mobile app.
MuleSoft Proprietary & Confidential
16/19
API Led Connectivity Three Layer Architecture
2012 Gotta Open Up to Partners
In 2012 Mythical entered into partnerships with two new partners. One partner wanted to communicate via XML and the other using CSV.
Since the Process API was expecting JSON, Mythical simply created an experience API for each partner to handle to data transformation between each Partner. This also allows the API owner to establish SLAs and apply policies on a per partner basis. Over the years many more experience and process APIs were created to on-board additional partners, provide access to additional applications, to leverage social apps and of course to integrate applications. The evolution resulted in the following diagram.
MuleSoft Proprietary & Confidential
17/19
API Led Connectivity Three Layer Architecture
2015 Bye Bye Old Apps
In 2015 the decision was made to retire the on-prem applications and move to a SaaS portfolio.
Because of the ALC 3LA none of the process or experience APIs need to be modified - only the System MuleSoft Proprietary & Confidential
18/19
API Led Connectivity Three Layer Architecture APIs were modified to remove the on-prem apps. But the bigger savings was none of the applications, integrations or Partners need to be updated since they were built on top of the data, process and experience APIs. That’s the beauty of adopting the API Led Connectivity approach.
Summary How easy is it for you to provide on-demand connectivity? If your business needed to provide an an API in order to capitalize on a new market could you do it quickly with your current approach & tools? In today’s interconnected world how quickly connect could be the determining factor in whether you are first in a new market opportunity or whether you are an also ran. Not only can API Led Connectivity accelerate your ability to rapidly deliver connected solutions, but it also provides an agile environment enabling isolation between the different layers: System, Process & Experience. To see a demo of the platform in action visit: http://tinyurl.com/alcdemo
MuleSoft Proprietary & Confidential
19/19
