Module 1: Introducing Application Networks and API-Led Connectivity

About

This is module 1 covering introduction to application networks and the API-led connectivity approach.

Objectives

• Explain what an application network is and its benefits
• Describe how to build an application network using API-led connectivity
• Explain what web services and APIs are
• Make calls to secure and unsecured APIs

Intro

Companies need to innovate themselves to stay competitive by providing their customers with on-demand services. MuleSoft is enabling companies to innovate and stay competetive by providing them the tools and an approach to accomblish this. This is using API-led connectivity to build application networks.

At the end of this module, you should be able to:

• Explain what an application network is and its benefits
• Describe how to build an application network using API-led connectivity
• Explain what web services and APIs are
• Make calls to secure and unsecured APIs

Identifying the problems faced by IT today

The challenges are not new - innovation and staying competitive have always been there. The difference is in the rate of change faced by organisation. IT cannot go fast enough. In the past there were fewer changes to the technology landscape, and they played out over many years. However, now IT is constantly being bombarded by changes. There has been an explosion of Applications, Mobile innovations, Social Media, Cloud offerings, Smart devices, and access to a large amount of data that come with them. These are the digital disrupters that require IT to change the way that they operate. To succeed, companies must be able to quickly respond to all these changes and leverage them as they continuously get introduced. The yellow line in the graph below represents the relatively fixed capacity available for to address these discrupters. Distance between the yellow line and the resources needed to overcome the challenges is know as the transformation delivery gap. The transformation delivery gap is the distance between the IT delivery capacity and the demands on IT presented by various innovations such as Cloud and SaaS, Big Data, Analytics, Mobile and Digital Competition. Digital pressures create a widening transformation delivery gap.

Unless something changes, the gap will continue to widen as the pace of technology continues to increase. As things stand today, IT cannot delivery projects to business fast enough to keep up with the pace of change. Hiring more people will only raise the capacity so much.

A new way of working to close the delivery gap

What’s needed is a more transformative change to the operating model. Instead of producing and delivering every project themselves, central IT needs to use its capacity to create re-usable assets that can be discovered and reused by the rest of the organisation. They need to produce the assets and enable the entire organisation to consume them through self-serve and deliver more of their own projects the right way. In this way, core system of record can be unlocked, while IT maintains control over them. Scale can be achieved by distributing the workload across the organisation. This enables the company to close the delivery gap and have the speed it needs to respond to market pressures: leveraging new technologies and innovating. Line of business developers can self-serve, building their own solutions using the core assets created and governed by IT, instead of waiting for IT to produce the assets for them when they have ideas and want to innovate rapidily. This gives the company the speed it needs to respond to market pressures, pushing new technologies and innovating.

Introducing a new IT operating model

Topic video

Using this new operating model members of Central IT/ Line of Business IT (LoB IT) can produce re-usable assets that can be made available to the rest of the organisation. This enables LoB IT to deliver some of their own projects. The key to this strategy is emphasis is on consumption as well as production. The whole point of the production of reusable assets, not just for easier maintainability and re-use by IT but for ease of consumption by other parties so they can self-serve and build their own applications. This production-consumption model is different to the traditional production model as it encourages the collection of active feedback from consumers, along with usage metrics to be fed back to the producers. This helps the producers produce create assets, that are easier to use, and benefit the needs of the consumers.

A common project-based approach using Traditional operating model

Project objective: Web app provides real-time order status and order history for sales team engaging with customers
Data resides in the following:

• Order data in eCommerce system
• Inventory data in SAP
• Customer data in SAP, Salesforce

A common approach to delivering new features is for developers to jump in and start writing code to aggregate data from both systems, then write more code to use inconjunction with the data from the e-commerce system. This would produce the order status and the order history data, this is then hooked up into a web app API used by the web app.

The result will be:

• On time and within budget
• Limited opportunity fo reuse
• Tight coupling = brittleness. Point to point connections are made directly to the data sources from within the application making them tightly coupled. If one changes, the whole application may break and will need to be updated. Because the connections are coded directly inside the application, they are difficult to secure and monitor.
• Difficult to govern. The connections are difficult to secure and monitor because they are coded directly to the code
• Meets business requirements - Although it meets the requirements, it falls short when the same data wants to be re-used
• Adding more features e.g. a mobile app, will require the repetition of the process used to create the desktop app, despite the functionality being almost identical
• This is inefficient and unleveraged way of doing things
• A different operating model that leverages reusability to build the same functionality over and over again

New operating model

Modern API: The core enabler of a new operating model Apps are built using composable APIs

• The APIs must fulfil a number of requirements to comply to the new operating model

  • Discoverable and accessible through self-service
  • Productised and designed for ease of consumption
  • Easily managed for security, scalability, and performance

• Modern APIs are the core enabler of a new operating model

• This is called API-led connectivity

• This approach uses groups of services instead of using one service

• Each API within the group of services is its own API that can be governed and monitored

• Diagram of API-led connectivity approach

  
  API led Connectivity Approach

• Access to the systems of record such as SAP, Salesforce and the eCommerce systems are built as System APIs

• Data is retrieved using these APIs, aggregated, transformed and ochestrated as needed using Process APIs

• It is then packaged and modified as needed as an Experience API, to best be consumed by the end-user, the Web application, in this case

• When a request for new features comes in, the connections to the systems of record (SAP, Salesforce & eCommerce) do not have to be recreated

• The developers can instead go straight to building a new Experience API for the new mobile app

• This delivers the data that’s needed by that app and in the right format

• This mobile API can make calls via the Experience API to the existing Process API to leverage the work already done to orchestrate the data retrieved form the systems of record/ data sources using the System APIs

• If there are request for connections to shipping sources, the connection can just be added in the form of System APIs, which retrieve data and be made available for use by the mobile app and all the applications within the network

• The end result

  • Enable and empower the entire organisation

  • Image of organisation empowerment

    
    Enable and Empower the Entire Organisation

  • Central IT typically unlocks the data in the core system of records, producing System APIs, whose access can be managed and governed
  • Line of Business IT (LoB IT) typically discovers and consumes data made available by these System APIs, and then create additional APIs
  • These are Process APIs, which compose data, transforming, enriching, aggregating and orchestrating it as needed
  • These Process APIs are now reusable by others as well
  • The developers building the functionality can discover, self-serve, reuse and consume all the APIs, and then build the Experience layer of the APIs
  • Everyone across the organisation is driving the production-consumption model of reusable and discoverable assets

• Analysis of the Traditional and Modern (API-led) delivery models

  Traditional	Modern (API-led)
  On time and within budget	On time and within budget
  Limited opportunity for reuse	Drives reuse vs build
  Tight coupling = brittleness	Designs in readiness for change
  Difficult to govern	Builds in governance, compliance, security, and scalability
  ? Meets business requirements	Meets the needs of your business

• C4E: Organising differently to drive API-led connectivity

  
  Center for Enablement C4E

  • C4E is a cross-functional team, which evangelises and drives the adoption of API-led connectivity
  • C4E ensures all new products are built using existing assets as much as possible, and producing new assets as needed for functionality or connections that do not yet exist
  • They are responsible for promoting the consumption of assets across the organisation
  • Their succes is based on asset consumption

  • C4E ensures that assets are

    • Productised and published
    • Consumable
    • Consumed broadly
    • Fully leveraged
  • Success of C4E measured on asset consumption

Achieving an application network

Topic video

Another advantage of using an API-led connectivity approach is that an application network is formed

This is the network of applications, data and devices connected with the APIs

The power of this network is that every time another node or API is added, the number of possible connections increases multiplictivly

The addition of every new node adds significant value to the rest of the network resulting in a network effect

Applications that exist in an organisation are usually siloed

Using API-led connectivity, connections are formed between applications through the APIs

This brings about an increase in speed, agility and innovation in production and consumption of apps and better respond to innovations and market pressures, making the business a market disrupter

An application network:

• Emerges bottoms-up via self-service through discoverability by the entire organisation

• Provides visibility, security and governability at every API node

• The emergence of an application network also provides speed, agility and innovation.

• Is recomposable: it bends, not breaks - built for change

Reviewing APIs and web services

Topic video

Deconstructing APIs

• A modern API is:

  • Discoverable and accessible through self-service
  • Productised
  • Designed for ease of consumption
  • Easy to manage for security, scalability and performance

• What is an API?

  • An Application Programming Interface

  • It provides the information for how to communicate with a software component, defining the

    • Operations (what to call)
    • Inputs (what to send with a call)
    • Outputs (what you get back from a call)
    • Underlying data types

  • It defines functionalities independent of implementations

    • You can change what’s going on behind the scenes without changing how people call it

  • What do people mean when they say API?

    • The could be referring to a number of things:

      • 1. An API interface definiton file (API specification)
        • Defines what you can call, what you send it, and what you get back
      • 2. A web service
        • The actual API implementation you can make calls to or the interface of that API implementation
      • 3. An API proxy
        • An application that controls access to a web service, restricting access and usage through the use of an API gateway
    • Modern APIs include all three of the above (API specification, web service and API proxy)

• What is a web service?

  • Different software systems often need to exchange data with each other

    • Bridging protocols, platforms, programming languages, and hardware architectures
  • A web service is a method of communication that allows two software systems to exchange data over the Internet

  • Systems interact with the web service in a manner prescribed by some defined rules of communication

    • How one system can request data from another system, what parameters are required, the structure of the return data, and more

  • The parts of a web service

    • The web service API

      • Describes how a client interact with the web service
      • It may or may not (though it should) be explicitly defined in a file
      • It could be any sort of text in any type of file but ideally should implement some standard API description language (or specification) e.g. RAML, OAS

    • The web service interface implementing the API

      • Is the code providing the structure to the application so it implements the API properly
      • This may be combined with the actual implementation code

    • The web service implementation itself

      • Is the actual code and application
      • In theory, this code is distinct from the code implementing the API from the second meaning (The web service interface implementing the API)
      • But in practice, the two code bases are sometimes combined

  • There are two main types of web services

    • SOAP web services

      • Traditional, more complex type
      • The communication rules are defined in an XML-based WSDL (Web Services Description Language) file

      • The communication require

        • The XML-based WSDL file
        • A protocol such as HTTP

    • RESTful web services

      • Recent, simpler type
      • Use the existing HTTP communication protocol

      • REST stands for Representational State Transfer

        • An architectural style where clients and servers exchange representations of resources usign the standard HTTP protocol
        • Stateless: The server does not remember any client state from previous requests
        • Clients can cache previous responses to avoid repeated network calls
      • Other systems interact with the web service using the HTTP protocol
      • The HTTP request method indicates which operation should be performed on the object identified by the URL

      • Examples of RESTful web service calls

        • Data and resources are represented usign URIs

        • Resources are accessed or changed using a fixed set of operations

          • (GET) /companies
          • (GET) /companies?country=France
          • (GET) /companies/3
          • (POST) companies with JSON XML in HTTP body
          • (DELETE) /companies/3
          • (PUT) companies/3 with JSON XML in HTTP body

        • RESTful web services request methods

          • GET retrieves the current state of a resource in some representation (usually JSON or XML)
          • POST create a new resource
          • DELETE deletes a resource

          • PUT replaces a resource completely

            • If the resource doesn’t exist, a new one is created

          • PATCH partially updates a resource

            • Just submitted data

        • Example RESTful web service response

          • JSON (JavaScript Object Notation)

            • A lightweight data-interchange format (without a lot of extra XML markup)
            • Human-readable results (usually JSON or XML)
            • Supports collections and maps

        • Learning about APIs

          • API documentation

            • Should include the list of all possible resources, how to get access to the API, and more

          • API portals

            • Accelerate onboarding by providing developers a centralised place for discovering all the tools they need to successfully use the API, which could include

              • Documentation, tutorials, code snippets, and examples
              • A way to register applications to get access to the API
              • A way to provide feedback and make requests
              • A way to test the API by making calls to it

          • Discover APIs in API directories and marketplaces

            • For example, APIs.guru, which has thousands of APIs

Calling RESTful web services

Topic video

Calling RESTful web services

• To call web services, you need to write code or have a tool to make the HTTP requests

  • Need to be able to specify the HTTP method, request headers, and request body

• Example tools include

  • An API portal with an API console
  • Advanced Rest Client
  • Postman
  • A cURL command-line utility

• Making calls to RESTful APIs

  • Unsecured APIs

    • The API may be public and require no authentication

  • Secured APIs

    • The API may be secured and require authentication
    • You may need to provide credentials and/ or a token
    • Often a proxy is created to govern access to an API

    • You can also secure an API with other authentication protocols

      • OAuth
      • SAML
      • JWT

• Getting responses from web service calls

  • RESTful web services return an HTTP status code with the response

  • The status code provides client feedback for the outcome of the operation (succeeded, failed, updated)

    • A good API should return status codes that align with the HTTP specification

      
      Good API Status Codes

• Common HTTP status code

  Code	Definition	Returned by
  200	OK - The request succeeded	GET, DELETE, PATCH, PUT
  201	Created - A new resource or object in a collection	POST
  304	Not Modified - Nothing was modified by the request	PATCH, PUT
  400	Bad request - The request could not be performed by the server due to bad syntax or other reason in request	All
  401	Unauthorised - Authorisation credentials are required or user doesn’t have access to the resource/ method they are requesting	All
  404	Resource not found - The URI is not recognised by the server	All
  500	Server error - Generic something went wrong on the server side	All

Successfully creating application networks using API-led connectivity

Topic video

Producing discoverable and consumable assets is key

• The key is to having a good application network is by creating assets that are discoverable and consumable

Designing for API success

• Create APIs that developers can find and want to use and share with others

  • Design the API for the business use cases it will fulfill, not to model the backend services or applications they expose
  • Focus on performance of client applications and user experience

• Take an API design-first approach!

• Get API design right before investing in building it

  • Define it iteratively getting feedback from developers on its usability and functionality along the way
  • Building the implementation of an API is time consuming and expensive to undo

API development cycle

• This is supported by Anypoint Platform

• There are three circles

  • 1. Phase 1: API specification
    • The output from this phase is the API specification

    • Stages

      • 1. Design stage
        • Receive requirements
        • Design the API specification
      • 2. Simulate stage
        • Create the prototype for implementation
        • The prototype is then made accessible to users who can provide feedback, which helps the designer determine if they’ve considered all the consumer’s needs
      • 3. Validation
        • The output is the contract or API specification
  • 2. Phase 2: API Implementation
    • Deals with the API Implementation, where the API specification is used to build and test API functionality
    • At the completion of this phase, the API is ready to be deployed as a web service
  • 3. Phase 3
    • Deals with pointing and managing the web service

    • It does the following:

      • Establishing the API version
      • Creating policies to secure it
      • Deploying and registering the web service with the API
      • Monitoring operations
      • Analysing and reporting on the usage
      • Troubleshooting and scaling to meet the resource demands
      • Responding to the changing needs of the organisation by repeating the lifecycle

Extras

Three-layered Connectivity Architecture Explained | Lightboard Series

• Breaking apart the implentation is key to having a coherent architecture

• As you bring about more features, you can make use of existing functionality

• The following principles are key to a good decomposition of the interactions/ architecture

  • DRY - Don’t Repeat Yourself
  • KISS - Keep It Simple and Succinct
  • YAGNI - You Ain’t Gonna Need It

• The following is a detail of the interaction

  • Client (Client Apps)

    • It is the originator of the request

      • Could be the system making an HTTP or SOAP call
      • Could also be something that’s triggering a file being created
    • Communicate with the Experience API using a protocol of some sort

    • Examples

      • Scheduler

        • Creating a file

      • Messaging queue

        • Pushing messages onto a queue
        • Listening to a JMS
  • HTTP or another protocol RPC to connect

  • System (which needs data to be sent to or requested from)

    • Experience API

      • Client experience (needs of the client)
      • Experience application or integration
      • It may make calls to both System and Process APIs

    • Process API

      • RESTful JSON is better than XML

        • Adding an extra field to XML, which has a schema, causes problems since schemas are inflexible
        • JSON can accomodate change and is flexible

      • Different them by different domains e.g. departments

        • Not trying to be too unified, and avoiding building functionality you may never need
        • You may not need all the CRUD functionalities
      • You can have the same names in the Process API as the System API so it’s good to name them accordingly e.g. p-orders.xml and s-orders.xml
      • It may make one or more calls to the System API

    • System API

      • Create functionality that talks to the systems of record
      • A good starting point for creating System APIs is per system rather than fine grained, and then break it apart more depending on complexity
      • It’s sheilded by the Process API so can be broken down and have the Process API processing the data that comes from the decomposed System APIs
  • Server (System of record Apps)
  • NB: the Client and Server Apps can be the same e.g. Salesforce but need to be split up in diagram representations by functionality

Summary

Companies today need to rapidly adopt and develop new technologies in order to stay relevant to customers & keep competitive

IT needs to be able to rapidly integrate resources and make them available for consumption

An API-led connectivity approach can help achieve this

To drive API-led connectivity, create a C4E (Center for Enableament)

A cross-functional team to ensure assets across the organisation are productised, published, and widely consumed

An application network is a network of applications, data, and devices connected with APIs to make them pluggable and to create reusable services

A web service is a method of communication that allows two software systems to exchange data over the Internet

An API is an application programming interface that provides information for how to communicate with a software component

The term API is often used to refer to any part of a RESTful web service

The web service API (definition or specification file)

The web service interface implementing the API

The web service implementation itself

A proxy for the web service to control access to it

RESTful web services use standard HTTP protocol and are easy to use

The HTTP request method indicates which operation should be performed on the object identified by the URL

Test your knowledge

1. What is a core characteristic of the Modern API?

API is designed first using an API specification for rapid feedback

2. What MuleSoft API-led connectivity layer is intended to expose part of a backend database without business logic?

System

3. What statement is part of MuleSoft’s description of an application network?

Creates reusable APIs and assets designed to be consumed by other business units

4. According to MuleSoft, what is the Center for Enablement’s role in the new IT operating model?

Creates and manages discoverable assets to be consumed by line of business developers

5. Refer to the exhibit. The API specification supports searching for articles on the searchworld.org site. What is the most idiomatic (used for its intended purpose) URL and method to retrieve articles about “einstein” in XML format?

Answer

6. What HTTP method in a RESTful web service is typically used to completely replace an existing resource?

PUT

Anki

Part 1: Getting started with Anypoint Platform

Introducing application networks and API-led connectivity

• Walkthrough 1-1: Explore an API directory and an API portal

What is communication to a system called?

• Question

  What is communication to a system called?

• Answer

  Request

What is communication from a system called?

• Question

  What is communication from a system called?

• Answer

  Response

Walkthrough 1-2: Make calls to an API

• Response methods

Links

• Anypoint Platform Development: Fundamentals - Notes - Part 1: Getting started with Anypoint Platform
• Anypoint Platform Development: Fundamentals - Introducing the course
• Module 2: Introducing Anypoint Platform

References