Module 7: Structuring Mule Applications

About

¹.

Objectives

• Create applications composed of multiple flows and subflows
• Pass events between flows using asynchronous queues
• Encapsulate global elements in separate configuration files
• Specify application properties in a separate properties file and use them in the application
• Describe the purpose of each file and folder in a Mule project
• Define and manage application metadata

Notes

Intro

Mule configuration files are the primary source for Mule projects but not the only files that can be used

Breaking the project into multiple flows makes it easier to understand the project

At the end of this module, you should be able to

• Create applications composed of multiple flows and subflows

• Pass events between flows using asynchronous queues

• Encapsulate global elements in separate configuration files

• Specify application properties in a separate properties file and use them in the applicatino

• Describe the purpose of each file and folder in a Mule project

• Define and manage application metadata

Encapsulating processors into separate flows and subflows

Topic video

• Break up flows into separate flows and subflows

  • Makes the graphical view more intuitive

    • You don’t want long flows that go off the screen
  • Makes XML code easier to read
  • Enables code reuse

  • Provides separation of concerns between an interface and implementation

    • We already saw this
  • Makes them easier to test

• Flows vs subflows

  
  Flows vs subflows

  • Flows can have their own error handling strategy, subflows cannot
  • Flows without event sources are sometimes called private flows
  • Subflows are executed exactly as if the processors were still in teh calling flows

• Creating flows and subflows

  • Several methods

    • Add a new scope: Flow or Sub Flow
    • Drag any event processor to the canvas - creates a flow
    • Right-click processor(s) in canvas and select ‘Extract to’

  • Use Flow Reference component to pass events to other flows or subflows

    • Metadata is automatically propagated and can be customised
    • When a flow or subflow all references to it are automatically renamed as well

  • Variables persist through all flows unless the even crosses a transport boundary

    • We saw this in the last module

    • The content of the event persist throughout all flows of the application unless it crosses the transport boundary

      • For example, in the previous module with the HTTP Request, when the goodByeFlow was triggered, a new event was created for that flow by the Listener; and when it completed, the payload and attributes in the original event were replaced with the HTTP response

  • Walkthrough 2-1: Create and reference subflows and private flows

    • Extract processors into separate subflows and private flows
    • Use the Flow Reference component to reference other flows
    • Explore event data persistence through subflows and private flows

Passing events between flows using asynchronous queues

Topic video

• Passing events between flows using asynchronous queues

  • What using Flow Reference, events are passed synchronously between flows

  • You may want to pass events asynchronously between flows to

    • Achieve higher leves of parallelism in specific stages of processing
    • Allow for more-specific tuning of areas within a flow’s architecture
    • Distribute work across a cluster
    • Communicate with another application running in the same Mule domain
    • Implement simple queueing that does not justify a full JMS broker
    • This can be accomplised using the VM connector

  • Using the VM connector

    • Use the connector for intra and inter application communication through asynchronous queues
    • Add the VM module to the project

    • Configure a global element configuration

      • Specify a queue name and type

        • Queues can be transient or persistant
        • By default, the connector uses in-memory queues
        • Transient queues are faster, but are lost in teh case of a system crash
        • Persistent queues are slower but reliable
      • Use operations to publish and/ or consume events

  • Walkthrough 2-2: Pass events between flows using the VM connector

    • Pass events between flows using the VM connector
    • Explore variable persistence with VM communication
    • Publish content to a VM queue and then wait for a response
    • Publish content to a VM queue without waiting for a response

Walkthrough steps

• When Publish Consume is used, the flow of the event similar to the Flow reference; the output is ‘GOODBYE maxine’, which is different from HTTP Request in that it doesn’t make use of the passed Response variable

• When using Publish, however, it is different: the output is ‘Hello’ instead of ‘GOODBYE Maxwell’. It also doesn’t go to the goodBye subflow, but goes to the Logger instead unlike Flow Reference and Publish Consume. The attributes, however, are HttpRequestAttributes since Publish is an asynchronous operation, and doesn’t wait

Extras

• Mule Application Structure Explained | Lightboard Series

  • Breaking code apart will provide the following benefits: reuse, testability, maintanability, coping with complexity and understandability

  • Use:

    • Flow reference: the Mule event will get passed to the reference flow, and when finished, the result comes back and continues on. So there is waiting or this is a Synchronous operation.
    • Async Scope: this is invoked asynchronously. The original execution continues, there is no breaking, but the async scope creates a new thread/ scope and goes on its own and doesn’t come back

    • VM or other messaging platform: VM queues can be invoked through the connector operations in the VM connector

      • VM Publish operation: similar to Async scope - when it hits the Publish operator, it bypasses on as it was, so the result doesn’t come back and merge in
      • VM Publish Consume: provides a synchronous mechanism. It’s more like a Flow Reference, than Async scope or VM Publish. It will wait around. When the thread comes in, it waits up to a certain point in time, and continues on when the results are available

Organising Mule application files

Topic video

• Separating apps into multiple configuration files

  • Just as we separated flows into multiple flows, we also want to separate configuration files into multiple configuration files
  • Monolithic files are difficult to read and maintain

  • Separating an application into multiple configuration files makes code

    • Easier to read
    • Easier to work with
    • Easier to test
    • More maintainable

• Encapsulating global elements in a configuration file

  • If you reference global elements in one file that are defined in various, unrelated files

    • It can be confusing
    • It makes it hard to find them

  • A good solution is to put most global elements in one config file

    • All the rest of the files reference them
    • If a global element is specific to and only used in one file, it can make sense to keep it in that file

• Creating multiple applications

  • You are also not going to want all your flows in one application/ project

  • Separate functionality into multiple applications to

    • Allow managing and monitoring of them as separate entities
    • Use different, incompatible JAR files
  • Run more than one application at a time in Anypoint Studio by creating a run configuration

• Sharing global elements between applications

  • A domain project can be used to share global configuration elements between applications, which lets you

    
    Domain Project

    • Ensure consistency between applications upon any changes, as the configuration is only set in one place
    • Expose multiple services within the domain on the same port
    • Share the connection to persistent storage
    • Call flows in other applications using the VM connector
  • Only available for customer-hosted Mule runtimes, not on CloudHUb or RTF

  • The general process

    • Create a Mule Domain Project and associate Mule applications with a domain
    • Add global element configurations to the domain project

• Walkhthrough 2-3: Encapsulate global elements ina separate configuration file

  • Create a new configuration file with an endpoint that uses an existing global element
  • Create a configuration file global.xml for just global elements
  • Move the existing global elements to global.xml
  • Create a new global element in global.xml and configure a new connector to use it

Organising and parametirising application properties

Topic video

• Application properties

  • Provide an easier way to manage connector properties, credentials, and other configurations
  • Replace static values

  • Are defined in a configuration file

    • Either in a .yaml file or a .properties file
  • Are implemented using property placeholders
  • Can be encrypted
  • Can be overridden by system properties when deploying to different environments

• Defining application properties

  • Create a YAML properties file in the src/main/resources folder: config.yaml

  • Define properties in the hierarchical YAML file

    db:
      port: "3306"
      user: "mule"
    

  • Create a Configuration properties global element

• Using application properties

  • In global element configurations and event processors: ${db.port}
  • In DataWeave expressions {port: Mule::p('db.port')}

• Overriding property values in different environments

  • Use system properties to override property values when deploying an application to a different environment (like dev, qa, production)

  • Set system properties (JVM parameters) from

    • Anypoint Studio

      • In Run > Run Configurations > Arguments

  • The command line for a standalone Mule instance

    mule -M-Ddb.database=training2 -M-Ddb.password=mule2
    

• Walkthrough 2-4: Use property placeholders in connectors

  • Create a YAML properties file for an application
  • Configure an application to use a properties file
  • Define and use HTTP and Salesforce connector properties

Organising Mule project files

Topic video

• Examining the folder structure for a Mule project

  
  Mule Folder Structure

  • The names of folders indicate what they should contain

  • src/test folders should contain files only needed at development time

    • Like schema and example files for metadata types, sample data for transformations
    • They are not included in the application JAR when it is packaged

• In Mule 4, Mule applications are Maven projects

  • Maven is a tool for building and managing any Java-based project that provides

    • A standard way to build projects
    • A clear definition of what a project consists of
    • An easy way to publish project information
    • A way to share JARs across several projects
  • Maven manages a project’s build, reporting and documentation from a central piece of information - the project object mode (POM)

  • A Maven build produces one or more artifacts, like a compiled JAR

    • Each artifact has a group ID (usually a reversed domain name, like com.example.foo), an artifact ID (just a name), and a version string

• The POM (Project Object Model)

  • Is an XML file that contains info about the project and configuration details used by Maven to build the project including

    • Project info like its version, description, developers, and more
    • Project dependencies
    • The plugins or goals that can be executed

• Walkthrough 2-5: Create a well organised Mule project

  • Create a project based on a new API in Design Center
  • Review the project’s configuration and properties files
  • Create an application properties file and a global configuration file
  • Add Java files and test resources files to the project
  • Create and examine the contents of a deployable archive for the project

Managing metadata for a project

Topic video

• Defining metadata

  • It is often beneficial to define metadata for an application

    • For the output structures required for transformations

    • For the output operations that can connect to data sources of different structures

      • Like the HTTP Request connector

  • For the output of connectors that are not DataSense enabled

    • And do not automatically provide metadata about the expected input and output

• Ways to access the Metadata Editor

  • From the Transform Message component
  • From the Metadata tab in the properties view for most event processors
  • From a project’s menu in the Package Explorer

Summary

Separate functionality into multiple applications to allow managing and monitoring of them as separate entities

Mule applications are Maven projects

• A project’s POM is used by Maven to build, report upon, and document a project

• Maven builds an artifact (a Mule deployable archive JAR) from multiple dependencies (module JARs)

Separate application functionality into multiple configuration files for easier development and maintenance

• Encapsulate global elements into their own separate configuration file

Share resources between applications by creating a shared domain

Definie application properties in a YAML file and reference them as ${prop}

Application metadata is stored in application-types.xml

Create applications composed of multiple flows and subflows for better readability, maintenance, and reusability

Use Flow Reference to call flows synchronously

Use the VM connector to pass events between flows using asynchronous queues

Test your knowledge

Refer to the exhibit. In the deployable archive’s/ classes folder, there are two properties files named dev.properties and prod.properties. The Mule application fails to deploy to CloudHub through Runtime Manager with the following error message.

• Answer: The env property is NOT set in the Runtime Manager in the the Mule application’s Properties tab

What reserved property can be defined and used in a Mule application to allow an HTTPS Listener to be accessed by external web clients after the Mule application is deployed to CloudHub?

• Answer: ${https.port}

Why must a Mule application’s deployable archive package all its dependencies in order to be deployed to CloudHub?

• CloudHub workeres CANNOT download ALL possible project dependencies a project may contain

A Mule application has two flows named parentFlow and childFlow. A variable is defined in parentFlow. What is the scope of the variable when the parentFlow calls childFlow using a Flow Reference?

• Answer: The variable is accessible in childFlow, can be changed, and changes are seen back in parentFlow

What can ONLY be done with VM connectors, and NOT with Flow References, in a single Mule application?

• Allow a flow to pass events to another flow asynchronously

In what file does the Mule project keep track of all its dependencies?

• pom.xml

Anki

Links

• Anypoint Platform Development: Fundamentals - Part 2 Building Mule applications with Anypoint Platform
• Module 6: Accessing and modifying Mule events
• Module 8: Consuming web services

References