AWS Step Functions — Combine Data from Parallel State

With AWS Steps Function, you can setup state machines that can handle simple or complex workflows since you have a few State types available for use. One great State, is the Parallel state that allows you to run two or more tasks in parallel. However you may have a question on how the output from each branch of the Parallel state, can be combined or passed on to the next state.

For this solution, you will use a combination of “ResultPath” in the Parallel branches and “Parameters” in the state that comes after the Parallel state. The reason ResultPath is important, is that it lets you assign the path of the value generated by the Parallel branch, so that you can be able to get the values as Parameters.

Below is an example of a State Machine that looks up user information, and in the parallel state provides the Name and Address in parallel.

Below is the JSON definition of the State Machine.

{
 "Comment": "Parallel Example.",
 "StartAt": "LookupCustomerInfo",
 "States": {
  "LookupCustomerInfo": {
   "Type": "Parallel",
   "Next": "ReturnCombinedData",
   "Branches": [{
     "StartAt": "Name",
     "States": {
      "Name": {
       "Type": "Pass",
       "Comment": "This can be any state that returns data, since it's a Pass, the values are static",
       "Result": {
        "fname": "Jack",
        "lname": "Johnson"
       },
       "ResultPath": "$.Name",
       "End": true
      }
     }
    },
    {
     "StartAt": "Address",
     "States": {
      "Address": {
       "Type": "Pass",
       "Comment": "This can be any state that returns data, since it's a Pass, the values are static",
       "Result": {
        "Number": "123",
        "Street": "Fake St",
        "Zip": "90210"
       },
       "ResultPath": "$.Address",
       "End": true
      }
     }
    }
   ]
  },
  "ReturnCombinedData": {
   "Type": "Pass",
   "Parameters": {
    "comment": "Combining the result",
    "UserDetails": {
     "Name.$": "$[0].Name",
     "Address.$": "$[1].Address",
     "StaticMessage": "This is a static message"
    }
   },
   "End": true
  }
 }
}

For this example, I am using Pass states to keep it simplified, however you can use any other State type. In the Parallel state, I have two branches that have a Pass state that is meant to grab the persons Name in while also getting the address. Once this is done, the Parallel state automatically returns the data in an array with the array having the following format:

[
   {“result of branch 1”},
   {“result of branch 2”},
   {“result of branch 3”},
   ......
] 

From the previous state machine we would get the following data from the Parallel state:

[
  {
    “Name”: {
      “fname”: “Jack”,
      “lname”: “Johnson”
    }
  },
  {
    “Address”: {
      “Number”: “123”,
      “Street”: “Fake St”,
      “Zip”: “90210”
    }
  }
],

In the state that comes after the parallel state, which I used a Pass state in this example, I used the Parameters field to combine the user information out from the array and into an object. In the Parameters field, setting the key value pair requires the following format:

“Name.$”: “$[0].Name”

Since this is an array, you would require to select the appropriate location within the array that has the data you need. Since we are setting the name, the data that has the name is in the first position “$[0]” since the name operation was done on the first branch of the array. For the address, it is in position “$[1]” of the array. In the above state machine, the final successful output would look like the following:

{
  “comment”: “Combining the result”,
  “UserDetails”: {
    “StaticMessage”: “This is a static message”,
    “Address”: {
      “Number”: “123”,
      “Street”: “Fake St”,
      “Zip”: “90210”
    },
    “Name”: {
      “fname”: “Jack”,
      “lname”: “Johnson”
    }
  }
}

With this result, you can then process this data by returning it to a user, inserting to a database, or whatever your heart desires.

Feel free to use the above definition to test and modify for your own use.