In the previous example, we saw how to invoke activities.
Next, let's consider how data flows through the system. Let's add one more activity to our example: we'll call a sortData activity on the result of the first activity, getData.
This is as easy as using a Python variable to store the result of one activity and pass it as an argument to the second activity.
Take a look at the generated JSON below: the result of getData in placed in a dictionary; that dictionary is then passed to the next state, and the right data is passed to the sortData activity.
You don't have to think about this while writing the workflow -- simply use variables as you would in Python.
AWS Step functions places limits on the size of data passed between states. This limit is currently 32kB.
When you're passing more data than this between tasks, it's a good idea to write the data to a datastore such as S3 or Redis. Then, pass reference to the data through the workflow instead.
To make this easier to implement, Cohesion provides lightweight open source wrappers for Lambda functions or Activity implementations. You may also need to configure your IAM roles for access to your chosen datastore.