User guide · Amazon ECS on AWS Fargate

AI-Powered Trace & Process Document-to-Adaptive BPMN & Process Graphs

Give it your standard operating procedure, the records of how real cases were resolved, or both. It works out the business flows they describe, which of your tools each step calls, who does what, and the decisions that connect the steps. You get the result as a knowledge graph, as BPMN diagrams you can run, and as an RDF graph, all in your own S3 bucket. You only upload files: the build starts by itself. It works for any business process you want to model; the examples in this guide use a tax reconciliation process only as a sample.

1What it does

You upload your inputs to S3, an AI pipeline running in your AWS account builds the graph, and the results go back to S3.

The examples are samples. The examples in this guide (a tax reconciliation procedure, the tools read_invoice and lookup_tax_rule, case ids such as INV-TAX001-001) come from one sample process. Use any process you like: invoice approval, customer onboarding, claims handling, incident response, or anything else that has a procedure and cases. You choose the name of each use case.

What a "flow" is

A flow is one end-to-end scenario a case can follow. It has a trigger (for example "the tax rule is missing"), a resolution (who fixes what, with which tool), and an outcome. Each flow in the graph holds:

Steps

In order: the starting inputs, every tool call, every step a person performs (reviews, approvals, supplying values), and the final report.

Tools

Which operations from your tool catalog each step calls, with their URL, method, parameters and the values they return.

Decisions and data

Where a case goes next and under which condition (for example lookup_tax_rule_result == "NOT_FOUND"), and which values each step needs and produces.

One graph per use case

Each business process you model is a use case with its own folder under input/ and under output/ in your bucket, for example input/tax-reconciliation/ and output/tax-reconciliation/ (a sample name; call yours whatever describes the process, such as invoice-approval or customer-onboarding). Each use case has its own graph, history and build status, and builds of one use case never touch another.

What the first build of a use case does

The agent looks at which inputs are present and picks the mode itself:

You uploadWhat happens
A playbook onlyThe playbook is split into its flows, and each flow is built from it. Best for a first graph from documented procedures.
Case traces onlyEach resolved case either becomes a new flow or is merged into the flow it belongs to. Best when you have history but no written procedure.
BothThe playbook builds the graph first, then each case updates the flow it matches, adding the variations seen in practice. This gives the most complete graph.

After that, every build of the use case is incremental: it reads only what is new in the input folder and updates the existing graph (section 6). Upload new cases whenever you have them and the build starts by itself: new flows are added, and existing flows gain the new paths, with each change recorded.

2What you upload

Your files go in the use case's folder under input/. Two file names are fixed, tools.json and sop.txt; the case-trace files can have any name.

s3:///input//
├──          tool catalog (OpenAPI), exactly this name   (required)
├──             the procedure (SOP) as plain text, exactly this name   (optional)
├──     case traces: any name, .json or .jsonl,     (optional)
└── …                           as many trace files as you like, sub-folders allowed

A build needs tools.json plus sop.txt, a trace file, or both. Keep adding trace files to the same folder over time (for example one file per month). Files that were already processed can stay: the agent remembers every case it has ingested and skips it.

Upload once, then wait. Upload everything a build needs in one go, then don't upload to that use case again until the build has finished (section 4 shows how to see that). You can upload to different use cases at the same time.

Optional

sop.txt

A plain-text procedure: its steps, checks, decisions, error codes, roles and corrections. Name the tools the steps use (for example [read_invoice]) when you can; that makes the mapping exact. One SOP per use case; for a second procedure, make a second use case.

Optional

Case traces

The record of how individual cases ran, in the format below: an id and a trace for each case. The agent summarizes every trace for you.

Required

tools.json

An OpenAPI 3 specification of the tools your process uses. Each operation needs an operationId, its request schema, and its response schema, ideally with the status values it can return.

Case-trace format

Every trace file is a JSON list (or .jsonl, one object per line). Each entry has exactly two keys: id, which names the case, and trace, which is the record of that run.

[
  {
    "id": "INV-TAX001-001",
    "trace": { "steps": [ { "name": "read_invoice", "output": { "status": "FOUND" } }, ... ],
               "decisions": [ ... ] }
  },
  {
    "id": "INV-TAX001-002",
    "trace": "The workflow called read_invoice and lookup_tax_rule returned NOT_FOUND ... "
  }
]

The agent reads each trace with an AI model and writes a summary of it, which it keeps with the results (output/ingested/). The graph is built from that summary. A trace longer than 300,000 characters is not ingested: split it or shorten it.

A case id is read once. Once a case has been ingested, the agent never reads that id again, even if its content changes. To correct a case, or to add another case for the same invoice, give it a new id (for example INV-TAX001-002).

What the tool catalog needs

Limits

WhatLimit
Flows in one use caseUp to 25. A use case is one process; a process with more flows should be split into several use cases (one folder each). When a graph holds more than 25 flows, a new case is compared only with the first 25.
Size of sop.txtUp to 200 KB of plain text (about 30,000 words). The whole procedure is read by the model in one piece, so a longer one gives poorer results. For a longer procedure, split it by process into several use cases.
Size of one traceUp to 300,000 characters. A longer trace is reported as failed and is not billed.

3One-time setup

Done once per AWS account by your administrator, when AI-Powered Trace & Process Document-to-Adaptive BPMN & Process Graphs is launched from AWS Marketplace.

  1. Subscribe to AI-Powered Trace & Process Document-to-Adaptive BPMN & Process Graphs in AWS Marketplace with the AWS account that will run it. Each task checks the subscription when it starts and stops if the account is not subscribed.
  2. Enable model access in the Amazon Bedrock console for both models, (Claude Sonnet) and (Claude Haiku), in the same region.
  3. Have a VPC ready with a subnet (two, in different Availability Zones, if you deploy the HTTP API) that can reach S3, Amazon Bedrock, Amazon ECR and CloudWatch Logs (a NAT gateway or VPC endpoints for private subnets), and a security group for the tasks that allows outbound HTTPS (443). The stack creates no security groups and no network.
  4. Launch the CloudFormation stack from the Marketplace listing in . Choose your bucket, , or let the stack create it, and fill in the VPC, subnets and security group (section 8). Accept the IAM capability: the stack creates the task roles and two small Lambda functions.

When the stack is created, uploading files to input/<use case>/ starts the builds. With an existing bucket, the stack adds its upload notification and keeps the bucket's other notifications; nothing else in the bucket changes.

What the stack lets the agent do
{
  "Version": "2012-10-17",
  "Statement": [
    { "Sid": "ReadInputs", "Effect": "Allow",
      "Action": ["s3:GetObject"],
      "Resource": "arn:aws:s3:::/input/*" },
    { "Sid": "NeverWriteInputs", "Effect": "Deny",
      "Action": ["s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::/input/*" },
    { "Sid": "ReadWriteOutputs", "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::/output/*" },
    { "Sid": "ListUseCases", "Effect": "Allow",
      "Action": ["s3:ListBucket"],
      "Resource": "arn:aws:s3:::" },
    { "Sid": "InvokeModel", "Effect": "Allow",
      "Action": ["bedrock:InvokeModel", "bedrock:InvokeModelWithResponseStream"],
      "Resource": ["arn:aws:bedrock:*::foundation-model/anthropic.*",
                   "arn:aws:bedrock:*:<ACCOUNT_ID>:inference-profile/*"] },
    { "Sid": "MarketplaceMetering", "Effect": "Allow",
      "Action": "aws-marketplace:MeterUsage", "Resource": "*" }
  ]
}

The agent never writes to input/: only your own uploads go there. The Lambda that starts builds can list input/, start the build task, and write one small status file, output/<use case>/trigger.json. No access keys are used anywhere.

Your data stays in your account. Inputs and results are in your S3 bucket, and the model is called through Amazon Bedrock in your account with the task's role.

4Running a build

Upload your files to the use case's input folder, wait for the build to finish, then read the results. The same steps work for the first build and for every later one.

  1. Upload the files (later builds: just upload the new trace files). The order doesn't matter:
    aws s3 cp  s3:///input//
    aws s3 cp  s3:///input//
    aws s3 cp  s3:///input//

    The build starts when the folder has tools.json and at least one more .txt, .json or .jsonl file. Until then nothing starts, and output/<use case>/trigger.json says what is missing.

  2. Wait until it has finished. Don't upload to that use case again meanwhile. The build runs as a task in your ECS cluster, and takes from a few minutes up to a few hours for a large first build (section 6). To follow it:
    • output/<use case>/jobs/ holds one status file per build: state is running, then succeeded or failed. It also says which kind of build ran, which cases were ingested, and how many model calls it made.
    • The task's log is in CloudWatch Logs, in the log group /ecs/<stack name>.

    If files do arrive while a build is running, the task builds again after it, so they are not lost; waiting is simply the reliable way.

  3. Read the results in output/<use case>/ (section 5).

Starting a build by hand

The stack's outputs include RunBuildNowCommand, an aws ecs run-task command that builds every use case with something new. You rarely need it: use it to retry after a failed build, or if you turned the upload trigger off (TriggerOnUpload=false).

Optional HTTP API

If you deployed the stack with DeployApi=true, an internal load balancer serves these actions (POST <ApiUrl>/invocations with a JSON body, from inside your network):

ActionPayloadWhat you get back
build{"action":"build","use_case":"…"}A job_id, and where the status file will be. Processes what is new in the use case's input/.
plan{"action":"plan","use_case":"…"}What a build would do right now, without running it: the kind of build, the new case ids, and whether the playbook or tool catalog changed. Free: no model calls.
status{"action":"status","use_case":"…","job_id":"…"}The state, the kind of build, the cases ingested or failed, the number of model calls, the files written, and any error.
use_cases{"action":"use_cases"}Every use case in the bucket, with its graph version, number of cases ingested and its last build.
flows{"action":"flows","use_case":"…"}Every flow in the graph, with its description and number of steps.
ask{"action":"ask","use_case":"…","flow_name":"…","question":"…"}An answer about one flow, based only on that flow's graph (for example "Who approves the invoice correction?").

A use case name is 2–63 lowercase letters, digits and -, and is the name of its folders. The API has no authentication of its own: allow trusted networks only.

5What you get

Everything the agent writes for a use case lives in output/<use case>/. Don't edit it by hand.

s3:///output//
├── trigger.json                            what the upload trigger did last: started a build, or what it is waiting for
├── manifest.json                           every input file and case ingested so far, and every build
├── registry/                               the knowledge graph
│   ├── flow_with_activity_details.json     ★ every flow, complete: steps, tools, data, decisions
│   ├── flow_registry.json                  flow names and descriptions
│   ├── flow_with_tool_registry.json        + the tools of each flow
│   ├── flow_with_activity_registry.json    + the ordered steps
│   ├── flow_with_edges_registry.json       + data and transitions (same content as ★)
│   ├── inverted_index.json                 which flows use each tool, step, condition, value
│   ├── diff_log.json                       what each input added or changed, and why
│   └── versions/<flow>/v1.json, v2.json …  every stored version of each flow
├── artifacts/
│   ├── flows.json                          the flows found in the playbook (playbook builds)
│   ├── bpmn/<flow>.bpmn                   BPMN 2.0 diagram per flow (Camunda 8)
│   └── kg/flow_with_activity_details.ttl   the graph as RDF / Turtle (pbOnto ontology)
├── tool_bindings.json                      the tool catalog as the graph uses it (to detect changes)
├── ingested/<case>.json                    each ingested case: its id and trace, and the summary made from it
├── history/v<N>/                           an earlier graph, kept when a changed playbook replaced it
├── jobs/<job_id>.json                      status of each build
└── lock                                    present only while a build is running

The knowledge graph

registry/flow_with_activity_details.json is the main result: one entry per flow, each with its steps in order. A step records its type, the tool it calls, the values it needs and produces, and its outgoing transitions with conditions. Every flow, step and transition also lists the inputs it came from (the playbook file, or the case ids).

BPMN diagrams

artifacts/bpmn/ holds one Camunda 8 process per flow. Tool calls become HTTP service tasks, people's steps become user tasks with forms, and decisions become gateways with their conditions. Open them in Camunda Modeler to review or run them.

RDF graph

artifacts/kg/flow_with_activity_details.ttl is the same graph in Turtle, against the pbOnto ontology. Load it into a triple store (for example Apache Jena Fuseki) to query it with SPARQL.

Change log

registry/diff_log.json has one entry per flow from the playbook and per case: whether it created a flow or updated one, what changed (new steps, new tools, new branches and their conditions), the evidence for each change, and any case that was skipped and why.

Example: one step of a flow

From a flow built from the sample tax-reconciliation SOP and five cases:

{
  "node_id": "lookup_tax_liability",
  "type": "tool_call",
  "requiredEntities": [{"name": "invoice_id"}, {"name": "expected_tax"}],
  "edges_out": {
    "produces_entities": ["lookup_tax_liability_result", "booked_amount", "liability_amount"],
    "transitions": { "to_actions": [
      { "target_node_id": "review_payment_discrepancy",
        "condition": "lookup_tax_liability_result == \"NOT_FOUND\"",
        "traces": ["playbook:sop_tax_reconciliation.txt", "INV-TAX008-001", "INV-TAX020-CN1"] },
      { "target_node_id": "present_reconciliation_results",
        "condition": "lookup_tax_liability_result == \"ERROR\"" }
    ]}
  }
}

Example: flows found in a playbook

The sample SOP produced these five flows:

FlowTriggerResolution
Full reconciliationEvery check passesReported as reconciled, nothing corrected
Missing tax ruleTax rule not found (TAX_001 / TAX_002)Custodian supplies the rule, it is written, clearance is re-run, ledger re-posted
Invoice tax correctionTax amount or arithmetic mismatch (TAX_003 / TAX_004)Steward approves, invoice corrected, bank line synced if the gross changed, ledger re-posted
Liability correctionLiability missing or wrong (TAX_008)GL Controller confirms, liability and payment written
Payment correctionPayment short, missing or over (TAX_009 / TAX_010)GL Controller confirms, payment written

6What to expect

How long a build takes

It depends on the size of your inputs. In our tests with the sample data:

  • a 10 KB SOP with five flows: about 4½ minutes
  • each case trace: about 30 to 45 seconds, including its summary
  • the SOP and two traces together, started by an upload: about 6½ minutes from upload to the end of the task

A new task takes about half a minute to start before it begins work. There is no time limit on a build, and a large first build can take an hour or more.

How the graph grows

  • Each build starts from the use case's graph in S3 and adds only the new cases to it.
  • A case that matches a flow is merged into it: nothing is deleted, and an older path that a newer case replaced is kept and marked as superseded.
  • A case that doesn't match any flow becomes a new flow.
  • To start a use case over, delete its output/<use case>/ folder. Everything is then built again from input/, and billed again.

What each build does

The agent compares the input folder with the use case's manifest and does only the work needed. With the HTTP API, {"action":"plan"} shows the decision before you build.

What is in input/What the build doesModel calls
A use case that was never builtNew: the playbook (if any) builds the graph, then every case updates it.Yes
New case idsUpdate: only the new cases are ingested. Cases already ingested are skipped, in the same file or in old files you left in the folder.For the new cases only
A changed tool catalogUpdate: every flow that uses a changed tool gets the new URL, method, parameters and response fields, and new BPMN and RDF files. The status lists the flows whose tool inputs or outputs were renamed, added or removed, so you can review their data and conditions.None
A changed (or newly added) playbookRebuild: the graph is built again from the new playbook, and every case ingested so far is applied again. The previous graph is kept in output/history/.Yes, like a first build
Nothing newUp to date: nothing runs.None

A case that fails (for example a temporary model error) is tried again by the next build. Removing a file from input/ doesn't remove its cases from the graph.

Review the graph before you rely on it. The flows are built by an AI model (Claude, through Amazon Bedrock). Results are usually close, but they can vary between runs and can contain mistakes, such as a missing branch or a condition worded differently from your procedure. Use the change log and the BPMN diagrams to review them, and correct a case by uploading it again with a new id.

Getting the best results

7Troubleshooting

The build ended with state: failed

The status file has the error message and a technical detail. The most common causes are below.

I uploaded files but no build started

Open output/<use case>/trigger.json: it says waiting and why. A build starts when the use case's folder has tools.json and at least one .txt, .json or .jsonl file. Check that the files are directly in s3:///input// (or in a sub-folder of it), that the use case name is 2–63 lowercase letters, digits and -, and that the tool catalog is named exactly tools.json. If trigger.json is missing altogether, the upload did not reach input/ of the stack's bucket, or the stack was launched with TriggerOnUpload=false.

No tool spec in the input folder: upload the OpenAPI tool spec as tools.json

The use case's folder has no tools.json (or it is somewhere else). Upload the tool catalog under exactly that name. More than one OpenAPI tool spec means two files named tools.json exist in the folder and its sub-folders; keep one.

No files in s3://…/input/ or AccessDenied

The use case's input folder is empty or misspelled, or the task's role can't read it. The stack grants s3:ListBucket on the bucket and s3:GetObject on input/*; if you changed the role, check those.

Nothing to build

A new use case needs sop.txt or at least one trace in its input folder, besides tools.json.

A trace is listed under not_traces in the build's plan

The entry is not {"id": …, "trace": …}: it has no id, no trace, or an empty trace. The message names the file and the entry number. Fix the entry and upload the file again; entries that were already ingested are skipped.

My corrected case was not picked up (changed_content_ignored)

That case id was already ingested, so the agent skips it. Give the corrected case a new id and upload it again.

AccessDeniedException from Bedrock

Model access isn't enabled for or in this region, or the role can't invoke it. Enable access in the Bedrock console and check the InvokeModel permission. A model id starting with us. only works in US regions; pick the inference profile for your region.

A build says busy

A build for the same use case is already running; the extra task leaves it alone and ends. Wait until the running build's status is succeeded or failed. A lock older than 9 hours is cleared automatically.

The task stops right after it starts, with a subscription message in the log

The AWS account running the stack is not subscribed to AI-Powered Trace & Process Document-to-Adaptive BPMN & Process Graphs in AWS Marketplace, or the task has no task role. Subscribe, then upload one of your files again to start the build, or start a build by hand (section 4).

A case shows skipped or error in the change log

That case did not change the graph. The entry says why, for example insufficient information when the trace doesn't say enough about how the case was resolved. Improve the trace, give it a new id, and upload it again; the other cases are unaffected. A case that ended in error is retried automatically by the next build.

8Settings reference

What you fill in when you launch the stack, and the variables the stack sets for the agent. The values shown are this deployment's.

Stack parameters

ParameterMeaning
BucketName, CreateBucketThe bucket for input/ and output/: . Let the stack create it (encrypted, private, kept when the stack is deleted) or use an existing one.
VpcId, SubnetIds, SecurityGroupId, AssignPublicIpWhere the build tasks run. Subnets need a route to S3, Bedrock, ECR and CloudWatch Logs. The security group must allow outbound HTTPS (443).
RootPrefixOptional folder above input/ and output/, for example my-team/ (ending in /).
TriggerOnUploadtrue (default): an upload starts the build. false: start builds by hand.
ModelId, HaikuModelIdClaude Sonnet and Haiku models or inference profiles. A us. profile only works in US regions; pick the profile for your region.
TaskSize, MaxTokens, LogRetentionDays, KmsKeyArnOptional: task size (small, medium, large), longest model answer, log retention, and your own KMS key for the bucket.
DeployApi, ApiSecurityGroupIdOptional internal HTTP API behind a load balancer, with your own security group for it.

Variables the stack sets for the agent

VariableValueMeaning
S3_BUCKETYour bucket, with input/<use case>/ and output/<use case>/
ROOT_PREFIX(empty)Optional folder above input/ and output/
TOOLS_FILE_NAMEThe name of the tool catalog in each use case's input folder
USE_CASEUse case for API calls that don't name one
MODEL_IDClaude Sonnet model or inference profile, used for every step
HAIKU_MODEL_IDClaude Haiku model or inference profile, used for the first pass over the tool catalog
MAX_TOKENS20480Maximum length of each model answer
MAX_TRACE_CHARS300000Longest trace the agent will read; a longer one is not ingested
AWS_REGIONThe stack's region