Bulk API

Rylee Soll Updated 2026-07-27 7 min read 1,305 words

features bulk-api execution scale Additional

Running the same workflow operation on hundreds or thousands of process instances one at a time is slow, error-prone, and hard to manage. Every failure needs manual error handling, and sequential execution wastes time.

The Workflow Engine Bulk API runs runtime operations on multiple process instances in parallel through a single method call. Each operation is isolated, so one failure never blocks the others.

With the Bulk API, you can create, query, command, or delete hundreds of process instances with one call. For example, approving a queue of 500 leave requests by executing the same command on each instance, with per-item success or failure reporting.

What it is

Think of it as an express lane for batch operations. Instead of walking each item through the regular counter one at a time - calling a single-instance method in a foreach loop with manual Task.WhenAll and per-item error handling - you hand the whole list at once. The system processes them in parallel up to a configurable limit, tracks each result independently, and reports back which succeeded and which failed.

For developers: The Bulk API is exposed as runtime.Bulk on WorkflowRuntime. It mirrors single-instance methods with collection-based inputs and returns a Dictionary<Guid, BulkTaskResult<TResult>> where every input key maps to its own result or exception.

using OptimaJet.Workflow.Core.Runtime;// Create 100 process instances from the same schemevar processIds = Enumerable.Range(1, 100)    .Select(i => Guid.NewGuid())    .ToList();Dictionary<Guid, BulkTaskResult<Empty>> results =    await runtime.Bulk.CreateInstanceAsync(processIds, "InvoiceApproval");

The Bulk API supports the following operations:

Per-instance error isolation

When throwOnError is false (the default), every operation runs independently. The returned dictionary contains a BulkTaskResult for each input identifier, with State set to Completed or Failed and the Exception property populated on failures:

var results = await runtime.Bulk.CreateInstanceAsync(processIds, "InvoiceApproval");foreach (var (processId, result) in results){    if (result.State == BulkTaskState.Failed)    {        Console.WriteLine($"Process {processId} failed: {result.Exception?.Message}");    }}

Set throwOnError to true to throw an AggregateException containing all errors when any operation fails. You can also configure defaults through runtime.Bulk.DefaultThrowOnError.

Concurrency control

The maxDegreeOfParallelism parameter controls how many operations run concurrently. The default is set by runtime.Bulk.DefaultMaxDegreeOfParallelism, which defaults to Environment.ProcessorCount. Lower it to reduce database connection pressure in high-contention environments:

var results = await runtime.Bulk.CreateInstanceAsync(    processIds,    "InvoiceApproval",    maxDegreeOfParallelism: 4);

Per-task progress tracking

Pass a taskFinalizedHandler to receive a callback as each individual operation completes:

var results = await runtime.Bulk.CreateInstanceAsync(    processIds,    "InvoiceApproval",    taskFinalizedHandler: (sender, args) =>    {        Console.WriteLine(            $"Process {args.LastFinalizedTaskId}: {args.LastFinalizedTaskResult.State}"        );    });

Why it matters

The Bulk API delivers these outcomes:

• Faster batch processing - Operations run in parallel up to the configured concurrency limit, not sequentially. Creating 500 process instances with one call is significantly faster than a foreach loop with 500 individual calls.

• Resilient to partial failures - One bad process ID does not derail the entire batch. Each item succeeds or fails independently. Failed items are reported with their exception details so you can retry or audit them later.

• Less integration code - No manual Parallel.ForEach with error handling, no Task.WhenAll with result aggregation, no try-catch blocks around every single call. The Bulk API handles parallelism, error isolation, and result mapping.

• Progress visibility - The optional taskFinalizedHandler callback fires after each item completes, enabling progress bars, logging, or early warning of failure patterns.

Who it is for

Evaluator (CEO, CTO, PM): Bulk operations reduce the time and code needed for batch workflows - data migrations, mass approvals, system cleanup. This means faster deployment of bulk processing features and fewer integration bugs.

Developer: The Bulk API replaces complex parallel execution patterns with a single method call and handles error isolation automatically. Use runtime.Bulk on WorkflowRuntime with collection-based inputs.

Enterprise architect: The concurrency limit prevents resource exhaustion. Each operation is isolated, so a corrupted process instance never blocks other operations.

When to use it

Use the Bulk API when you need to perform the same workflow operation on many process instances at once:

• Bulk data migration or import - Create hundreds or thousands of process instances from the same scheme using CreateInstanceAsync with a list of process IDs or CreateInstanceParams objects.

• Mass approval or escalation - Collect available commands from multiple instances, then execute the same command (approve, reject, escalate) across all of them.

• Batch cleanup - Delete completed, abandoned, or expired process instances by calling DeleteInstanceAsync with a list of IDs, instead of looping with individual delete calls.

• Batch scheme update - Call UpdateSchemeIfObsoleteAsync to bring a set of in-flight instances up to the latest scheme version, useful after deploying a process change.

How it compares

The table below shows the tradeoffs between the Workflow Engine approach and the alternative.

The Bulk API standardizes a pattern you would otherwise write from scratch every time, with proper error isolation and concurrency control already built in.

Frequently asked questions

What operations does the Bulk API support?

The Bulk API supports create instance, get process instance with parameters, get process instances tree, get available commands, execute command, execute command with restriction check, update scheme if obsolete, delete instance, and process existence check.

Does the Bulk API require a NEO license?

No. The Bulk API on WorkflowRuntime (runtime.Bulk) is available in all editions. The HTTP endpoints that wrap bulk operations are part of the RPC API and subject to its licensing.

How does the Bulk API handle partial failures?

By default, every operation runs independently. The returned dictionary contains a BulkTaskResult for each input - some complete successfully, others report a failure. One failed item does not cancel the rest. Set throwOnError to true if you prefer an exception that collects all failures after all tasks complete.

What is the default level of parallelism?

The default is set by runtime.Bulk.DefaultMaxDegreeOfParallelism, which defaults to Environment.ProcessorCount. Each operation runs on its own task, and the Bulk API limits concurrent execution to this value to avoid overwhelming the database.

Do I need to change my application code to handle bulk results differently?

The Bulk API returns the same data types as single-instance methods, organized per input identifier. You iterate over the results dictionary to check success or failure for each item. A single method call replaces what would otherwise be a manual loop with parallel execution and error handling code.

See also

• Clustering

  Horizontal scale-out across multiple servers for high-throughput scenarios.

• Persistence Providers

  Which databases Workflow Engine supports and how to configure them.