Autoscaling with KEDA

How Vantage 3.0 self-hosted uses KEDA, Prometheus, and Kubernetes HPAs to scale supported workloads.

KEDA is the optional, recommended autoscaling engine for Vantage 3.0 self-hosted. When you enable it, the Vantage component charts create ScaledObject resources for supported workloads. KEDA evaluates their triggers and manages Kubernetes HorizontalPodAutoscaler (HPA) resources that scale the workloads up and down.

The Vantage workload charts require KEDA 2.17.x. Use KEDA 2.17.3. KEDA 2.18 and later cannot process some ScaledObject fields generated by the current Vantage charts.

How autoscaling works

Stage	Component	Responsibility
1	Vantage workloads	Consume CPU and expose application metrics such as queue length and active worker threads.
2	Kubernetes resource metrics and Prometheus	Supply CPU utilization and Vantage application metrics to the configured triggers.
3	KEDA operator	Watches the chart-generated ScaledObject resources and evaluates their triggers.
4	Kubernetes HPA	Uses the replica recommendation produced for each metric and selects the largest recommendation.
5	Kubernetes scheduler	Places the requested replicas on eligible nodes with available resources.

KEDA changes workload replica counts. It does not add Kubernetes nodes, increase quotas, or make an ineligible node eligible for a workload. Coordinate KEDA with your node autoscaler or other capacity-management process.

Scaling strategies

The generated ScaledObject resources use different trigger types for different workload characteristics.

Workload pattern	Trigger	Why it is used
Synchronous APIs and orchestration services	CPU utilization	Request load generally appears as CPU consumption while the service handles traffic.
Asynchronous processing and TechCore workers	Prometheus queue and worker metrics	Backlog and worker activity indicate pending work more directly than CPU utilization.

The generated thresholds, replica boundaries, queries, and HPA behavior differ by workload and Vantage chart version. Treat the generated ScaledObject resources in your installation as the source of truth.

CPU utilization triggers

CPU triggers compare a workload’s average CPU utilization with its configured target. CPU utilization is calculated relative to the workload’s CPU requests, so the Kubernetes resource metrics API must be healthy and the workload must retain valid CPU requests.

Confirm that resource metrics are available:

kubectl top nodes
kubectl top pods -n $install_namespace

Prometheus triggers

Asynchronous worker ScaledObject resources use Vantage application metrics from Prometheus. The principal signals are:

application_worker_priority_queue_length{namespace="<install-namespace>"}
application_worker_active_threads{namespace="<install-namespace>"}

• application_worker_priority_queue_length reports work waiting for a worker type.
• application_worker_active_threads reports active processing threads. Some generated worker configurations use it as a second Prometheus trigger alongside queue depth: it scales the worker up preemptively when no processing threads remain free, so a newly arriving task is picked up immediately instead of waiting for a queue backlog to accumulate.

The chart-generated queries add workload-specific service and worker selectors and may use rolling time windows. Do not replace them with the two broad inspection queries above.

Some ScaledObject resources contain more than one trigger. Any trigger can request a scale-out, and the HPA uses the largest replica recommendation. When a workload scales unexpectedly, inspect every trigger rather than only the first one.

Scale worker types in proportion to your skills

TechCore processing is divided into worker types, such as OCR, Extraction, and Classification, and each worker type scales independently on its own queue. Your deployed skills determine which worker types carry the most work: an OCR-heavy skill mix drives demand for OCR workers, while an extraction-heavy mix drives demand for Extraction workers.

Because documents move through these worker types in stages, capacity imbalance between stages appears as waiting rather than as errors. If one stage is under-provisioned relative to the stage that feeds it (for example, ample OCR capacity but too few Extraction workers), documents clear the first stage quickly and then pause for several minutes while the downstream queue drains and its workers scale out. Watch per-worker-type queue depth during a representative load test with your production skill mix, and discuss adjustments to the relative replica boundaries with your ABBYY account team.

Prerequisites

Before enabling KEDA, confirm that:

1. KEDA 2.17.3 is installed and its operator and metrics API server are healthy.
2. The Kubernetes resource metrics API returns CPU data.
3. Prometheus is reachable at http://prometheus-operated.observability.svc.cluster.local:9090.
4. The Vantage ServiceMonitor exists and Prometheus reports the Vantage /metrics-text targets as UP.
5. Prometheus returns current values for the Vantage worker metrics.
6. The cluster has enough schedulable capacity for the additional replicas, including nodes labeled for TechCore workers.

The Prometheus service name, namespace, and port in item 3 form a fixed endpoint for the generated ScaledObject resources.

Use these checks before continuing:

kubectl get deployment --all-namespaces | grep keda-operator
kubectl -n observability get service prometheus-operated
kubectl -n $install_namespace get servicemonitor

In Prometheus, confirm that these queries return series for your install namespace:

application_worker_priority_queue_length{namespace="<install-namespace>"}
application_worker_active_threads{namespace="<install-namespace>"}

For Prometheus installation and service-mesh requirements, see Monitoring with Prometheus.

Enable autoscaling

Enable KEDA in the Vantage configuration and enable the chart-created ServiceMonitor through the separate top-level observability stanza:

vantage:
  keda:
    enabled: true

observability:
  prometheus:
    namespace: observability
    serviceMonitor:
      create: true

The observability stanza is not part of vantage or the Vantage custom resource. It controls monitoring resources rendered directly by the vantage-selfhosted chart.

Apply the values through your existing vantage-selfhosted Helm release. ArgoCD creates the component applications and their ScaledObject resources asynchronously.

Verify autoscaling

List the generated resources:

kubectl -n $install_namespace get scaledobjects.keda.sh
kubectl -n $install_namespace get hpa

Wait for the ScaledObject resources to become ready:

kubectl -n $install_namespace wait \
  --for=condition=Ready scaledobjects.keda.sh \
  --all \
  --timeout=5m

Inspect an individual resource and its HPA:

kubectl -n $install_namespace describe scaledobject <scaledobject-name>
kubectl -n $install_namespace get scaledobject <scaledobject-name> -o yaml
kubectl -n $install_namespace describe hpa <hpa-name>

Use the status carefully:

• Ready=True means KEDA accepted the trigger configuration.
• Active=True means at least one trigger is currently requesting scaling activity.
• Active=False can be normal when there is no traffic or queued work.
• An HPA metric value of <unknown> indicates that Kubernetes cannot obtain a current value for that metric.

During a controlled workload test, watch the workload, HPA, and node capacity together:

kubectl -n $install_namespace get deployment,hpa,pod --watch

Confirm that requested replicas become Running. A rising HPA desired count with pods remaining Pending is a capacity or scheduling problem, not a KEDA trigger problem.

Understand the generated behavior

The Vantage workload charts supply workload-specific settings for:

• Trigger thresholds and Prometheus queries.
• Minimum and maximum replica boundaries.
• Scale-up rate and stabilization behavior.
• Scale-down rate and stabilization behavior.

The configuration is designed to scale up quickly when backlog appears and to scale down cautiously through a stabilization window: a cool-down that keeps replicas running until the queue has stayed quiet for a set period. Model and training workers use a longer cool-down than lightweight API services because removing their replicas discards cached models, and reloading those models makes the next task noticeably slower. Treat the window durations in your generated resources as reference values for that chart version, not tuning targets.

Exact values are product configuration, not a universal capacity recommendation. They can change with Vantage chart versions as workload characteristics and validation data evolve.

Do not copy ScaledObject manifests or thresholds from another environment, and do not edit generated resources directly. ArgoCD can restore the chart-rendered configuration during reconciliation or upgrade. Use only supported Vantage chart values; contact your ABBYY account team before applying additional autoscaling overrides.

Plan cluster capacity

Autoscaling cannot compensate for an undersized or constrained cluster. Plan for:

• The maximum document arrival rate and acceptable queue wait time.
• Average and high-percentile processing duration for representative documents and skills.
• Pod startup and model warm-up time.
• CPU and memory requests for every concurrently scaling workload.
• Node-autoscaler provisioning time and maximum node-pool size.
• Namespace quotas, cluster limits, and registry pull throughput.
• TechCore node labels, taints, accelerators, and training-worker isolation.
• Failure tolerance when a node or availability zone is unavailable.

Measure queue depth, processing latency, pod startup time, HPA desired replicas, and pending pods during a representative load test. Use those observations when discussing environment-specific tuning with ABBYY.

Troubleshooting

Symptom	Likely area to inspect
No ScaledObject resources appear	Confirm vantage.keda.enabled: true and inspect the generated ArgoCD applications.
Ready=False or ScaledObjectCheckFailed	Confirm KEDA 2.17.3 and inspect the ScaledObject events for an unsupported trigger field.
HPA metrics show <unknown>	Check the Prometheus service, ServiceMonitor, scrape targets, mesh policy, and query results.
HPA requests more replicas but pods remain Pending	Check node capacity, quotas, resource requests, TechCore labels, taints, and persistent-volume scheduling.
A workload scales when its queue appears quiet	Inspect all triggers; another Prometheus or CPU trigger may be active.
Scaling behavior changes after an upgrade	Compare the generated ScaledObject with the versioned chart configuration and confirm both Vantage charts use the same version.

For detailed corrective steps, see Troubleshooting KEDA and Prometheus.

What’s next

Monitoring with Prometheus

Configure scraping and verify the metrics used by KEDA.

Prerequisites

Review KEDA, Prometheus, and cluster-capacity requirements.

Troubleshooting

Diagnose unhealthy ScaledObjects and unavailable metrics.

Compatibility

Check supported versions before upgrading KEDA or Vantage.