Workload Configuration

Add Pod Annotations

Add the following annotations to your Pod metadata to configure GPU workload requirements.

Note: If multiple containers in a Pod need GPUs, provide per-container GPU counts. Use tensor-fusion.ai/container-gpu-count to specify GPU count per container. tensor-fusion.ai/container-gpus is written back by the system to record the GPU IDs per container and generally should not be set manually.

Enable TensorFusion via Label

tensor-fusion.ai/enabled is a label (not an annotation) used to explicitly enable or disable TensorFusion. It is typically set under metadata.labels:

metadata:
  labels:
    tensor-fusion.ai/enabled: "true"

Relationship with tensor-fusion.ai/enabled-replicas: enabled turns TensorFusion on/off at the workload level; enabled-replicas controls how many replicas are enabled for a canary rollout after it’s on. When enabled is false, enabled-replicas has no effect.

Annotation Reference

Basic Annotations

Advanced Annotations

Example Config

kind: Deployment
apiVersion: apps/v1
metadata: {}
spec:
  template:
    metadata:
      labels:
        tensor-fusion.ai/enabled: "true"
      annotations:
        tensor-fusion.ai/inject-container: python # could be comma split if multiple containers using GPU
        tensor-fusion.ai/tflops-limit: '20'
        tensor-fusion.ai/tflops-request: '10'
        tensor-fusion.ai/vram-limit: 4Gi
        tensor-fusion.ai/vram-request: 4Gi
        tensor-fusion.ai/qos: medium
        tensor-fusion.ai/workload-profile: default-profile # WorkloadProfile has lower priority as template
        tensor-fusion.ai/is-local-gpu: 'true'
        tensor-fusion.ai/gpu-count: '1' # GPU device number per TensorFusion Worker
    spec: {}

Configure WorkloadProfile Custom Resource

For advanced features like auto-scaling, create a WorkloadProfile custom resource and reference it in your Pod annotations.

apiVersion: tensor-fusion.ai/v1
kind: WorkloadProfile
metadata:
  name: example-workload-profile
  namespace: same-namespace-as-your-workload
spec:
  # Specify AI computing resources needed
  resources:
    requests:
      tflops: "5"
      vram: "3Gi"
    limits:
      tflops: "15"
      vram: "3Gi"
  # Specify the number of vGPU workers, usually the same as Deployment replicas
  replicas: 1

  # Schedule the workload to the same GPU server that runs GPU worker for best performance
  isLocalGPU: true

  # Specify pool name (optional)
  poolName: default-pool

  # Specify QoS level (defaults to medium)
  qos: medium

  # Specify the number of GPU devices per vGPU worker (optional, default to 1)
  gpuCount: 1

  # Specify the GPU/NPU model (optional)
  gpuModel: A100

  # Auto-scaling configuration options (optional)
  autoScalingConfig: {}

Then reference this profile in your Pod annotation:

tensor-fusion.ai/workload-profile: example-workload-profile

For more details on WorkloadProfile schema, see the WorkloadProfile Schema Reference.