CUDA 13.4 memory limits change GPU sharing decisions

CUDA 13.4 puts memory enforcement into the sharing decision

NVIDIA's October 27, 2026 technical announcement, as identified in the supplied reference, describes memory-enforced GPU sharing in Kubernetes with CUDA 13.4 and the GPU Fractioning Operator. The operational change is the prospect of sharing GPU execution while enforcing memory budgets, rather than merely advertising additional schedulable GPU shares. That creates another option between conventional time-slicing and hardware partitioning.

The distinction matters because a Kubernetes allocation is not automatically a GPU memory boundary. Conventional time-slicing can place several workloads on one device without giving each workload a protected portion of its memory. One allocation-heavy job can therefore disrupt its neighbours even when the scheduler considers every pod correctly placed.

Memory enforcement addresses that specific risk. It does not, by itself, establish dedicated compute, predictable tail latency or hardware-level fault isolation. Operators deciding between time-slicing, NVIDIA Multi-Process Service, or MPS, and Multi-Instance GPU, or MIG, must separate those requirements before changing their deployment policy.

For rFactorial, owned by Muscat-based Cognition-AI, the connection is private-cloud resource management. Clastiq Orchestrator covers compute, GPU, storage and networks; Clastiq Lite brings self-service AI and GPU sharing to teams. Compatibility with CUDA 13.4 or the GPU Fractioning Operator is not established by the supplied product context and must be verified separately.

Four mechanisms, four different contracts

Start with the resource contract offered to the application, not the number of GPU resources displayed by Kubernetes. Memory capacity, execution opportunity, performance interference and fault containment are separate dimensions.

Conventional time-slicing is attractive when workloads leave execution gaps and occasional slowdown is acceptable. Replicated GPU resources are scheduling opportunities, not additional physical capacity. Increasing the replica count does not create more memory or compute, and requesting multiple advertised shares should not be interpreted as a proportional performance entitlement.

The new memory-enforced path changes the memory contract. Its operational value depends on the supported hardware and software matrix, the enforcement scope, and the behaviour when a workload reaches its budget. Confirm those details against NVIDIA's announcement and deployment documentation rather than inferring them from the phrase “GPU fractioning.”

MPS is different from simply rotating access. It can permit concurrent execution from multiple processes and provides resource controls whose availability and semantics depend on the implementation. Compute limits should not be read as guaranteed throughput. Kubernetes support restrictions also need checking independently of what standalone MPS can do.

MIG creates hardware-defined instances on supported GPUs, with dedicated resources according to the selected profile. That provides a stronger boundary than ordinary software sharing, but profile sizes constrain placement. It can leave unusable fragments of capacity when application requirements do not match available instances.

When a memory limit changes the answer

Memory-enforced sharing changes the MIG-versus-MPS decision when memory exhaustion was the principal reason to avoid ordinary time-slicing, while variable execution speed remains acceptable. Examples include internal experimentation, bounded batch inference and development jobs with known peak allocations.

It changes less when the requirement is a dependable response-time envelope. A workload that stays within its memory budget can still submit long-running kernels or compete for shared execution and memory-system resources. Protecting allocation capacity does not establish a maximum queueing delay for another workload.

Ask three questions in order. Must each job be prevented from allocating beyond a defined budget? Must each tenant receive a stronger hardware resource boundary? Must the service satisfy a latency objective under concurrent demand? Different answers can justify different GPU pools rather than one organisation-wide sharing setting.

For trusted internal workloads, memory-enforced time-sharing may remove enough operational risk to avoid partitioning every device. For compatible applications whose kernels benefit from overlap, MPS remains worth testing. For stronger separation requirements, MIG remains relevant even if software memory limits become reliable.

Neither MIG nor a memory cap should be treated as a complete security architecture. Host access, container privileges, driver exposure, identity, network policy and data handling still require controls. Distinguish protection against accidental resource exhaustion from protection against a malicious tenant.

Kubernetes GPU sharing with hard limits needs precise language

A standard Kubernetes GPU resource request normally selects an advertised device resource. It does not automatically translate into a requested number of gigabytes of GPU memory. CPU and host-memory limits likewise do not cap device-memory allocations.

For the new sharing path, establish exactly where the memory budget is declared, which component applies it and what happens if the enforcement component fails. Check whether the limit applies to a pod, container, process or another allocation boundary. Multi-process jobs make that distinction operationally important.

Also verify how the supported configuration treats allocation pools, framework caching, child processes and memory allocated through different APIs. Do not assume that a successful test with one framework proves enforcement for every CUDA application. Record the tested allocation paths alongside the supported runtime versions.

Use discovery before copying configuration examples:

# Inventory GPU devices and the node driver.
nvidia-smi -L
nvidia-smi

# Inspect resources actually advertised to Kubernetes.
kubectl get nodes -o custom-columns='NODE:.metadata.name,ALLOCATABLE:.status.allocatable'

# Discover installed APIs before applying operator-specific manifests.
kubectl api-resources | grep -Ei 'gpu|fraction|nvidia'

These commands are inventory checks, not a configuration recipe for the GPU Fractioning Operator. They do not prove memory enforcement, runtime compatibility or application isolation. Use the exact manifests and resource names documented for the installed release, and validate them in a non-production pool.

Test exhaustion, contention and latency separately

A useful qualification suite needs three independent experiments. Combining them into one benchmark makes it difficult to identify whether failure comes from memory pressure, execution contention or application behaviour.

First, test memory exhaustion. Run a protected workload at steady load while a neighbouring test job allocates memory in controlled increments beyond its assigned budget. Observe the allocation failure, application exit, restart behaviour and effect on the protected workload. Repeat with the frameworks and allocation paths your teams actually use.

The acceptance criterion is not simply that the aggressive job fails. Confirm that it fails at the intended boundary, that its neighbour remains functional, and that memory becomes available after cleanup. Test repeated failures to expose leaked resources, restart storms or delayed recovery.

Second, test competing workloads. Pair a compute-heavy job with a latency-sensitive service, then repeat with a memory-bandwidth-heavy neighbour. Measure both workloads. A sharing policy can improve aggregate throughput while making one service operationally unusable; a single device-utilisation percentage will hide that result.

Third, test latency under realistic arrivals. Capture p50, p95 and p99 latency, throughput and error rate during steady traffic and bursts. Include model loading, warm-up and concurrent starts. Compare each sharing mode with a dedicated-device baseline using the same application versions, inputs and measurement window.

Predeclare acceptance thresholds. A batch pool might tolerate a longer completion time if more jobs finish per device-hour. An interactive pool might reject any configuration that breaches its p99 objective. Memory enforcement passes qualification only for the specific workload class and service contract tested.

An illustrative capacity and cost calculation

Consider a hypothetical private-cloud batch pool with four GPUs, each exposing 80 GiB of usable device memory for this illustration. Each GPU has an illustrative allocated infrastructure cost of $3 per provisioned hour. These are planning assumptions, not NVIDIA or Clastiq benchmarks.

There are 12 identical jobs. Each job runs for one hour on a dedicated GPU and reaches a measured peak of 18 GiB. The operator sets an illustrative 22 GiB memory budget per job to allow headroom. Three jobs require 66 GiB per device, leaving 14 GiB outside those budgets. Actual usable capacity and required reserves must be measured.

With exclusive allocation, four jobs run at once and the batch requires three waves, or three hours. That consumes 12 provisioned GPU-hours during the batch window, with an allocated cost of $36.

Suppose qualification shows that three memory-bounded jobs sharing each GPU all finish in 1.5 hours, without errors. All 12 jobs can then complete in one wave across four GPUs. The batch consumes six provisioned GPU-hours during its active window, with an allocated cost of $18. Makespan falls by 50%, while individual job duration rises by 50%.

That distinction is the decision. Aggregate throughput increases from four to eight completed jobs per hour, but each user waits longer after execution starts. The arrangement fits a batch deadline better than an application requiring dedicated-speed completion.

Now add the ownership reality: on-premises hardware does not stop costing money when the batch ends. Over a fixed 24-hour day, four GPUs at the same accounting rate still represent $288. The $18 difference is released capacity at an internal rate, not necessarily a cash saving. Financial value requires additional useful work, avoided expansion or measurable reductions in variable costs.

Measure useful work rather than apparent occupancy

The right economic question is not just whether GPU utilisation rises. It is whether more accepted work completes within its service objectives per owned GPU-hour. Retries, failed allocations and missed deadlines can make a busy device economically worse than a quieter one.

Track queue time, completion time, successful jobs, GPU-memory peaks, allocation failures and latency by workload class. Separate time waiting for placement from time executing after placement. Otherwise a policy that reduces queueing but slows execution can appear either beneficial or harmful depending on the dashboard selected.

For inference, compare cost per accepted request or useful output unit at a defined latency target. For batch workloads, compare completed jobs and deadline attainment. Keep model, context length and concurrency explicit; changing those variables can overwhelm gains from a different sharing mechanism.

The Clastiq guide on DeepSeek context costs provides a related reading path for context-cost planning. Use the guides available through Clastiq for deeper infrastructure planning rather than treating this decision framework as a replacement for release-specific configuration documentation.

Place orchestration around a verified resource contract

Clastiq Orchestrator's verified role is private-cloud orchestration across compute, GPU, storage and networks. Clastiq Lite's verified role is self-service AI and GPU sharing for teams. Those capabilities connect the sharing decision to infrastructure delivery, but do not establish support for any particular NVIDIA operator, enforcement interface or GPU model.

A practical deployment workflow begins with an inventory: GPU models, memory capacity, drivers, runtimes, Kubernetes versions and existing sharing configuration. Next, classify workloads by trust, peak memory, concurrency, latency objective and failure tolerance. Only then choose candidate sharing modes and run the qualification suite.

Design separate service classes around the results. One class might offer shared development capacity with variable performance; another might provide memory-bounded batch execution; a third might require hardware partitioning or dedicated devices. These are operating-model recommendations, not claims about built-in Clastiq templates or controls.

For self-service, users need a visible contract: allocation size, permitted runtime, expected contention, timeout policy and escalation route. Platform teams need ownership records, approved images, change control and capacity accounting. Verify which requirements are implemented by the orchestration product and which depend on Kubernetes, NVIDIA components or organisational processes.

Before exposing the new memory-enforced path through any platform, obtain a supported compatibility statement and test upgrades and rollback. Do not advertise a “hard GPU limit” unless its scope and failure behaviour are documented. Product-specific questions can be directed to info@cognitionai.cloud.

Keep upgrades and placement reversible

Treat the CUDA 13.4 sharing path as a separate qualification target, not an automatic upgrade to an existing time-slicing configuration. Driver, operator and runtime changes can affect scheduling, observability and recovery as well as application execution.

Use a small canary pool with explicit placement rules. Preserve a known-good pool for workloads that fail qualification. Test node drain, pod termination, enforcement-component restart and recovery after a deliberately failed allocation before expanding access.

Changing the advertised resource shape can strand pending workloads or alter packing efficiency. Record the existing configuration, admission policies and resource names before rollout. Define rollback success in application terms: workloads resume, memory is reclaimed and latency returns to the accepted baseline.

The final choice is therefore conditional. Memory limits can make shared execution acceptable where exhaustion was the blocker. They cannot substitute for evidence about compute contention, hardware boundaries or service quality.

What to do this week

1. Inventory one candidate GPU pool. Record hardware, drivers, runtimes, advertised Kubernetes resources and supported NVIDIA configurations.
2. Define three workload contracts. Separate development, batch and latency-sensitive requirements, including memory budgets and acceptable failure behaviour.
3. Run the three-part qualification suite. Test memory exhaustion, competing workloads and latency against a dedicated-device baseline.
4. Calculate capacity value honestly. Report accepted work per GPU-hour and distinguish released capacity from reductions in cash expenditure.
5. Verify support before self-service rollout. Confirm the relevant product and NVIDIA compatibility, then document canary placement, rollback and user-facing limits.

Explore Clastiq Orchestrator for private-cloud orchestration across compute, GPU, storage and networks, and the rFactorial product family for the broader portfolio owned by Cognition-AI. Keep the orchestration decision tied to tested infrastructure behaviour rather than assumptions about the underlying sharing mechanism.

Sources

• NVIDIA technical announcement: memory-enforced GPU sharing with CUDA 13.4
• NVIDIA GPU Operator documentation: GPU sharing
• Clastiq Insights: DeepSeek context costs
• Clastiq product information and guides