A Dubai demonstration becomes a capacity question
BroadNet’s account of Seamless Middle East in Dubai places its conversational AI demonstration on September 22–24, 2026. The relevant development is a regional supplier presenting conversational customer engagement, not evidence of a newly released on-premises inference product. For operators serving GCC businesses, the infrastructure question is whether engagement bursts can become predictable, isolated workloads on locally controlled hardware.
The supplied BroadNet page is the sole primary-source reference for this article. Its event account has not been independently corroborated here by two additional sources. It does not establish deployment architecture, model size, accelerator requirements or measured throughput. Treat the demonstration as a reason to investigate demand, not as a capacity specification or a verified local-hosting announcement.
That distinction matters for an operator with a quarter rack to a few racks. A successful demonstration can run one conversation at a time. A production service must absorb simultaneous arrivals, preserve tenant boundaries and remain affordable when demand falls. The bottleneck is often queueing and memory headroom rather than whether a model can generate an acceptable answer.
Size conversations, not a chatbot instance
Start with a service definition. A completed customer interaction might include several user turns, retrieval requests, model generations and a handoff to a person. Counting HTTP requests alone hides that work. Define completion explicitly: resolved, transferred, abandoned or failed. Keep those outcomes separate in both operational reporting and billing.
Collect representative input lengths, output limits, conversation depth and arrival patterns. Include Arabic and English, code-switching and the formats actually present in customer records. Different tokenisation and context lengths can change memory demand and execution time. Do not infer bilingual capacity from an English-only prompt set.
Concurrency also needs two measures: open conversations and active generations. Thousands of sessions waiting for a customer reply need not consume accelerator compute continuously. Their retained state may still consume memory or storage. Conversely, a few hundred customers replying together can create an inference burst even when average daily traffic looks modest.
Separate prefill, generation, retrieval and application latency. Otherwise, a slow vector lookup can be mistaken for insufficient GPU capacity, while a saturated decode stage can be hidden behind an acceptable daily average.
Benchmark the burst before buying capacity
Build a replay workload from approved, sanitised examples or synthetic conversations with equivalent length distributions. Test steady traffic, sudden bursts and sustained overload. Reproduce realistic pauses between turns, but also run a synchronised-reply scenario. That exposes the shared queue pressure a campaign or service incident can create.
Measure time-to-first-token from the customer-facing gateway, not merely from model admission. Record queue wait separately. Track p50, p95 and p99 latency, completion rate, cancellations, retries and output tokens per second. A fast first token is not enough if subsequent generation stalls or the application requires a complete response before displaying anything.
Use a compact acceptance sheet. The following values are illustrative test objectives, not BroadNet results or Clastiq Orchestrator benchmarks.
| Measure | Illustrative objective | Operator response to failure |
|---|---|---|
| Gateway p95 first token | At most 2 seconds | Inspect queue and prefill time |
| Completed interactions | At least 98% | Separate overload from application faults |
| Tenant waiting queue | At most 20 requests | Reject or defer excess work |
| Cross-tenant interference | Within agreed latency budget | Change placement or reservations |
Repeat tests with one tenant producing maximum permitted context lengths. Equal request counts do not imply equal resource consumption.
Make admission control tenant-aware
A shared endpoint without tenant-specific admission control lets one campaign consume everyone’s latency budget. Put authenticated tenant identity into the request path before scheduling work. Maintain separate limits for concurrent generations, queued requests, input tokens, output tokens and request rate. Establish a maximum queue residence time as well as a queue length.
When the limit is reached, make the outcome deliberate: return a retryable response, offer a human handoff or defer an asynchronous job. Do not silently accumulate requests until clients time out and retry. That creates duplicate work precisely when capacity is scarce.
A short policy sketch helps application and infrastructure teams agree on the contract:
# Illustrative admission policy, not a Clastiq API schema
tenant: retailer-a
max_active_generations: 8
max_queued_requests: 20
max_queue_wait_ms: 1500
max_input_tokens: 4096
max_output_tokens: 512
external_fallback: denied
These limits belong at the serving gateway or scheduler. Kubernetes resource quotas alone do not understand conversation queues. An infrastructure platform supplies placement and tenancy controls; the inference stack must enforce request-level fairness.
Partition only after measuring memory pressure
On a small estate, partitioning can prevent a lightly loaded tenant from reserving an entire accelerator. Where hardware supports isolated device partitions, test the model within the partition’s actual memory and compute allocation. Elsewhere, time-sharing or process-level sharing may improve occupancy but carries different isolation and scheduling properties.
Model weights are only part of the memory footprint. Runtime buffers, attention caches and concurrent contexts determine whether the service stays within its allocation. A configuration that fits during startup can fail under long conversations. Test near the intended context and concurrency limits, not only with short prompts.
Reserve capacity for infrastructure work and recovery. If every accelerator is committed at peak load, draining a node or replacing a failed device immediately becomes a customer incident. Specify degraded-mode behaviour before selling the service: lower concurrency, a smaller approved model or tenant-specific load shedding. External fallback is another deployment path, not a substitute for an agreed recovery policy.
Worked example: capacity is not the same as margin
Consider an illustrative estate of eight accelerators operated for a 30-day month. It provides 8 × 24 × 30 = 5,760 installed GPU-hours. Suppose two accelerators remain reserved for recovery and operational headroom, leaving six in the serving pool. At an assumed 60% average utilisation of that pool, occupied capacity is 6 × 720 × 0.60 = 2,592 GPU-hours.
For this planning example, utilisation means occupied equivalent accelerator time, not a vendor-specific compute counter. Across all eight installed devices, that is 2,592 ÷ 5,760 = 45%. Reporting only the serving pool’s 60% would conceal the cost of reserve capacity.
Assume an all-in infrastructure cost of $1.50 per installed GPU-hour, including allocated host, storage, power and facility costs. Monthly infrastructure cost is therefore 5,760 × $1.50 = $8,640. If the service completes 1,200,000 interactions, infrastructure cost per completed interaction is $8,640 ÷ 1,200,000 = $0.0072. This excludes application support, taxes and commercial margin.
If completions fall to 600,000 while provisioned cost remains unchanged, the figure doubles to $0.0144. Multiplying only occupied GPU-hours by $1.50 would understate the bill: idle and reserved hardware still costs money. Replace every assumption with measured workload results and the operator’s actual cost base before setting prices.
Meter work and outcomes separately
Chargeback needs a defensible allocation rule. In the example, suppose a tenant accounts for 25% of weighted serving consumption. Allocating the entire $8,640 infrastructure pool on that basis assigns $2,160 to the tenant. If it completes 300,000 interactions, its allocated infrastructure cost is again $0.0072 per completion.
That proportional method is only fair if the weighting reflects meaningful cost drivers. Long contexts, reserved partitions and high availability commitments can make two tenants with equal completion counts materially different. A practical tariff can combine a reservation charge with measured consumption, while reporting cost per completed interaction as an efficiency metric.
Instrument the gateway, serving engine and infrastructure layer with a common tenant identifier. Retain counts for admitted, rejected, cancelled and completed work. Record retries and external calls so an apparent improvement in local GPU efficiency cannot hide spend elsewhere. Avoid putting raw prompts into billing records; usage metadata should not become a second customer-data repository.
Compare local execution with external calls
Run the same approved conversation set through a local model and an external model service. Compare answer quality, end-to-end latency, completion rate and total cost under identical output limits. Document model differences: a cheaper local response is not an equivalent substitute if it sends more customers to human agents.
For external calls, include input and output token charges, retrieval costs, network dependencies and contractual limits. For local execution, include reserve capacity, model operations, storage, power and the staff time required to keep the service available. Measure burst behaviour on both paths; neither local ownership nor an external API guarantees acceptable latency.
Use completed business outcomes as the comparison denominator, with quality thresholds established before testing. If external inference is permitted only for particular data classes, measure that restricted routing policy rather than an unrestricted configuration. A hybrid design must be evaluated as deployed, including redaction, routing latency and any repeated local work before escalation.
Sovereignty requires a map of every data path
GCC relevance is more specific than proximity to Dubai. An operator in Oman serving customers across the region must identify where each tenant’s data is processed, stored, observed and backed up. Local compute does not automatically satisfy sector rules or cross-border transfer requirements. Those require tenant-specific legal and contractual review.
Map the customer channel, messaging provider, gateway, authentication service, retrieval store, model endpoint, telemetry system and support workflow. Mark external boundaries and record which fields cross them. A locally hosted model can still receive data through an overseas channel, send traces to an external monitoring service or expose records through remote support access.
Give routing policies explicit defaults. If a tenant prohibits external processing, overload must not trigger an external call. Enforce that requirement through network egress controls as well as application configuration. Test denied routes and document the customer-visible behaviour. Sovereignty is an operational property of the complete service, not the physical location of one GPU.
Keep storage and power inside the capacity plan
Ceph can provide shared storage for model artefacts, tenant volumes and supporting services, but those workloads have different access patterns. Separate latency-sensitive retrieval from bulk model distribution and backup activity. Test simultaneous model loading after a restart; a storage bottleneck can extend recovery even when spare compute is available.
Capacity planning must account for replication or erasure-coding overhead, recovery traffic and free-space thresholds. A three-copy replicated pool offers roughly one-third of raw capacity before other overhead and operational headroom. Establish retention and deletion rules for conversations, embeddings, logs and backups, with access scoped by tenant and purpose.
Measure whole-estate power rather than relying on accelerator nameplate figures. At an illustrative average IT draw of 5 kW, a 720-hour month uses 3,600 kWh. With a measured facility PUE of 1.4, attributed facility energy becomes 5,040 kWh. At $0.10 per kWh, that is $504. If the all-in GPU-hour rate already includes electricity, do not add it again. Check cooling limits and peak circuit loading alongside monthly energy.
Where Clastiq Orchestrator fits
Clastiq Orchestrator addresses the infrastructure control layer on an operator’s own hardware: bare-metal provisioning, Kubernetes and VM tenancy, GPU partitioning, Ceph storage, per-tenant metering and chargeback, and quota and policy engineering. For this investigation, its role is to make tested resource allocations repeatable and attributable across a small estate.
Keep the boundary clear. An orchestrator does not establish model quality or independently guarantee first-token latency. Operators still need an inference runtime, application-level admission control and outcome instrumentation. Infrastructure quotas should align with serving limits, while metering ties reservations and consumption back to the tenant responsible for them.
Air-gap deployments require another preparation track: approved model artefacts, container images, dependencies and update procedures must be available inside the boundary. Clastiq’s air-gap deployment scope is relevant here, but external fallback would be incompatible with a genuinely disconnected service. Cognition-AI, based in Muscat, Oman, also provides human-led local support. Define escalation paths and operating responsibilities rather than assuming an unprovided support SLA.
What to do this week
- Create an evidence ledger. Record BroadNet’s September 22–24, 2026 demonstration as a single-source regional signal. Keep deployment claims unverified until supported by technical documentation.
- Build a burst replay. Include Arabic, English, long contexts, simultaneous replies and cancellations. Capture gateway first-token latency, queue wait and completed outcomes.
- Set tenant limits. Agree active-generation, queue, token and timeout budgets. Test an overloaded tenant beside a normally loaded one.
- Price the installed estate. Include idle capacity, recovery reserves, storage and facility costs. Compare local and external routes using quality-qualified completed interactions.
- Audit the boundary. Trace channels, retrieval, telemetry, backups and support access. Verify that prohibited external routes remain blocked during overload and failure.
Clastiq Orchestrator runs this infrastructure on the operator’s own hardware: request a demo.