Follow the workload from data creation to action
Teams often begin this decision with a preferred platform. The useful starting point is the path taken by the workload. Where is the data created? Where must the result arrive? How much data has to move? What happens during a connection failure? Who can operate and recover the system at each location?
Draw that path for one representative request. It might begin with a camera beside a production line, measurements inside a laboratory, documents in a business system, or a user prompt in an internal application. Mark every transfer, storage point, identity boundary, transformation, model call, approval, and final action. The deployment options become easier to compare once the physical and organizational route is visible.
Latency should be measured across that route. Network travel, serialization, queueing, data loading, inference, application logic, authorization, and result delivery all contribute. A low model runtime does not compensate for a slow or unreliable path around it. The same map reveals data movement that may affect bandwidth, cost, retention, privacy, and governance.
The NIST Fog Computing Conceptual Model describes decentralized computing close to IoT devices and the data they produce. That idea is useful for placement, but proximity alone does not settle the design. The workload still needs capacity planning, integration, monitoring, updates, and a defined failure mode.
Each deployment model places work and responsibility differently
Edge execution is focused computing close to a data source or physical process. It can reduce data movement and keep a time-sensitive function operating during a wider network interruption. Edge hardware usually has tighter limits on power, cooling, memory, storage, accelerator choice, and service access. A fleet of edge devices also creates a fleet-management problem: software distribution, configuration, health checks, credentials, logs, replacement, and version control.
On-premise deployment runs a broader workload inside infrastructure controlled by the client. It may support several applications, shared storage, accelerators, internal identity systems, local data services, and scheduled scientific or business workloads. The organization gains direct control over capacity and change timing. It also accepts responsibility for facilities, hardware lifecycle, operating systems, drivers, runtimes, security controls, backups, monitoring, and recovery unless those duties are contracted explicitly.
Cloud deployment provides managed services and resources that can be provisioned or scaled through a provider. It may suit variable demand, distributed users, managed data services, or teams that do not want to own every infrastructure layer. The design must still account for data transfer, provider regions, service limits, identity, configuration, availability dependencies, egress paths, and the work retained by the customer.
Hybrid deployment assigns different parts of the system to different locations. A camera gateway might filter video locally while a central platform manages models and reporting. Confidential measurements might stay on-premise while approved metadata supports a cloud collaboration service. Hybrid is useful when the split has a clear reason and clear owners. An accidental mixture of platforms can add failure points and leave routine operations divided between teams.
Average demand hides the difficult part of the workload
Size the architecture from a representative workload, including peaks. Record model size, input size, request rate, concurrency, batch behavior, memory use, accelerator use, storage growth, and any preprocessing or post-processing. Interactive requests, continuous video, scheduled training, large document jobs, and laboratory analysis place very different demands on the same nominal hardware.
Concurrency often changes the answer. One request may fit comfortably while several simultaneous users exhaust memory or create a queue that misses the response target. A continuous stream may be predictable but leave little room for maintenance or a second model. A periodic workload may tolerate waiting and benefit from shared resources. Test cold starts, model loading, cache behavior, sustained operation, and the effect of competing workloads.
Scaling also has an operating form. Cloud resources can expand, but applications, quotas, data services, budgets, and downstream systems must be designed for that expansion. On-premise capacity can be predictable, but adding hardware takes procurement, installation, and validation. Edge capacity may require changing many devices. Choose the scaling method that matches likely demand and the organization's ability to manage it.
Compare full operating cost, not a single hardware or service price. Relevant items may include equipment, support, power, cooling, rack space, connectivity, data transfer, software licences, staff time, spare capacity, replacement cycles, and recovery arrangements. The balance depends on workload shape and ownership. Cloud is not automatically cheaper, and local infrastructure is not automatically the economical choice.
Security follows identities, controls, and ownership
A server inside a building is not secure because of its address. The NIST Zero Trust Architecture states that trust should not be granted solely from physical or network location. Local systems still need identity controls, authorization, segmentation, patching, secrets management, logging, incident response, physical protection, and tested recovery.
Cloud use also leaves substantial work with the customer. The AWS shared responsibility model and Microsoft's shared responsibility guidance describe how duties vary by service model. Customers continue to own areas such as their data, identities, access decisions, and configuration. A managed service changes the boundary; it does not remove the need to govern that boundary.
Write an ownership matrix for the chosen architecture. It should identify who manages physical equipment, firmware, operating systems, drivers, containers, model artifacts, application code, data stores, network rules, identities, backups, monitoring, incident response, and vendor relationships. Shared responsibility becomes risky when each party assumes the other will act.
The NIST Cybersecurity Framework 2.0 groups cybersecurity outcomes under Govern, Identify, Protect, Detect, Respond, and Recover. Those functions apply across deployment locations. Placement changes how they are carried out and who performs the work.
Choose the failure behavior before choosing the platform
Connectivity failure is an operating condition, not an edge case. Decide whether the function should continue locally, buffer work, return a visible degraded state, switch to a manual path, or stop safely. Define how long queued data can remain useful, how duplicates are handled after reconnection, and whether the downstream system can accept a replay.
Infrastructure failure needs the same treatment. Consider a lost node, full disk, expired certificate, unavailable identity service, model load failure, damaged edge device, failed accelerator, interrupted update, or incompatible dependency. The system should expose the condition in a place where an owner can see it. Recovery should have a tested route, with known data consequences.
Updates deserve a deployment design of their own. State how software, models, policies, and configurations move between development, test, and production. Define approval, rollback, compatibility checks, and evidence of what version ran. For edge fleets, include devices that are offline during an update. For on-premise accelerators, include driver and runtime compatibility. For managed services, include provider changes and deprecation notices.
The Google Cloud Well-Architected Framework, last reviewed in January 2026, treats operations, security, reliability, cost, and performance as connected design concerns for cloud workloads, including hybrid and multi-cloud deployments and workloads migrated from on-premise systems. That framing is useful because no placement decision remains purely technical after deployment.
A representative test should stress the chosen boundary
Benchmark the real application on candidate infrastructure. Use the intended model, input pipeline, data volumes, concurrency, integrations, identity flow, logging, and storage. Run long enough to observe heat, memory growth, queues, scheduled jobs, network variation, and maintenance activity. Synthetic load is useful for controlled comparison, but confirm the final design with representative data and user behavior.
Measure response time at the user or process boundary, throughput, resource use, queue growth, failure rate, recovery time, data movement, and operating effort. Inspect what happens when demand exceeds capacity. A system that becomes slower in a visible and controlled way creates a different risk from one that drops work or returns partial results without warning.
The test should also include the people who will own the service. Ask them to deploy a version, review a health alert, restore a backup, rotate a credential, investigate a failed request, and recover from an interrupted connection. If these tasks depend on the project team remaining permanently available, the operating model is incomplete.
Local scientific computing shows why governance and placement meet
A GAIAA scientific-computing case involved researchers who needed Python, notebooks, machine learning, and GPU workloads for confidential measurements that had to stay on local infrastructure. The design recorded the user, project, governed inputs, code version, runtime, and resource limits for each request. Isolated workers executed jobs locally.
Results were immutable and validated before publication as separate analysis records. The path connected measurements to results and kept model release behind review. Local placement addressed the stated data boundary, but the dependable part of the design came from job identity, isolation, version records, validation, controlled resources, and ownership.
This case does not establish that on-premise infrastructure is the default for scientific work. It shows how a requirement to keep data local affects the rest of the system. Capacity, scheduling, security, updates, result publication, and recovery still need explicit engineering.
Finish with an architecture decision that someone can operate
The final choice should be recorded in terms that can be tested. Name the data boundary, decision location, timing requirement, workload profile, expected concurrency, connection assumptions, integration points, security controls, failure behavior, update route, recovery method, and owner for each layer. Record the rejected options and the reason they lost.
Revisit the choice when the workload, data classification, service limits, model size, user population, or operating team changes. Deployment location is not a permanent identity. It is a design decision tied to current conditions.
Choose infrastructure only after the workload, data boundary, timing, integration, failure behavior, and ownership are written down. GAIAA can build and test candidate edge, on-premise, cloud, or hybrid paths using representative operation and a support model that can continue after launch.