Decentralized Intelligence Part 2: The Architecture Behind the Approach
This post is Part 2 of a four-part blog series, Decentralized Intelligence. Part 1 established the problem: the most valuable data in regulated industries can't be centralized, and the solution is a federated approach. Here we get into the architecture, and what it makes possible when organizations start building intelligence networks with their peers.
Federated computing decentralizes the entire AI execution lifecycle — analytics, training, inference, agents, optimization, and workflows — while keeping data under local control.
The easiest way to understand how a federated approach works is to see how it can answer a question that couldn't be answered before.
Let’s take the example of a plant operator who manages three factories in a network. For two years, they have wanted to know: how does raw-material batch variation correlate with downstream defect rates across all three plants?
It sounds like an easy reporting question, but it isn't. Batch records and defect logs sit inside each plant's control boundary. The information may be proprietary, governed by different teams, stored in different systems, and represented using different schemas. Copying everything into a centralized data lake may be undesirable, prohibited, or simply impractical.
As a result, the organization has been sitting on years of potential insight because the data-access problem was never fully solved.
Now, she uses a federated computing platform to find the answer.
Before we walk through what happens for this particular query, it's worth understanding what's actually running underneath it.
What Federated Computing Actually Is
Most organizations have encountered parts of this architecture before, including edge computing, federated learning, data mesh, distributed analytics, privacy-enhancing technologies, and AI agents.
Federated computing brings these concepts together into a broader operating architecture. In a federated system, computation, analytics, models, and agents operate across distributed environments while the underlying data remains locally governed.
Federated learning is one application of this architecture. It asks: Can multiple parties train a model together without centralizing their training data?
Federated computing asks a broader question: Can organizations run useful intelligence across distributed data, systems, and organizational boundaries without requiring the underlying data to be centralized?
That intelligence may involve model training, but it can also include inference, analytics, simulations, optimization, planning, data transformation, or agentic workflows.
Taken together, this is the foundation for decentralized intelligence — a network of distributed data and compute resources that can behave like a coordinated intelligence system without becoming one centralized data and compute system.
What’s Running Under the Hood
The federated computing architecture rests on three layers.
1. Federated Data Mesh
Federated data mesh is where the data lives. The old model treated data as something to copy: move it, replicate it, transform it, and spend years governing the copies. The new model treats data as something to call. Each data asset has an owner, access controls, and a defined set of permitted operations. Approved computation runs near the data. The query goes to the data. The data stays put. This architectural foundation is what makes multi-party collaboration possible at scale.
This is related to the concept of a data mesh, but extends it further. A conventional data mesh decentralizes ownership of data products. A federated data layer also enables decentralized execution against those data products, allowing intelligence to operate across organizational boundaries without requiring the underlying data to be consolidated. That distinction becomes particularly important when collaboration spans not just departments and business units, but different companies, hospitals, research institutions, pharma, factories, or other independent organizations.
The result is the foundation for a federated intelligence network where participants can contribute data and computational capability while retaining control over their underlying assets.
2. Decentralized and Autonomous Compute
Decentralized and autonomous compute is how the work gets done. For decades, someone had to configure the infrastructure by picking the instance type, setting up the cluster, and deciding whether a workload belongs on cloud, edge, or local hardware.
In a federated architecture, that changes. A user or agent states a goal, such as minimize cost, prioritize privacy, keep execution within a geographic boundary, or maximize accuracy. The platform determines the execution path. But, autonomous doesn't mean uncontrolled. Every workload still needs an authorized identity, an approved execution environment, policy boundaries, logging, and an audit trail. In sensitive environments, execution may also be tied to stronger verification mechanisms such as cryptographic workload identity or attestation before protected resources or encryption keys become available.
The goal is therefore not merely autonomous computation. It is policy-aware decentralized compute in which infrastructure complexity can disappear from the user experience while governance remains explicit and enforceable.
3. Intelligence Orchestration
Intelligence orchestration is the coordinating layer. A federated AI system may involve large reasoning models, smaller specialist models, deterministic tools, analytical code, retrieval systems, databases, agents, and human approvals all operating across different environments.
The orchestrator determines not only what should execute next, but also what is permitted to run, where it’s permitted to run, which data it can run against, on whose authority it operates, which models or tools it can use, and what information is allowed to leave the execution environment.
Routing a request between a large model and a specialist model is useful. But, in a federated environment, orchestration also has to understand security scopes, organizational boundaries, data permissions, execution policies, and disclosure controls.
The orchestration layer is effectively the nervous system of the federated network. Orchestration, policy, and audit are coordinated from one place, which is what makes the system governable. Data, compute, and execution stay distributed and locally owned.
Grounding Federated Computing in Reality
Now that you know what's under the hood, here's what actually happens when the plant operator runs their query.
1. An orchestrator interprets the intent. The orchestrator identifies the relevant domains: plants, raw-material batches, suppliers, process information, and defect data. The user does not need to know which database contains each field or which system each plant uses.
2. The orchestrator decomposes the question into executable subtasks: The system determines that it needs to:
- Establish the plant scope
- Identify the appropriate data assets
- Reconcile data definitions
- Execute the statistical analysis
- Validate the outputs
- Combine permitted results
The original question becomes a workflow.
3. The orchestrator discovers the relevant datasets. The orchestrator queries a federated catalog spanning the three plants.
Plant one may call an identifier batch_id.
Plant three may call the equivalent field lot_number.
The system can understand that relationship because the mapping was established when those data products were onboarded to the network — potentially proposed through AI-assisted harmonization, reviewed by a person responsible for the data, approved, and then retained for reuse.
The important point is that the network does not have to rediscover the same semantics for every question. Approved schemas, mappings, metadata, and policies become reusable infrastructure.
4. The security and policy scope determines the execution pattern. The orchestrator recognizes what datasets and tools the user and agent can access based on access control policies. It therefore selects an execution pattern in which the analysis runs locally within each plant's permitted compute environment.
5. Approved computation executes locally. At each plant, an approved workload operates against the relevant batch, supplier, process, and defect records. Intermediate row-level information remains within the local environment. Only outputs permitted by that plant's policies are eligible to leave.
6. Permitted results and combined and validated. The system receives the approved outputs from the participating environments and produces a network-level answer that includes correlations across the three plants, confidence measures, caveats, and supporting evidence. The result can then become input into the next analytical or agentic step.
Perhaps the operator asks, “Which suppliers appear most strongly associated with the variation?” or “Would adjusting a specific process parameter reduce the defect rate?” The same federated infrastructure can support the next iteration.
7. Maintain the audit of all executions. The platform maintains a record of how the result was produced, which incorporates:
- Which identity initiated the request
- Which data assets participated
- Which code, models, agents, and tools executed
- Where execution occurred
- Which policies were applied
- What information was returned
- Who approved actions that required human authorization
One question. Three plants. No data movement. That's decentralized intelligence.
Trust is the Foundation, Not a Layer You Add Later
Federated computing is explicitly designed to operate across security and organizational boundaries. That makes trust foundational.
Several controls are particularly important.
Identity and authorization. Every operation must be associated with an identity and an authority. The network has to understand who is making a request, which organization they represent, which data they are allowed to access, and which operations they are permitted to execute.
Data protection. Raw data remains within its governed environment, while any information leaving that environment is subject to the disclosure controls established by the data owner and the network.
Execution verification. Data owners need to know what is going to execute against their infrastructure and what is actually executed. In environments using confidential computing or other trusted execution mechanisms, cryptographic attestation can further verify the identity and state of an execution environment before access to protected data or keys is granted.
Model and agent governance. Models, agents, tools, and code should be versioned and tied to provenance. A result should be reproducible, and a problematic model or agent should be revocable rather than quietly continuing to operate throughout the network.
Human accountability. Sensitive workflows may require named approvers and explicit participation. Access must also be revocable. If a participant withdraws or a permission changes, that decision should be enforced at execution time.
Every layer participates in establishing trust.
The complexity can be hidden. The accountability cannot.
The Value of Intelligence Networks
Over time, the system accumulates shared operational intelligence in the form of mappings, metadata, policies, trusted execution paths, models, agents, and reusable workflows. That is where the architecture begins to create compounding value. Adding another question, model, or workflow becomes easier. Eventually, adding another participant to the network becomes more valuable because they gain access not just to one application, but to an increasingly capable ecosystem of intelligence.
It becomes a federated intelligence network. The boats don't leave port, and the intelligence reaches them anyway.
This architecture is already running in production. If your organization is sitting on data it can't use, Rhino Federated Intelligence is built for exactly that problem. Reach out to our team to schedule a demo today.
In our next installment, we’ll explore federated intelligence networks further—how they work, and how they create real, compounding value for the organizations building and participating in them.
This post is Part 2 of a four-part blog series, Decentralized Intelligence, publishing every two weeks.
