Are you AI-ready? Part 3 — Architecture & Infrastructure: the demo that quietly becomes your architecture

Third of six articles on the dimensions of the Mentis READY Framework. Part 1 covered Strategy & Value — deciding what AI is for. Part 2 covered Data Foundations — whether the data each use case needs is reachable and lawful. This one is about where that use case will actually run.

Nobody decides their architecture

Here is how AI architecture happens in most mid-market organisations: it doesn’t. A department signs up for an AI feature inside a SaaS tool it already uses. A vendor demo impresses the board and becomes a pilot, and the pilot’s stack becomes the de facto standard because it is the only thing running. Someone connects a public API to a spreadsheet. Eighteen months later the organisation has five AI systems on four clouds, none of which talk to each other, none of which were chosen — they accumulated.

The Architecture & Infrastructure dimension asks a simple question: if your best use case were approved tomorrow, could your landscape run it in production? Not in a demo. In production, integrated with the systems where the work actually happens, with the data staying where it is allowed to be, operated by people who can tell when it degrades.

What “AI-ready architecture” actually means

It does not mean a platform purchase. For a mid-market organisation it means four concrete things:

1. An integration surface. The systems that hold your data and your workflows — ERP, CRM, DMS, core business applications — reachable through APIs or at least stable exports. An AI system that cannot reach the workflow it is supposed to improve is a demo with a budget.
2. A place to run models. A deliberate posture on cloud, EU cloud, private cloud or on-premise — chosen per workload by data sensitivity, not per vendor by sales calendar.
3. A sovereign path. For the workloads that cannot leave the EU — or the building — a proven way to run them anyway. In regulated sectors this is the difference between an AI portfolio and an AI wish list.
4. A way to operate what you deploy. Someone who knows which models are running, what they cost, how they are monitored, and how they are updated or rolled back. Call it MLOps if you like; it is mostly discipline.

Notice again what is not on the list: a data-science platform, a Kubernetes migration, a two-year “modernisation programme”. The first production use case needs a path, not a platform.

The five maturity levels of Architecture & Infrastructure

The most common profile we see in Belgian and Luxembourgish organisations is a lopsided one: level 3 or 4 on classic IT — solid ERP, decent APIs — and level 1 on everything AI-specific, because the questions were never asked. That is normal, and it is fixable without a rebuild.

Three questions that tell you where you are

These are the three the scorecard asks for this dimension.

Could your IT landscape support an AI system in production tomorrow? “Legacy systems, few or no APIs” is level 1 — and the honest answer more often than annual reports suggest. The jump that matters is to level 3: not API-first everywhere, just reachable where your priority use cases live. Map the integration surface of the top three use cases from your portfolio (Part 1) and you usually find one that is deliverable now.

If a use case required data to stay in the EU — or on-premise — could you deliver? “We don’t know” is its own answer, and a common one. “No — we depend on US SaaS providers” is at least a known constraint; you can plan around a wall you can see. The organisations that answer “a sovereign path is proven in production” did not get there by buying a bigger platform — they got there by running one contained workload on infrastructure they control and learning what it takes.

How do you decide build versus buy for AI? The level-2 answer — “vendors effectively decide for us” — is rarely said out loud but frequently true: whoever demos first, wins. The fix is not an architecture board with a binder of reference patterns; for a mid-market organisation it is one page of criteria — data sensitivity, differentiation, integration depth, exit cost — applied consistently, with someone empowered to say no.

The sovereign path, without the ideology

“Sovereign AI” has become a slogan, so let us be concrete. Sovereignty is not a political preference; it is a requirements question, and it is a spectrum:

• Public cloud, EU region — fine for a great deal of workloads, provided the contractual and transfer questions are actually answered rather than assumed.
• EU-controlled cloud or private cloud — for data whose processing must stay under European jurisdiction, not just on European soil.
• On-premise — for the workloads where the documents, the vectors derived from them, and the logs may not leave the building: patient data, case files, deal rooms, defence-adjacent work.

What changed in the last two years is that the third option became practical. Open-weight models running on your own hardware are now good enough for the enterprise use cases that dominate real portfolios — document assistants, retrieval-augmented generation with citations, classification, extraction. Not good enough for everything; good enough for the use cases that regulated organisations actually rank first. The cost of a capable on-premise inference setup has fallen to the price range of an ordinary server project, and the engineering is a known quantity.

The mistake is treating the tiers as a ladder to climb. They are a routing decision: classify the data first (Part 2), then send each workload to the cheapest tier that satisfies its constraints. Most portfolios end up mixed — and should.

Where the law lands on infrastructure

GDPR is the binding constraint most architectures quietly violate. Transfers of personal data outside the EU are regulated under Chapter V, and the case law on US transfers has made “our vendor has a data centre in Frankfurt” an insufficient answer — jurisdiction over the provider matters, not just the location of the disk. If your AI pipeline sends personal data to an API, that is a transfer question someone must be able to answer in writing.

The EU AI Act adds an operational requirement: high-risk systems must keep logs (record-keeping under Article 12) and support human oversight and post-market monitoring. In practice that rewards architectures you control — you cannot produce audit trails from a black-box SaaS feature that does not expose them.

Sector rules stack on top: financial entities carry ICT risk and exit-strategy obligations under DORA, health data carries its own residency expectations. None of this forbids the cloud; all of it rewards knowing, per workload, where the data flows and who can compel access to it.

What to do in the next 30 days

If you recognise yourself at level 1 to 3:

• Map the integration surface of your top three use cases. Which systems, which APIs or exports, which owners. One week, one page. This tells you which use case is deliverable first — often not the one everyone assumed.
• Classify before you route. Take the data inventory from Part 2 and mark what may leave the EU, what may not, and what may not leave the building. The sovereignty decision then makes itself, workload by workload.
• Write the build-versus-buy page. Four or five criteria, one owner, applied to the next three vendor conversations. The goal is not to build more — it is to stop defaulting.
• Do not buy a platform. Prove one use case on the smallest stack that satisfies its constraints. The platform decision is easier, cheaper and better-informed after production, not before.

Where this sits in the bigger picture

Architecture & Infrastructure is the third of six dimensions. Next: Governance & AI Act compliance — the register, the risk classifications, and the deadlines that are no longer in the future. Then People & Operating Model, and Security & Trust. Together they produce the maturity radar at the heart of the AI Readiness Assessment, our four-week, fixed-price diagnostic for mid-market organisations in regulated sectors.

Want your own reading? The free AI Readiness Scorecard asks the three architecture questions above — and fifteen more across the other dimensions — and gives you your maturity radar in four minutes. No account, no sales call attached; if you want a second opinion on your results, leave an email and we will write back within a business day.

Mohamed Ben Haddou is the founder of Mentis Consulting (Brussels, ULB spin-off, since 2005) and an Independent AI Expert for the European Commission.