You installed an AI desktop app. Here's what actually happens to your data — and the 20-minute check every firm should run.

A desktop AI assistant — Claude Desktop, the ChatGPT desktop app, Microsoft Copilot, and the growing field around them — is not a website in a window. It is a program installed on your computer, running with the permissions of a program installed on your computer. That one difference is the entire subject of this post.

I’ve written this for the people who have to answer for where client data goes: managing directors, and above all the professionals who carry a statutory duty of confidentiality — tax advisors, lawyers, doctors, and the firms and practices around them. No code, no jargon. If you can read your own engagement letter, you can read this.

Monday, 9:14 a.m.

Someone on your team installs a desktop AI app. The onboarding is friendly. It offers to “help you work across your files” and to “see what’s on your screen,” and because that is obviously the point of the thing, they click Allow. Then they paste in a client’s email — a real one, with a real name and a real tax question — and ask for a polished reply. Ten seconds later they have a good draft. They send it. Nothing broke. Nothing turned red.

But in those ten seconds, three boundaries were crossed, and nobody wrote any of them down: the client’s data left the device for a vendor’s cloud; it very likely left the EU for processing in a third country; and it did so through a personal account your firm has no visibility into and no contract governing. This isn’t a story about a breach. It’s a story about a normal Monday.

What a desktop AI app actually is

A chat window on a website can only see what you type into it. A desktop app is a different category of thing. To be useful, it asks for reach into your computer — and reach, once granted, tends to be forgotten. Five things are worth understanding plainly, and each one comes with something to do about it.

It holds permissions you granted once and never revisited. At install, these apps commonly request access to files, folders, the screen, sometimes the microphone. People grant them in the first two minutes and never look again. What to do: open your operating system’s privacy settings (macOS: System Settings → Privacy & Security; Windows: Settings → Privacy) and look at what each AI app can actually reach. Turn off anything you don’t consciously need. Five minutes, and the highest-value five minutes in this post.

Its “connectors” turn one question into a cloud transfer. The selling point of modern assistants is that they plug into your mail, calendar, files, or CRM. The mechanics matter: to answer a question about “this client,” the app often has to send the surrounding context — the email thread, the document, the record — up to the vendor’s cloud to be processed. Each of those is a transfer of personal data that no human reviewed and no system logged. What to do: treat every connector as a door. Open only the ones you have a specific reason to open, and revoke the rest. A connector you’re not actively using is pure risk with no upside.

Your conversation history lives in two places at once. What people type into these tools is generally retained in the vendor’s cloud, tied to the account — and often cached locally on the machine as well. That’s fine right up until the laptop is lost, sold, shared with a colleague, or synced to a personal device. What to do: find the tool’s data controls (usually under “Data controls,” “Personalization,” or “Privacy”), learn how to view and delete history, and check whether your chats are used to improve the model — on many consumer tiers that setting is on by default and can be switched off. Do it once, deliberately, per tool.

A personal account means no contract and no oversight. This is the big one. When someone uses a private ChatGPT or Claude login for work, there is usually no data processing agreement — in German law, no Auftragsverarbeitungsvertrag — between the vendor and your firm, and your administrators can’t see that the account exists, let alone what passed through it. That is the textbook definition of shadow AI, and in most organizations it isn’t the exception; it’s the normal state of affairs. What to do: decide, as a firm, which tools are sanctioned, and provide business or team accounts for them — the tiers that come with a proper agreement and admin controls. A sanctioned tool with a contract behind it is what turns shadow AI back into something you can stand behind.

The app you approved is not the app running next month. These products auto-update, and their capabilities and defaults shift with them. A connector or a data setting you reviewed in spring may behave differently by autumn. What to do: don’t treat the review as one-and-done. Put a recurring note in the calendar — quarterly is plenty — to re-check permissions and settings on the tools your team actually uses.

Why the stakes are higher if you keep secrets for a living

For most companies, mishandled personal data is a GDPR question: a business risk, measured in fines and remediation. Serious, but corporate.

For a Berufsgeheimnisträger — a tax advisor, a lawyer, a doctor, and the people in their firm — it is a different order of thing. Section 203 of the German Criminal Code (§203 StGB) makes the unauthorized disclosure of a client’s or patient’s protected secret a criminal matter, with personal liability attached to the individual who discloses it. That is stricter than GDPR, and it is aimed at a person, not just a balance sheet. A client’s tax situation sitting inside an ungoverned tool stops being a productivity question and becomes a disclosure question.

I’m going to be careful here, because this is exactly the kind of line where confident internet advice does damage: what actually counts as a disclosure under §203, and what your professional duty of confidentiality (Verschwiegenheitspflicht) requires when a third-party tool is in the loop, is a question for your chamber (Kammer) and your own lawyer — not for a blog. What I can say plainly is that the stakes here are personal and criminal, not merely regulatory, and that this alone is reason enough to treat “which AI tools touch client data, and under what contract” as a governance question your firm answers on purpose.

The 20-minute self-check

None of this requires a project. It requires twenty minutes and the willingness to look. Do it with your team, not to them — the goal is an honest picture, and the person who installed something they shouldn’t have will only tell you if it isn’t a trial.

1. 01

   Inventory what's actually installed

   List every AI tool on your team's machines — desktop apps, browser extensions, plug-ins inside Office or your CRM. Ask people directly and without blame; the software your firm licensed is not the same as the software running. You can't govern what you haven't named.

   ⏱ 5 min per person

2. 02

   Check the account behind each tool

   For every tool: is it logged in with a personal account or a firm/business one? Is a data processing agreement (Auftragsverarbeitungsvertrag) in place? Is the "use my data to improve the model" setting off? A personal account with no agreement is the first thing to fix.

   ⏱ 3 min per tool

3. 03

   Audit the connectors

   Look at what each tool is allowed to reach — mail, files, calendar, CRM — both in the app's own settings and in your OS privacy panel. Revoke everything that isn't consciously, currently needed. An unused connector is risk with no benefit.

   ⏱ 5 min

4. 04

   Adopt the interim rule today

   Until a tool is sanctioned — a contract and controls behind it — nothing with a name in it. No client names, no case details, no patient data into an ungoverned tool. Anonymize, or wait. Blunt, but it holds while you sort out the rest.

   ⏱ decide once

5. 05

   Find the history and retention settings

   Learn how to view and delete conversation history, check whether chats are cached locally on the device, and ask the lost-laptop question out loud: if this machine walked out the door tonight, what client data would walk out with it?

   ⏱ 3 min per tool

6. 06

   Write one paragraph of policy

   One written paragraph — which tools are allowed, which accounts, and the no-names rule — beats zero pages of good intentions. It gives your team a clear line and gives you something to point to. It does not need to be long to be real.

   ⏱ 15 min, once

The interim rule in the middle of that list is the one to adopt today, ahead of any other decision: nothing with a name in it goes into an ungoverned tool. Anonymize, or wait until there’s a sanctioned path. It’s a blunt rule, and blunt is what works while you sort out the rest.

There’s a printable one-page version of this check — A4, plain, forward it to a colleague or hand it round a partner meeting: the 20-minute self-check as a one-pager.

The honest limit of a checklist

Here’s the part most vendors won’t put on a slide: good hygiene reduces your risk, but it produces no evidence. You can revoke every connector, switch off every training toggle, and move everything onto business accounts — and you still will not be able to answer the one question that actually gets asked in an audit, a Kammer inquiry, or an EU AI Act review: what did your systems do with this person’s data, and when?

Checklists don’t answer that. Logs do.

That distance is where the work we do actually lives. The self-check above is real, and worth doing this week. But past a certain level of exposure — regulated client data, statutory confidentiality, an EU AI Act obligation coming into force — the answer isn’t a longer checklist. It’s an architecture: sensitive data kept inside a boundary you control, with a gateway that logs every time something crosses it. We’ve laid out how that’s built, in plain terms, in the sovereign-AI blueprint.

If you’re not sure which side of that line your firm is on, that is precisely what a fixed-price Sovereignty Assessment is for. You walk away with a data map, a target architecture, and a costed roadmap — yours to keep, whether you build it with us or not.