There is a claim about tools that most of us accept without ever examining it: that a tool is neutral, and only its use is good or bad. The hammer takes no view on whether you build a house or break a window. The spreadsheet is indifferent to what goes in the cells. On this account the tool stays innocent, and the responsibility belongs entirely to whoever picks it up.
I found this convincing for a long time. Working on AI systems is what changed my mind, because the claim falls apart precisely where it matters most. A tool that actively shapes what its user can see and decide is doing more than sitting there waiting to be used, and calling it neutral stops describing anything real.
Tools Shape What You See
A map is a tool. It shows you what’s there. Except it doesn’t. It shows you what the mapmaker decided was worth showing. A colonial-era map of West Africa shows administrative boundaries, resource deposits, and trade routes. It does not show sacred groves, seasonal grazing patterns, or the territories of communities that didn’t interest the colonial administration. The map doesn’t lie. It just decides, silently, what counts as geography.
A survey instrument is a tool. It collects data about what people think. Except the questions determine the answers. Ask farmers whether they support cluster farming and you get a yes or no. Ask them what concerns they have about collective agricultural arrangements and you get a different kind of knowledge entirely: stories about past cooperatives that collapsed, worries about losing control over planting decisions, calculations about risk that the survey designer never considered. The survey doesn’t distort. It just decides, silently, what counts as a valid response.
A diagnostic algorithm is a tool. It identifies diseases. Except it identifies the diseases it was trained to recognise, in the populations it was trained on, using the symptoms that were labelled in its training data. A skin cancer detection model trained primarily on lighter skin will miss conditions on darker skin. Not because the tool is biased in any intentional sense. Because the data it learned from encoded a particular definition of what skin looks like, and that definition, silently, became the tool’s understanding of the problem.
In each case, the tool makes choices. Not consciously. Not maliciously. But structurally. It foregrounds certain information and backgrounds other information. It makes some questions easy to ask and other questions invisible. It doesn’t force anyone to decide anything. It just shapes the landscape of what feels decidable.
The Framing Effect
This goes deeper than the familiar argument about biased data, though biased data is part of it. The more fundamental issue is that tools frame problems, and framing determines outcomes.
When a participatory budgeting platform asks residents to submit proposals and then uses AI to categorise them into themes, the categories become the frame. A proposal that fits neatly into “transportation” or “green spaces” gets amplified. A proposal that cuts across categories, or that addresses something the system doesn’t have a category for, gets compressed, reclassified, or lost. The tool didn’t reject the proposal. It just made certain kinds of thinking easier to express than others.
When a credit scoring algorithm evaluates a loan applicant based on mobile phone data, the variables it uses define what creditworthiness means. Regular payments to a utility company signal reliability. Frequent small transfers to multiple family members might signal instability to one algorithm and community embeddedness to another. The tool doesn’t decide who deserves credit. It decides what “deserving” looks like, and that decision was made by whoever chose the variables, long before any individual applicant was scored.
When an agricultural advisory app recommends a planting window based on satellite-derived weather data, it defines what relevant knowledge looks like. Satellite data is relevant. The farmer’s observation that the soil on the north side of the field behaves differently after early rains is not relevant, because the tool has no way to receive it. The tool doesn’t override the farmer’s knowledge. It just operates as if that knowledge doesn’t exist.
Neutral for Whom?
The neutrality claim usually holds up from the perspective of the tool’s designers. They didn’t intend to exclude anyone. They didn’t deliberately choose variables that disadvantage particular groups. They built the best system they could with the data available and the problem definition they started with.
But intention is not the same as effect. A tool built by people in one context, using data from that context, encoding assumptions from that context, and deployed in a different context doesn’t become neutral just because nobody meant harm. It carries its origins with it. The question isn’t whether the designers were biased. The question is whether the tool’s embedded assumptions match the reality of the people using it.
In my experience, this mismatch is sharpest when tools cross borders. An AI system developed in a well-resourced research environment and deployed in a low-resource community carries assumptions about data availability, infrastructure, literacy, connectivity, and institutional support that may not hold. The tool works. It just works within a definition of the problem that was written somewhere else, by someone else, for someone else.
What Follows
If tools aren’t neutral, then the people who design them are making choices that affect outcomes, whether they acknowledge it or not. This doesn’t mean every tool is an instrument of oppression. Most tools are built with good intentions and produce genuinely useful results. But it means that “we just built a tool, how people use it is up to them” is an incomplete account of what happened.
A more honest account would say: we built a tool that frames problems in a particular way, foregrounds particular kinds of information, and makes particular kinds of decisions easier. We made those choices based on our understanding of the problem, which was shaped by our context, our data, and our assumptions. Those choices will interact with the contexts where the tool is deployed, and the interactions may produce effects we didn’t anticipate.
That’s not a confession. It’s just a more accurate description of what building a tool involves. And it opens up a set of questions that the neutrality story forecloses: whose definition of the problem does this tool encode? What kinds of knowledge does it make visible, and what kinds does it make invisible? Who was in the room when the design choices were made, and who wasn’t?
These aren’t questions you ask once and then move on. They’re questions that apply every time a tool crosses from the context where it was built to a context where it’s used. Which, in a world where AI systems are built in a handful of countries and deployed everywhere, is nearly every time.