Build versus buy for device intelligence

Most articles with this title are written by a vendor and reach the conclusion the vendor was always going to reach. This one is written by a vendor too, so read it with that in mind.

What follows is the argument as honestly as we can make it, including the parts that do not help us.

Start by taking the build seriously

If your team built device intelligence in-house, you almost certainly did it for good reasons.

You wanted control over what was collected. You did not want a third party in the path of a request that has to complete. You had a specific problem, in a specific flow, and nothing on the market solved that exact problem at a price that made sense. So an engineer spent a few weeks on it, and it worked.

That was a reasonable decision and in many cases it was the right one. Building a device identifier is not hard in the way vendors like to imply. You collect a set of browser and hardware attributes, you hash them into something stable, you store it, and you compare on the next visit. A competent engineer gets a working version in under a month.

What you get at the end of that month is real. An identifier tuned to your own traffic. No per-call cost. No vendor in the request path. Nobody else holding your data. Those are genuine advantages and no product gives them back to you.

The cost that is not in the business case

The build has two prices, and business cases usually carry one.

The first price is the build itself, and as above, it is smaller than people expect.

The second price arrives every quarter afterwards, and it has three parts.

Decay. Browsers ship. Privacy defaults change. Attributes that were stable last year get reduced, partitioned or removed. A signal set is accurate on the day it goes live and drifts from then on. This is not a defect in the build. It is what happens to any control pointed at a surface that other people keep changing.

Ownership. In-house device layers are usually one engineer’s project. Eighteen months later that engineer is on payments infrastructure, or at another company. The system still runs. Nobody knows exactly how, and the first person who has to touch it has to relearn it before they can change anything.

Coverage. Resolving a device is the tractable part of this problem. Answering where a session actually is when the network path is masked, or whether behaviour is automated, or whether the system should decline to answer at all, are separate problems. Each has its own maintenance curve, and a build that solved the first one rarely solved the other three.

None of these are about engineering skill. The team that built it could build all of it. The question is whether they should keep spending their quarters on it.

The problem neither of us has solved

Here is the part that does not flatter us.

Ask anyone with an in-house device layer how accurate it is, and the honest answer is usually that nobody knows. There is no independent reference to score it against. When the system says two sessions came from the same device, there is rarely anything to check that against except the system’s own output.

That makes the accuracy number partly a property of the method rather than the performance.

We have the same problem. It is the hardest problem in this category and buying a product does not automatically solve it. A vendor with a published figure has usually measured against something, and what they measured against matters more than the figure itself.

The only reference worth much is your own outcomes. Confirmed fraud. Chargebacks. Accounts you banned and did not have to unban. Things that resolved, in your data, where somebody eventually found out what was true.

That is available to you and it is not available to us until you run something.

What actually decides it

Not build quality. Three questions, and they take about a day to answer honestly.

Who owns it today? Not who built it. Who would pick up the pager if it started returning nonsense on Saturday. If the answer is a name, the build is in better shape than most. If the answer takes a while to arrive, that is the finding.

When was the signal set last revised? Not the last commit. The last time somebody deliberately reviewed which attributes are being collected and whether they still carry information. If that was more than a year ago, the system is answering a question the browser stopped asking.

What is it scored against? The hard one. Most systems in production cannot answer it at all, in-house and bought alike.

Three good answers means the build is doing its job and the case for changing anything is weak. Two means it needs an owner more than it needs a vendor.

What we would actually suggest

Not replacing it.

Run a second read alongside what you built, on one flow, with no enforcement. Nothing switched off, nothing rerouted, no customer sees anything different. Then look only at the cases where the two systems disagree.

Where two systems agree, they tell you nothing new. The disagreements are the entire experiment, and your own confirmed outcomes decide which read was right.

If the in-house build wins, that is a genuinely useful thing to have proved, and it is worth more than the exercise cost. You now have an answer to the question the business case could not answer, and the next person who suggests buying something has data to argue with.

The honest limit

A second read is a second dependency. It costs a request, it adds a failure mode, and it needs somebody to own the integration.

If your build has a named owner, a revision cadence, and something to measure against, you probably do not need us. That is a real answer and we would rather give it than pretend otherwise.

The question was never really build or buy. It is what happens to a device layer in year three, and whether anyone in the building is still responsible for it.