The small cogs and the Seeing stone

I wrote Palantir and Machines and the small cogs as twin articles for a reason. You might have noticed that there is an underlying tension where those two ideas meet.

The kind of aggregation that enables the small cogs to benefit from the Machines is the same aggregation that enables the Surveillance state we are afraid of.

That’s the true issue with the current direction of both the AI world and the authoritarian veer that governments worldwide are experiencing.

Large language models are genuinely revolutionary, but strong limits need to exist, not just in the usual security and alignment work that we discuss often (I did in Why are they talking about creating Gods) but also in how the data can be shared with other people and systems. It’s pretty clear that the expectation people have is that their chats are private until explicitly shared, and that expectation is driving a lot of the behavior. Why would you share financials and health, or discuss potential criminal cases, as if you were talking to a consultant, a GP or a lawyer, if the expectation of privacy you have with those professionals doesn’t carry over to the machines?

The answer is that you wouldn’t.

The answer to why you would let the Machines control part of your house is that that data can be local, accessible to you only, and the aggregation becomes much, much more difficult, but not impossible.

Self-hosting top shelf models is too costly at the moment and unrealistic for any small cog. What we have to hope is that the models keep improving not only in the intelligence dimension but in the efficiency one too, and that the hardware can catch up before the Seeing stone catches you.