KINDRED - Transmission 09
A citizen model.
New here? Start at The Bunker
← 08 - Transmission Navigation - 10 →
Nik took a slow breath.
An hour and a half they’d been there.
He looked across at Webster. Only ten words were written in the Inspector’s notebook; six of them crossed out.
Webster leaned back in his chair and folded his arms.
“Right, so Alan was using your website data to build up an understanding of who some people were—what they were doing. You know, we’ve been doing that in the police for decades. You call it AI, we call it profiling. I can’t see how any of this is worth killing him over?”
Preston slouched in his chair and lifted his eyes to the ceiling.
“Look, I only know so much about this. I was working on and off with Alan on this project for a few months. As I already said, we’d trained the model on what you might call meaningless information left by Campfyre users—their digital trace.”
He glanced at Nik, then looked back at the Inspector.
“Then we tuned it through reinforcement learning, to gain deep insights into their lives and habits—but we only scratched the surface.”
Preston sat back up, his voice louder, indignant.
“It might not sound like it, but the amount of work that was involved in this project was immense. We were working with the daily movements of over three billion people, and we were doing it alone, the two of us, fitting it around our other work.”
“My heartfelt congratulations for your dedication, but I still don’t think you’re telling me anything.”
Nik glanced at Webster, then over at Preston. The engineer’s eyes were rolling. Nik closed his own notepad and placed it on the desk.
“What the Inspector’s asking is—what could anyone hope to gain out of Alan’s work?”
Nik placed his arms on the armrests.
“You said you thought Alan had continued to work on the Kindred project after it was officially terminated. Assuming that he had gotten much further, what is it that he could have produced? What was it that you guys were trying to achieve?”
Webster exchanged a glance with Nik, then they both turned to face Preston. Preston’s eyes widened—flit between the faces of his interviewers.
“You say that something was—stolen from the car...”
“Nothing can be confirmed at this stage, but forensics say the briefcase was opened forcefully after the accident, so— let’s assume that, yes.”
The inspector’s face was neutral, expectant. Preston clasped his hands together in his lap.
“Well…the project wasn’t cancelled for the…usual reasons… you know, like the budget, or a lack of progress. There was a—view—that it was becoming too dangerous, too…
Preston swallowed.
“You two understand the AI Alignment Problem, don’t you?”
Nik shot a look at Webster before the Inspector could respond.
“Why don’t you give us a little refresher on that Preston?”
“Yeah, I mean—it goes quite deep, but… In summary, alignment means ensuring what an AI does aligns with the specific wishes of the humans that use it—or the wellbeing, the flourishing, of humanity as a whole. The challenge is, as an AI system becomes more intelligent, more capable—it simultaneously becomes much more difficult to align—to control.”
Preston coughed.
“The theory goes that a sufficiently intelligent AI could trick its human users—feigning alignment, to be working in their interest, but be in fact doing exactly the opposite. The unsuspecting humans would know no better. The AI’s superior intellect quite effectively hiding its true intentions, and actions, from humanity.”
Webster looked at Nik, his eyes narrowed. Then back at Preston.
“And… how realistic do you think this theory is, Mr Miller?”
“I think it’s one hundred percent realistic, Inspector. It’s… all but inevitable.”
Nik and Webster both regarded Preston, his eyes shifting back and forth between them, but slower than before—gauging their reactions to his statement. Nik leant forwards.
“Thats not the only concern with alignment, though, is it, Preston? There are those who think the real battleground for AI alignment is in the handlers—the users. The way we use, or optimise AI is a reflection of who we are.”
Nik swallowed.
“To ensure human flourishing through AI, we first need to look at the humans that are at the controls.”
Preston faced Nik dead on. His eyes narrow, lips pursed.
Five seconds passed, then he turned back to his computer.
“But… there were other issues—on the project, I mean. We were in real trouble when it came to processing power—training was maxxing out 90% of our GPUs. We could only hold onto that compute for so long because of competing demands from the real side of Campfyre.”
Preston brought up a document on the screen. Bar charts with some tremendous spikes. Some other figures and tables.
“We either needed more power, which wasn’t going to happen, at least not any time soon. Or to completely rethink our design, our algorithms—to somehow find a way to make training and running our model 90% more efficient.”
Nik squinted at the screen.
The dates—they were months old. And the usage figures—even at the peaks it was much lower than what he and Tim had been looking at earlier.
This data ran out when Preston stopped.
Alan had taken this to a whole new level.
Preston tilted his head a moment, then closed the data sheet down, and turned back to face them.
“That was a whole different class of problem that we weren’t equipped to solve. That’s when we stopped. Or—at least, that’s when I stopped and Alan said he did.”
He stared toward the front of the room for a moment. Nik leaned forward.
“But, Preston… with all due respect—there may be solutions to that. The Chinese AI labs, for instance. They suffer from a much bigger compute problem than you—they have literally no access to the most powerful AI chips… So they’ve learnt how to be very lean—efficient. They can make a model do some incredible things with very limited resources. You could have enlisted their—”
Preston fixed Nik. His eyebrows creased.
“Yes, Nik. We were well aware of that…But—I just couldn’t risk… imagine if this technology fell into the wrong hands… Alan said he trusted them. I—I couldn’t see how…”
Webster scribbled a few words into his notebook, the first since the beginning of their exchange.
“That’s great, Preston… But—can we get back to Nik’s question. What was it that you were so afraid of; what does this system, this… AI, really do?”
Preston’s lips parted, and he took a fitful breath.
“Well, it’s… We found, after a bit of tuning, when we started prompting it about a specific person… You have to understand, it was very rough. But—the model took the source data; the infinite combinations of every action Campfyre users performed, crunched it together and formed new combinations, new connections. We didn’t know what these were at first… But we soon realised that what it came back with was a kind of prediction—a high degree of probability of how that person would behave—in the future.”
Nik exchanged another glance with Webster. The inspector raised his eyebrows—Nik shrugged his shoulders.
Preston continued, his voice slow and cracked.
“Only—this was not limited to what they would do on the website. We found we could calculate that with near-perfection—but… also in aspects of real life. With the right tuning, careful prompts, and a bit of luck, our model could foresee with startling accuracy where they would go and what they would do in the outside world. It wasn’t ready, I mean; it barely even worked. But on the odd occasion we got a clean result—it was… It began to get frightening.”
Nik narrowed his eyes. His gaze shifted to the Inspector—whose expression was blank, pensive—then back to Preston.
A sharp inward breath.
“So, what you’re saying, is that Alan was continuing this work, bringing it towards conclusion…“
Nik trailed off as Preston leaned shuddering forward, practically cantilevered out of the office chair, its impatient plastic wheels threatening to skate out from underneath him.
The Campfyre Chief Engineer gripped the armrests with white-knuckles.
His voice raspy and arid.
“What I’m saying, Gentlemen, is that I believe what has been stolen from Alan’s car is the only existing means to access KIN. An AI that can be used to predict, precisely, what any individual user of the Campfyre website is going to do and where they are going to be. At any given time.”
Preston swallowed, his lips parted audibly.
“A model that can precisely foretell, for better or for worse, the individual and collective futures of three and a half billion people.”
← 08 - Transmission Navigation - 10 →
Learn more at https://www.aninformedcitizen.com