You may not have been paying close attention to the noise surrounding the news that Anthropic has tweaked its text-extruding LLM Claude so that it will now extrude text strings with "watermarks." But the whole business is worth a moment of your attention.
First, we need to know what the "watermark" is, because it's not what you think it is. It's not some sort of sort-of-visible marking that anyone with eyeballs can spot. Claude will not extrude text that has "Claude" printed on it in large-but-transparentish print. Nor will there be invisible text or a special note in the metadata.
No, as Anthropic explained, the "watermark" will actually be a change in how the program strings words together. And as Jason Koebler points out at 404 (headlined with the quote in this post's title), it demonstrates the degree to which Anthropic has no real respect for or interest in the actual work of writing. Just in case you had imagined otherwise.
The watermark is not visible to readers. No hidden characters, no additional text. Just a "nudge" in the selection of the next probable word. Here's part of their explanation.
Large language models like Claude work by generating one word at a time. Each time the model decides on the next word, it chooses among a list of potential candidates, ultimately selecting the most sensible or likely based on the preceding text. Take the sentence “The weather today was cold and…”. The next word is very unlikely to be “sugary.” But it is quite likely to be “overcast” or “grey.” Under most circumstances, it doesn’t matter much to the reader which of these latter two words the model ultimately chooses—the meaning of the sentence is largely the same either way. In cases like this, the choice is settled by a random number.
Watermarking uses low-stakes choices like these—which occur many times over a piece of generated text—to leave a pattern in Claude’s responses. That pattern is undetectable to the reader, but is detectable to anyone who has a key that encodes it.
Emphasis mine.
But "overcast" and "grey" are different words, with different meanings. Koebler points to an essay by John Gruber of Daring Fireball, and another by journalism academic Jeff Jarvis, in which he claims Anthropic “devalues writing” which confirm that, yes, word choice does matter. In making this choice, “Anthropic declares words fungible, language random, choice meaningless,” Jarvis writes.
But Koehler rightly sees an even more troubling foundational issue here.
It does not matter to me, necessarily, whether the output of Claude’s garbage AI text is one way or is a slightly different way. But it does matter to me that AI data scientists at huge tech companies think that word choice doesn’t matter, or that it is possible to statistically use synonyms wherever without fucking with the meaning of a sentence.
Anthropic's post talks about writing as if it's a random generation of words or numbers. It is, in one Anthropic example, just a random roll of the dice.
What Anthropic makes abundantly clear is that the question of how human writing happens, the company isn't even on their radar. They are working on an entirely different problem-- how to get a pair of dice to generate a sentence that more-or-less mimics strings of words in their training data. It is pointless to discuss chatbots as if they are just not far enough down the path to qualify as real writing tools because they are not even on that path. Anthropic wants AI-generated code to be exactly correct, but with words, one is pretty much as good as a bunch other.
The "watermark," visible only to Anthropic-created software, is meant to comply with the letter of the European Union's AI Act if not the actual spirit (See, Anthropic? Sometimes the exact wording matters!). And programmers have already figured out how to avoid it.
It all reminds me of the great Mark Twain quote: The difference between the almost right word and the right word is really a large matter - 'tis the difference between the lightning-bug and the lightning.
But mostly it's the umpteenth reminder that AI LLMs don't write in any human sense of the word, nor do their corporate overlords actually aspire for them to write in any human sense of the word.
No comments:
Post a Comment