Data that you can prove was generated by humans is now exceedingly valuable ...and most of that comes from the days before LLMs. The situation is a bit like how steel manufactured before the nuclear age is valuable.
I will blame overlong copyright term lengths. 70 years after authors death or 95 years after publication, allowing most recent work to enter the commons effectively after a century, or more, from now [0].
This is the rare case when Europe is even worse. Metropolis, the 1927 Fritz Lang film, is out of copyright in the United States but will still be in copyright in Germany until 2047: 120 fucking years.
It’s preposterous, and offensive to anyone’s intelligence to claim that this is about incentivizing production; does anyone seriously believe there is a potential artist out there who would avoid making their magnum opus if it could only be under copyright for 119 years?
The problem is, copyright law is no longer about artists, if it ever was: it’s about corporations, i.e. maximizing the value corporations can extract from intellectual property.
Given the argument over LLMs consuming books illegally, I think publishers could be a little concerned that an LLM that combined partial previews on every modern work on a subject might be a destroyer of the market for the average book on the subject with the license to do so having been properly granted via this feature.
Right, that's what I'm saying. For whatever reason it seems publishers decided they don't want their preview-only books as part of the full-text search across all books. If they decide that, Google has to comply.
This isn't like web search where web pages are publicly available and so Google can return search results across whatever it wants. For books, it relies on publisher cooperation to both supply book contents for indexing under license and give permissions for preview. If publishers say to turn off search, Google turns off search.
Among the less-important things I'd like to send back in time to my past-self:
"The trend in digitized book passages will reverse, and they will become harder and harder to find with time, so clip your own copies of everything you like to quote."
I just checked and yes, search inside of books with previews is still possible.
(a) when you search books.google.com and find a book with a preview, it opens their new book viewer - the search is at the bottom of the page. You can also click "View All" to see all references of your search in that book.
(b) if you go to the book homepage (clicking X in the top right of the book viewer if that opened), there's still a "Search Inside Book" next to the "Preview" button under the title.
So, if you search for some text that occurs at the end of one chunk, will it then preview a following chunk? And could chaining these chunks give you the entire book?
If so, I could see someone doing this to exfiltrate books.
You're talking about in-book search (TFA is about search across all books), and yes that was indeed once a known technique for extracting whole or nearly whole books.
That's why publishers responded by excluding sections of books from search (it will list the pages but you can't view them), and individual Google accounts became limited in how many extra pages they were ever allowed to see of an individual book beyond the standard preview pages.
But then LibGen, Z-lib, and Anna's Archive became popular and built up their collections...
If search gives you a preview with a few surrounding words, it is fairly simple to abuse search with quotation marks to extract bigger and bigger sections of the books, potentially till you have the whole book.
Since I pretty much only use Google Books for public domain books, old magazines, and newspapers I haven't noticed any problem with it. Maybe it's not as dead as this person thinks.
Sure, just like Aaron Swartz was persecuted for "recklessly damaging a protected computer" and "wire fraud" not for any other reasons at all and btw the State wasn't at all involved in murdering him, he did that to himself, probably because he felt guilty for having damaged so many computers....
I’m genuinely curious how you feel about LLMs being trained on pirated material. Not being snarky here.
Your comment reflects the old “information wants to be free” ideals that used to dominate places like HN, Slashdot, and Reddit. But since LLMs arrived, a lot of the loudest voices here argue the opposite position when it comes to training data.
I’ve been trying to understand whether people have actually changed their views, or whether it’s mostly a shift in who is speaking up now.
why would that change anything? copyright is still a tax on the whole of society for the benefit of rich people and corporations. it opposes innovation, evolution and progress
maybe a short copyright would be fine (10 year fixed?) but copyright as-is seems indefensible to me
> copyright is still a tax on the whole of society for the benefit of rich people and corporations. it opposes innovation, evolution and progress
The original reason for copyright, patents, and trademarks made sense.
We want people to create and share. And unlike the old guild solutions from Europe, copyright and patents were a tradeoff to encourage the arts and science.
But what's a good tradeoff? Thats a big copyright question. 17 years? 34 years? Life of author? 75 years? How about individual non-commercial use? Or abandoned works?
And patents aren't even in scope, but we see similar abuses against the raison d'etra of them. Patents were supposed to entail a full reproduction of invention. Now, its a game of how incomplete can we make the filing while still getting protection. Or worse yet, really dumb shit has been patented like 1 click or the XOR patent, or that asshole Chakrabarty who patented living organisms.
There were good reasons for a fair copyright and patent law for furtherance of the art and sciences. That narrative was lost long ago. Now, only the violators can really push ahead. And they can't talk about it.
(Trademark law has never really had much complaints, aside trademarking a color. If you buy from XYZ company, you want to buy from them, not a counterfeit. And it relates back to coats of arms, again, representing a family or a charge.)
Personally, my opinion doesnt matter. I'm a nobody who doesnt work in AI fields.
But as a pirate, I specialize in finding hidden, hard to find, or otherwise lost sources. They're not making anybody any money, and I absolutely do not sell anything thats not mine (freely given).
But having every commercial work available for ingestion into an LLM is an amazing way to train an AI. However if you're going to use piracy at scale to train, you should also not be able to sell the LLM or access to it.
And yeah, that wrecks every corporate LLM strategy. Boo fucking hoo.
Do creators need paid for content they create? Ideally, yes! Do they deserve iron-fisted control of your hardware (DRM) to enact their demands? Fuck no!
Ideally, the LLMs would be FLOSS, full weights published, lists of content used to reproduce, etc. We could prune bad content and add more good. But the problem again is whoever does this must violate copyright cause copyright in the way its implemented is terrible.
In reality, I like the RIAA's congressional solution. You send a check for how many plays you did to BMI/ASCAP and you're good. That could be extended to books and shows. If that were done, you could have a New-Flix service that literally has every show and movie in existence. You just pay a reasonable cost per month to access the whole of video humanity.
They don't do full text search anymore esp for copyrighted books. I wonder if this is not a regression but an intent to give them a let up in the AI race.
The YT transcripts are linked to on the YT page itself. If they remove that, it is trivial to use a local STT model to transcribe the video. If they make it impossible to download a video, you could just have a microphone record all of the sound, and so on. Once you have the transcription of anything, summarizing is trivial. I have a local script that does this and I use it all of the time. Also produce diagrams for YT summaries. Hours saved, per day.
The left results are contemporary, the right are decades old. That includes editions of the same book --- surely the newer edition is going to be preferred by most readers.
I guess. That's not immediately clear to me. However, browsing around on Google Books suggests to me that it is the corpus which changed, not the algorithms.
> surely the newer edition is going to be preferred by most readers.
Why? Where different editions exist, the reader will want to know which one they're getting, but they're unlikely to systematically prefer newer editions.
But also, Google Books isn't aimed at "readers". You're not supposed to read books through it. It's aimed at searchers. Searchers are even less likely to prefer newer editions.
> they're unlikely to systematically prefer newer editions
That seems wrong to me. Generally when a new edition of something is put out it's (at least nominally) because they've made improvements.
("At least nominally" because it may happen that a publisher puts out different editions regularly simply because by doing so they can get people to keep buying them -- e.g., if some university course uses edition E of book B then students may feel that they have to get that specific edition, and the university may feel that they have to ask for the latest edition rather than an earlier one so that students can reliably get hold of it, so if the publisher puts out a new edition every year that's just different for the sake of being different then that may net them a lot of sales. But I don't think it's true for most books with multiple editions that later ones aren't systematically better than earlier ones.)
> But I don't think it's true for most books with multiple editions that later ones aren't systematically better than earlier ones.
Most books with multiple editions are books that have been translated multiple times. It is definitely true that later translations aren't systematically better than earlier ones.
Google Books is long dead. If you click on the author's name in one of the results, it will search inauthor:"Author's Name" and this search will return garbage because it chokes on double quotes. This has been true for at least a couple of years; Google Books is not compatible with itself. Changing the double quotes to single quotes fixes it. Also, lately, when you filter only for books that have Full View some results that have Full View get dropped for no intelligible reason.
Nobody is looking at it. I wouldn't be surprised if the preview search was switched off by accident.
For me Books is only useful (and it is very useful) for books out of copyright, 100+ years old. Sometimes they aren't at archive.org.
I hate Google, but I think it's a bit absurd to criticize them on this if somehow it's over AI. The only reason Google created Books may even have been AI, but they were hoping to have the books open to everyone, and the publishers and authors whose full text is being blocked are literally the people who stopped it from happening. Maybe they spoke up about AI, too. I find it even hard to even criticize that Google doesn't take care of Books - it has no purpose or profit potential for them anymore, it's obviously charity that they don't take it down completely.
I suspect it's actually the opposite. Standard inverted index text search is incredibly cheap and mature. Vector search requires generating embeddings and running approximate nearest neighbor queries, which is significantly more compute intensive than simple keyword matching. If they switched, it wasn't to save on compute costs.
> The largest truly open library in human history
[0] https://en.wikipedia.org/wiki/Anna%27s_Archive
“Our mission is to organize the world’s information and make it universally accessible and useful”
https://about.google/company-info/
Data that you can prove was generated by humans is now exceedingly valuable ...and most of that comes from the days before LLMs. The situation is a bit like how steel manufactured before the nuclear age is valuable.
Almost certainly, this is something that publishers requested the removal of, under threat of requiring previews to be removed entirely.
Books that are out of copyright still have full search and display enabled.
So blame publishers, not Google.
[0] https://en.wikipedia.org/wiki/Copyright_law_of_the_United_St...
It’s preposterous, and offensive to anyone’s intelligence to claim that this is about incentivizing production; does anyone seriously believe there is a potential artist out there who would avoid making their magnum opus if it could only be under copyright for 119 years?
This post which was on the front page today is relevant: https://alexwennerberg.com/blog/2026-01-25-slop.html
This isn't like web search where web pages are publicly available and so Google can return search results across whatever it wants. For books, it relies on publisher cooperation to both supply book contents for indexing under license and give permissions for preview. If publishers say to turn off search, Google turns off search.
More on the HathiTrust project: https://en.wikipedia.org/wiki/HathiTrust
Though I don't know how many of the HathiTrust books are the "preview" kind the Reddit post mentions. Maybe none are?
"The trend in digitized book passages will reverse, and they will become harder and harder to find with time, so clip your own copies of everything you like to quote."
(a) when you search books.google.com and find a book with a preview, it opens their new book viewer - the search is at the bottom of the page. You can also click "View All" to see all references of your search in that book.
(b) if you go to the book homepage (clicking X in the top right of the book viewer if that opened), there's still a "Search Inside Book" next to the "Preview" button under the title.
Then it would have been hella useful.
If so, I could see someone doing this to exfiltrate books.
That's why publishers responded by excluding sections of books from search (it will list the pages but you can't view them), and individual Google accounts became limited in how many extra pages they were ever allowed to see of an individual book beyond the standard preview pages.
But then LibGen, Z-lib, and Anna's Archive became popular and built up their collections...
"Hey, remove search?"
"OK, it was costing money anyways."
"But a few days ago they removed ALL search functions for any books with previews, which are disproportionately modern books." <emphasis mine>
Check out library genesis, Anna's archive, and scihub for content.
Piracy isnt theft if buying isnt ownership.
William Tyndale was put to death for translating the Bible into English, which would have been an act to make information open and accessible.
That's not what he was put to death for. See https://www.catholic.com/magazine/print-edition/tyndales-her... and https://www.chinakasreflections.com/did-the-roman-catholic-c...
Your comment reflects the old “information wants to be free” ideals that used to dominate places like HN, Slashdot, and Reddit. But since LLMs arrived, a lot of the loudest voices here argue the opposite position when it comes to training data.
I’ve been trying to understand whether people have actually changed their views, or whether it’s mostly a shift in who is speaking up now.
maybe a short copyright would be fine (10 year fixed?) but copyright as-is seems indefensible to me
The original reason for copyright, patents, and trademarks made sense.
We want people to create and share. And unlike the old guild solutions from Europe, copyright and patents were a tradeoff to encourage the arts and science.
But what's a good tradeoff? Thats a big copyright question. 17 years? 34 years? Life of author? 75 years? How about individual non-commercial use? Or abandoned works?
And patents aren't even in scope, but we see similar abuses against the raison d'etra of them. Patents were supposed to entail a full reproduction of invention. Now, its a game of how incomplete can we make the filing while still getting protection. Or worse yet, really dumb shit has been patented like 1 click or the XOR patent, or that asshole Chakrabarty who patented living organisms.
There were good reasons for a fair copyright and patent law for furtherance of the art and sciences. That narrative was lost long ago. Now, only the violators can really push ahead. And they can't talk about it.
(Trademark law has never really had much complaints, aside trademarking a color. If you buy from XYZ company, you want to buy from them, not a counterfeit. And it relates back to coats of arms, again, representing a family or a charge.)
But as a pirate, I specialize in finding hidden, hard to find, or otherwise lost sources. They're not making anybody any money, and I absolutely do not sell anything thats not mine (freely given).
But having every commercial work available for ingestion into an LLM is an amazing way to train an AI. However if you're going to use piracy at scale to train, you should also not be able to sell the LLM or access to it.
And yeah, that wrecks every corporate LLM strategy. Boo fucking hoo.
Do creators need paid for content they create? Ideally, yes! Do they deserve iron-fisted control of your hardware (DRM) to enact their demands? Fuck no!
Ideally, the LLMs would be FLOSS, full weights published, lists of content used to reproduce, etc. We could prune bad content and add more good. But the problem again is whoever does this must violate copyright cause copyright in the way its implemented is terrible.
In reality, I like the RIAA's congressional solution. You send a check for how many plays you did to BMI/ASCAP and you're good. That could be extended to books and shows. If that were done, you could have a New-Flix service that literally has every show and movie in existence. You just pay a reasonable cost per month to access the whole of video humanity.
Alas. Guess I'll have to build it myself.
https://en.wikipedia.org/wiki/Z-Library
Protest this by pirating, until copyright terms are reduced to make copyright once again a net benefit for society.
Here are two screenshots taken on Jan 20 and Jan 23 https://bsky.app/profile/adamnemecek.bsky.social/post/3mdbup...
They don't do full text search anymore esp for copyrighted books. I wonder if this is not a regression but an intent to give them a let up in the AI race.
Similarly, a year ago or so ChatGPT could summarize YouTube videos. Google put a stop to that so now only Gemini can summarize YouTube videos.
Why? Where different editions exist, the reader will want to know which one they're getting, but they're unlikely to systematically prefer newer editions.
But also, Google Books isn't aimed at "readers". You're not supposed to read books through it. It's aimed at searchers. Searchers are even less likely to prefer newer editions.
That seems wrong to me. Generally when a new edition of something is put out it's (at least nominally) because they've made improvements.
("At least nominally" because it may happen that a publisher puts out different editions regularly simply because by doing so they can get people to keep buying them -- e.g., if some university course uses edition E of book B then students may feel that they have to get that specific edition, and the university may feel that they have to ask for the latest edition rather than an earlier one so that students can reliably get hold of it, so if the publisher puts out a new edition every year that's just different for the sake of being different then that may net them a lot of sales. But I don't think it's true for most books with multiple editions that later ones aren't systematically better than earlier ones.)
Most books with multiple editions are books that have been translated multiple times. It is definitely true that later translations aren't systematically better than earlier ones.
Nobody is looking at it. I wouldn't be surprised if the preview search was switched off by accident.
For me Books is only useful (and it is very useful) for books out of copyright, 100+ years old. Sometimes they aren't at archive.org.
I hate Google, but I think it's a bit absurd to criticize them on this if somehow it's over AI. The only reason Google created Books may even have been AI, but they were hoping to have the books open to everyone, and the publishers and authors whose full text is being blocked are literally the people who stopped it from happening. Maybe they spoke up about AI, too. I find it even hard to even criticize that Google doesn't take care of Books - it has no purpose or profit potential for them anymore, it's obviously charity that they don't take it down completely.
Which tends to be kind of poop compared to true text search.