I’ve been telling people for years that the entire 21st century is at risk of being a lost century. Even personally I can’t guarantee my data will be with me 20 years from now even though I back it up. If you care about a photo or document, print it and throw it it a box. As I get older I find more of an obsession with physical media from a preservation point of view. Because I know my books and pictures will be around 50 years from now. Digital files not so much.
I agree with you in general, as most people don’t use physical media. However, those of us that do, are probably pretty secure in our legacy.
I have digital files that have been with me for over a quarter of a century, first through repeated copies to new media formats, then to more sophisticated backup systems. In the past few years, I’ve been alternating backing up to cloud services and then to local USB disks; the backup program is a statically compiled, monolithic program with few dependencies. Recently, I found a solution to the encrypted restore by survivors. I even have a README with instructions.
I’m secure in the knowledge that my 3TB of painstakingly curated collection of foot porn will be available to future researchers, for the betterment of mankind.
I’ve been telling people for years that the entire 21st century is at risk of being a lost century. Even personally I can’t guarantee my data will be with me 20 years from now even though I back it up. If you care about a photo or document, print it and throw it it a box. As I get older I find more of an obsession with physical media from a preservation point of view. Because I know my books and pictures will be around 50 years from now. Digital files not so much.
LOCKSS and KISS, though. Flash chips don’t last forever but are pretty durable, and so are optical media as long as they’re the right material. SSDs decay and HDDs fail, but for magnetic platter media even if the head or motor crashes there’s always the old magnetic microscope in a pinch. USB’s not going anywhere, and if you have four or five copies that you don’t completely neglect and don’t store in the same physical place, presumably you’ll have the chance to notice and take corrective measures if any of them start failing or are at risk.
I don’t actually know that an individual book or picture will still be around in 50 years; Fire, flooding, insects, acidic paper, low-quality ink maybe— Digital stuff’s fragile, but so is physical stuff. Stick it in the attic, and the heat’ll speed up any chemical reactions and probably make it cozier for insects; Stick it in the basement, and the condensation will get you mildew and rot. By contrast, having a flash drive accidentally survive a trip through a washer and dryer is a pretty common occurrence, and I’ve yet to lose a drive even with that level of negligence. Material compatibility’s one of the very most basic parts of a set of very precise manufacturing techniques, tin whiskers seem pretty rare these days, the really scarily insidious stuff like hydrogen embrittlement is super improbable, and most biological forms of decay haven’t adapted to eating cured epoxy and monocrystalline silicon yet.
At least I sorta know how a flash cell or hard drive platter is meant to be structured; Who knows what weird organic reactions and unstable or slowly diffusing molecules are happening in the pile of chemical pigments on a sheet of likely-acidic bleached cellulose and cheap ink or toner, and whether it will still be legible to human eyes in however many years? Plus, a printed photo or document starts fading the very instant it’s created, and it gets a little worse every time you touch it with sweaty human hands or look at it while exhaling moist human breath and corrosive enzymatic saliva droplets under a white LED lamp or G-type star shooting out ionizing UV rays. Digital failures tend to be catastrophic, but at least up until the moment it fails, you can make sure that it is the exact same picture or text— And you can make many, many copies very cheaply, all of them very physically durable compared to paper, and know that they are all the exact same picture and text.
That said, I absolutely agree with your overall assessment that most of the information in the early 21st century, including most of the public Internet/WWW, most likely either will be or already is… Maybe not technically lost, per se, given how much caching and saving happens on private clients, but certainly rendered inaccessible.
Ideally I’d really love to see a return of microfiche, actually, using modern polymers and metallization. I’ve been meaning to look into that for a while now. At a reasonable scale for optical viewing, you could fit… much, much more content than you might expect, and do it several times over, in an entirely reasonable number of pages. Your comment actually spurred me to finally think of a practical way of printing that— for years before, I’d been trying to idly figure out a process based on photomasks and nanoparticles suspended in resin, which had always felt like a very messy and tricky idea, but I just thought of another idea– So thanks for providing some inspiration there.
Even data from the Apollo missions was found to be either degrading (tapes) or the formats were forgotten and the systems that could read them were gone. They had to do research into rediscovering how to read the data and hunt around for the few antique systems remaining to read the tapes.
There is a will but there aren’t enough people with enough brain power to actually do the steps needed.
Should it endure? I don’t know, maybe the last few decades should be forgotten.
Crazy how we are now talking about losing everything from the past few decades of human history because of a story about videographers losing their video data on a quality control issue.
Crazy how a single event sometimes reminds people of bigger problems, huh?
Digital media, where we store basically everything we care about, is hugely, hugely volatile, unreliable, and fragile. But you never notice it until you’re reminded of it, and then you really notice it. This story reminded people of it.
The reminder to stay grounded is probably also healthy, but I do think you’re missing the point of this comment thread.
PCM, ASCII, and straight RGBA bitmap encodings aren’t going anywhere. By extension, derived formats like WAV, UTF-8, and word processor files and webpage HTML are mostly fine too. The formats are structurally simple enough that even if the associated file extensions were somehow to be forgotten, all you’d need to do to invent them again is hand the file to a bored nerd over the weekend.
I think you kinda got the BBC and NASA problems backwards. The BBC’s had a couple of prominent incidents where digital “preservation” that was supposed to be eternal couldn’t even be opened anymore after a couple of years, like their Domesday Book/Project application thingy. They’ve also lost a bunch of old shows, like early Dr. Who episodes, I think. NASA didn’t just forget how to read the Apollo tapes; they overwrote them to reuse the tapes, as was their standard practice at the time. The original signal and tapes were very HD (or analog), but most of the videos we have today are from the TV camera that they pointed at their own TV screen last-minute when they realized they didn’t have an adapter for broadcast— The equivalent of a grainy cell phone photo of a screenshot, basically.
The BBC and NASA incidents happened in an era before computers were a ubiquitous commodity product. So, everyone and their cat was basically inventing their own obscure single-implemention proprietary file formats at that time. Nowadays we have established technical standards, as well as formats that have already sorta stood the test of time based on their utility and simplicity— and millions of people who already know how to read them— so that particular vector for bitrot isn’t really as much of an issue anymore.
…That said, I think I sorta missed your point. What you’re really saying is that stewardship of digital records is much trickier and riskier than stewardship of physical records— and that results in stuff being lost. And that is absolutely true.
I’ve been telling people for years that the entire 21st century is at risk of being a lost century. Even personally I can’t guarantee my data will be with me 20 years from now even though I back it up. If you care about a photo or document, print it and throw it it a box. As I get older I find more of an obsession with physical media from a preservation point of view. Because I know my books and pictures will be around 50 years from now. Digital files not so much.
I used to think this, but now, less so.
I agree with you in general, as most people don’t use physical media. However, those of us that do, are probably pretty secure in our legacy.
I have digital files that have been with me for over a quarter of a century, first through repeated copies to new media formats, then to more sophisticated backup systems. In the past few years, I’ve been alternating backing up to cloud services and then to local USB disks; the backup program is a statically compiled, monolithic program with few dependencies. Recently, I found a solution to the encrypted restore by survivors. I even have a README with instructions.
I’m secure in the knowledge that my 3TB of painstakingly curated collection of foot porn will be available to future researchers, for the betterment of mankind.
Thank you for your service !
We’re in the presence of greatness 🫡
LOCKSS and KISS, though. Flash chips don’t last forever but are pretty durable, and so are optical media as long as they’re the right material. SSDs decay and HDDs fail, but for magnetic platter media even if the head or motor crashes there’s always the old magnetic microscope in a pinch. USB’s not going anywhere, and if you have four or five copies that you don’t completely neglect and don’t store in the same physical place, presumably you’ll have the chance to notice and take corrective measures if any of them start failing or are at risk.
I don’t actually know that an individual book or picture will still be around in 50 years; Fire, flooding, insects, acidic paper, low-quality ink maybe— Digital stuff’s fragile, but so is physical stuff. Stick it in the attic, and the heat’ll speed up any chemical reactions and probably make it cozier for insects; Stick it in the basement, and the condensation will get you mildew and rot. By contrast, having a flash drive accidentally survive a trip through a washer and dryer is a pretty common occurrence, and I’ve yet to lose a drive even with that level of negligence. Material compatibility’s one of the very most basic parts of a set of very precise manufacturing techniques, tin whiskers seem pretty rare these days, the really scarily insidious stuff like hydrogen embrittlement is super improbable, and most biological forms of decay haven’t adapted to eating cured epoxy and monocrystalline silicon yet.
At least I sorta know how a flash cell or hard drive platter is meant to be structured; Who knows what weird organic reactions and unstable or slowly diffusing molecules are happening in the pile of chemical pigments on a sheet of likely-acidic bleached cellulose and cheap ink or toner, and whether it will still be legible to human eyes in however many years? Plus, a printed photo or document starts fading the very instant it’s created, and it gets a little worse every time you touch it with sweaty human hands or look at it while exhaling moist human breath and corrosive enzymatic saliva droplets under a white LED lamp or G-type star shooting out ionizing UV rays. Digital failures tend to be catastrophic, but at least up until the moment it fails, you can make sure that it is the exact same picture or text— And you can make many, many copies very cheaply, all of them very physically durable compared to paper, and know that they are all the exact same picture and text.
That said, I absolutely agree with your overall assessment that most of the information in the early 21st century, including most of the public Internet/WWW, most likely either will be or already is… Maybe not technically lost, per se, given how much caching and saving happens on private clients, but certainly rendered inaccessible.
Ideally I’d really love to see a return of microfiche, actually, using modern polymers and metallization. I’ve been meaning to look into that for a while now. At a reasonable scale for optical viewing, you could fit… much, much more content than you might expect, and do it several times over, in an entirely reasonable number of pages. Your comment actually spurred me to finally think of a practical way of printing that— for years before, I’d been trying to idly figure out a process based on photomasks and nanoparticles suspended in resin, which had always felt like a very messy and tricky idea, but I just thought of another idea– So thanks for providing some inspiration there.
This is a hot take
Even data from the Apollo missions was found to be either degrading (tapes) or the formats were forgotten and the systems that could read them were gone. They had to do research into rediscovering how to read the data and hunt around for the few antique systems remaining to read the tapes.
https://en.wikipedia.org/wiki/Digital_dark_age
If there is a will to maintain it, it will endure. If there is no will, should it endure?
There is a will but there aren’t enough people with enough brain power to actually do the steps needed. Should it endure? I don’t know, maybe the last few decades should be forgotten.
Mistakes are usually the stuff that should be remembered
Fair
Crazy how we are now talking about losing everything from the past few decades of human history because of a story about videographers losing their video data on a quality control issue.
Crazy how a single event sometimes reminds people of bigger problems, huh?
Digital media, where we store basically everything we care about, is hugely, hugely volatile, unreliable, and fragile. But you never notice it until you’re reminded of it, and then you really notice it. This story reminded people of it.
The reminder to stay grounded is probably also healthy, but I do think you’re missing the point of this comment thread.
The Apollo mission data and BBC TV recordings weren’t considered important enough at the time to preserve them, it wasn’t until decades later that people realized they were but by then the BBC had destroyed or overwritten much of them and NASA had forgotten how to read much of the data. Then there was the notorious loss of many master recordings by great artists in a fire because the company was just too cheap and lazy to store them properly.
PCM, ASCII, and straight RGBA bitmap encodings aren’t going anywhere. By extension, derived formats like WAV, UTF-8, and word processor files and webpage HTML are mostly fine too. The formats are structurally simple enough that even if the associated file extensions were somehow to be forgotten, all you’d need to do to invent them again is hand the file to a bored nerd over the weekend.
I think you kinda got the BBC and NASA problems backwards. The BBC’s had a couple of prominent incidents where digital “preservation” that was supposed to be eternal couldn’t even be opened anymore after a couple of years, like their Domesday Book/Project application thingy. They’ve also lost a bunch of old shows, like early Dr. Who episodes, I think. NASA didn’t just forget how to read the Apollo tapes; they overwrote them to reuse the tapes, as was their standard practice at the time. The original signal and tapes were very HD (or analog), but most of the videos we have today are from the TV camera that they pointed at their own TV screen last-minute when they realized they didn’t have an adapter for broadcast— The equivalent of a grainy cell phone photo of a screenshot, basically.
The BBC and NASA incidents happened in an era before computers were a ubiquitous commodity product. So, everyone and their cat was basically inventing their own obscure single-implemention proprietary file formats at that time. Nowadays we have established technical standards, as well as formats that have already sorta stood the test of time based on their utility and simplicity— and millions of people who already know how to read them— so that particular vector for bitrot isn’t really as much of an issue anymore.
…That said, I think I sorta missed your point. What you’re really saying is that stewardship of digital records is much trickier and riskier than stewardship of physical records— and that results in stuff being lost. And that is absolutely true.