discussion Legacy Attachment System Corruption – Partial Recovery and Archive Release

[Ttian]

Can't really think of a single forum that has gone with no drama. It just seems inevitable eventually.

 

[BgileMunro]

I am also mainly interested in the latest updates page. If that can be put back to "normal", I'm gonna be pretty happy, even if you have to start over with little or no history.

 

[fidgetspinner]

Can't really think of a single forum that has gone with no drama. It just seems inevitable eventually.

Not the point. The whole forced thread thing is just annoying. Admin can figure this shit out without always forcing us in the loop.

 

[Flamypaisen]

Sheesh what bad luck huh,
Thank you for your efforts and good luck with the hard work it will take to fix this.

 

[Jack Of Blades]

Not the point. The whole forced thread thing is just annoying. Admin can figure this shit out without always forcing us in the loop.

and if we keep public out of the loop and decide to nuke the forum to remake it the community is mad.

 

[Ttian]

and if we keep public out of the loop and decide to nuke the forum to remake it the community is mad.

No matter what, you can never keep the general public happy lol.

 

[Joquer]

what do you mean by (Is there a donation station planned? Or perhaps a better, easier to manage option- those of us so willing and able inventory our goodies, gawdies, and lawdies-help-me-jeebus! in a format easily interpreted and accessible to All Ye Lewd Powers That Be, and you let us know, ahem... When & Where to Stick It!)

I have maybe 3 or so terabytes of okay to magnificent quality video, art, comics, AVNs, games, etc., in a release date-range heavily referencing 2019 to current, but some dating back to 2010 or so, and all of it fits into LewdCorner's content criteria and would be a boost to your rebuilding efforts. I am certain others on the site have similar repositories of perverted wealth and they would be willing to donate for the greater-good. This would allow the site to get back on it feet quicker, provide the much-desired content causing so much restlessness amongst the natives, and, within the content, perhaps even introduce work delicious enough to make even the saltinest naysayer forget there ever was a server issue.

By 'Donation Station', I mean an internet address where such files as described above could be uploaded in an efficient manner to be scanned and reviewed for appropriateness of use by the site.
By the remainder of the your quoted bit I posit: If the LC Administration feels that accepting a large amount of content from its community up front for the purpose of 'restocking' various sections with content that will be 'refreshed', if not previously unseen on the site prior, is an idea worthy of investigating, I am saying:
Perhaps having an en masse upload fest that overwhelms the LC Team and creates more work than benefit may not be the best course. A better way might be for the LC Team or Member to develop a process for users to submit samples of the work they have to donate,, which the LC Team would review, decide which content they would like to accept, then respond to user with an address and instructions for uploading said content.

For example: I have a nice collection of Magistrate's faux-mag. I would create a contact sheet of the collection and as well as other pieces, provide atrribution and brief description for each, combine all into a single pdf, and submit it to the designated LC account.
At some point in the future II would recieve a response telling me which content was wanted, with instructions telling me where to upload (i.e., ('Where to stick it') and how to name each file, etc.. I send a response back once I have completed this, a staff member does a content to review ensuring the instructions were followed and the content is what was described, and it goes live.

In the above scenario, the 'heavy lifting' of backfilling LC's stock is spread across a willing membership, allowing your Team to focus your rebuild efforts. The individuals donating, essentially provide you with menus of content, you order the pieces you want, tell us where you want it delivered, and how you want it placed, we do it, you confirm it, and set it live at your convenience.

eZpZ, no?

 

[fidgetspinner]

and if we keep public out of the loop and decide to nuke the forum to remake it the community is mad.

Have you considered not nuking it? or... you know... a banner announcement would suffice. no need to force thread every damn day.

 

[Joquer]

I see where you are going, but that's not really how database formats usually work, to be able to open the db, and being able to still read it. In 99% of cases where an SQL database gets messed up by actual corruption and not the fuck up of the one writing the SQL statement, you will have either full corruption, like full pages being corrupted, data being partially written or truncated, or the DB format being inconsistent, and before getting actually corrupt data you will likely get checksum mismatches and index errors. To have the bits of the binary realign itself that perfectly, is extremely unlikely, possible, but I would say it's probably way more likely that I get hit by lightning in the next 10 minutes, while sitting in my room in the sun. It is much more likely that the whole format file is messed up, not just the rows having switched ids or hashes, and you can't read much data fully anymore. So in every normal case the underlying hashed data changed (the attachments), or settings changed and a different hash is now used. Like I mean it, the chances for this happening with any modern engine are almost zero, even more so if they used common filesystems, since almost all of them split up their files physically on the storage in some way.
They can have index corruption, like secondary index corruption behaving like switched data, but the file bits will not randomly have changed, and these cases you can rebuild the index, and in primary index corruption you would in most cases not even get much data back anymore anyway, since where the data lives is not actually known reliably anymore. For this to happen it would require page level corruption (modern DBs usually store their data in pages with a fixed size), where all row and field metadata (headers, offsets, pointers and indices) are still valid and untouched, but the actual data bits and bytes are swapped, this is literally like winning the Roulette table 4 times in a row while betting on 2.

I believe you are taking my examples too literally. It was not my intent to explain perfectly what happened, but rather to provide examples of how it could happen. Without a doubt you understand systems much better than I do, and I am unable to engage the conversation at that level.

I believe you might be focusing on the wrong part of this particular equation, however. It seems as though you are holding what was described about the database as separate and exclusive of devastation of the server crash. I say this because your description above and in your original response reference how the db works under normal conditions, and corruption.

What I was attempting to show with the two examples offered was A) how relative data can become disassociated, and 2) how critical failures, even on a minute scale such as sudden power up and down of a car stereo system can unexpected and unpredictable results. Combining the two examples with the context provided at the top of the thread I believe answers your question. Of course, assuming my interpretation of the original event is correct, i.e., that db corruption had little to do with the server implosion, but the server implosion played 'Johnny Appleseed' with the db.

Another bit of information we interpreted differently was what was meant with regard to the hash files. What I thought was communicated was what I described- that there was a table in the database that a field storing a copy of a file's hash and another field in the db that linked to that file. In my interpretation the hash is simply another attribute describing the file, and the file does not physically exist in the db. In this context, the file's defined fingerprint has not changed, What has changed is the transcribed copy of that fingerprint may now exist in a different row, or even ddifferent table entirely, within the remnants of a once proud db while the file it describes rests serenely in folder far away from the destruction, or perhaps it, too, is scattered to across multiple other files and lost forever.

I fully accept the fact that I may be wrong about the perspectives and context of your interpretation, and that those i have taken are incorrect. I simply wanted to define and clarify the intent of my response and square what I was hearing from you so you would understand where I was coming from. I will gladly read your response, but this is the sum of what I have to share on this topic.

 

[Jack Of Blades]

Have you considered not nuking it? or... you know... a banner announcement would suffice. no need to force thread every damn day.

tried coming up with ideas, we are leaning on archiving lewdcorner ie archive.lewdcorner.com, there is no other option the entire xenforo system is fucked and Haywoodspartan even looked into all of it.

 

[Nicest_Incest]

It hasn't though and mime has tried her hardest to get more members interested.

I'll miss all those A.I videos though may they rest in peace.

Maybe when the dust settles after the server upgrade and shit is fixed I'll up what I grabbed from ATF over the past couple years (mostly just clips, nowhere near everything but just what I found that made me nut, maybe 100 or so, but are mostly VAM with a few AI).

 

[Mr.LewdRim]

tried coming up with ideas, we are leaning on archiving lewdcorner ie archive.lewdcorner.com, there is no other option the entire xenforo system is fucked and Haywoodspartan even looked into all of it.

It seems the backend is that damaged then its the only solution to moving forward. Enough people will post. Ran enough forums for 15+ years things happen people will cry ultimately if people want the community it will survive either way.

 

[Phoenix2011]

Sounds like you got dealt a bad hand and played it the best you could. I'm sure people at least appreciate your honesty and openness.

 

[Jack Of Blades]

@Jack Of Blades; I do hope that you have user security at least partially in mind… any uploaded archive needs to be stripped of user data and DM’s.

I would prefer that whatever is lost is just lost, most of us don’t need a readymade file of whatever we did here available to whomever wants to go digging.

Pulling only the images and running them through a second scrubbing to ensure a lack of metadata is the only appropriate way to archive the loss. Or when the new site is ready move this to archive.lc.whatever with all the content as is but read only, thus you can only see what you could anyway.

it wouldnt be a downloadable file, the site would be archive.lewdcorner.com, it would be LC but frozen in time.

 

[Jack Of Blades]

Everything still behind a login I hope.

yes lol

 

[Ttian]

it wouldnt be a downloadable file, the site would be archive.lewdcorner.com, it would be LC but frozen in time.

If its preserved, it won't be forgotten. Thanks for the effort!

 

[Leuchtmittel]

it wouldnt be a downloadable file, the site would be archive.lewdcorner.com, it would be LC but frozen in time.

I thought the links for the attachments are lost, so the archive site could still not show these images/animations which are broken now.
What's the point of the archive site then?

 

[Jack Of Blades]

I thought the links for the attachments are lost, so the archive site could still not show these images/animations which are broken now.
What's the point of the archive site then?

to keep an archive of what was posted so users can restore what can be salvaged, we need to build a new forum cause our current xenforo build is corrupted.

 

[nydd]

... as Little Mac has shown time and time again he has no recovery.

A sad state of affairs indeed.
Seriously though, I always choose stability. When bugs creep up the madness caused can be endless. Go for a stable route is my call.

 

[Uh nona miss]

As long as we can still be degens i think we're fine as long as in the end we can get our stories