Last night, Lemmygrad went down due to a hardware glitch on our VPS provider. Unfortunately, they were unable to do anything to help me fix the issue, and asked that I reinstall from scratch 🙁 . They did inform me that the hardware issue at fault is fixed at least.

I do daily backups, so we lost a day of posts.

Pictures are in the process of being restored now, but it should take a day or so to restore them all.

Let me know if there are any other ongoing issues so I can fix them. Again, sorry everyone.

  • Capitalist Tears
    link
    fedilink
    arrow-up
    12
    ·
    2 years ago

    Being involved in devops, I can relate to how much this sucks and I appreciate you taking the time to fix it. o7

    • Muad'DibberOPMA
      link
      fedilink
      arrow-up
      8
      ·
      2 years ago

      Thanks comrade, ya these can be nightmare scenarios. I at least have several backup locations, so at most we lose a day’s worth of data.

    • panic
      link
      fedilink
      arrow-up
      6
      ·
      2 years ago

      I’m not involved in this type of work but I would just pull the plug, I can’t understand the amount of patience you need for this shit

      • Muad'DibberOPMA
        link
        fedilink
        arrow-up
        3
        ·
        2 years ago

        Everything will be going fine for weeks, and then: oopsie poopsie everything’s gone!

    • Capitalist Tears
      link
      fedilink
      arrow-up
      5
      ·
      2 years ago

      Anecdotally, my instances have poofed into thin air even on AWS, so this seems not too rare. You use a managed database or something you’ve setup yourself?

      • Muad'DibberOPMA
        link
        fedilink
        arrow-up
        4
        ·
        2 years ago

        Set up ourselves. Its postgres running in a docker container, with daily backups.

  • panic
    link
    fedilink
    arrow-up
    10
    ·
    2 years ago

    The only explanation is that someone posted cringe so big they went to mess with the VPS. Let’s act like it never happened but this better be the last time /s

  • Leninismydad
    link
    fedilink
    arrow-up
    9
    ·
    2 years ago

    Honestly how dare you 😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾😾🤖😾😾😾😾😾😾😾😾😾😾

  • CriticalResist8A
    link
    fedilink
    arrow-up
    8
    ·
    2 years ago

    No worries, these things happen. Look on the bright side, thankfully it was just one day loss and not more.

  • panic
    link
    fedilink
    arrow-up
    8
    ·
    2 years ago

    we lost a day of posts

    Spongebob_suffering.jpeg

  • chinawatcherwatcher
    link
    fedilink
    arrow-up
    7
    ·
    2 years ago

    nice to get an update, i was confused yesterday. every time i consistently can’t connect to lemmygrad i assume the worst hahaha

  • whoami
    link
    fedilink
    arrow-up
    6
    ·
    2 years ago

    thanks for your hardwork. I was concerned there was some type of ddos or something

  • fruityloop
    link
    fedilink
    arrow-up
    5
    ·
    2 years ago

    I was worried that the site got nuked for good because the outage lasted this long, very glad to know it’s not the case! I can imagine the horror you felt when this happened, data loss is a nightmare (I have to deal with this stuff at work). rip to the posts that got lost in the ether but you did what you could.