No, that’s exactly this, i kinda do feel sorry for buddhists (and hinduists and euro pagans and every other legit users of swastika) getting dragged in that crap, but this had become a meme decades ago and we STILL have nazis around who will unironically use this excuse. Note that followers of those religions in the euro culture circles tend to either completely avoid swastikas by now or use very stylized versions.