Have you tried a 1/4 wave vertical with 4 radials sloping towards the ground? (all wire)? or even a vertical dipole fed halfway up? J-poles show significant coupling to their feedline and mounting arrangements and, as you discovered, are taller than they need to be for their gain.
You don’t have to have conversations with those people - start your own net on a repeater about gardening or robotics or astronomy, find other local people you’d like to work simplex with, ask POTA or SOTA contacts if you can come along next time.
You can change the hobby for the better, if you actively participate.