using superglue would probably be your best defense imo, only drawback is that moving the pins would be more difficult but you could also crack the superglue with some patience
edit: with patches then i agree with saying it'd be best to sew them on, even iron-on patches are known to fall off/loosen eventually