Not on the lease

Yellow jackets (predatory wasps in the genus Vespula, don't know the exact name) have been flying into the wall outside my window.
Last night at the insect lecture, someone said yellow jackets get aggressive in the fall and there's nothing to do but spray them.
This family is outside, and I never open the window, so I will let them be. I hope they don't come through the wall!