My FN shoots the same way, consequently I tend to shoot lower groups with it.
It is a 40S&W, so for a long time I thought I was flinching downward anticipating the “snappy” recoil, but I think this explains the low groups with particular gun. I loaned it to a buddy that mainly shoots a Barretta M9, he loved it. I loaned him another gun that has the 6 o’clock sight picture, he couldn’t hit the broad side of a barn! Note: loaning was in the lane beside me at the shooting range.