Aesthetics of the watch play a big factor as well. Colors and case material can make a large difference in size appearance.
Generally speaking I prefer the 42mm. But I also just bought a 35mm JLC master ultra thin, so I like watches to look proportionally right on my wrist. With 44mm you have RBII, novelties, gp? and schumi. All of which I think look different sizes on the wrist, with RBII looking smallest out of the bunch which is why I prefer it. But I'm also considering a novelty 44mm because of the style.
Basically, don't get pushed around by size too much, if a watch really speaks to you buy it, You'll be happy. If it feels to big, it'll sell easy as AP's are poplar here. I was afraid ROO's would be too big when, I then bought one and ended up with 6 or so at one time, you get used to it [emoji106]