I had the same issues as you.
I went for a 432 (44mm Luminor) I could score pretty cheap and I like it. Altough it is pretty hefty on my wrist and I can't wear it with most of my shirts, no matter if french cuff or normal cuffs it is still one of my favorite pieces for casual wear.
After some time I also scored a 338 (42mm Radiomir) and man I am in love. Snugs perfectly on my wrist, the dial is the same size as my 44mm 432, it fits under all cuffs and suits, just perfect.
Now for recommodations:
Basically you have to choose between five options:
1.) Basic Luminors (40 and 44mm) The standard case, IMHO compared to the 44mm the 40mm look dispropotionate due to having the same height.
2.) New 42mm acciaio Luminor. super slim.
3.) 1950 Case Luminor comes in 42mm, slim.
4.) Radiomir wire lug case: Comes in 42mm, super slim.
5.) Radiomir 1940 case: comes in 42mm, slim.
6.) Submersible: comes in 42mm, looks bulky IMHO.
If I could get a third PAM I would opt for a 42mm Luminor with the new movement (slim case). I like the 676, but the sunburst dial kills it for me. The 392 could be an option, but it is a bit taller due to the different case.
I personally like the 1950 case, but unfortunately it does not come in 44mm.