A slightly fitted swim trunk is a good choice (even so, be careful that it isn't tight). You don't want anything that is too long or too baggy. Longer, to-the-knee board shorts are out of style, except for real surfers.
It should go without saying, but swim shorts and shorts are not interchangeable. Swim trunks are similar to shorts worn as clothing on land, but are made from light, fast-drying materials (usually nylon or polyester, sometimes all cotton) and feature a tighter-fitting lining inside the shorts. Colors, patterns and inseam lengths can vary widely.
You should not just grab whatever style you see first. Men are likely to have only one or two swim shorts in their wardrobe at a time, so it's worth finding ones you really like and feel good in. When shopping, look for a trimmer cut with either a fixed waist or a drawstring waist. You don't need an absolutely perfect tailored fit, as long as the suit stays up when you dive in (most suits with a cord at the waist help with that).
As to colors and patterns, think stripes, plaids, bright solids or some not-too-crazy floral patterns. Avoid busy, printed, colorful, matched outfits, as they are not the best choice if you want to look stylish. Stay with a simple T-shirt top.
Also, consider shorts with a wide waistband and adjustable tabs, allowing you to optically flatten your torso.
This is a chance for you to have fun. Pools and beaches are among the few places where well-dressed men can still look sharp while wearing bold graphics and bright colors. Though you may not want to wear a flower-printed blazer, at the beach you can get away with a lot. Still, you mentioned a possible "company" pool party. That changes things. A business context calls for a quiet approach ... even at the pool. Don't go wild with attention-grabbing prints and too-clever message T-shirts. When you're trying to look professional, a conservative (not exaggerated) length, conventional styles and dark solid colors or restrained stripes are for you. For example, navy blue is always safe.