Switches do not yield any security improvements.
First, whenever a switch doesn't know where a packet should go (ie, the MAC address isn't inside the switch's cache) it will broadcast the packet to all the interfaces. This could be the reason you're seeing connection attempts to other IP addresses -- if you see SYN packets but no later packets, odds are that when the SYN packets arrive the switch doesn't know where to send them, but thereafter it is sending packets to the correct destination.
Second, almost all switches can be spoofed. If another machine on your network claims to have your MAC address, a switch may send packets to you, may send them to the other machine, and may send them to both machines.
Third, almost all switches can be forced into hub mode. Because of the limited size of the MAC caches, if a switch is flooded with random MAC entries, it will not know where to send *any* packets, and so it will revert to sending all packets to all ports -- ie, what a hub normally does.
Basically, switches are useful for performance reasons, but not for security reasons.