TIME_WAIT is defined by /proc/sys/net/ipv4/tcp_fin_timeout. Default is 60 seconds.
set your time wait lower: 'sysctl -w net.ipv4.tcp_fin_timeout=30'
It is normal to have a socket in the TIME_WAIT state for a long period of time. The time is specified in RFC793 as twice the Maximum Segment Lifetime (MSL). MSL is specified to be 2 minutes. So, a socket could be in a TIME_WAIT state for as long as 4 minutes. Some systems implement different values (less than 2 minutes) for the MSL.