calculate the transmission coefficient