One you have tagged the man with Eagle Vision head over to the Southern side of the street by the stairs and wait him go East. He will follow the street a good while, going up the slope ahead of him. You can use the crowd for blending. If there is not a Convoy coming then you can duck into a Stalking Zone just off where he is walking. If there is a Convoy coming then head to the southern side of the nearby buildings and head East from there.
The "gentleman" will continue East until he comes across a wooden bridge across a shallow stream. He will then turn South and continue on. You can easily keep up with him by ducking into the Stalking Zone nearby the stream. Wait for him to cross then go yourself. Use the Cattails as Stalking Zones to avoid being seen. The l;arge house ahead is the brothel you are looking for. Take up a corner position and watch him go inside.