I find frequently that the Remote Desktop Agent will stop responding – but I can ssh to the machine and if I use the gui to restart the ARD agent I can access it again. To restart it through ssh type the following:
/System/Library/StartupItems/RemoteDesktopAgent/RemoteDesktopAgent restart
Source:
Apple – Support – Discussions – Reset (stop and start) Apple Remote …

