FYI for anyone running into java.io.FileNotFoundException: File file:/tmp/hadoop-USER/mapred/system/JOBID/jobToken does not exist. when running Hadoop MapReduce jobs.
Your JobTracker needs access to an HDFS NameNode to share information with TaskTrackers. If the JobTracker is misconfigured and cannot connect to a NameNode it will write to local disk instead. Or at least it will in hadoop-1.0.1. The result is TaskTrackers fail to find the jobToken in HDFS, or on their local disks, and will throw a FileNotFoundException.
I ran into this because I did not have a fs.default.name property in my JobTracker’s mapred-site.xml. I was putting it in hdfs-site.xml instead.
I have not found messages in the JobTracker (or TaskTracker) logs that explicitly indicate this misconfiguration.