Home / ubuntu / Questions / 495898
Accepted
A J
Asked: 2014-07-10 21:28:50 +0800 CST

Problem to copy a file from HDFS to local system [copyToLocal: Permission denied]

  • 772

I have installed and executed an mapreduce program successfully in my system(Ubuntu 14.04).

I can see the output file as,

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -ls /user/hadoopuser/MapReduceSample-output
Found 3 items
-rw-r--r--   1 hadoopuser supergroup          0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_SUCCESS
drwxr-xr-x   - hadoopuser supergroup          0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_logs
-rw-r--r--   1 hadoopuser supergroup     880838 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/part-00000

And I can open it on terminal using following command,

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -cat /user/hadoopuser/MapReduceSample-output/part-00000

I can see the output file on terminal, but I can't see the full result because my output has large amount of lines.

UPDATION

I have tried to copy the output file into local machine,

hadoopuser@avvenire-PC:/usr/local/hadoop$ bin/hadoop dfs -copyToLocal  /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/Downloads/SampleDataHadoop

I got response as,

copyToLocal: Permission denied

Expecting solution.

14.04

2 Answers

  1. Best Answer

    The commands you use are executed as the normal user. And the location you copy to is NOT owned by your user.

    So the answer is to do this as the administrator: put sudo in front of the command. ie:

    hadoopuser@arul-PC:/usr/local/hadoop$ sudo bin/hadoop dfs -copyToLocal 
/user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/Downloads/SampleDataHadoop/
    • 0

  2. Even if you use sudo at the begining of your copyToLocal command, you still need to mind that if your file in HDFS is stored somewhere in a user-owned directory you need to be that user to copy it.

    For example, if your HDFS file is in /user/root/.... your user after sudo should be root - if it is /user/keanureeves your user should start with that user.

    Here's a sample to have a vision on it:

     sudo -u root hdfs dfs -copyToLocal  /user/root/XXXX/20210921/* /home/YYYY/

    (I was using wildcard so if you need the directory only, remove that)

    good luck

    • 0