pyspark issue in Cloudera quickstart VM | env: /usr/lib/lib/python3.5 permission denied
Modified on: Tue, 24 May, 2016 at 12:33 PM
By Default the cloudera quickstartvm has python 2.6.6 however that is an old version so for our project requirement we need python 3.5.
I downloaded python 3.5 and did -- ./configure, make, make install. Now python is installed in usr/lib/lib/python3.5.
changed the spark-env.sh --
When i try to start pyspark with python3.5 I get error
env: /usr/lib/lib/python3.5 permission denied
I tried chmod 755 also chmod 777 for all the folder path /usr/lib/lib/python3.5 it does not work I still see same error.
This is very urgent as I have to do a POC. Immediate help needed.
Resolved --> The pyspark_python path should be pointing to pyhton executable.
export PYSPARK_PYTHON=/usr/lib/bin/python3.5 in spark-env.sh resolved the issue.
Did you find it helpful?
Sorry we couldn't be helpful. Help us improve this article with your feedback.