Query: 

By Default the cloudera quickstartvm has python 2.6.6 however that is an old version so for our project requirement we need python 3.5.

I downloaded python 3.5 and did -- ./configure, make, make install. Now python is installed in usr/lib/lib/python3.5.

changed the spark-env.sh --

PYSPARK_PYTHON=/usr/lib/lib/python3.5

When i try to start pyspark with python3.5 I get error 

env: /usr/lib/lib/python3.5 permission denied

I tried chmod 755 also chmod 777 for all the folder path /usr/lib/lib/python3.5 it does not work I still see same error.

This is very urgent as I have to do a POC. Immediate help needed.


Resolved -->  The pyspark_python path should be pointing to pyhton executable.
export PYSPARK_PYTHON=/usr/lib/bin/python3.5 in spark-env.sh resolved the issue.

203035