Dear Sai,
Greetings!
We cannot access any directory directly from spark shell.
Spark shell is only used for writing the codes and executing it.
You do not have to go to Kafka directory to run it.
For Kafka:
For Kafka we have to use Kafka commands, there is no interactive shell.
From the Cloudera manager, you can check the active Kafka brokers and zookeeper servers.
Zookeeper IP's
Check the below examples for listing topics creating topics, start console producer and consumer etc.
For listing topics
kafka-topics --list --zookeeper ip-20-0-21-161.ec2.internal:2181
Please check below commands, same is working and is able to produce and consume the message
Creating Topic:
kafka-topics --create --zookeeper ip-20-0-21-161.ec2.internal:2181 --replication-factor 1 --partitions 1 --topic lovelyb1
Producer
kafka-console-producer --broker-list ip-20-0-31-210.ec2.internal:9092 --topic lovelyb1
Consumer
kafka-console-consumer --zookeeper ip-20-0-21-161.ec2.internal:2181 --topic lovelyb1 --from-beginning
producer
consumer
Hope this helps.
Please let us know if you have any query.