Dear Sai,

Greetings!

We cannot access any directory directly from spark shell.

Spark shell is only used for writing the codes and executing it.

You do not have to go to Kafka directory to run it.


For Kafka:


For Kafka we have to use Kafka commands, there is no interactive shell.

From the Cloudera manager, you can check the active Kafka brokers and zookeeper servers.

 
image

Zookeeper IP's

image 
Check the below examples for listing topics creating topics, start console producer and consumer etc.

 
For listing topics

kafka-topics --list --zookeeper  ip-20-0-21-161.ec2.internal:2181

 
image

 
image


Please check below commands, same is working and is able to produce and consume the message


Creating Topic:

kafka-topics --create --zookeeper ip-20-0-21-161.ec2.internal:2181 --replication-factor 1 --partitions 1 --topic lovelyb1

Producer

kafka-console-producer --broker-list ip-20-0-31-210.ec2.internal:9092 --topic lovelyb1

Consumer

kafka-console-consumer --zookeeper ip-20-0-21-161.ec2.internal:2181 --topic lovelyb1 --from-beginning

image

producer
 

image
 
consumer

image
 
Hope this helps.

Please let us know if you have any query.