Install Scala, Apache Spark was written in Scala. I am unsure if this is needed but I installed it anyways.
brew install scala
Install Apache Spark
brew install apache-spark
To run Spark shell.
# Scala Spark-shell # Python pyspark
Adding some paths for shell to ease usage. I use OhMyZsh follow your shells directives.
export SPARK_VERSION=3.0.1 export SPARK_HOME="/usr/local/Cellar/apache-spark/$SPARK_VERSION/libexec" export PYTHONPATH="/usr/local/Cellar/apache-spark/$SPARK_VERSION/libexec/python/:$PYTHONP$" alias sparkpath="cd $SPARK_HOME/sbin"
We did some of this to make it easier to run some of Sparks dashboards.
In one terminal run Spark in another run:
You should now be able to bring up
http://localhost:8080 and http://localhost:4040
This should get you on your way to learning and playing with Spark.