gettyimages/docker-spark
gettyimages/docker-spark has 667 GitHub stars and 358 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-15 22:35 UTC. Last push 2021-12-30.
Measured star growth
- in 24h
- 0
- in 7 days
- Update delayed
- in 15 days
- Update delayed
- in 1 month
- Update delayed
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
Docker build for Apache Spark
Developer ToolsREADME.md
Originalspark
A debian:stretch based Spark container. Use it in a standalone cluster with the accompanying docker-compose.yml, or as a base for more complex recipes.
docker example
To run SparkPi, run the image with Docker:
docker run --rm -it -p 4040:4040 gettyimages/spark bin/run-example SparkPi 10
To start spark-shell with your AWS credentials:
docker run --rm -it -e "AWS_ACCESS_KEY_ID=YOURKEY" -e "AWS_SECRET_ACCESS_KEY=YOURSECRET" -p 4040:4040 gettyimages/spark bin/spark-shell
To do a thing with Pyspark
echo -e "import pyspark\n\nprint(pyspark.SparkContext().parallelize(range(0, 10)).count())" > count.py
docker run --rm -it -p 4040:4040 -v $(pwd)/count.py:/count.py gettyimages/spark bin/spark-submit /count.py
docker-compose example
To create a simplistic standalone cluster with docker-compose:
docker-compose up
The SparkUI will be running at http://${YOUR_DOCKER_HOST}:8080 with one worker listed. To run pyspark, exec into a container:
docker exec -it docker-spark_master_1 /bin/bash
bin/pyspark
To run SparkPi, exec into a container:
docker exec -it docker-spark_master_1 /bin/bash
bin/run-example SparkPi 10
license
MIT