Projects
View on GitHub
amplab/keystone
amplab/keystone has 473 GitHub stars and 117 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-21 17:43 UTC. Last push 2017-04-18.
Measured star growth
- in 24h
- 0
- in 7 days
- Collecting history
- in 15 days
- Collecting history
- in 1 month
- Collecting history
Net star changes over overlapping periods, not daily totals. Missing measurements are left blank. Stars measure attention, not software quality.
About
Simplifying robust end-to-end machine learning on Apache Spark.
Machine LearningREADME.md
OriginalKeystoneML
The biggest, baddest pipelines around.
Example pipeline
Build the KeystoneML project
./sbt/sbt assembly
make # This builds the native libraries used in KeystoneML
Example: MNIST pipeline
# Get the data from S3
wget http://mnist-data.s3.amazonaws.com/train-mnist-dense-with-labels.data
wget http://mnist-data.s3.amazonaws.com/test-mnist-dense-with-labels.data
KEYSTONE_MEM=4g ./bin/run-pipeline.sh \
keystoneml.pipelines.images.mnist.MnistRandomFFT \
--trainLocation ./train-mnist-dense-with-labels.data \
--testLocation ./test-mnist-dense-with-labels.data \
--numFFTs 4 \
--blockSize 2048
Running with spark-submit
To run KeystoneML pipelines on large datasets you will need a Spark cluster. KeystoneML pipelines run on the cluster using spark-submit.
You need to export SPARK_HOME to run KeystoneML using spark-submit. Having done
that you can similarly use run-pipeline.sh to launch your pipeline.
export SPARK_HOME=~/spark-1.3.1-bin-cdh4 # should match the version keystone is built with
KEYSTONE_MEM=4g ./bin/run-pipeline.sh \
keystoneml.pipelines.images.mnist.MnistRandomFFT \
--trainLocation ./train-mnist-dense-with-labels.data \
--testLocation ./test-mnist-dense-with-labels.data \
--numFFTs 4 \
--blockSize 2048
Explore related projects
Loading…