mcanini/SysML-reading-list
mcanini/SysML-reading-list has 288 GitHub stars and 40 forks. GitHot measured 0 stars in 24 hours. Last checked 2026-11-20 01:56 UTC. Last push 2025-06-09.
Crecimiento de estrellas medido
- in 24h
- 0
- in 7 days
- Recopilando historial
- in 15 days
- Recopilando historial
- in 1 month
- Recopilando historial
Cambios netos en períodos que se solapan, no totales diarios. Los datos no disponibles se dejan en blanco. Las estrellas indican interés, no calidad.
Acerca de
Systems for ML/AI & ML/AI for Systems paper reading list: A curated reading list of computer science research for work at the intersection of machine learning and systems. PR are welcome.
Aprendizaje automáticoREADME.md
OriginalMarco's SysML reading list
A curated reading list of computer science research for work at the intersection of machine learning and systems. PR are welcome.
Review
A Berkeley View of Systems Challenges for AI https://arxiv.org/pdf/1712.05855.pdf
Strategies and Principles of Distributed Machine Learning on Big Data https://arxiv.org/abs/1512.09295
Background
Deep learning Nature volume 521, 2015 https://www.nature.com/articles/nature14539
Measurement
Multi-tenant GPU Clusters for Deep LearningWorkloads: Analysis and Implications https://www.microsoft.com/en-us/research/uploads/prod/2018/05/gpu_sched_tr.pdf
Frameworks
PyTorch: An Imperative Style, High-Performance Deep Learning Library NeurIPS 2019 https://arxiv.org/pdf/1912.01703
TensorFlow: A System for Large-Scale Machine Learning OSDI 2016 https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf
Ray: A Distributed Framework for Emerging AI Applications OSDI 2018 https://www.usenix.org/system/files/osdi18-moritz.pdf
Tuning
HyperDrive: Exploring Hyperparameters with POP Scheduling MiddleWare 2017 https://dl.acm.org/citation.cfm?id=3135994
Ease.ml: Towards Multi-tenant Resource Sharing for Machine Learning Workloads VLDB 2018 http://www.vldb.org/pvldb/vol11/p607-li.pdf
Automating Model Search for Large Scale Machine Learning SoCC 2015 http://dl.acm.org/authorize?N91362
Google Vizier: A Service for Black-Box Optimization KDD 2017 https://dl.acm.org/citation.cfm?id=3098043
Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization Journal of Machine Learning Research 18 (2018) https://arxiv.org/pdf/1603.06560.pdf
Hyperopt: a Python library for model selection and hyperparameter optimization Computational Science & Discovery, 8(1) 2015 http://iopscience.iop.org/article/10.1088/1749-4699/8/1/014008
Auto-Keras: Efficient Neural Architecture Search with Network Morphism https://arxiv.org/pdf/1806.10282v2.pdf
Runtime execution
Cavs: An Efficient Runtime System for Dynamic Neural Networks ATC 2018 https://www.usenix.org/system/files/conference/atc18/atc18-xu-shizhen.pdf
TVM: An Automated End-to-End Optimizing Compiler for Deep Learning OSDI 2018 https://www.usenix.org/system/files/osdi18-chen.pdf
PipeDream: Fast and Efficient Pipeline Parallel DNN Training https://arxiv.org/pdf/1806.03377.pdf
STRADS: A Distributed Framework for Scheduled Model Parallel Machine Learning EuroSys 2016 https://dl.acm.org/citation.cfm?id=2901331
Dynamic Control Flow in Large-Scale Machine Learning EuroSys 2018 https://dl.acm.org/citation.cfm?id=3190551
Improving the Expressiveness of Deep Learning Frameworks with Recursion EuroSys 2018 https://dl.acm.org/citation.cfm?id=3190530
Continuum: A Platform for Cost-Aware, Low-Latency Continual Learning SoCC 2018 https://dl.acm.org/citation.cfm?id=3267817
KeystoneML: Optimizing Pipelines for Large-ScaleAdvanced Analytics ICDE 2017 https://amplab.cs.berkeley.edu/wp-content/uploads/2017/01/ICDE_2017_CameraReady_475.pdf
Owl: A General-Purpose Numerical Library in OCaml https://arxiv.org/pdf/1707.09616.pdf
Distributed learning
MAST: Global Scheduling of ML Training across Geo-Distributed Datacenters at Hyperscale USENIX 2024
Large Scale Distributed Deep Networks NIPS 2012 https://ai.google/research/pubs/pub40565.pdf
Managed Communication and Consistency for Fast Data-Parallel Iterative Analytics SoCC 2015 http://dl.acm.org/authorize?N91363
Ako: Decentralised Deep Learning with Partial Gradient Exchange SOCC 2016 https://lsds.doc.ic.ac.uk/sites/default/files/ako-socc16.pdf
Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters ATC 2017 https://www.usenix.org/system/files/conference/atc17/atc17-zhang.pdf
Parameter Hub: a Rack-Scale Parameter Server for Distributed Deep Neural Network Training SoCC 2018 https://dl.acm.org/citation.cfm?id=3267840
MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systems ML Systems Workshop at NIPS 2016 https://arxiv.org/pdf/1512.01274.pdf
Scaling Distributed Machine Learning with the Parameter Server OSDI 2014 https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-li_mu.pdf
Project Adam: Building an Efficient and Scalable Deep Learning Training System OSDI 2014 https://www.usenix.org/system/files/conference/osdi14/osdi14-paper-chilimbi.pdf
Orpheus: Efficient Distributed Machine Learning via System and Algorithm Co-design SoCC 2018 https://dl.acm.org/citation.cfm?id=3267810
Petuum: A New Platform for Distributed Machine Learning on Big Data KDD 2015 https://arxiv.org/pdf/1312.7651.pdf
GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism https://arxiv.org/pdf/1811.06965.pdf
Serving systems and inference
DeepCPU: Serving RNN-based Deep Learning Models 10x Faster ATC 2018 https://www.usenix.org/system/files/conference/atc18/atc18-zhang-minjia.pdf
Clipper: A Low-Latency Online Prediction Serving System NSDI 2017 https://www.usenix.org/system/files/conference/nsdi17/nsdi17-crankshaw.pdf
Research for Practice: Prediction-Serving Systems ACM Queue 16(1), 2018 https://queue.acm.org/detail.cfm?id=3210557
InferLine: ML Inference Pipeline Composition https://arxiv.org/pdf/1812.01776.pdf
PRETZEL: Opening the Black Box of Machine Learning Prediction Serving Systems OSDI 2018 https://www.usenix.org/system/files/osdi18-lee.pdf
Olympian: Scheduling GPU Usage in a Deep Neural Network Model Serving System MiddleWare 2018 https://dl.acm.org/citation.cfm?id=3274813
Low Latency RNN Inference with Cellular Batching EuroSys 2018 https://dl.acm.org/citation.cfm?id=3190541
SERF: Efficient Scheduling for Fast Deep Neural Network Serving via Judicious Parallelism SC 2016 https://ieeexplore.ieee.org/document/7877104
NoScope: Optimizing Neural Network Queries over Video at Scale VLDB 2017 https://dl.acm.org/citation.cfm?id=3137664
Scheduling Optimus: An Efficient Dynamic Resource Scheduler for Deep Learning Clusters EuroSys 2018 https://dl.acm.org/citation.cfm?id=3190517
SLAQ: Quality-Driven Scheduling for Distributed Machine Learning SoCC 2017 https://dl.acm.org/authorize?N46878
Proteus: agile ML elasticity through tiered reliability in dynamic resource markets EuroSys 2017 https://dl.acm.org/citation.cfm?id=3064182
Gandiva: Introspective Cluster Scheduling for Deep Learning OSDI 2018 https://www.usenix.org/system/files/osdi18-xiao.pdf
Topology-Aware GPU Scheduling for Learning Workloads in Cloud Environments SC 2017 https://dl.acm.org/citation.cfm?id=3126933
Algorithmic aspects in scalable ML
The Case for Learning Index Structures https://arxiv.org/pdf/1712.01208.pdf
Hemingway: Modeling Distributed Optimization Algorithms ML Systems Workshop at NIPS 2016 https://arxiv.org/pdf/1702.05865.pdf
Asynchronous Methods for Deep Reinforcement Learning ICML 2016 http://proceedings.mlr.press/v48/mniha16.pdf
Don't Use Large Mini-Batches, Use Local SGD https://arxiv.org/pdf/1808.07217.pdf
GeePS: Scalable deep learning on distributed GPUs with a GPU-specialized parameter server EuroSys 2016 https://dl.acm.org/citation.cfm?id=2901323
ImageNet Training in Minutes ICPP 2018 https://dl.acm.org/citation.cfm?id=3225069
Semantics-Preserving Parallelization of Stochastic Gradient Descent IPDPS 2018 https://ieeexplore.ieee.org/abstract/document/8425176
HOGWILD!: A Lock-Free Approach to ParallelizingStochastic Gradient Descent NIPS 2011 https://papers.nips.cc/paper/4390-hogwild-a-lock-free-approach-to-parallelizing-stochastic-gradient-descent.pdf
QSGD: Communication-Efficient SGD via Randomized Quantization NIPS 2017 https://papers.nips.cc/paper/6768-qsgd-communication-efficient-sgd-via-gradient-quantization-and-encoding.pdf
Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent NIPS 2017 https://papers.nips.cc/paper/7117-can-decentralized-algorithms-outperform-centralized-algorithms-a-case-study-for-decentralized-parallel-stochastic-gradient-descent.pdf
Slow and Stale Gradients Can Win the Race: Error-Runtime Trade-offs in Distributed SGD AIStats 2018 https://arxiv.org/pdf/1803.01113.pdf
Probabilistic Synchronous Parallel https://arxiv.org/pdf/1709.07772.pdf
AI Testing and Verification
DeepXplore: Automated Whitebox Testing of Deep Learning Systems SOSP 2017 https://dl.acm.org/authorize?N47145
Programmatically Interpretable Reinforcement Learning ICML 2018 https://arxiv.org/pdf/1804.02477.pdf
AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation SP 2018 https://ieeexplore.ieee.org/document/8418593
Interpretability and Explainability
“Why Should I Trust You?”Explaining the Predictions of Any Classifier KDD 2016 https://www.kdd.org/kdd2016/papers/files/rfp0573-ribeiroA.pdf
Learning to Explain: An Information-Theoretic Perspective on Model Interpretation ICML 2018 https://arxiv.org/pdf/1802.07814.pdf
A Unified Approach to Interpreting Model Predictions NIPS 2017 https://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf
The Mythos of Model Interpretability WHI 2016 https://arxiv.org/pdf/1606.03490.pdf
Model Management
MISTIQUE: A System to Store and Query Model Intermediates for Model Diagnosis SIGMOD 2018 https://dl.acm.org/citation.cfm?id=3196934
MODELDB: A System for Machine Learning Model Management HILDA 2016 https://mitdbg.github.io/modeldb/papers/hilda_modeldb.pdf
Model Governance: Reducing the Anarchy of Production ML ATC 2018 https://www.usenix.org/system/files/conference/atc18/atc18-sridhar.pdf
The Missing Piece in Complex Analytics: Low Latency,Scalable Model Management and Serving with Velox CIDR 2015 http://www.bailis.org/papers/velox-cidr2015.pdf
Bandana: Using Non-volatile Memory for Storing Deep Learning Models SysML 2019 https://arxiv.org/abs/1811.05922
Hardware
Deep learning with limited numerical precision ICML 2015 http://proceedings.mlr.press/v37/gupta15.pdf
In-Datacenter Performance Analysis of a Tensor Processing Unit ISCA 2017 https://dl.acm.org/citation.cfm?id=3080246
Serving DNNs in Real Timeat Datacenter Scale with Project Brainwave IEEE MICRO 38(2), Mar./Apr. 2018 https://ieeexplore.ieee.org/document/8344479
Security aspects
Efficient Deep Learning on Multi-Source Private Data https://arxiv.org/pdf/1807.06689.pdf
Chiron: Privacy-preserving Machine Learning as a Service https://arxiv.org/pdf/1803.05961.pdf
MLCapsule: Guarded Offline Deployment of Machine Learning as a Service https://arxiv.org/pdf/1808.00590.pdf
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware https://arxiv.org/pdf/1806.03287.pdf
Privado: Practical and Secure DNN Inference https://arxiv.org/pdf/1810.00602.pdf
ML Platforms (Applied)
Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective HPCA 2018 https://research.fb.com/publications/applied-machine-learning-at-facebook-a-datacenter-infrastructure-perspective/
Machine Learning at Facebook: Understanding Inference at the Edge HPCA 2019 https://research.fb.com/publications/machine-learning-at-facebook-understanding-inference-at-the-edge/
Meet Michelangelo: Uber’s Machine Learning Platform https://eng.uber.com/michelangelo/
Introducing FBLearner Flow: Facebook’s AI backbone https://code.fb.com/core-data/introducing-fblearner-flow-facebook-s-ai-backbone/
TFX: A TensorFlow-Based Production-Scale Machine LearningPlatform http://dl.acm.org/authorize?N33328
Horovod: fast and easy distributed deep learning in TensorFlow https://arxiv.org/pdf/1802.05799v3.pdf
ML for Systems
Resource Central: Understanding and Predicting Workloads for Improved Resource Management in Large Cloud Platforms SOSP 2017 https://dl.acm.org/authorize?N47144
Adaptive Execution of Continuous and Data-intensive Workflows with Machine Learning MiddleWare 2018 https://dl.acm.org/citation.cfm?id=3274827
AuTO: Scaling Deep Reinforcement Learning to Enable Datacenter-Scale Automatic Traffic Optimization SIGCOMM 2018 https://dl.acm.org/citation.cfm?id=3230551
Neural Adaptive Video Streaming with Pensieve SIGCOMM 2017 https://dl.acm.org/citation.cfm?id=3098843
Neural Adaptive Content-aware Internet Video Delivery OSDI 2018 https://www.usenix.org/system/files/osdi18-yeo.pdf
Workshops
Systems for ML and Open Source Software Workshop at NeurIPS 2018 http://learningsys.org/nips18/acceptedpapers.html
SysML 2018 http://www.sysml.cc/2018/index.html
Engineering Dependable and Secure Machine Learning Systems 2019 https://sites.google.com/view/edsmls2019/program
Engineering Dependable and Secure Machine Learning Systems 2018 https://sites.google.com/edu.haifa.ac.il/edsmls/program
Workshop on Distributed Machine Learning 2017 https://distributedml2017.wordpress.com/schedule/
ML Systems Workshop at NIPS 2016 https://sites.google.com/site/mlsysnips2016/accepted-papers
Upcoming 2019
ColumnML: Column Store Machine Learning with On The Fly Data Transformation VLDB 2019
Continuous Integration of Machine Learning Models: A Rigorous Yet Practical Treatment SysML 2019
Seer: Leveraging Big Data to Navigate the Complexity of Performance Debugging in Cloud Microservices ASPLOS 2019
RLgraph: Flexible Computation Graphs for Deep Reinforcement Learning SysML 2019 https://arxiv.org/pdf/1810.09028.pdf
Courses on ML Systems
CMU: Deep Learning Systems (Graduate Level) https://dlsyscourse.org/lectures/ Slides + Recorded Lectures + Assignments Included
MIT: TinyML and Efficient Deep Learning Computing (Graduate Level) https://hanlab.mit.edu/courses/2024-fall-65940 Slides + Recorded Lectures + Assignments Included
Stanford: CS329S: Machine Learning Systems Design (Undergraduate Level) https://stanford-cs329s.github.io/syllabus.html Slides Only, Focus is on ML Systems Lifecycle, more Industry/Engineering Oriented
For adding/updating the list
- Fork the repository
- Update this file
- Send a pull request