What is Apache Submarine?
Submarine allow Data Scientists to create end-to-end machine learning workflow. (Working in progress)
- Data Analytics
- Algorithm development
- Job scheduling
- Model Serving
Apache Submarine Workbench (working in progress) is a WEB system for data scientists. Data scientists can interactively access notebooks, submit/manage jobs, manage models, create model training workflows, access data sets, and more through Submarine Workbench.
Machine Learning Fromework
Submarine supports them in the form of plugins for different machine learning frameworks. Three machine learning frameworks, tensorflow, pytorch are currently supported. (Working in progress: Horovod/MXNet)
Resource Management Independent
Submarine also designed to be resource management independent, no matter if you have Kubernetes, Apache Hadoop YARN or just a container service, you will be able to run Submarine on top it.
Submarine can run in hadoop yarn with docker features
Submarine developed a submarine operator to allow submarine to run in kubernetes.
Submarine can also be run on one or more servers with the docker runtime environment installed. You only need to configure the server list in submarine-site.xml, submarine can automatically build these server groups into a cluster, and run submarine services and jobs in this cluster. This feature is very suitable for small-scale user scenarios.
Want to learn more about roadmap and release plan of Submarine? Please go to roadmap
Apache Submarine is Apache2 Licensed software. Please check out the source repository and how to contribute. Apache Submarine has a very active development community. Join to our Mailing list and report issues on Jira Issue tracker.