Apache Submarine

Apache Submarine is a unified AI platform which allows engineers and data scientists to run Machine Learning and Deep Learning workload in distributed cluster.

Technologies


ONE PLATFORM


What is Apache Submarine?




ONE PLATFORM

Submarine allow Data Scientists to create end-to-end machine learning workflow. (Working in progress)

  • Data Analytics
  • Algorithm development
  • Workflow
  • Job scheduling
  • Model Serving


Workbench

Apache Submarine Workbench (working in progress) is a WEB system for data scientists. Data scientists can interactively access notebooks, submit/manage jobs, manage models, create model training workflows, access data sets, and more through Submarine Workbench.


Machine Learning Fromework

Submarine supports them in the form of plugins for different machine learning frameworks. Three machine learning frameworks, tensorflow, pytorch are currently supported. (Working in progress: Horovod/MXNet)


Resource Management Independent

Submarine also designed to be resource management independent, no matter if you have Kubernetes, Apache Hadoop YARN or just a container service, you will be able to run Submarine on top it.

YARN

Submarine can run in hadoop yarn with docker features

Kubernetes (In-Progress)

Submarine developed a submarine operator to allow submarine to run in kubernetes.

Docker (In-Progress)

Submarine can also be run on one or more servers with the docker runtime environment installed. You only need to configure the server list in submarine-site.xml, submarine can automatically build these server groups into a cluster, and run submarine services and jobs in this cluster. This feature is very suitable for small-scale user scenarios.

Roadmaps

Want to learn more about roadmap and release plan of Submarine? Please go to roadmap

Apache Submarine is Apache2 Licensed software. Please check out the source repository and how to contribute. Apache Submarine has a very active development community. Join to our Mailing list and report issues on Jira Issue tracker.