Fig. 1 | Scientific Reports

Fig. 1

From: Enabling scalable single-cell transcriptomic analysis through distributed computing with Apache spark

Fig. 1

Apache Spark workflow. The Spark driver program works as a master and as an entry point for all the Spark jobs. The master submits jobs to the worker nodes. The cluster manager keeps the track of the nodes, and the jobs distributed to them, several cluster managers are Yet Another Resource Negotiator (YARN), Kubernettes, mesos and standalone (in our case). The worker/slave nodes are the actual machines where the tasks are executed, and they report back to the cluster manager.

Back to article page