I'd like to startup an Apache Spark cluster after boot using the following command:
sudo ./path/to/spark/sbin/start-all.sh
Then run this command when the system prepares to reboot/shutdown:
sudo ./path/to/spark/sbin/stop-all.sh
How can I get started? Is there a basic template I can build on?
I've tried to use an extremely simple (file: /lib/systemd/system/spark.service
):
[Unit]
Description=Spark service
[Service]
ExecStart=sudo ./path/to/spark/sbin/start-all.sh
Which doesn't work.
Your
.service
file should look like this:Now, take a few more steps to enable and use the
.service
file:Place it in
/etc/systemd/system
folder with a name likemyfirst.service
.Make sure that your script is executable with:
Start it:
Enable it to run at boot:
Stop it:
Notes
You don't need to launch Spark with
sudo
in your service, as the default service user is already root.Look at the links below for more
systemd
options.Moreover
Now what we have above is just rudimentary, here is a complete setup for spark:
To setup the service:
Further reading
Please read through the following links. Spark is a complex setup, so you should understand how it integrates with Ubuntu's init service.
Copy-paste this into a terminal (as root) to create
/root/boot.sh
and run it on boot:To modify the parameters, for example to use a different
$bootscript
, just set that variable manually and skip that line when copying the commands.After running the commands, you can edit
/root/boot.sh
using your favorite editor, and it will run on next boot. You can also immediately run it by using:Adding startup items in systemd is complex and cumbersome. To make this more convenient, I have wirte a tool add_service which provides a simple way to quickly add startup item in systemd.
Install:
Usage:
Examples: