There are many hundreds of other attributes that can be configured for the different services in Hopsworks, and they can be found in the respective Chef cookbook for the service.
#CHEF RECIPE EXAMPLE FOR MAC INSTALL#
We also configure the workers to install Cuda and give 16 CPUs/48 GBs of RAM to the YARN nodemanagers. In the example, below, we configure the database, NDB, to have 16 threads and 16GB of memory. If you want to configure the installation of services in Hopsworks, you will need to edit the cluster definition before it is installed. You can, of course, add more workers and install different services on different VMs.
You can have a single host installation of Hopsworks where there is only 1 group section for the head VM or, below, we have a 3-node Hopsworks cluster where the head VM will contain most of the services with the execution of runing jobs - they will run on the GPU workers (2 of them are in this cluster definition). The installation scripts populate this YML file with IP addresses for services, auto-generated credentials for services, and configuration options for services.Īn example of a cluster definition file is provided below that will install Hopsworks on 3 servers (VMs). Both Hopsworks Cloud Installer and Hopsworks Installer will create a cluster definition file in cluster-defns/hopsworks-installation.yml that is used for installation. A Hopsworks installation is performed using a cluster definition in a YML file. A custom installation of Hopsworks allows you to change the set of installed services, configure resources for services, configure username/passwords, change installation directories, and so on.
0 kommentar(er)
