Setting Up the Engine
DataSynapse HPCCA uses a prepared VM scale set/auto scaling group or Instance Group based on VM images with security settings aligned with your environment, Grid Libraries preinstalled, and the Engine Configuration set and tested. This provides you with the opportunity to know that the Engine VM is set up the way you need.
The following instructions for DataSynapse GridServer® Engine setup are a general case. Due to the inclusion of customer-editable startup scripts (User Data on AWS, VM Extension scripting on Azure, and startup script on GCP), the specific setup of the Engine VM's image depends on your particular use case. A good way to test your particular scenario is to set the
MINENGINES to 1 in the cloudprovider.properties file and see if the Engine starts normally when the Director starts.
Creating VM Scale Set on Azure
For details about creating a VM scale set on Azure, see Create a virtual machine scale set in the Azure portal. Ensure that the following conditions are met when creating a VM scale set:
-
Ensure that the VM size selected has vCPUs same as
AZURE_VM_CORESproperty in thecloudprovider.propertiesfile. -
For Orchestration mode, select Uniform mode.
-
Select the same Virtual Network on the Networking tab as the one you have used when creating the image for Engine VM.
-
On the scaling tab, set the scaling policy to Manual.
Creating Auto scaling Group on AWS
For details about creating an auto scaling group on AWS, see Creating an Auto Scaling group using a launch template. Ensure that the following conditions are met when creating an auto scaling group:
-
Instance weights must be specified for the instance types. For more information about instance weighting, see Configuring instance weighting for Amazon EC2 Auto Scaling.
-
Weight of the instance type must be equal to the number of vCPUs present in instance type. You can select instance types with different vCPUs.
-
You must create an auto scaling group with allocation strategy as capacity-optimized. For more information, see Auto Scaling groups with multiple instance types and purchase options.
-
When creating an auto scaling group on AWS portal, ensure that the Minimum capacity property and the Desired capacity property are set to 0. DataSynapse HPCCA maintains the minimum VMs in auto scaling group by using the
MINENGINESproperty in thecloudprovider.propertiesfile. -
Select the same security group in Launch Template as that used when creating the image for Engine VM.
-
Set the scaling policies as None.
-
Select the same network and subnet in the auto scaling group as that used when creating the image for DataSynapse GridServer® Engine VM.
Creating an Instance Group on Google Cloud Platform (GCP)
For details about creating an instance group on GCP, see Creating managed instance groups. Ensure that the following conditions are met when creating an instance group from GCP console:
-
In Create an instance group, select New Managed Instance Group (stateless) option.
-
Set location to multiple zones and select the required region in which engine VMs are to be launched.
-
Select an instance template. For more information, see Creating a Google Cloud Platform (GCP) Engine VM Image.
-
Disable Instance redistribution.
-
Click the delete auto scaling configuration button.