Performing a Clean Shut Down and Power On of a PowerFlex Cluster shows how to shut down and power on the PowerFlex cluster.
So, a “clean” shutdown and power-on means to perform these activities safely, preserving the data stored in the SDS devices.
We used the following public Dell KB as a base for our post:
Important: Our PowerFlex cluster has the 4.5.2 version!
1- Check the clients/SDC activity. Access the MDM primary node by SSH and apply the following command. With that, we can see all connected SDCs and if they are performing any IOPS:
scli --login --management_system_ip pfmp.lab.local --username admin
watch -n1 scli --query_all_sdcAs we can see in the following picture, we have five SDCs, some of which are performing some writes IOPS:
2- Additionally, we need to check the system’s health, filtering by “IOPS”:
watch -n1 "scli --query_all | grep 'IOPS'"
3- The next step is deactivating a Protection Domain (PD). It will cause all IOPS to stop (we will have downtime).
Since we have two protection domains, we need to disable each one:
scli --query_all | grep -i "protection"
scli --inactivate_protection_domain --protection_domain_name PD01
scli --inactivate_protection_domain --protection_domain_name PD02
While a protection domain is inactivated, the following activities can take place behind the scenes:
- Determine if any current rebuild/rebalance activities are taking place. If so, the shutdown will be delayed (unless it is forced) until they are finished;
- Block future rebuild/rebalance activities;
- Temporarily disable application I/O and disable access to volumes;
- Move the DRL mode of all SDSs to harden in preparation for restarting the server;
- Reload all SDSs before re-enabling data access.
To inactivate a protection domain through the PowerFlex Manager UI, follow the actions below:
1. On the menu bar, click Block –> Protection Domains;
2. In the list of protection domains, select the relevant protection domain and click More Actions –> Inactivate;
3. In the Inactivate Protection Domain, enter the user password and click Inactivate;
4. Verify that the operation has finished successfully, and click Dismiss.
4- Validate that the Protection Domain has been deactivated (the expected output is “Inactive”):
scli --query_protection_domain --protection_domain_name PD01 | grep –i “Oper”
scli --query_protection_domain --protection_domain_name PD02 | grep –i “Oper”
5- Now, we can shut down all PowerFlex nodes.
Since we are using a Remote Desktop Manager, we can apply the “shutdown –h now” command on all nodes simultaneously. If you are not using a Remote Desktop Manager or something related to it, access each node and execute the shutdown!
6- To perform a cluster power-on, first, we need to power on each PowerFlex node.
Verify the system’s health and check if all SDSs and SDCs are up and connected:
scli --query_all_sds
scli –query_all_sdc | grep –i “connected”
7- Activate the Protection Domain (PD). In our example, since we are two Protection Domains, we need to activate both:
scli --activate_protection_domain --protection_domain_name PD01
scli --activate_protection_domain --protection_domain_name PD02
That’s it 🙂
Now, all Volumes are available again!