One of the old IWStack clusters is experiencing problems
Posted: Sun Jul 12, 2015 3:47 pm
We are investigating random HA events which restart some VMs. It seems to prefer Ubuntu as before.
If you have a VM on the x cluster (hosts starting with x, like x01, x02, ..., x11) please consider starting another VM and migrate data. It is very unlikely to land in the same cluster as we are vacating it, same as we did with the y cluster.
These 2 were created as big clusters with an inadequate storage formula for so many small volumes and there are various issues with this, the performance degraded over time. We are now deploying new clusters (the u from Ubuntu series), smaller, with a different storage formula and, for almost a year, had no issues with those nor is it looking like performance is degrading.
Migration happens all the time, but it takes very long and needs downtime, this is why it is better you are moving your data yourself.
These incidents are still rare and affect few people, maybe 20-30 VMs on x03 have been impacted now, but it is not a risk worth taking, in my view.
If you have a VM on the x cluster (hosts starting with x, like x01, x02, ..., x11) please consider starting another VM and migrate data. It is very unlikely to land in the same cluster as we are vacating it, same as we did with the y cluster.
These 2 were created as big clusters with an inadequate storage formula for so many small volumes and there are various issues with this, the performance degraded over time. We are now deploying new clusters (the u from Ubuntu series), smaller, with a different storage formula and, for almost a year, had no issues with those nor is it looking like performance is degrading.
Migration happens all the time, but it takes very long and needs downtime, this is why it is better you are moving your data yourself.
These incidents are still rare and affect few people, maybe 20-30 VMs on x03 have been impacted now, but it is not a risk worth taking, in my view.