iwstack knowledgebase
Posted: Sun Sep 08, 2013 6:40 am
Hello !
I will try to keep this updated on things you should know about our iwstack implementation
1. IPv6 limitations.
Our implementation lacks IPv6 support. While cloudstack has it, we didnt think it was mature enough to be used in production. This does not mean it is completely not supported, it will work for shared networks but not for isolated networks because:
-The firewall works only with IPv4;
-The Virtual Router is IPv6 incapable.
We will be upgrading it when we will be confident it is production-ready. It may take a few months.
In the meantime, if you need IPv6, please go with the shared networks and/or use a reverse proxy for sites or a 6to4 tunnel and redirect all calls over IPv4 towards the isolated network.
2. Adding more IPv4 works only with the Advanced Router (isolated networks)Solved, now you can add multiple IPs to the shared network instances too.
Routing more IPs can be done only with the virtual router, please do not try to add more IPs to your shared network VMs, it will result in an error. Your interface will show the added IPs in the total number, but they are not really added as the router for the shared network will not recognize the ownership.
3. rDNS (PTR) records can only be done manually until our panel is ready.
We consider the cloud services as being semi-permanent instances that are working only when needed. We give incentives to the people to use them only when needed by not charging when they are down (except storage and IP(s) reserved if any). Thus, hosting domains or mail servers here, while possible is not the main intended usage so adding PTR or rDNS records is not a priority. They can be done manually after you are requesting it in a ticket, but please do this only when you need it long term for permanent instances. If you need hosting+mail you may be better off with one of our other VPS offerings, please check our KVM/Xen and even OVZ offers, for lower prices, but less features.
3.1. Once you destroy your instance in the shared network, the IP(s) are released and might be taken by someone else.
Please be careful when destroying instances within the shared network, the IPs you had will be released both IPv4 and IPv6, they may be taken by someone else and may no longer be available, even with our help to manually allocate.
Be sure you considered this aspect if you have domains pointed there or you otherwise need those particular IPs and decide to destroy the instance on which they are allocated.
4. While you can always upload your own ISO and volumes, if you need to download them later, make sure you tick the extractable box !
When you link an ISO or upload a template/volume, make sure you tick the extractable box otherwise you will not be able to download the volumes or templates generated from those. It can be changed later in the database but the error will still continue to plague and re-appear later. The setting is inherited to child volumes/templates.
Same, if you need to use the password change option from within the UI make sure you load the necessary cloud-set-guest-password script at every start of the VM and tick the password enable option. I maintain versions for centos/debian/ubuntu at where xxxxxx can be centos, debian or ubuntu. If your user is different than root, you will need to change the script replacing root with whatever user you defined.
5. Whenever you are creating a private network (VLAN) please make sure you allow the task to complete in the background.
Creating an isolated network is a complex procedure. It deploys a virtual router, it configures it and registers the network. Unfortunately, registration in the database happens before the virtual router creation finishes. This means you can proceed to the creation of the virtual machine on a network which is not ready yet. As a result, the firewall, NAT, VPNs, VPCs will not work, the ui will register the changes, but they will not be passed to the virtual router.
This can be avoided by leaving enough time for the virtual router to be created and started. It should not take longer than 10 minutes after the UI declares that the network is ready.
If, however, it happened and your firewall changes, load balancing etc, are not working, please delete any VMs you may have (you can make templates to redeploy fast if you already made important changes to them (unlikely since the vm will not be able to connect to the internet)) and delete the network as well. It will also take some time before is deleted, so please be patient. You can then proceed from scratch with the virtual network creation.
6. Taking snapshots of the VMs with more than one disk will not work if the snapshots are taken in the same time.
This is a KVM limitation and will probably not be present in (future) Xen implementations.
What happens is that KVM can only snapshot one disk at a time, until the operation is not completed any new requests for snapshots will be ignored. In the UI they will remain in a "creating" status and will also be impossible to delete them, we will need to delete them from the database. This bug will be fixed (deletion of "creating" snapshots), however, the virtual disks of a KVM will have to be saved at different times to allow for the completion of the previous task. This may lead to inconsistencies (but even if saving them in the same time inconsistencies appear due to some data being in memory and not yet on the disk) so I recommend having your database installation (for example) solely on the data disk or make a big disk at start to include the OS and the database (best option if it will not grow too much). If you take the snapshots manually, please allow it to complete before going to the next, if you are taking them automatically, please allow some half an hour between the same VM different disks snapshots if they are large.
7. Resizing the disks can be done with the API, however, due to data loss risk at re-partitioning, it is not supported.
The extraordinary freedom to install any OS that works on x86 hardware and KVM excellent isolation and privacy comes at a price.
The underlying cloudstack infrastructure will have limited access and knowledge of your operating system's partitioning scheme, MBR/GPT, filesystem, etc and, as such probably fail in resizing disks resulting in data loss. We wish to avoid the problems with the users which lost data due to resizing of existing drives and we do not support this method.
The recommended way is to stop the vm, attach a recovery ISO of your choice along with the desired size disk, boot up and call parted or another tool to repartition and copy partitions from one disk to another. You can then detach the old disk and use the new one instead (for the root disk this needs more steps).
In order to avoid these problems and to keep your IP in case you will need to resize the root disk anyway, here are some tips:
- Try to estimate the space needs at the start and leave at least some 20% free just in case;
- If you anticipate you will need frequent resizing anyway, put all your changing data on the data disk which can be safely resized through the "attach recovery iso and disk then boot the ISO and dump the data" method described above, this also means you will only need to take a snapshot of that disk for back-up purposes;
- The resizing of the root disk is more complicated. I recommend to avoid it, use a data disk for your frequently changing data and make a tiny root disk for boot only. For example create a custom "root" disk when you create the VM with only, say, 200 MBs and then a big data disk which will keep the OS and the data if both change frequently. This will also make the back-ups simple as well as restoring from a snapshot maintaining data consistency.
I will try to keep this updated on things you should know about our iwstack implementation
1. IPv6 limitations.
Our implementation lacks IPv6 support. While cloudstack has it, we didnt think it was mature enough to be used in production. This does not mean it is completely not supported, it will work for shared networks but not for isolated networks because:
-The firewall works only with IPv4;
-The Virtual Router is IPv6 incapable.
We will be upgrading it when we will be confident it is production-ready. It may take a few months.
In the meantime, if you need IPv6, please go with the shared networks and/or use a reverse proxy for sites or a 6to4 tunnel and redirect all calls over IPv4 towards the isolated network.
2. Adding more IPv4 works only with the Advanced Router (isolated networks)Solved, now you can add multiple IPs to the shared network instances too.
Routing more IPs can be done only with the virtual router, please do not try to add more IPs to your shared network VMs, it will result in an error. Your interface will show the added IPs in the total number, but they are not really added as the router for the shared network will not recognize the ownership.
3. rDNS (PTR) records can only be done manually until our panel is ready.
We consider the cloud services as being semi-permanent instances that are working only when needed. We give incentives to the people to use them only when needed by not charging when they are down (except storage and IP(s) reserved if any). Thus, hosting domains or mail servers here, while possible is not the main intended usage so adding PTR or rDNS records is not a priority. They can be done manually after you are requesting it in a ticket, but please do this only when you need it long term for permanent instances. If you need hosting+mail you may be better off with one of our other VPS offerings, please check our KVM/Xen and even OVZ offers, for lower prices, but less features.
3.1. Once you destroy your instance in the shared network, the IP(s) are released and might be taken by someone else.
Please be careful when destroying instances within the shared network, the IPs you had will be released both IPv4 and IPv6, they may be taken by someone else and may no longer be available, even with our help to manually allocate.
Be sure you considered this aspect if you have domains pointed there or you otherwise need those particular IPs and decide to destroy the instance on which they are allocated.
4. While you can always upload your own ISO and volumes, if you need to download them later, make sure you tick the extractable box !
When you link an ISO or upload a template/volume, make sure you tick the extractable box otherwise you will not be able to download the volumes or templates generated from those. It can be changed later in the database but the error will still continue to plague and re-appear later. The setting is inherited to child volumes/templates.
Same, if you need to use the password change option from within the UI make sure you load the necessary cloud-set-guest-password script at every start of the VM and tick the password enable option. I maintain versions for centos/debian/ubuntu at
Code: Select all
http://board.prometeus.net/myfiles/xxxxxx5. Whenever you are creating a private network (VLAN) please make sure you allow the task to complete in the background.
Creating an isolated network is a complex procedure. It deploys a virtual router, it configures it and registers the network. Unfortunately, registration in the database happens before the virtual router creation finishes. This means you can proceed to the creation of the virtual machine on a network which is not ready yet. As a result, the firewall, NAT, VPNs, VPCs will not work, the ui will register the changes, but they will not be passed to the virtual router.
This can be avoided by leaving enough time for the virtual router to be created and started. It should not take longer than 10 minutes after the UI declares that the network is ready.
If, however, it happened and your firewall changes, load balancing etc, are not working, please delete any VMs you may have (you can make templates to redeploy fast if you already made important changes to them (unlikely since the vm will not be able to connect to the internet)) and delete the network as well. It will also take some time before is deleted, so please be patient. You can then proceed from scratch with the virtual network creation.
6. Taking snapshots of the VMs with more than one disk will not work if the snapshots are taken in the same time.
This is a KVM limitation and will probably not be present in (future) Xen implementations.
What happens is that KVM can only snapshot one disk at a time, until the operation is not completed any new requests for snapshots will be ignored. In the UI they will remain in a "creating" status and will also be impossible to delete them, we will need to delete them from the database. This bug will be fixed (deletion of "creating" snapshots), however, the virtual disks of a KVM will have to be saved at different times to allow for the completion of the previous task. This may lead to inconsistencies (but even if saving them in the same time inconsistencies appear due to some data being in memory and not yet on the disk) so I recommend having your database installation (for example) solely on the data disk or make a big disk at start to include the OS and the database (best option if it will not grow too much). If you take the snapshots manually, please allow it to complete before going to the next, if you are taking them automatically, please allow some half an hour between the same VM different disks snapshots if they are large.
7. Resizing the disks can be done with the API, however, due to data loss risk at re-partitioning, it is not supported.
The extraordinary freedom to install any OS that works on x86 hardware and KVM excellent isolation and privacy comes at a price.
The underlying cloudstack infrastructure will have limited access and knowledge of your operating system's partitioning scheme, MBR/GPT, filesystem, etc and, as such probably fail in resizing disks resulting in data loss. We wish to avoid the problems with the users which lost data due to resizing of existing drives and we do not support this method.
The recommended way is to stop the vm, attach a recovery ISO of your choice along with the desired size disk, boot up and call parted or another tool to repartition and copy partitions from one disk to another. You can then detach the old disk and use the new one instead (for the root disk this needs more steps).
In order to avoid these problems and to keep your IP in case you will need to resize the root disk anyway, here are some tips:
- Try to estimate the space needs at the start and leave at least some 20% free just in case;
- If you anticipate you will need frequent resizing anyway, put all your changing data on the data disk which can be safely resized through the "attach recovery iso and disk then boot the ISO and dump the data" method described above, this also means you will only need to take a snapshot of that disk for back-up purposes;
- The resizing of the root disk is more complicated. I recommend to avoid it, use a data disk for your frequently changing data and make a tiny root disk for boot only. For example create a custom "root" disk when you create the VM with only, say, 200 MBs and then a big data disk which will keep the OS and the data if both change frequently. This will also make the back-ups simple as well as restoring from a snapshot maintaining data consistency.