Migrate ODA VM to another ODA (with vdisks)

There wasn't any helpful blog entries out there for migrating a virtualized ODA VM that contains vdisks to another virtualized ODA.  Below is what I used in order to migrate the VMs over.  I'm using ACFS snapshots in order to successfully migrate my VMs.

Always remember to test, test, and test before using this on a production ODA/VM.

1.  Execute the "oakcli show vm <vm_name>" in order to see the files that make-up the VM.

[root@oda1dom0] # oakcli show vm TESTVM
The Resource is : TESTVM
        AutoStart       :       never
        CPUPriority     :       100
        Disks           :       |file:/OVS/Repositories/demorepo/.A
                                CFS/snaps/TESTVM/VirtualMachine
                                s/TESTVM/612aeba3b38f4527a4500e
                                c202a08e3f.img,xvda,w|
        Domain          :       XEN_PVM
        DriverDomain    :       False
        ExpectedState   :       online
        FailOver        :       false
        IsSharedRepo    :       true
        Keyboard        :       en-us
        MaxMemory       :       16384M
        MaxVcpu         :       4
        Memory          :       16384M
        Mouse           :       OS_DEFAULT
        Name            :       TESTVM
        Networks        :       |type=netfront,bridge=net1|
        NodeNumStart    :       1
        OS              :       OL_5
        PrefNodeNum     :       1
        PrivateIP       :       None
        ProcessorCap    :       0
        RepoName        :       demorepo
        State           :       Online
        TemplateName    :       otml_VM_OL7U2_ORA_BASE
        VDisks          :       |oakvdk_vdisk_demorepo|
        Vcpu            :       4
        cpupool         :       default-unpinned-pool
        vncport         :       5902



2.  Be sure to cat the vm.cfg file in order to display where the vdisks are that are attached to the source ODA VM.

[root@oda1dom0]# cat /OVS/Repositories/demorepo/.ACFS/snaps/TESTVM/VirtualMachines/TESTVM/vm.cfg

vif = ['bridge=net1']
name = 'TESTVM'
extra = 'NODENAME=TESTVM'
builder = 'hvm'
cpus = '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71'
vcpus = 4
memory = 16384
cpu_cap = 0
vnc = 1
serial = 'pty'
disk = [u'file:/OVS/Repositories/demorepo/.ACFS/snaps/TESTVM/VirtualMachines/TESTVM/612aeba3b38f4527a4500ec202a08e3f.img,xvda,w', u'file:/OVS/Repositories/demorepo/.ACFS/snaps/oakvdk_vdisk1/VirtualDisks/oakvdk_vdisk1,xvdb,w!', u'file:/OVS/Repositories/demorepo/.ACFS/snaps/oakvdk_vdisk2/VirtualDisks/oakvdk_vdisk2,xvdc,w!']
maxvcpus = 4
maxmem = 16384


3.  Create *.img file snap by executing the below command on SOURCE ODA_BASE. 
acfsutil snap create -p <vm_name> <snap_vm_repo_name> <vm_repo_mountpoint>

i.e. acfsutil snap create -p TESTVM TESTVM_repo_snap /u01/app/sharedrepo/demorepo

4.  Create vdisk snap on SOURCE ODA_BASE.

acfsutil snap create -p oakvdk_vdisk1 vdisk1_snap /u01/app/sharedrepo/demorepo
acfsutil snap create -p oakvdk_vdisk2 vdisk2_snap /u01/app/sharedrepo/demorepo

5.  Copy the files using gzip, rsync, tar, etc. to TARGET ODA (onto DOM0).  Below example is using rsync.

rsync --archive --compress --hard-links --human-readable \
--inplace --no-whole-file --numeric-ids \
/u01/app/sharedrepo/demorepo/.ACFS/snaps/TESTVM_repo_snap/VirtualMachines/TESTVM \
192.168.100.10:/OVS/vm_stage

6. Tar up the file as to create a file to be used for importing the vmtemplate as part of the next steps.

tar cvSzf /OVS/vm_stage/TESTVM/TESTVM.tgz -C /OVS/vm_stage/TESTVM vm.cfg adafadfalasdfjladsflk60993qadfa.img

7.  Import the bundle onto TARGET ODA's ODA_BASE.  This should be done as root user.

[root@oda1db0 ~]# oakcli import vmtemplate TESTVM-template -files /OVS/vm_stage/TESTVM/TESTVM.tgz -repo appdevrepo -node 0


8.  Clone VM using "oakcli clone vm" to layout the directories for the VDISKS to be copied directly to proper location from SOURCE ODA to TARGET ODA.

oakcli clone vm TESTVM -vmtemplate TESTVM-template -repo appdevrepo -node 0

9.  Copy the vdisks to their final destination on the TARGET ODA's ODA_BASE.  Below examples use rsync as the copying tool.

rsync --archive --compress --hard-links --human-readable \
--inplace --no-whole-file --numeric-ids \
/u01/app/sharedrepo/demorepo/.ACFS/snaps/vdisk2_snap/VirtualDisks/oakvdk_vdisk2 \
10.17.253.70:/u01/app/sharedrepo/appdevrepo/.ACFS/snaps/TESTVM/VirtualMachines/TESTVM

rsync --archive --compress --hard-links --human-readable \
--inplace --no-whole-file --numeric-ids \
/u01/app/sharedrepo/demorepo/.ACFS/snaps/vdisk1_snap/VirtualDisks/oakvdk_vdisk1 \
10.17.253.70:/u01/app/sharedrepo/appdevrepo/.ACFS/snaps/TESTVM/VirtualMachines/TESTVM

Note: I encountered the below error when I went to startup the VM.

OAKERR:7007 Error encountered while starting VM

The fix for this is to remove line "extra = 'xxxxxx' " from the vm.cfg (known Oracle documented issue)

10.  You should now be able to start the VM and open the vmconsole.

oakcli start vm TESTVM; oakcli start vmconsole

11.  After the VM has been successfully started and validated, you can now delete the ACFS snaps that were taken originally from SOURCE ODA_BASE.

acfsutil snap delete TESTVM_repo_snap /u01/app/sharedrepo/demorepo

acfsutil snap delete vdisk1_snap /u01/app/sharedrepo/demorepo

acfsutil snap delete vdisk2_snap /u01/app/sharedrepo/demorepo


That should cover it.  This is the procedure that I used to successfully migrate VMs from one virtualized ODA to another virtualized ODA.  I hope this helps!