Cloudmin ISCSI on Ubuntu 12.04

Cloudmin
1

Hi All,

I’m a longtime user of webmin and a recent user of cloudmin. I have purchased a cloudmin pro license and I am trying to setup KVM hosts using an LVM ISCSI target as storage. The end result (hopefully) is to setup a host failover group that shares storage. If I understand this correctly, cloudmin will create a new logical volume on the volume group shared by the ISCSI server when I create a new VM, share that logical volume via ISCSI and deploy the VM. My setup is as follows:

I have three servers named host1, host2 and store. They are all interconnected on a private network and each has a public interface. I want to share an LVM (called storage) over ISCSI. I have setup the iscsitarget server by following the instructions here: http://www.virtualmin.com/documentation/cloudmin/iscsi, however, i keep getting the following error when trying to make a new KVM machine:

Copying 437.79 MB image file to host system host2.example.com …
… already in host cache

Creating virtual system with KVM …
… creation failed : iSCSI disk creation failed : iSCSI server is not running on store.example.com

I have checked the config files and everything is enabled/working. I have tested the system by manually mounting a shared logical volume (called test). The setup works manually but gives the above error when creating a system from cloudmin.

After much googling, I decided to ask because the documentation on the subject (with reference to cloudmin) is sparse. Another thought I had after looking at the cloudmin ISCSI documentation page was that it was looking for the wrong type of ISCSI server (ie. ISCSI Server & ISCSI Target modules in webmin). I might be wrong here as I have no previous experience with ISCSI.

Any help would be greatly appreciated. Thanks.

2

Bump
+1 same issue here, did you ever solve it, I installed everything, set it up in webmin, but still cant put anything on the iscsi host/target with same error message

3

If you login to Webmin on the iSCSI server system and go to Hardware -> iSCSI Target (on Debian / Ubuntu) or iSCSI Server (on CentOS / Redhat) , does it show that the iscsi server is running?

If not, I would be interested to know which Linux distribution and version you are running on the server system.

4

Hi Jaimie,

No it does not indicate that it is running, but when i check via SSH it says its running, see both attachments.

5

All my HOST systems are Ubuntu Server 64 12.04.3 LTS
3 are running kernel: 3.2.0-52-generic #78-Ubuntu SMP Fri Jul 26 16:21:44 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
1 (SMVMCLOUD) is running 3.5.0-39-generic #60~precise1-Ubuntu SMP Wed Aug 14 15:38:41 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

here is what I did on smvmcloud (my soon to be shared iscsi share)

sudo apt-get update;sudo apt-get upgrade
sudo apt-get update;sudo apt-get dist-upgrade
sudo reboot # to ensure i am on the latest kernel
sudo apt-get install open-iscsi iscsitarget iscsitarget-dkms

in the webmin panel for this server I refreshed the modules so I could see
iscsi client
iscsi target

I them configured each module to indicate the correct paths for binaries:
eg: client
/sbin/iscsiadm
/sbin/iscsi-iname
/etc/iscsi/ifaces

target
/usr/sbin/ietadm

i then ensured that he services were running by entering a shell and running
sudo service iscsitarget status
response was:

  • iSCSI enterprise target is running
    then:
    sudo service iscsi-network-interface status
    response:
    iscsi-network-interface start/running

then: sudo service open-iscsi status
response:

  • iscsid is running
  • Current active iSCSI sessions:
    iscsiadm: No active sessions.

next in my cloudmin master, i choose “system storage”> “iscsi servers”
my chosen server SMVMCLOUD was already listed so I opened it and saved it accepting the LVM group listed.

Next I went to my cloumin master again and chose HOST SYSTEMS -> KVM HOST SYSTEMS

I opened my cloudmin master SMVMHOST and set Create disk images in: to Devices on iSCSI server
, then chose SMVMCLOUD as my iScsi server, and hit save.

Next I created a new VM (empty system from ubuntu server install cd) on SMVMHOST (hoping to have its drive on the iSCSI server)

for KVM hosting system chose: SMVMHOST
for Location for disk image i chose: Host system default
2gm ram, 10gm disk, default network and hit create system

it copies the install CD to the HOST, then
when it got to the create harddisk stage:

Copying 638.66 MB CD image to system smvmhost …
… already in host cache

Creating virtual system with KVM …
… creation failed : iSCSI disk creation failed : iSCSI server is not running on smvmcloud

The next test was to use the new move hardisk feature and I grbbed a logical volume for a VM that was shutdown hosted on SMVMHOST and tried to move it using new feature Move Disk:
From inside the Edit Disk page
Move Disk to: Storage location: iSCSI Server smvmcloud and clicked “Move Now”

Moving disk for /dev/vda to storage iSCSI server smvmcloud …
… move failed : Failed to create new disk : iSCSI server is not running on smvmcloud

Next test: I entered webmin on SMVMCLOUD -> Hardware-> iSCSI Client->iSCSI connections
got:
No iSCSI connections are currently active.
Tried to connect to SMVMCLOUD and it said:
Failed to add iSCSI connection : No iSCSI targets were found on smvmcloud
Next I went to iSCSI Target and setup a target-> Add a new iSCSI target.
set target name to: target_test (iqn.2013-08.smvmcloud:target_test)
Device to share 1: Logical volume (chose one I had made for this 400GB formatted ext4)
chose block based and hit create
restarted iscsitarget from shell
back to iSCSI Client connections to connect to SMVMCLOUD

iSCSI server hostname or IP: smvmcloud
iSCSI server port: Default
iSCSI interface: Choose automatically
Connect to target: target_test on 192.168.80.236
Login to server: Use default authentication method

and got: Failed to add iSCSI connection : iscsiadm: no records found!
tried again using IP instead of hostname smvmcloud and it worked.

I can see the connection in the list and syslog shows:

Aug 27 15:31:28 smvmcloud kernel: [ 3641.858449] iSCSI Enterprise Target Software - version 1.4.20.3
Aug 27 15:31:28 smvmcloud kernel: [ 3641.858499] iscsi_trgt: Registered io type fileio
Aug 27 15:31:28 smvmcloud kernel: [ 3641.858502] iscsi_trgt: Registered io type blockio
Aug 27 15:31:28 smvmcloud kernel: [ 3641.858504] iscsi_trgt: Registered io type nullio
Aug 27 15:34:32 smvmcloud kernel: [ 3825.540333] scsi4 : iSCSI Initiator over TCP/IP
Aug 27 15:34:33 smvmcloud kernel: [ 3825.798288] scsi 4:0:0:1: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4
Aug 27 15:34:33 smvmcloud kernel: [ 3825.798709] sd 4:0:0:1: Attached scsi generic sg3 type 0
Aug 27 15:34:33 smvmcloud kernel: [ 3825.799147] sd 4:0:0:1: [sdc] 838860800 512-byte logical blocks: (429 GB/400 GiB)
Aug 27 15:34:33 smvmcloud kernel: [ 3825.799314] sd 4:0:0:1: [sdc] Write Protect is off
Aug 27 15:34:33 smvmcloud kernel: [ 3825.799319] sd 4:0:0:1: [sdc] Mode Sense: 77 00 00 08
Aug 27 15:34:33 smvmcloud kernel: [ 3825.799707] sd 4:0:0:1: [sdc] Write cache: disabled, read cache: disabled, doesn’t support DPO or FUA
Aug 27 15:34:33 smvmcloud kernel: [ 3825.801312] sdc: unknown partition table
Aug 27 15:34:33 smvmcloud kernel: [ 3825.801791] sd 4:0:0:1: [sdc] Attached SCSI disk
Aug 27 15:34:33 smvmcloud kernel: [ 3825.852648] iscsi_trgt: scsi_cmnd_start(1059) Unsupported 85
Aug 27 15:34:33 smvmcloud kernel: [ 3825.853050] iscsi_trgt: cmnd_skip_pdu(472) 74000000 1c 85 0
Aug 27 15:34:33 smvmcloud kernel: [ 3825.853790] iscsi_trgt: scsi_cmnd_start(1059) Unsupported 85
Aug 27 15:34:33 smvmcloud kernel: [ 3825.854184] iscsi_trgt: cmnd_skip_pdu(472) 75000000 1c 85 0
Aug 27 15:34:33 smvmcloud iscsid: connection2:0 is operational now

tried the move again:

Moving disk for /dev/vda to storage iSCSI server smvmcloud …
… move failed : Failed to create new disk : iSCSI server is not running on smvmcloud

So from what I can tell I have the client and server working but not the cloudmin components. Any help at this point would be super…
Franco

6

It looks like Webmin is looking in the wrong place for the iscsitarget server PID file. The expected path is /var/run/iscsi_trgt.pid , which is configured in /etc/webmin/iscsi-target/config . You should check where the PID file is actually located on your system.

7

AHA!!! recent version of iscsitarget on ubuntu DO NOT USE the old pidfile, as that was a part of a bug, it now looks for and uses /var/run/ietd.pid

I will reassess everything with this new information, I set this in the module config and immediately got buttons and options i never had before. Good call and thanks. I will post back my working steps later today!

Franco

8

Ok, great - I will make Webmin look for that PID file as well in the next release. Till then, you can easily change the path Webmin uses by editing /etc/webmin/iscsi-target/config .

9

Hmm, this is still proving troublesome,

there is no way to attach an existing iSCSI target as a drive in the Cloumin Manage Disks GUI, only to create one.
From the comman line there is vague references in the modify-disk docs but no example of how to attch an iSCSI drive to a vm from command line either.

I have done it the hard way (using webmin, create a target on server, using webmin create a connection on client, then mount the resulting local device file, but this is not cloudmin aware/compatible.

Franco

10

i discovered that adding the device by path eg:
/dev/disk/by-path/ip-192.168.80.236:3260-iscsi-iqn.2013-08.smvmcloud:sparcsgit.sparcsonline.com_0_img-lun-1

in the GUI makes it iSCSI… :smiley:

11

Yes, there isn’t really a nice UI for this yet, sorry. But I plan to add one in the next release.

Till them, the steps to take are :

  1. Login to Webmin on the host system, and go to Hardware -> iSCSI Client
  2. Click on iSCSI Connections, and add the connection to the iSCSI server (if it isn’t already active).
  3. Once added, login to Cloudmin and select the VM from the left menu.
  4. Go to Resources -> Manage Disks, and click “Add existing file or device as a disk”
  5. In the “Other device or file” field, enter the device on the host system for the iSCSI drive. This will be like /dev/disk/by-path/ip-something

Once the disk is added, it should appear exactly like a iSCSI disk that was created by Cloudmin itself.

12

I have been unable to move a disk from lv to iscsi using cloudmin, but I have done it manually.
I am unable to create a VM from a KVM image on iSCSI host
I am unable to create an image of a VM using iSCSI

13

Sorry Jaime, just trying to document where I am having sucess and failure. I realize that some of this functionality may not yet be ready…

Moving disk for /dev/vda to storage iSCSI server smvmcloud …
… move failed : Failed to create new disk : LVM logical volume creation failed : Failed to read from fastrpc.cgi :

14

Make sure that TCP ports 10000 to 10010 are open on the iSCSI server system smvmcloud - failure to do this can cause RPC errors.

15

They are open in the sense that the linux firewall is allowing all traffic on everything and port 10000 appears to be listening according to netsat.

16

is this a bug? When viewing the VMs in the list, iSCSI hosted drives say unlimited for size, even if they are defined as 15gb?

17

That does sound like a bug. Can you attach a screenshot, or sent it to me at jcameron@virtualmin.com ?

Also, was this iSCSI drive created entirely within Cloudmin? Or is it an existing drive that was attached?

18

Hi Jaimie, the three iSCSI VMS i have exhibit this behaviour:

if there is a CD attached it shows that size instead of the iSCSI drive. (the top one in my attachment)

If not it shows unlimited. (bottom two in screenshot)

2 of the three were made by creating an empty system using an ISO directly on the iSCSI server host. the third was created by cloning one of the first two.

Franco

19

Changing the HOST to be iSCSI also breaks its normal LVG/LV functions.

Here is a simple clone on a HOST with iSCSI enabled(Not an issue when iSCSI is off and set to LV):
Creating clone of KVM system sparcs-fmd.localhost named sparcsfmd2 …
Allocating new IP addresses …
… allocated 192.168.80.8
Copying virtual disks …
Copying disk /dev/lv_storage/sparcs-fmd_localhost_img …
… copy failed : “/dev//sparcs-fmd_localhost_img”: Invalid path for Logical Volume The origin name should include the volume group. Run `lvcreate --help’ for more information.
Refreshing status of original system …
… done. New status is : Down

… failed : Copy of critical disk failed : “/dev//sparcs-fmd_localhost_img”: Invalid path for Logical Volume The origin name should include the volume group. Run `lvcreate --help’ for more information.

20

same job on same host but I set it to LVG instead of iSCSI:

Creating clone of KVM system sparcs-fmd.localhost named sparcsfmd …
Allocating new IP addresses …
… allocated 192.168.80.7
Copying virtual disks …
Copying disk /dev/lv_storage/sparcs-fmd_localhost_img …
… snapshotted to /dev/lv_storage/sparcsfmd_sparcsonline_com_img
… done
Adding DNS entry sparcsfmd.sparcsonline.com. for IP address 192.168.80.7 …
… done

Mounting new instance’s filesystem …
Refreshing status of original system …
… done. New status is : Down

Refreshing status of new system …
… done. New status is : Down

Re-fetching current status of host system smvmbackup64 …
… done. New status is : Webmin

… clone sparcsfmd.sparcsonline.com was partially created : No Linux partitions found in disk image /dev/lv_storage/sparcsfmd_sparcsonline_com_img