Also another thing is that XCP-ng it is finally able to support UEFI boot which was nice.

@FATeknollogee said in Hypervisors: revisit your choices!:

One downside to KVM (in virt-manager) is lack of snapshots for UEFI VMs.

yes it is.

@JaredBusch said in Dashrender why did you migrate to Hyper-V from XenServer:

The XCP-NG team is a team that had a horrible business model that they were trying to implement around XenServer (XOA). Great concept, poor business model.

I wish them well but they are fighting a few things...

Citrix couldn't make any real money even when they charged more and people were taking the product seriously.

Last time I checked they were just replacing some management components and packaging some storage stuff. They are not investing in upstream and there's a lot of... changes coming in hardware that are going to require non-trivial investments for hypervisors to remain relevant.

The real problem with Xen is upstream investment is drying up. Citrix has pulled back, Amazon and other cloud providers have moved on to KVM, SuSE doesn't even market virtualization (SAP HANA support, containers, OS is as close to bare metal as they get). Outside of some people in ARM/automotive virtualization I haven't seen anyone picking it up for net new projects. In the enterprise Oracle is the only champion of it these days. KVM won the open source hypervisor war (although at this point does anyone really care?)

@Danp said in XCP-NG/XenServer tapdisk error:

It's a bug... see https://mailchi.mp/7ed52f9a2151/important-noticeopenvswitch-issue

Sure enough, that was it.

So to answer my own question:
"Which hosts belong in what pool when running local storage?"

The answer is none - at least with xenserver. Don't use pools when using local storage.

PS. I had a look on the guest side of thing just now because that is what Microsoft talked about.

Most OSs are virtualization aware. I had a look at debian running as guest under Xen with a clean install without any Xen guest tools. Debian installation automatically sense it's running on virtualized hardware and sets it's I/O scheduler to "none", thereby letting the host handle whatever I/O scheduling needed. This also makes sense because the guest doesn't know what kind of storage the host is using.

Thanks guys.

Unfortunately the link @dbeato provided is how you add a new disk to xenserver when you want it to be Storage Repository - a place to store VM partitions. So if you have a disk already xenserver will wipe it clean and put LVMs or EXT3 with VDI files on it.

When it's passed through as a block device to a VM it will have whatever filesystem the VM formats it with.

The problem with the info in the link @black3dynamite provided is that it's for xenserver 5.x so it doesn't work straight up with Xenserver 7.x (I'm running 7.4).

What I ended up doing was adding a raid 1 array instead of just a disk. The principle is the same though, just another name on the block device.

The array /dev/md0 is passed through to the VM as a block device.

I did it by adding a rule to /etc/udev/rules.d/65-md-incremental.rules almost at the end.

KERNEL=="md*", SUBSYSTEM=="block", ACTION=="change", SYMLINK+="xapi/block/%k", \ RUN+="/bin/sh -c '/opt/xensource/libexec/local-device-change %k 2>&1 >/dev/null&'"

This rule will pass all md arrays to the VMs as Removable Storage in Xenserver (so you can attach it to whatever VM you want).

Note that * in KERNEL=="md*" is a wildcard. So this will match the devices /dev/md0, md1 md2 etc. Just replace md* with whatever block device you want to pass through.

The array is 2TB so I don't know if this works with bigger arrays.
After trying some larger drives I can verify that it works fine with larger than 2TB arrays.
Also the disks were empty so I'm not sure if xenserver will wipe the disk when you set this up the first time.
After some experimenting it looks like Xenserver will not touch the drive.

I'll add the complete file for reference.

KERNEL=="td[a-z]*", GOTO="md_end" # This file causes block devices with Linux RAID (mdadm) signatures to # automatically cause mdadm to be run. # See udev(8) for syntax # Don't process any events if anaconda is running as anaconda brings up # raid devices manually ENV{ANACONDA}=="?*", GOTO="md_end" # Also don't process disks that are slated to be a multipath device ENV{DM_MULTIPATH_DEVICE_PATH}=="?*", GOTO="md_end" # We process add events on block devices (since they are ready as soon as # they are added to the system), but we must process change events as well # on any dm devices (like LUKS partitions or LVM logical volumes) and on # md devices because both of these first get added, then get brought live # and trigger a change event. The reason we don't process change events # on bare hard disks is because if you stop all arrays on a disk, then # run fdisk on the disk to change the partitions, when fdisk exits it # triggers a change event, and we want to wait until all the fdisks on # all member disks are done before we do anything. Unfortunately, we have # no way of knowing that, so we just have to let those arrays be brought # up manually after fdisk has been run on all of the disks. # First, process all add events (md and dm devices will not really do # anything here, just regular disks, and this also won't get any imsm # array members either) SUBSYSTEM=="block", ACTION=="add", ENV{ID_FS_TYPE}=="linux_raid_member", \ RUN+="/sbin/mdadm -I $env{DEVNAME}" # Next, check to make sure the BIOS raid stuff wasn't turned off via cmdline IMPORT{cmdline}="noiswmd" IMPORT{cmdline}="nodmraid" ENV{noiswmd}=="?*", GOTO="md_imsm_inc_end" ENV{nodmraid}=="?*", GOTO="md_imsm_inc_end" SUBSYSTEM=="block", ACTION=="add", ENV{ID_FS_TYPE}=="isw_raid_member", \ RUN+="/sbin/mdadm -I $env{DEVNAME}" LABEL="md_imsm_inc_end" SUBSYSTEM=="block", ACTION=="remove", ENV{ID_PATH}=="?*", \ RUN+="/sbin/mdadm -If $name --path $env{ID_PATH}" SUBSYSTEM=="block", ACTION=="remove", ENV{ID_PATH}!="?*", \ RUN+="/sbin/mdadm -If $name" # Next make sure that this isn't a dm device we should skip for some reason ENV{DM_UDEV_RULES_VSN}!="?*", GOTO="dm_change_end" ENV{DM_UDEV_DISABLE_OTHER_RULES_FLAG}=="1", GOTO="dm_change_end" ENV{DM_SUSPENDED}=="1", GOTO="dm_change_end" KERNEL=="dm-*", SUBSYSTEM=="block", ENV{ID_FS_TYPE}=="linux_raid_member", \ ACTION=="change", RUN+="/sbin/mdadm -I $env{DEVNAME}" LABEL="dm_change_end" # Finally catch any nested md raid arrays. If we brought up an md raid # array that's part of another md raid array, it won't be ready to be used # until the change event that occurs when it becomes live KERNEL=="md*", SUBSYSTEM=="block", ENV{ID_FS_TYPE}=="linux_raid_member", \ ACTION=="change", RUN+="/sbin/mdadm -I $env{DEVNAME}" # Added line # Pass-through of all /dev/md* arrays. # Will end up as Removable Storage that can be assigned to a VM. KERNEL=="md*", SUBSYSTEM=="block", ACTION=="change", SYMLINK+="xapi/block/%k", \ RUN+="/bin/sh -c '/opt/xensource/libexec/local-device-change %k 2>&1 >/dev/null&'" LABEL="md_end"

@black3dynamite I used the old one, and it worked. I succesfully installed xo now. Normally, I'd like to know what went wrong, but it's too late and friday 🙂

@brrabill said in XenServer Not Booting:

@dustinb3403 said in XenServer Not Booting:

The culprit is likely that you'd didn't point your logs to a different directory, and those logs being written destroyed the USB.

No logs written to USB.

We spent forever going over all that, remember?

I don't remember, but in any case XS reading/writing to the USB killed your boot device

@krisleslie said in Adding additional Drive to XenServer host, get error:

@black3dynamite i use 1 hdd 😞

I meant virtual disks. VM1 with virtual disk 1 = OS and virtual disk 2 = data

Also we could achieve hyperconvergence "the other way" (unlike having a global shared filesystem like Gluster or Ceph) but use fine grained replication (per VM/VM disk). That's really interesting (data locality, tiering, thin pro etc.). Obviously, we'll collaborate to see how to integrate this in our stack 🙂

@danp his installation is pre-service creation.

IMO time to rebuild and get current.

@scottalanmiller said in Home Network Setup:

@jmoore said in Home Network Setup:

@scottalanmiller said in Home Network Setup:

@dashrender said in Home Network Setup:

The whole crux of my ask was - the desire to buy as few Windows Server CALs as possible.

This is unrelated to the question asked, though.

you know i have noticed you and dash really communicate differently. not good or bad, just different. then you both have trouble understanding the other. from the many threads i have read with you two, that is the common theme i have seen.

I'd assume part of it is that I am highly literal. That tends to be a root of many communications issues for me in general.

yeah i think your right you are literal. i had to adjust my communication with you. that was my fault though, i am used to having to be so unliteral with my users because i would lose them that i got into that bad habit lol. i know for me, i was not explaining my thoughts in a well laid out way and that made me harder to understand and threw you off. did i do better that time?

So when I first install XenServer and I "tick" thin provisioning, behind the scene it's making the drive ext, so I assume if I didn't choose thin provisioning it would have made it LVM?

@storageninja said in The argument for official support vs third party support:

@dashrender said in The argument for official support vs third party support:

Same problem for the vendors. If you are dealing with unpatched spares, so are they. Having worked for some of the big ones, I know that their supply chains struggle to get parts, too. Heck, IBM couldn't deliver a server internally in more than six months, imagine how hard it is to get support parts!

Shit like this just blows my mind.

Parts Bins, internal supplies for labs, and customer supply chains are all completely different (well IBM may have been a gong show). Dell and HPE staff can't just go grab something off the line, with Mfg you have to account for the costs and someone gets to pay (and often at a premium to prevent abuse) for those internal servers.

Parts Bins and stocking those are different, and supply chain for a OEM might actually be different in the us than EMEA.

At IBM< we were an external customer, even though we were inside IBM. We showed up just like any external enterprise customer. So their inability to support was universal.

@luismc said in Port - How to go about setting up a client to be virtualized?:

@dashrender said in Port - How to go about setting up a client to be virtualized?:

As for Sharepoint - You don't get that much storage in Sharepoint in O365, so you'll likely be paying for more storage there. Sharepoint also has a number of files per site limit, just something else to watch out for.

For future reference to anyone wondering, I spoke to three different O365 reps today and they said the limit has been bumped to 1 million items so we'll be good for a while!

That's a bit more!

@matteo-nunziati said in I tried NAS4Free, FreeNAS, OMV, ... Lots of suggestions to move to Linux. I'm willing. Point me please.:

@scottalanmiller said in I tried NAS4Free, FreeNAS, OMV, ... Lots of suggestions to move to Linux. I'm willing. Point me please.:

@matteo-nunziati said in I tried NAS4Free, FreeNAS, OMV, ... Lots of suggestions to move to Linux. I'm willing. Point me please.:

@scotth said in I tried NAS4Free, FreeNAS, OMV, ... Lots of suggestions to move to Linux. I'm willing. Point me please.:

@matteo-nunziati So, CentOS 7 , EXT4, LVM with compression turned on.
After all, I'm using consumer / smb desktop hardware. From what I've read, ZFS would require more resources than I have available if I wanted dedupe. My storage box will only have 8 GB RAM.

Generally go XFS but even ext4 is good. ZFS requires something like 1gb per raw tb of storage but mostrly for raid/compression. It raises up to 5gb per tb if you want dedup.
So 8gb=8tb of raw (pre raid) storage.
ZFS is nice byt you usually can go w/out its features in smb

The memory is for dedupe. It needs very little for compression and RAID. Those use CPU, but not RAM (much).

@scottalanmiller said in I tried NAS4Free, FreeNAS, OMV, ... Lots of suggestions to move to Linux. I'm willing. Point me please.:

@matteo-nunziati said in I tried NAS4Free, FreeNAS, OMV, ... Lots of suggestions to move to Linux. I'm willing. Point me please.:

@scotth said in I tried NAS4Free, FreeNAS, OMV, ... Lots of suggestions to move to Linux. I'm willing. Point me please.:

@matteo-nunziati So, CentOS 7 , EXT4, LVM with compression turned on.
After all, I'm using consumer / smb desktop hardware. From what I've read, ZFS would require more resources than I have available if I wanted dedupe. My storage box will only have 8 GB RAM.

Generally go XFS but even ext4 is good. ZFS requires something like 1gb per raw tb of storage but mostrly for raid/compression. It raises up to 5gb per tb if you want dedup.
So 8gb=8tb of raw (pre raid) storage.
ZFS is nice byt you usually can go w/out its features in smb

The memory is for dedupe. It needs very little for compression and RAID. Those use CPU, but not RAM (much).

No you are right 1gb is for dedup

I do this a lot 😉

@penguinwrangler said in Enterprise backup: Citrix XenServer, RHEV, oVirt, KVM:

@matteo-nunziati Yes, I treat them independently. Which I would love to have the Veeam Appliance which can control them but money is too tight for that.

Let us know when the money isn't so tight 😉

@RestoronixSean