Poor network bandwidth on VM (failover cluster)
-
This is a separate thread to a thread on the same server but different problem. http://mangolassi.it/topic/7368/hyper-v-failover-live-migration-failed-error-21502
Currently, everything seems to be working fine... at least on the hypervisor side. The VM server is meant for file server and it need to perform in its top notch performance during business hours. However, the performance is barely acceptable. When saving a blank file (excel) on to the VM it takes couple of seconds with a progress bar. This action should be instant. What can I do to improve the file server performance?
-
This could be related to the disk performance and not the network performance.
Just because the document is being written to a network share doesn't mean that is the issue.
-
Before I go digging... what is the storage situation? Local storage?
-
@DustinB3403 said:
This could be related to the disk performance and not the network performance.
Just because the document is being written to a network share doesn't mean that is the issue.
How is this related to disk performance? Please enlighten me.
@scottalanmiller said:
Before I go digging... what is the storage situation? Local storage?
Not sure what you mean but it is configured with vSAN.
-
-
So when you say saving the file is slow... this is when you are saving the file locally on the server or remotely from a workstation?
-
@scottalanmiller said:
So when you say saving the file is slow... this is when you are saving the file locally on the server or remotely from a workstation?
2 scenarios. 1st I noticed that opening files, saving, deleting, moving seem laggy. 2nd is when I remote into the VM via RDC it seem very laggy and delay. The mouse seem fine but clicking anything seem delayed. This is how I came to the thought of poor network bandwidth
-
When a file is written to your vSAN, the process doing the writing most likely won't reply that the process is completed until all nodes in the vSAN report that the file has been written.
If there is a bottleneck on the communication between the vSAN servers, this could introduce your delay.
I believe you said you are using x-over cables between the servers for the vSAN, so you likely don't have a switch related problem there. But you could still have bandwidth/latency issues there.
-
@LAH3385 said:
@scottalanmiller said:
So when you say saving the file is slow... this is when you are saving the file locally on the server or remotely from a workstation?
2 scenarios. 1st I noticed that opening files, saving, deleting, moving seem laggy. 2nd is when I remote into the VM via RDC it seem very laggy and delay. The mouse seem fine but clicking anything seem delayed. This is how I came to the thought of poor network bandwidth
In the case of RDP - what do the system resources utilization look like?
-
@Dashrender said:
When a file is written to your vSAN, the process doing the writing most likely won't reply that the process is completed until all nodes in the vSAN report that the file has been written.
If there is a bottleneck on the communication between the vSAN servers, this could introduce your delay.
I believe you said you are using x-over cables between the servers for the vSAN, so you likely don't have a switch related problem there. But you could still have bandwidth/latency issues there.
How to check if the bandwidth is saturated on cross over? It does not seem to use much at all.
@Dashrender said:
@LAH3385 said:
@scottalanmiller said:
So when you say saving the file is slow... this is when you are saving the file locally on the server or remotely from a workstation?
2 scenarios. 1st I noticed that opening files, saving, deleting, moving seem laggy. 2nd is when I remote into the VM via RDC it seem very laggy and delay. The mouse seem fine but clicking anything seem delayed. This is how I came to the thought of poor network bandwidth
In the case of RDP - what do the system resources utilization look like?
resources utilization on the my PC, hypervisor, or VM?
-
@LAH3385 said:
@Dashrender said:
In the case of RDP - what do the system resources utilization look like?
resources utilization on the my PC, hypervisor, or VM?
That would depend, are you RDPing into the hypervisor or VM? in either case the hypervisor could be at fault. I'd start by looking at the hypervisor. I know in VMWare you can see the resource utilization of all on a chart, so you should be able to see everything you need from within the hypervisor I would guess.
-
@LAH3385 said:
@scottalanmiller said:
So when you say saving the file is slow... this is when you are saving the file locally on the server or remotely from a workstation?
2 scenarios. 1st I noticed that opening files, saving, deleting, moving seem laggy. 2nd is when I remote into the VM via RDC it seem very laggy and delay. The mouse seem fine but clicking anything seem delayed. This is how I came to the thought of poor network bandwidth
Could be. Does the image update quickly?
-
@LAH3385 said:
@Dashrender said:
When a file is written to your vSAN, the process doing the writing most likely won't reply that the process is completed until all nodes in the vSAN report that the file has been written.
If there is a bottleneck on the communication between the vSAN servers, this could introduce your delay.
I believe you said you are using x-over cables between the servers for the vSAN, so you likely don't have a switch related problem there. But you could still have bandwidth/latency issues there.
How to check if the bandwidth is saturated on cross over? It does not seem to use much at all.
Starwind console should have graphs available for all kinds of resources utilisation. Btw, you don't need crossover cable on 1Gbit and faster ethernet cards.
-
@Dashrender said:
@LAH3385 said:
@Dashrender said:
In the case of RDP - what do the system resources utilization look like?
resources utilization on the my PC, hypervisor, or VM?
That would depend, are you RDPing into the hypervisor or VM? in either case the hypervisor could be at fault. I'd start by looking at the hypervisor. I know in VMWare you can see the resource utilization of all on a chart, so you should be able to see everything you need from within the hypervisor I would guess.
I just gave the VM more virtual processor from 1 to 4. It does seem more responsive while RDP. But the progress bar when saving still there. The graph on Performance (Task Manager) does not seem to be indicating anything out of ordinary. The graph goes up and down like a heart beat on cross over ethernet and wobbling on internet.
@scottalanmiller said:
@LAH3385 said:
@scottalanmiller said:
So when you say saving the file is slow... this is when you are saving the file locally on the server or remotely from a workstation?
2 scenarios. 1st I noticed that opening files, saving, deleting, moving seem laggy. 2nd is when I remote into the VM via RDC it seem very laggy and delay. The mouse seem fine but clicking anything seem delayed. This is how I came to the thought of poor network bandwidth
Could be. Does the image update quickly?
How do I check? It should be within couple of seconds.. or every second.
@marcinozga said:
@LAH3385 said:
@Dashrender said:
When a file is written to your vSAN, the process doing the writing most likely won't reply that the process is completed until all nodes in the vSAN report that the file has been written.
If there is a bottleneck on the communication between the vSAN servers, this could introduce your delay.
I believe you said you are using x-over cables between the servers for the vSAN, so you likely don't have a switch related problem there. But you could still have bandwidth/latency issues there.
How to check if the bandwidth is saturated on cross over? It does not seem to use much at all.
Starwind console should have graphs available for all kinds of resources utilisation. Btw, you don't need crossover cable on 1Gbit and faster ethernet cards.
I couldn't find where the graph would be located at. But this is the setting on Synchronization priority
-
What antivirus (if any) is running on that server? What happens if you disable it?
-
@marcinozga said:
What antivirus (if any) is running on that server? What happens if you disable it?
No AntiVirus. If any.. it would be w/e built-in into Windows server 2K12 R2
-
You could also try to move that synchronization slider a few notches towards the middle. That should give you a balance of sync and client access speed. It looks like you have it set to just focus on syncing. This could likely be what is hurting you.
-
@dafyre said:
You could also try to move that synchronization slider a few notches towards the middle. That should give you a balance of sync and client access speed. It looks like you have it set to just focus on syncing. This could likely be what is hurting you.
I moved it to 9/10 client access. Very little to no different.
-
@LAH3385 said:
@dafyre said:
You could also try to move that synchronization slider a few notches towards the middle. That should give you a balance of sync and client access speed. It looks like you have it set to just focus on syncing. This could likely be what is hurting you.
I moved it to 9/10 client access. Very little to no different.
Might be a wise thing to set it back closer to the defaults.
Look at the Perfmon Counters for Disk Read / Writes and Queue Length for both of your servers that are running Starwind?
-
@dafyre said:
@LAH3385 said:
@dafyre said:
You could also try to move that synchronization slider a few notches towards the middle. That should give you a balance of sync and client access speed. It looks like you have it set to just focus on syncing. This could likely be what is hurting you.
I moved it to 9/10 client access. Very little to no different.
Might be a wise thing to set it back closer to the defaults.
Look at the Perfmon Counters for Disk Read / Writes and Queue Length for both of your servers that are running Starwind?
What am I looking for? How long should I run the test?
