Windows Server 2012 Policy-Based QoS

I think you are misunderstanding how QoS works.

Lets say you have two applications running on a server (app A and app B) and either one is maxing the bandwidth. You decide you want to make sure App A gets more of the bandwidth.

In this case you would define two polices (lets say using exe name), you give App A a DSCP value of 10 and App B 5. You assign a throttle value of 10 MB/s to App A and 5 Mb/s to App B.

The important bit When App A is running on its own it won't be limited to 10MB/s and when App B is running on its own it won't be limited 5MB/s. Only when the two are running together and there is bandwidth contention will the throttle limit be effective.

This happens because it isn't a bandwidth limiting mechanism it is quality of service, the design implementation is to prioritise traffic rather then limit it when there is available bandwidth.

Take a look at the article here for more information but the part relating to above is this

When you assign a DSCP value to a QoS policy, you are essentially creating a queue for outbound network traffic. By default the traffic passing through the queue is not throttled. QoS only limits the traffic when bandwidth contention becomes an issue. In those types of situations lower priority queues yield to higher priority queues.

In some situations it is possible that a high priority queue could choke out a lower priority queue if a large amount of traffic passes through the higher priority queue. That being the case, the dialog box shown in the figure above gives you the opportunity to throttle the queue’s outbound traffic. Doing so implements a bandwidth cap that prevents the queue from consuming an excessive amount of bandwidth. The throttle rate can be specified in terms of either kilobits per second or megabits per second.

You may want to look at the bandwidth/qos settings on the vm configuration in hyper-v manager for true bandwidth control

I've worked with QoS in the networking world for years. I've not worked with it in Microsoft Windows Server (any version), but I would think the concepts are the same. Quality of Service is intended to control bandwidth. You can prioritize certain types of traffic over other types (voice and video over web browsing) when there is congestion, but what you want is traffic policing/shaping. With policing, you set a max bandwidth, and anything above that is dropped. With shaping, the excess traffic is queued and transmitted later, all while ensuring the bandwidth cap isn't exceeded.

https://technet.microsoft.com/en-us/library/hh831679.aspx#bkmk_bandwidth

When Maximum Bandwidth is set for a workload, the workload can never exceed the ceiling even if no other workloads in the system are using the network bandwidth. When Minimum Bandwidth is set for a workload, the workload can use as much bandwidth as it can - until network congestion occurs.

I can't tell whether Microsoft's Maximum Bandwidth feature is intended to use policing or shaping, but the verbiage on that TechNet article makes me thing what you want to do is possible. However, I can't seem to get it working on my Windows 10 PC either. :/

Update 10/23/15 I found an article detailing the very process you want to implement (traffic police a single executable), but I've tried this on Server 2008 and Windows 10 to no avail, even after multiple restarts. Others seemed to have it working, except for the last commenter on that thread, so this should work... https://practicalsbs.wordpress.com/2014/08/05/onedrive-throttling-upload-speed/

Using powershell on Win Server 2012 R2 worked for me, there was about 5 min delay for policy to take new value, but worked.

I run psping in bandwidth mode as server on one machine and set policy on port 444 to throttle at 10Mbit/sec, 443 port remains unthrottled

PS C:\windows\system32> New-NetQosPolicy -Name "Port444" -IPPort 444 -ThrottleRateActionBitsPerSecond 10Mb
Name           : Port444
Owner          : Group Policy (Machine)
NetworkProfile : All
Precedence     : 127
IPProtocol     : Both
IPPort         : 444
ThrottleRate   : 10.486 MBits/sec

From another machine, here's diff between 443 and 444 port bandwidth:

PS D:\Common\Sysinternals> .\psping.exe -b -l 1K -n 10s -r -4  10.10.10.10:443

PsPing v2.10 - PsPing - ping, latency, bandwidth measurement utility
Copyright (C) 2012-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

Setting warmup count to match number of outstanding I/Os: 16
TCP bandwidth test connecting to 10.10.10.10:443: Connected
10 seconds (16 warmup ops) receiving 1024 bytes TCP bandwidth test: 205087100%

TCP receiver bandwidth statistics:
  Received = 10, Size = 1024, Total Bytes: 1175108608,
  Minimum = 101.67 MB/s, Maximum = 113.49 MB/s, Average = ***112.05 MB/s***



PS D:\Common\Sysinternals> .\psping.exe -b -l 1K -n 10s -r -4  10.10.10.10:444

PsPing v2.10 - PsPing - ping, latency, bandwidth measurement utility
Copyright (C) 2012-2016 Mark Russinovich
Sysinternals - www.sysinternals.com

Setting warmup count to match number of outstanding I/Os: 16
TCP bandwidth test connecting to 10.10.10.10:444: Connected
10 seconds (16 warmup ops) receiving 1024 bytes TCP bandwidth test: 1850664100%

TCP receiver bandwidth statistics:
  Received = 10, Size = 1024, Total Bytes: 13563904,
  Minimum = 1.01 MB/s, Maximum = 1.47 MB/s, Average = ***1.24 MB/s***