In the VM's network settings, I have only one card - the host machine's network adaptor.
The fix took some googling to find but turned out to be very easy - disable "large send offload" for the network card:
1) On the host machine, go Control Panel -> Network and Internet -> Network and Sharing Center -> Change Adapter Settings.
2) Right-click the network card and choose "Properties"
3) Click "Configure"
4) Click on the "Advanced" tab
5) Click on "Large Send Offload" (IPv4 & IPv6) and set the value to "Disabled"
That solved the problem for me - the speed went from about 30kbps to 20Mbps. Hope this information can help someone else.