Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[murraydr44]

The DGL-5500 was moved to the bridged HG610 modem and checked with netperfrunner.sh with Streamboost turned off, then on.    DGL5500A1_FW113B04.bin firmware was already installed.    The Netgear router was replaced with a TP-Link WR1043ND with Barrier Breaker and netperf  package installed.   Here is the ssh console output for both tests:
StreamBoost off:
root@c43r1:/tmp# sh netrunner.sh
2015-02-27 22:26:08 Testing netperf.bufferbloat.net (ipv4) with 4 streams down and up while pinging gstatic.com. Takes about 60 seconds.
 Download:  9.33 Mbps
   Upload:  10.17 Mbps
  Latency: (in msec, 60 pings, 0.00% packet loss)
      Min: 11.266
    10pct: 100.546
   Median: 125.374
      Avg: 124.064
    90pct: 151.046
      Max: 162.234
     
StreamBoost on at 15/10 Mbps
root@c43r1:/tmp# sh netrunner.sh
2015-02-27 22:28:30 Testing netperf.bufferbloat.net (ipv4) with 4 streams down and up while pinging gstatic.com. Takes about 60 seconds.
 Download:  10.45 Mbps
   Upload:  8.23 Mbps
  Latency: (in msec, 61 pings, 0.00% packet loss)
      Min: 10.831
    10pct: 11.799
   Median: 17.216
      Avg: 18.297
    90pct: 25.538
      Max: 32.835
root@c43r1:/tmp#

[dtaht]

Your numbers don't make any sense in the sqm-scripts case. In all my testing - which admittedly, was not on the exact gear you are testing - I got numbers for sqm comparable or comparable or superior to qos-scripts. (qos-scripts, btw, uses hfsc + fq_codel, sqm-scripts uses htb + fq_codel)

I have seen devices with lousy timestamping facilities, which would affect fq_codel, but not htb or hfsc.

What was the exact hardware, kernel version, and chaos calmer release?

Secondly the script you used was contributed by a cerowrt contributor - and is well liked - but I typically use netperf-wrapper driven from a host, not from a router, to not heisenbug the tests.

[dtaht]

from looking at it it seems possible that you ran qos-scripts, and then sqm-scripts, back to back, without a reboot. It is possible that that is what messed up
- the two configure the system very differently and perhaps you had a conflict. So disable qos-scripts entirely, do a clean reboot, and try sqm-scripts again, please.

in any case, I am happy to see qos-scripts kicked streamboost's ass on this gear on this test.

But it would really not surprise me if the chipset support in this kernel release was missing, or had several very slow implementations of, core kernel features.

[dtaht]

is this attached to a DSL modem?

[dtaht]

OK, so it is attached to DSL modem, running PPOE.  Of everything you tested only sqm-scripts has the advanced settings available to correctly compensate for that - but I am not a DSL expert and there are some fiddly bits dependent on the DSL technology to get it right. You would be better off discussing this on the cerowrt-devel list as that is where the 3 really smart DSL guys are....

Secondly, it is unclear what your actual bandwidth is, and also you need to know that in order to get the shaper right. Easiest way to start to derive that is actually to do a big download - alone, and a big upload - (netperf -l 60 -H somewhere-close-like-netperf-eu,netperf-east,netperf-west.bufferbloat.net -t TCP_STREAM for the one direction and -t TCP_MAERTS ) alone, plunk in up/down numbers to the shaper 85% that (you can try higher but the first objective is to get a good, low latency result), and see what happens during a simultaneous ping.

AND if you don't have your dsl compensation right, simultaneous up/down tests like netperfrunner will behave badly. I really wish this wasn't this hard,
ISPs should just publish their framing overheads and a recommended setting for shapers, but we are not there yet.

[mrjezza]

Over the weekend I got my ASUS RT N-16 out of the basement and did some basic testing with OpenWRT, firstly with Barrier Breaker 14.07 and qos-scripts and secondly with Chaos Calmer nightly/auto-build (downloaded 27/2/15) and sqm-scripts.  I have ADSL2+ and my actual down/up is around 17.5-18.0Mbps / 0.85-0.90Mbps.

My testing ultimately became pinging my local council's website while simultaneously saturating downloads and uploads.  Baseline ping with no activity was 15-18ms which occasionally jump into the mid-20s.

With no qos-scripts or sqm-scripts, the latency during just download would jump into the 100s of ms; on just upload it would go up to over 1000ms.

I found no marked performance difference between qos-scripts and sqm-scripts once properly tuned (to the best of my tuning abilities).  I was able to set them to around 95% and 90% of measured download and upload respectively while keeping pings averaging around 30ms during total down/up saturation.  (I'm not sure what to make of the occasional blip up the high-40s or low-50ms in light of the blips into the mid-20s under no load.)

Under sqm-scripts however ping were dropping during total down/up saturation.  Not sure what this means but I was using a CC nightly rather than an official build so I'll probably revisit at a later time.  I ultimately went back to using BB - the CC web gui was super unstable and I saw no reason to keep sqm-scripts over qos-scripts.

Next step will be a comparison between the N-16 with qos-scripts and DGL5500 on 1.13B04 with streamboost.  Will run one of the scripts to test this time however.

Is there a user friendly traffic monitoring package for luci? I miss being able to login to the DGL5500 and having a nice dashboard showing which devices are using bandwidth?

Edit: Somehow I forgot to mention that I was genuinely amazed with the qos-scripts results. There was no set up apart from punching in accurate bandwidth data, but the performance was so good. I have run other custom firmwares before, with supposedly strong QoS/packet scheduling/shaping/whatever systems, but they were nowhere near as effective.

[dtaht]

sqm-scripts explicitly *deprioritizes* icmp (pings) - which would you rather drop, ping or data packets?

The whole prioritizing ping idea is wrongheaded for many reasons. Look for several of my comments about it (icmp) here: http://www.bufferbloat.net/projects/cerowrt/wiki/Wondershaper_Must_Die

I am perfectly fine with qos-scripts (unless you need ipv6 or framing overhead fixes), however, and I am glad you are amazed. I do note you might be able to do better with sqm-scripts if you get your PPPoe and DSL compensation right, and looking at ping loss is the wrong statistic, certainly. Look at the loss in other (such as voip) packets or overall throughput.

But thank you for pointing out that a naive user will think losing ping is bad.

[mrjezza]

Wasn't aware sqm-scripts was dropping ping on purpose. I thought it was just some issue with CC not being fully developed yet and me running a nightly.

Makes perfect sense to drop it now that you've explained it, but FWIW I was following these instructions and the brief guide for interpreting them:  http://www.bufferbloat.net/projects/cerowrt/wiki/Quick_Test_for_Bufferbloat

You should probably explain it there because the last line of the section about interpreting results suggests could be read by a "naive user" to suggest that if it is now dropping icmp packets then there might be something wrong.

[dtaht]

I have to note that "ping loss considered harmful" sparked a very interesting set of threads on the bloat aqm and cerowrt-devel mailing lists.

Thanks.

http://www.ietf.org/mail-archive/web/aqm/current/msg00953.html

[murraydr44]

2 msec were shaved off netperfrunner.sh results by using a PC for the testing:
TL-WR1043NDv1.8 router running BB1407 luci-qos-scripts set at 15/10 Mbps down/up connected to a bridged Huawei HG610 modem on TekSavvy 15/10 VDSL2 line.
tunnelbroker.net ipv6 tunnel established prior to testing

1) Test Environment: Intel G3420 Windows 8.1 VirtualBox Hosting LinuxMint 17.1

murraydr44@G3420Linux ~ $ cd CeroWrtScripts-master
murraydr44@G3420Linux ~/CeroWrtScripts-master $ ls
betterspeedtest.sh  config-cerowrt.sh  netperfrunner.sh  README.md
cerostats.sh        LICENSE            networkhammer.sh  tunnelbroker.sh
murraydr44@G3420Linux ~/CeroWrtScripts-master $ sh netperfrunner.sh
2015-03-07 13:50:10 Testing netperf.bufferbloat.net (ipv4) with 4 streams down and up while pinging gstatic.com. Takes about 60 seconds.
 Download:  13.23 Mbps
   Upload:  8.69 Mbps
  Latency: (in msec, 61 pings, 0.00% packet loss)
      Min: 12.500
    10pct: 13.500
   Median: 16.600
      Avg: 17.044
    90pct: 21.900
      Max: 28.200
murraydr44@G3420Linux ~/CeroWrtScripts-master $ sh betterspeedtest.sh
2015-03-07 13:51:33 Testing against netperf.bufferbloat.net (ipv4) with 5 simultaneous sessions while pinging gstatic.com (60 seconds in each direction)
.............................................................
 Download:  13.71 Mbps
  Latency: (in msec, 61 pings, 0.00% packet loss)
      Min: 11.000
    10pct: 12.300
   Median: 14.000
      Avg: 14.866
    90pct: 17.400
      Max: 27.000
.............................................................
   Upload:  9.23 Mbps
  Latency: (in msec, 61 pings, 0.00% packet loss)
      Min: 10.400
    10pct: 11.900
   Median: 12.700
      Avg: 13.159
    90pct: 15.100
      Max: 17.600
murraydr44@G3420Linux ~/CeroWrtScripts-master $

2) Test Environment: TL-WR1043NDv1.8 router instead of driven by a PC
login as: root
[email protected]'s password:
BusyBox v1.22.1 (2014-09-20 22:01:35 CEST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 BARRIER BREAKER (14.07, r42625)
 -----------------------------------------------------
  * 1/2 oz Galliano         Pour all ingredients into
  * 4 oz cold Coffee        an irish coffee mug filled
  * 1 1/2 oz Dark Rum       with crushed ice. Stir.
  * 2 tsp. Creme de Cacao
 -----------------------------------------------------
root@c43r1:~# cd /tmp
root@c43r1:/tmp# ls
TZ                   log                  resolv.conf
betterspeedtest.sh   luci-modulecache     resolv.conf.auto
dhcp.leases          luci-sessions        resolv.conf.ppp
dnsmasq.d            netperfrunner.sh     run
etc                  netserver.debug_862  state
hosts                opkg-lists           sysinfo
lock                 overlay              usr
root@c43r1:/tmp# sh betterspeedtest.sh
2015-03-07 20:11:05 Testing against netperf.bufferbloat.net (ipv4) with 5 simultaneous sessions while pinging gstatic.com (60 seconds in each direction)
............................................................
 Download:  13.93 Mbps
  Latency: (in msec, 61 pings, 0.00% packet loss)
      Min: 11.282
    10pct: 12.201
   Median: 13.479
      Avg: 14.033
    90pct: 14.658
      Max: 43.795
............................................................
   Upload:  9.3 Mbps
  Latency: (in msec, 61 pings, 0.00% packet loss)
      Min: 11.215
    10pct: 11.369
   Median: 12.340
      Avg: 12.340
    90pct: 12.987
      Max: 13.602
root@c43r1:/tmp# sh netperfrunner.sh
2015-03-07 20:13:20 Testing netperf.bufferbloat.net (ipv4) with 4 streams down and up while pinging gstatic.com. Takes about 60 seconds.
 Download:  13.63 Mbps
   Upload:  8.81 Mbps
  Latency: (in msec, 61 pings, 0.00% packet loss)
      Min: 10.936
    10pct: 13.121
   Median: 15.116
      Avg: 15.417
    90pct: 17.206
      Max: 30.054
root@c43r1:/tmp#

[dtaht]

Thank you for doing all this testing! We have tried very hard to make independent, auditable tools available to the general public to do exactly what you are doing against more devices and qos/aqm/fq/shaping systems. We had certainly hoped more vendors would try them before shipping product that featured "Now with traffic shaping" prominently on the box.

Since you have switched to a pc, which looks to be running linux (?), you can get graphable results from the https://github.com/tohojo/netperf-wrapper tool. You do (sometimes) need a custom compiled netperf, and you have to install python-matplotlib and python-qt4 - but it lets you collect tons of data and compare them later, graphically (or via text as you are now). netperf-wrapper can also be made to work on OSX using macports.

As for the differences you get on the home router driving the test, and a PC:

Note - netperf.bufferbloat.net is a round robin server for the servers in the eu, usa east coast, and usa west coast. We have unpublished servers all over the world, which we can't afford the bills on opening up to all - but if you want access, let me know offline. But if you want to ensure you are always testing the same server, use netperf-eu or netperf-east or netperf-west for your testing.

A) Anything with a weaker CPU will have more trouble driving a test than a PC. Also, using the westwood tcp variant was quite common on home routers until recently - it is less aggressive than cubic, and you can figure out which tcp is in use with cat /proc/sys/net/ipv4/tcp_congestion_control

B) The bufferbloat effort has also resulted in tremendous improvements in Linux cubic TCP, with improvements landing in every release since 3.3. What is in 3.19 is pretty amazing compared to 3.3. Google, redhat, and the entire netdev team for linux have done a wonderful job overall. I am so glad Van Jacobson finally got his hands on some big data and he (and so many others) have helped create more full understanding of TCP - and why it needed optimization - across the industry.

C) The range of results you are getting are about what we get with fq_codel on other benchmarks at your rates, so you are golden. I note that the increase in RTT delay is mostly due to the congested upstream path. Were you running at a higher rate overall (say 20mbit/20mbit) you would see a much smaller increase in delay on the measurement flows, a nearly unnoticeable one at higher rates - only the fat flows would have any inherent delay to them (fq_codel aims for 5ms, and usually bounces around 20ms) - and that can only be measured (at the moment) via analyzing separate packet captures. - and doesn't matter for most traffic.

Now that your test rig is stable, I'd love to see you retry streamboost. (and try netperf-wrapper!)

D) It is still unclear if you need to optimize for DSL framing on your link, and/or whether you are achieving an optimum. You can try increasing the up (or the down) settings until you start seeing latencies climb (and it usually is quite abrupt, going from 15mbit to 16mbit might quintuple the latencies)

 E) One further bit of fun you can have is enabling ECN on your device(s) - netperf-west has ecn enabled - and you can have full throughput with zero packet loss, if you so choose. sqm-scripts half enables ECN by default (can be both ways if you wish)  it is a single sysctl on linux,mac,and windows, documented here:

http://www.ietf.org/proceedings/89/slides/slides-89-tsvarea-1.pdf

[dtaht]

For a netperf-wrapper example on fiber, see: http://zlkj.in/fiber-sqm

[FurryNutz]

Just FYI, SB is owned and developed by Qualcomm, not D-Link. Any and all information regarding SB should be directed to Qualcomm for concerns about testing and development. D-Link integrated it for the DGL-5500 product only. I presume they may not be using it again at this time.

Thank you for all the testing and information regarding 3rd party FW. We hope that both Qualcomm and D-Link will gain knowledge from this. It's great information you are sharing.

[FurryNutz]

An interesting article:
http://www.dslreports.com/shownews/Bufferbloat-A-Big-Problem-With-a-Fairly-Simple-Solution-133536

[FurryNutz]

http://www.dslreports.com/forum/r30023180-Comparison-