D-Link Forums

The Graveyard - Products No Longer Supported => D-Link Storage => DNS-323 => Topic started by: lazzaro4000 on February 16, 2010, 03:36:03 AM

Title: Highest disk transfer performances: who/how much/how ?
Post by: lazzaro4000 on February 16, 2010, 03:36:03 AM
Hi there.

Despite of the 1 Gb component used (a Dlink DGS-1005D switch, the DNS-323, the gigabit ethernet port of the Gigabyte EX38-DQ4 motherboard, 2 x 1.5 mt cat5E cables), I cannot see more than 15 Mbyte/s transfer speed moving big files (4 Gbyte) from pc to nas and viceversa.

What about the maximum disk performances of the nas-323 itself ?

Can you report your maximum disk transfer performaces and the related system components ?

Many thanks.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: AKFubar on February 16, 2010, 03:47:32 AM
Running Gigabit wired network to NAS.  I have 2 x WD Caviar Green drives installed in NAS.

Using NAS Performance Tester 0.4 here are my results:

Running a 200MB file write on drive s: 3 times...
Iteration 1:     18.03 MB/sec
Iteration 2:     18.29 MB/sec
Iteration 3:     16.98 MB/sec
------------------------------
Average (W):     17.76 MB/sec
------------------------------
Running a 200MB file read on drive s: 3 times...
Iteration 1:     23.53 MB/sec
Iteration 2:     27.06 MB/sec
Iteration 3:     27.53 MB/sec
------------------------------
Average (R):     26.04 MB/sec
------------------------------
YMMV
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: lazzaro4000 on February 16, 2010, 03:53:19 AM
Excuse me for my silly question but, is this Nas Performance Tester 0.4 a "standard" way to measure the transfer speed ? Is it free ?

Many thanks
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: AKFubar on February 16, 2010, 03:59:34 AM
The tester is free and is a benchmarking tool for network performance.  You can download the tool here:
http://www.808.dk/?code-csharp-nas-performance
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 16, 2010, 04:40:18 AM
@AKFubar - try pushing the file size up to the max (2GB) - you'll probably see over 30MB/sec on a read.  Are you running jumbo frame - because if you're not those are stellar results.

Please note - you might want to reduce the number of iterations to two - and - if NASTester appears to be hung or Windows says it's not responding, leave it to run, the problem is that it doesn't refresh the screen between iterations.

@lazzaro4000 - the throughput is affected by the hardware at the other end of the connection - so it's a good idea to get that information as well, and I no longer have the same setup that I did a few years back - my best results are documented somewhere in these forums, if you're curious you can try searching.

In a nutshell (and as close as I can remember) with 2x250GB Seagates in a RAID1 in the NAS, via a Netgear GS108T to an IBM xSeries 206 server running 2x250GB Maxtors also in a RAID1, I was able to just pip 30MB/sec on a read and mid 20's on a write - this was with 9k jumbo frame.

The IBM has since been upgraded to a RAID5 so I've seen a significant drop in throughput - I'm putting together another 206 server (and trying to get memory on eBay) so I may yet post detailed test results.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 16, 2010, 04:56:18 AM
The tester is free and is a benchmarking tool for network performance.  You can download the tool here:
http://www.808.dk/?code-csharp-nas-performance

If I might be allowed the liberty.

NASTester should not be used as a benchmarking tool for network performance - but rather for NAS performance - the differences are subtle but important.

NASTester creates test files of the selected size and then times the transfer from disk to disk, this is repeated as many times as set, and then the results averaged, after which the entire process is done in the reverse direction.  The time taken to transfer files will be affected by such things as file size, disk speed, disk fragmentation, buffering, the processor and the amount of memory available, the efficiency of the network stack and yes, the network itself.

As you can see there is quite a bit outside of the network involved there - network performance testers do exist and they usually do their transfers from memory rather than disk to disk - in case you're curious, I've clocked the DNS-323 network performance at 400mbps using Ixia's QCheck.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: gunrunnerjohn on February 16, 2010, 05:43:21 AM
in case you're curious, I've clocked the DNS-323 network performance at 400mbps using Ixia's QCheck.
How do you install the QCheck endpoint on the DNS-323?
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 16, 2010, 07:03:00 AM
There's a downloadable arm endpoint - you'll need fun_plug or at least telnet access to run it - I believe there is a thread somewhere in here with a link to it, but I don't seem to be able to find it.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: beefy314 on February 16, 2010, 07:10:46 AM
Using net meter, I'm getting on the average of 20-22Mbps transferring a single 4gb file to the NAS
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: AKFubar on February 16, 2010, 08:19:03 AM
Hi Fordem and thanks for your comments.  Actually I do have jumbo frames set at 4000 with the above results.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 16, 2010, 09:00:22 AM
Using net meter, I'm getting on the average of 20-22Mbps transferring a single 4gb file to the NAS

This post highlights three of the biggest problem areas in a thread of this nature ...

1) - the units being used.

20-22Mbps - is that megabit per second or megabytes per second - please note - the first few posts used MByte/sec and MB/sec - if we don't use a common unit the thread becomes meaningless.

2) - the test tools being used.

Net Meter (at least the one I found on the internet with Google - may not be the same one - and which by the way, reports its results in mbps, presumably megabits-per-second since that is the norm for bandwidth measurements) is a bandwidth measuring tool and it presumably reports what it sees - unfortunately the traffic it's measuring is a bit more than just that data that's being written to the disk - data packets on a network include network "overhead" such source & destination addresses and in some cases "padding" to fill out frames.  This prevents us from doing a simple "divide by 8" to convert bits to bytes.  This is the very reason that NAStester was written.

3) - Details

Despite my raising this point in an earlier post - there is a total lack of information about what's in the DNS-323, the network itself and the equipment at the other end of the transfer. ::)
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: cklassen on February 16, 2010, 10:28:54 AM
regarding "the units being used".

the standard is:

MB/mB = megabyte
Mb/mb = megabit

so unless you are not abiding to this standard, than 'the units being used" is self explanatory and implied

regards,
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: gunrunnerjohn on February 16, 2010, 10:48:54 AM
MB/mB = megabyte
Mb/mb = megabit
Way too much potential for confusion, why not just specify mbyte or mbit, no chance for misinterpretation.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: AKFubar on February 16, 2010, 11:09:26 AM
Running a 2000MB file write on drive s: twice...
Iteration 1:     17.73 MB/sec
Iteration 2:     17.83 MB/sec
------------------------------
Average (W):     17.78 MB/sec
------------------------------
Running a 2000MB file read on drive s: twice...
Iteration 1:     29.02 MB/sec
Iteration 2:     25.34 MB/sec
------------------------------
Average (R):     27.18 MB/sec
------------------------------
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 16, 2010, 01:19:48 PM
regarding "the units being used".

the standard is:

MB/mB = megabyte
Mb/mb = megabit

so unless you are not abiding to this standard, than 'the units being used" is self explanatory and implied

regards,

Thank you for the explanation cklassen - unfortunately you neglected to include the Mbps that beefy314 used in his post and to which I specifically referred (did you notice I quoted him?), leaving us to interpret that.

Nevertheless - you have given me another opportunity to highlight the need both to be absolutely clear in the units used, and to describe the network.

Beefy314 didn't tell us what sort of network he was using, he just says 20-22Mbps - which we will have to assume is megabits per second, since you claim that Mb should be megabits.

Now - over a 100mbps ethernet LAN my DNS-323 delivers something in the region of 80 megabits per second throughput - so beefy314's 20-22 megabits per second could be considered as appallingly low.

On the other hand - I do know that the same DNS-323 will deliver 20-22 megabits per second over an 802.11g (54mbps) wireless network, and over gigabit ethernet it is quite capable of 20-22 megabytes per sec.

Maybe it's just coincidental that the particular numbers posted by beefy are acheivable, with different units, over different types of network, but it sure does add to the confusion doesn't it?
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: AKFubar on February 16, 2010, 01:32:59 PM
Using net meter, I'm getting on the average of 20-22Mbps transferring a single 4gb file to the NAS

I agree that units used can be confusing.  However is not unreasonable to presume beefy314 is referring to Mega Bytes since he is referring to "file" transfers and files are usually measured in bytes/megabytes etc..   :)
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: tom-p on February 16, 2010, 02:12:33 PM
Industry standard measurements.

Disk transfer rates/speed - MB/sec (megabytes)
LAN/WAN transfer rates/speed - Mb/sec or Mbps (megabits)

1 Mb = 0.125 MB
or
1 MB = 8 Mb

Therefore 1Gig = 1000Mb = 125 MB, your Gig connection maximum theoretical speed is 125MB/s, account for packet/protocol overhead your down to 100MB/s, account for practical throughput your down to 85MB/s, then account for OS/NIC vendor inefficiency, CPU overhead etc you may not ever exceed 65MB/s.

Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 16, 2010, 03:23:36 PM
I agree that units used can be confusing.  However is not unreasonable to presume beefy314 is referring to Mega Bytes since he is referring to "file" transfers and files are usually measured in bytes/megabytes etc..   :)

Unfortunately - and a perusal of the threads in this forum that discuss the speed (or lack therof) of the DNS-323 will bear witness - presumptions as to what a lay person means are an exercise in futility.

I now invite you to use the search function and see for yourself.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 16, 2010, 03:51:25 PM
Running a 2000MB file write on drive s: twice...
Iteration 1:     17.73 MB/sec
Iteration 2:     17.83 MB/sec
------------------------------
Average (W):     17.78 MB/sec
------------------------------
Running a 2000MB file read on drive s: twice...
Iteration 1:     29.02 MB/sec
Iteration 2:     25.34 MB/sec
------------------------------
Average (R):     27.18 MB/sec
------------------------------

Interesting - your results go in the opposite direction to mine - maybe this relates to your jumbo frame size.

If you have the time and the interest (and your network can support it) - maybe you might like to try a larger frame size.  I'll warn you though - this can become an exercise in "one-up-man-ship" - merely a competition to see who can generate the highest numbers, because the best frame size would be the one that maximizes throughput with the file sizes you use on a day to day basis.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 16, 2010, 03:58:56 PM
Industry standard measurements.

Disk transfer rates/speed - MB/sec (megabytes)
LAN/WAN transfer rates/speed - Mb/sec or Mbps (megabits)

1 Mb = 0.125 MB
or
1 MB = 8 Mb

Therefore 1Gig = 1000Mb = 125 MB, your Gig connection maximum theoretical speed is 125MB/s, account for packet/protocol overhead your down to 100MB/s, account for practical throughput your down to 85MB/s, then account for OS/NIC vendor inefficiency, CPU overhead etc you may not ever exceed 65MB/s.



Thank you Tom - I see some "accepted rules of thumb" coming into play here.

I have two difficulties with your post ...

- the first is that it approaches a discusion on disk transfer performance (please refer to the title of the thread) from a network throughput angle and as such completely ignores the bottlenecking that can occur at the disk interfaces and the perfomance degradation that can be caused by purely disk related issues, such as fragmentation.

- the second is that it suggests that disk transfer speeds of as much as 65MB/sec can be acheived by a NAS on a gigabit network, and this, whilst not incorrect, is based upon an invalid assumption (that of the available processor performance), and in fact is what has led to the large number of discussions on disk transfer speeds as it relates to the DNS-323 - you're now giving false hope to all those who read here.

I've had my DNS-323 for over three years and I know without a doubt that it's maximum performance does not even remotely approach 65MB/sec - a memory to memory transfer that completely ignores the disk subsystem can barely crack 500 mb/sec.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: tom-p on February 17, 2010, 10:04:22 AM
Hi foredem, maybe I should have been a little more clear to say that my previous post was not specific to the DNS-323 but to data transfer speeds over GigE LANs in general.  I was trying to set expectations around the fastest possible speeds over GigE network connections regardless of NAS device.  Higher end NAS systems that can support faster disks, R5 or R10 over numerous spindles, with built in caches will likely not exceed 65MB/s.

Agree that CPU, interfaces and fragmentation play a factor in data throughput and given the home use nature of the DNS-323 and 2 disk limit people should not expect 65MB/s.  I'll leave the actual performance data for the DNS-323 to those on this thread that are benchmarking with the various utilities available.

Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 17, 2010, 10:56:09 AM
How about this  ;) ...

1 Mb = 0.125 MB
or
1 MB = 8 Mb

Therefore 100Mb = 12.5 MB, your 100mbps connection's maximum theoretical speed is 12.5MB/s, account for packet/protocol overhead your down to 10.0MB/s, account for practical throughput you're down to 8.5MB/s.

Anything faster than that may be considered as gigabit performance - it may be less than you anticpated (or were hoping for) but, since it is faster than 100 mbps can deliver, it is, nevertheless, gigabit performance.

What I'm actually trying to do here is to set an expectation level because I've noticed that users seem to expect figures in the 65MB/sec vicinity (or at least significantly higher than the DNS-323 delivers).

Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: can80an on February 17, 2010, 03:27:17 PM

What I'm actually trying to do here is to set an expectation level because I've noticed that users seem to expect figures in the 65MB/sec vicinity (or at least significantly higher than the DNS-323 delivers).


I'd be thrilled to death to get transfer speed anywhere close to what I get on PC to PC transfer across my gigabit network.  Using NASTester on my newly purchased DNS-323 I am getting 17.77 write and 27.45 read.  I also have a Linux server set up which I can get 47.55 write and 50.29 read.

Reading the specs before buying the unit gave me the impression that I would be getting speeds at around what I was already getting.  Am I disappointed?  Sure I am, but I have no intention of taking the unit back.  It's a great little unit and does pretty much everything I need it to do.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 17, 2010, 03:40:56 PM
can80an

First - I am not entirely certain I understand you ...

Reading the specs before buying the unit gave me the impression that I would be getting speeds at around what I was already getting.

Does this mean you had the impression you'd be getting the same speeds you were "PC-PC"?  Because if it does, I'd like to know exactly where you read those specs.

It's been a number of years since I dumped the box my DNS-323 came in, but as far as I can recall, the spec on the box didn't promise the speeds you're reporting now - compared to my best results I'd say you're doing pretty good, and are unlikely to see anything significantly faster.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: can80an on February 17, 2010, 05:09:33 PM
can80an

First - I am not entirely certain I understand you ...

Does this mean you had the impression you'd be getting the same speeds you were "PC-PC"?  Because if it does, I'd like to know exactly where you read those specs.

Nope, sorry I wasn't clearer, all I meant was that when I saw it had a gigabit ethernet port I 'assumed' that I would get the speed I was getting across my network.  I don't recall reading any spec that quoted transfer rates, but this is what my computer vendor has on their web site:

Ports
• 1 x 10/100/1000 Gigabit Ethernet Port
• 1 x USB Print Server Port
• Power

Had I come here first my initial expectation would definitely been different.  From the comments I've read, I do understand my transfer rates are about the best I can get and I'm OK with that or I would have already returned the unit.

Didn't mean to get off on the wrong foot with my first post ...  :-)
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 17, 2010, 06:16:59 PM
Thanks for the clarification.

If I recall correctly (it's been over three years), when I ordered mine I did have some idea of what the advertised speed was, and I know there were some figures quoted on the box, but with the usual "upto" disclaimer of course.

At that time, jumbo frame was not supported, and with it's addition, the claimed speeds were not only delivered, but easily surpassed.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: can80an on February 17, 2010, 07:48:06 PM
If I recall correctly (it's been over three years), when I ordered mine I did have some idea of what the advertised speed was, and I know there were some figures quoted on the box, but with the usual "upto" disclaimer of course.

At that time, jumbo frame was not supported, and with it's addition, the claimed speeds were not only delivered, but easily surpassed.

Hmm, interesting you mention 'figures quoted on the box', I just checked mine and it does indeed quote some figures of 'up to 23MBps-Read and 15MBps-Write'.  I'm almost bang on those numbers.

My PC network card supports 1500, 4000 and 9000 jumbo packets.  I've tested all of them and found 4000 gives me the best performance.
Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: fordem on February 18, 2010, 04:32:38 AM
Hmm, interesting you mention 'figures quoted on the box', I just checked mine and it does indeed quote some figures of 'up to 23MBps-Read and 15MBps-Write'.  I'm almost bang on those numbers.

Now we're getting someplace - these are D-Link's stated numbers and most of us know that what the "up to" really means is - you should expect less.  I think those are the old "pre jumbo frame" numbers and the addition of jumbo frame is what allows you to get where you are - 15~20% over.

Title: Re: Highest disk transfer performances: who/how much/how ?
Post by: AKFubar on February 18, 2010, 05:51:37 AM
Hmm, interesting you mention 'figures quoted on the box', I just checked mine and it does indeed quote some figures of 'up to 23MBps-Read and 15MBps-Write'.  I'm almost bang on those numbers.

My PC network card supports 1500, 4000 and 9000 jumbo packets.  I've tested all of them and found 4000 gives me the best performance.

Yup same here.  I'm pretty sure I have squeezed just about all the speed that I can from my current setup. :)