Welcome!
Important information
-- Spectre and Meltdown vulnerabilities
-- Change in MX sources

News
-- MX Linux on social media: here
-- Mepis support still here

Current releases
-- MX-17.1 Final release info here
-- antiX-17 release info here

New users
-- Please read this first, and don't forget to add system and hardware information to posts!
-- Here are the Forum Rules

Yet another new inxi feature: disk speeds!

Message
Author
User avatar
oops
Forum Regular
Forum Regular
Posts: 203
Joined: Tue Apr 10, 2018 5:07 pm

Re: Yet another new inxi feature: disk speeds!

#31 Post by oops » Thu May 31, 2018 6:04 pm

Hi,
Yes, a lot of parameters are interdependent, and can have an impact to the final results. However the initiative is mostly to have an over view of the read and write speed in the real and normal circumstances, and the dispersions should not be very important if the computer is not overloaded during this test. This can give an approximate idea of the speeds.
PS: "...speed differences based on 1 big block written vs 1000 small blocks,... " Into Unix system blocks are usually small for the writing, so the 4k test is for me more pertinent. But with a spectrum (smalls and large blocks) is better.
Thank you for your input and investigations.
$ inxi -S : System: Host:XEON Kernel: 4.16.0-xeon-01.efi x86_64 bits: 64 Desktop: Xfce 4.12.3 - Distro: MX-17.1_x64 Horizon


User avatar
h2-1
Forum Regular
Forum Regular
Posts: 386
Joined: Sat Nov 15, 2008 3:16 pm

Re: Yet another new inxi feature: disk speeds!

#32 Post by h2-1 » Thu May 31, 2018 8:18 pm

Thanks for the suggestion too, though there's enough issues with it to probably put it off for a long time. I tested a few different things, changed size of blocks, number of writes, etc, and the variance is well over 2x between fastest and slowest, on one disk it went from around 35 MB/s to 85MB/s (depending on test type, block size, counts), which makes it basically meaningless in any larger sense. Basically what you will learn, more or less, from what I can see, is the following:

1. how well the filesystem on that partition handles writes
2. how well it handles small vs large block writes
3. how well it handles, say, 25000 consecutive small writes vs say, 2500. Since the former will basically never really happen in the real world, it's not real data, but the difference between these two is quite striking
4. how well it handles a single large file being written at once (which is actually far more typical of normal user actions I think): dd if=/dev/zero of=/tmp/TEST_100MB bs=100M count=3 conv=fdatasync ; rm /tmp/TEST_100MB

With results differing by over 100% in just these tests I ran, I'd say the notion that one can get really meaningful write data that reflects the actual disk performance may be an illusion, that's my tentative guess, though I could pick certain blocksize/repetition combinations that tended towards roughly an average, but I could also pick others that tended towards another number, totally different, so I'd say, as an ultra crude measure, it's useful, and may one day get used, but as a real technical test, it's not, except to show you the relative performance differences for different drives based on a range of tests, which might be the right way to do those tests. A test that can result in this large of a variance cannot be considered an accurate benchmark in my opinion.

However, it's also worth remembering that ssd's don't really like having a lot of data written to them over and over, so I'm not sure I'd want to really add such a feature in general, but I do think it's an interesting way to determine certain types of performances for write, but objectively, read I think matters more since read is what happens when you load a program, open a file, or whatever.
smxi/sgfxi site (manuals, how-to's, faqs) :: script forums :: Check out inxi sys info script!

User avatar
Adrian
Forum Veteran
Forum Veteran
Posts: 8449
Joined: Wed Jul 12, 2006 1:42 am

Re: Yet another new inxi feature: disk speeds!

#33 Post by Adrian » Thu May 31, 2018 8:35 pm

but I do think it's an interesting way to determine certain types of performances for write, but objectively, read I think matters more since read is what happens when you load a program, open a file, or whatever.
Agreed with your points. One question, is there a big common file for most of the distros (that's not loaded in memory) that can be tested in such a read test, something like "time foo=$(cat <vmlinuz)" While the kernel is in the memory, I think it would read the file from drive... Not sure how relevant results that would give, probably not very consistent either.

User avatar
rokytnji.1
Forum Regular
Forum Regular
Posts: 736
Joined: Sun Apr 13, 2014 9:06 pm

Re: Yet another new inxi feature: disk speeds!

#34 Post by rokytnji.1 » Thu May 31, 2018 9:54 pm

Just fodder for the Mill.

Code: Select all

$ pinxi -Mdxxxzy80
Machine:
  Type: Laptop System: LENOVO product: 2347DS2 v: ThinkPad T430 
  serial: <filter> Chassis: type: 10 serial: <filter> 
  Mobo: LENOVO model: 2347DS2 serial: <filter> UEFI [Legacy]: LENOVO 
  v: G1ET41WW (1.16 ) date: 05/25/2012 
Drives:
  HDD Total Size: 298.09 GiB used: 75.13 GiB (25.2%) 
  ID-1: /dev/sda vendor: Hitachi model: HGST HTS545032A7E380 size: 298.09 GiB 
  speed: 3.0 Gb/s rotation: 5400 rpm serial: <filter> rev: AC90 scheme: MBR 
  Optical-1: /dev/sr0 vendor: Optiarc model: DVD RW AD-7740H rev: 1.S1 
  dev-links: cdrom,cdrw,dvd,dvdrw 
  Features: speed: 24 multisession: yes audio: yes dvd: yes 
  rw: cd-r,cd-rw,dvd-r,dvd-ram state: running 
Hard drive is out of a gutted IBM laptop that gave up the ghost. I did not even bother to change the operating system on it. Just slap it in a go.
The Windows 7 Enterprise drive that was original with this laptop went into the laptop bag.

User avatar
h2-1
Forum Regular
Forum Regular
Posts: 386
Joined: Sat Nov 15, 2008 3:16 pm

Re: Yet another new inxi feature: disk speeds!

#35 Post by h2-1 » Fri Jun 01, 2018 12:48 am

All working, good to see! This set of features, speed/rotation won't work for everything, but it's ok as a start.
smxi/sgfxi site (manuals, how-to's, faqs) :: script forums :: Check out inxi sys info script!

User avatar
oops
Forum Regular
Forum Regular
Posts: 203
Joined: Tue Apr 10, 2018 5:07 pm

Re: Yet another new inxi feature: disk speeds!

#36 Post by oops » Fri Jun 01, 2018 3:24 am

h2-1 wrote:
Thu May 31, 2018 8:18 pm
... except to show you the relative performance differences for different drives based on a range of tests, which might be the right way to do those tests. A test that can result in this large of a variance cannot be considered an accurate benchmark in my opinion.
Right, and it is the main goal, the ability to compare some hdds, USB keys, etc .... and the variance for USB keys, can be with the same file system, more higher than 2 times.
...but objectively, read I think matters more since read is what happens when you load a program, open a file, or whatever...
Even when the system read files, you have also some writing (except with noatime parameter).
$ inxi -S : System: Host:XEON Kernel: 4.16.0-xeon-01.efi x86_64 bits: 64 Desktop: Xfce 4.12.3 - Distro: MX-17.1_x64 Horizon


User avatar
h2-1
Forum Regular
Forum Regular
Posts: 386
Joined: Sat Nov 15, 2008 3:16 pm

Re: Yet another new inxi feature: disk speeds!

#37 Post by h2-1 » Fri Jun 01, 2018 2:21 pm

Particularly with USB keys, you would NOT want to run this type of test on one, because the flash memory used has a very finite write life.

You're correct that the system always is writing files here and there, but in general, that's stuff like log files, various lock files, small stuff, that is relatively speaking basically nothing, which is why the tests proposed do very little to determine anything meaningful in my opinion about actual realworld performance.

As noted, I did log in the main inxi issue post on github this stuff, but after looking at it, I don't see adding this type of test to inxi, first, because you can actually damage your flash drive by doing this type of heavy write to it, which I hadn't thought of, SSDs have decent controllers in general that are designed to avoid damaging the flash memory, but flash drives are technically far more simple.

Particularly for USB devices I don't think inxi should make it easy to run such tests because it can actually help make them fail, which is certainly not one of the missions of inxi.

In almost all cases, the primary data inxi wants is the realworld, actual, current, data, but I think this read speed test thing is maybe one of the first times I've found a case where probably it's best to use the bad theoretical data, the static stuff that is either there or not there, than to use a real empirical write/read test.

In terms of a single large file, or group of files, to test against read, I don't know, it's hard to generalize in bsds/linux like that. That would certainly be the best way, but I don't actually know how to do a read test timer technically. But that would be where I think it would be most profitable to go.

Writes I think are really outside of inxi's program scope, I think that's best left to users to do if they want on their own, since there are so many variants, and there is no 'right' method to test real write speeds, just generalized assumptions about what is happening in writes. From my tests, I think the best one might be 50M as the value, done 1 time.

Also note, something I thought of last night, even those speeds are almost meaningless because they also reflect the actual motherboard speeds, which can vary very widely as well, and also the CPU itself, there are so many variables that really the term 'disk write speed'', the more I look at it, is really basically a fiction, except for comparing different partitions on different disks on the same system, but outside of that, it's really not very meaningful.

More profitable I think is hard data, like, longer term, board sata speed, disk sata speed, and for legacy systems, pata, just to avoid errors and bad assumptions.

I can say from USB, that's almost totally useless, because I've always found that usb writes are radically dependent on the kernel/os setup, sometimes usb transfers are downgraded in priority in the kernel, for example, and they slow to a crawl, it means virtually nothing in my experience in terms of actual performance, there are so many variables involved that it's I think self deception to even think you can get realistic data from USB in the first place, except in ideal circumstances, again, file systems, data sizes, number of writes, all these significantly vary on the same device to where after thinking on it, I'm almost certainly not going to add it to inxi because inxi already has enough problem with OEM generated junk data, I don't think I'd want to get into generating my own, lol.

But I'm glad we could sort of think through this and see what the possible scenarios are.

The one I do like however is figuring out using raw system tools how long it takes to load a large file, but i have no idea how to do that. Oh, wait, maybe dd can use the large file as if and /dev/null as of, that could work. I'll look into that.

I think basically re writes, you can safely assume write speed will almost always be less than read speed.

By the way, by chance, I've been learning about SSD internals a bit, and it's far more complex than you might think there too, for example, I believe many SSDs initially take data to a very fast storage (TLS I believe), which when filled, starts to write to the actual long term storage, so you get all kinds of performance variances depending on how large that 'cache' is, and how fast the internal logic transfers data to the long term storage, etc. Really, there are so many variables, that the more I look at it, the more variables I see, and the easier to see just how 'subjective' this data would actually be, in terms of, block size selected, counts of writes, etc. A test that does 25600 block writes for example doesn't represent anything 99% of users would ever do, which is why I'd lean more towards something you'd actually do with a real system, like 10 MB blocks written 10 x. That's roughly what I'd recommend to people who want to test their disk speeds, but I'm now pretty convinced write speed is not within inxi program scope, it's too invasive, and potentially destructive to flash drives in particular, to really think of using in inxi.
smxi/sgfxi site (manuals, how-to's, faqs) :: script forums :: Check out inxi sys info script!

User avatar
oops
Forum Regular
Forum Regular
Posts: 203
Joined: Tue Apr 10, 2018 5:07 pm

Re: Yet another new inxi feature: disk speeds!

#38 Post by oops » Fri Jun 01, 2018 7:30 pm

--- oops double post ----
Last edited by oops on Sat Jun 02, 2018 5:57 am, edited 2 times in total.
$ inxi -S : System: Host:XEON Kernel: 4.16.0-xeon-01.efi x86_64 bits: 64 Desktop: Xfce 4.12.3 - Distro: MX-17.1_x64 Horizon


User avatar
oops
Forum Regular
Forum Regular
Posts: 203
Joined: Tue Apr 10, 2018 5:07 pm

Re: Yet another new inxi feature: disk speeds!

#39 Post by oops » Sat Jun 02, 2018 5:14 am

h2-1 wrote:
Fri Jun 01, 2018 2:21 pm
1-) ... so you get all kinds of performance variances depending on how large that 'cache' is, and how fast the internal logic transfers data to the long term storage, etc.

2-)That's roughly what I'd recommend to people who want to test their disk speeds, but I'm now pretty convinced write speed is not within inxi program scope, it's too invasive, and potentially destructive to flash drives in particular, to really think of using in inxi.
1-) This parameter neutralizes all caches: conv=fdatasync , but of course, and even for the reading, the saturation phénomen may occurs from some others components of the computer.
2-) Of course, your are right, it is potentially destructive (to flash drives in particular) but less than if the SWAP for example, is always actives, but for me this test is a one shot test, no need to do this kind of test recursively with the same device.
FYI: It was only a suggestion about a write process over view into inxi, but for me it does not matter if it is not included into inxi, don't worry about that.
Anyway thank you for your implication in this inxi project.

PS: An other interesting "one shot system-info tool" (not for inxi is bootchat2)

Code: Select all

# -----------------------man bootchart2----------------------------------------
...
# pour man bootchart2 sequence demarrage /var/log/bootchart.png
# ADD:  initcall_debug printk.time=y quiet init=/sbin/bootchartd
#Then after bootup run pybootchartgui to get an interactive chart rendering tool (a image.png).  If you  want  to  chart the initrd, add:  rdinitrd=/sbin/bootchartd  to the kernel command line.
...

GRUB_CMDLINE_LINUX_DEFAULT=" initcall_debug printk.time=y quiet init=/sbin/bootchartd rdinitrd=/sbin/bootchartd"
A sample image:
# then update-grub
A sample image:
Image

Image
$ inxi -S : System: Host:XEON Kernel: 4.16.0-xeon-01.efi x86_64 bits: 64 Desktop: Xfce 4.12.3 - Distro: MX-17.1_x64 Horizon


User avatar
h2-1
Forum Regular
Forum Regular
Posts: 386
Joined: Sat Nov 15, 2008 3:16 pm

Re: Yet another new inxi feature: disk speeds!

#40 Post by h2-1 » Sat Jun 02, 2018 11:40 pm

oops, thanks for all the possible ideas, that's what helps build inxi. Sometimes someone will say something, an idea, suggestion, and I won't implement because of, as in this case, being able to see too many problems right away with the basic idea, but since the actual ideas are good, as they are in this case, it usually sticks in my head until some future point when a better solution might come up, at which point I'll often implement the original idea, or a part of it, or in a slightly different way, but it was almost always some small or large suggestion that helped launch the final fix (long ago, for example, some guy posted an issue about inxi help wrapping on narrow terminals. Now, since I never use narrow terminals, I didn't care, nor was it at that point possible to really do dynamic wrapping, but later on, it was, so I did what you now see, dynamically sized output features, which were then ported to inxi perl, and became even better, and fixed many small or large problems with the original method. So it's often a process.

In this case, to avoid forgetting these types of ideas, methods, tips, tricks, suggestions, I add ones that seem to have some value or interesting idea behind them to the github new inxi features issue thread. Currently 2 of the now I think 13 have been done over the last months (weather unit selector/options and basic disk rotation speed). Sometimes the solution isn't very good, like rotation speed, but it's a start, and then it might get better in the future. Sometimes it evolves rapidly, though taking a huge amount of research / test time until it's close to best in class (like disk vendor, which took almost 1 week to stabilize, and ongoing fine tunings to get more accurate). Other times the feature is so easy to do, that I add it while talking to the person who asked for it (see pinxi --admin -C in current pinxi, soon inxi 3.0.11 for an example of such an easy to do feature)

In other words, don't think your input is ignored if it doesn't appear right away in inxi, or that it isn't greatly appreciated.
smxi/sgfxi site (manuals, how-to's, faqs) :: script forums :: Check out inxi sys info script!

Post Reply

Return to “antiX”