Welcome!
Important information
-- Spectre and Meltdown vulnerabilities
-- Change in MX sources

News
-- MX Linux on social media: here
-- Mepis support still here

Current releases
-- MX-17.1 Final release info here
-- antiX-17 release info here

New users
-- Please read this first, and don't forget to add system and hardware information to posts!
-- Here are the Forum Rules

Computer is lagging when there is high I/O on ext. HDD

Report Bugs, Issues and non- pacakage Requests
Message
Author
User avatar
postcd
Forum Novice
Forum  Novice
Posts: 51
Joined: Mon Oct 29, 2018 2:02 pm

Computer is lagging when there is high I/O on ext. HDD

#1 Post by postcd » Tue Nov 06, 2018 5:26 am

Hello,

i seen this a few times for last week of my usage of MX Linux, that when there was high activity on my external HDD, whole system was lagging (including delays like 3 seconds in reaction of system menus like MX menu).

Image
Above image probably reflect this activity that is going on on that external drive:
- checking large amount of data using torrent client on that ext. drive
- around 4MB/s download to encrypted container on that ext. drive
CPU utilization like 20%

Is this normal activity that the work space/apps reaction time is significantly badly affected even CPU, RAM and system drive is not fully utilized?

If i need to execute any specific commands at a time of lagging, let me know, or i apologize if this is bad section for this topic.
OS: 4.15.4-1~mx17+1 (2018-02-23) x86_64 ; XFCE 4 ; MB: G31M ES2L rev. 2.0 ; GPU: AMD Radeon RX 560 ; full HW specs. as of Oct 30/2018

User avatar
fehlix
Forum Guide
Forum Guide
Posts: 2030
Joined: Wed Apr 11, 2018 5:09 pm

Re: Computer is lagging when there is high I/O on ext. HDD

#2 Post by fehlix » Tue Nov 06, 2018 6:51 am

postcd wrote:
Tue Nov 06, 2018 5:26 am
Is this normal activity that the work space/apps reaction time is significantly badly affected even CPU, RAM and system drive is not fully utilized?
That's my experience too. Bad I/O can bring the system down.
E.g. plugging in a bad USB stick, can slow down the whole system significantly. :snail:
Gigabyte Z77M-D3H, Intel Xeon E3-1240 V2 (Quad core), 32GB RAM,
GeForce GTX 770, Samsung SSD 850 EVO 500GB, Seagate Barracuda 4TB

User avatar
Adrian
Forum Veteran
Forum Veteran
Posts: 8995
Joined: Wed Jul 12, 2006 1:42 am

Re: Computer is lagging when there is high I/O on ext. HDD

#3 Post by Adrian » Tue Nov 06, 2018 9:18 am

It's normal to some extent, I think you can improve this if you know a program causes it by lowering the IO priority with ionice. https://linux.die.net/man/1/ionice

User avatar
thomasl
Forum Regular
Forum Regular
Posts: 107
Joined: Sun Feb 04, 2018 10:26 am

Re: Computer is lagging when there is high I/O on ext. HDD

#4 Post by thomasl » Wed Nov 07, 2018 11:42 am

Copying $BIG_FILES to/from USB devices is one of the few areas in MX where I sometimes wish I was back in Windows (I occasionally do boot into Windows just to copy stuff between two USB sticks (those are especially bad) because MX is just so slow).

What helps sometimes but not always is using the nocache utility. This avoids contaminating the i/o cache with stuff that presumably doesn't need to be cached in the first place. See this post: viewtopic.php?f=104&t=46213#p460366 for details.

YMMV.
Dual-boot MX17.1/64 frugal root persistence + Windows 7 on Lenovo Edge72 i5-3470S/12GB and Tosh R950 i5-3340M/8GB
“In foreign countries they fear baldness. They are so rich in foreign countries, they can afford to fear all kinds of silly things.”

clicktician
Forum Regular
Forum Regular
Posts: 192
Joined: Sat May 02, 2015 4:35 pm

Re: Computer is lagging when there is high I/O on ext. HDD

#5 Post by clicktician » Wed Nov 07, 2018 1:07 pm

NTFS and exFat? Are you stuck with those?
USB is a CPU mastered buss, so it makes sense that you'd notice a slow down if there was high contention on a USB filesystem especially an encrypted one.

I don't dual boot or have to deal with Windows much, so I got rid of NTFS and exFat after I discovered how much they were bogging down my processes.
I run 5 I/O tests on each of 8 filesystems whenever I think about upgrading to a kernel. NTFS is consistently the least performant FS on MX17. And write speeds are especially poor.
This isn't the fault of MX. NTFS and exFat are patented, proprietary filesystems and only Microsoft knows how they really work. Linux implementations are reverse-engineered.

If you can move to a more performant file system, the problem may just vanish. Here are two quick charts from my test bed DB.
Here, NTFS (light blue bars) is dead last in overall IOPS/sec running in all the kernels. I should mention that JFS is the least CPU intensive, but it tends to fragment a bit more than the other Linux FSs.
fs_speed_chart.jpg
On this second chart, I've highlighted the NTFS write speeds in red compared to EXT4 and F2FS.
ext4_f2fs_ntfs_detail.jpg
You do not have the required permissions to view the files attached to this post.
Son, someday all this will belong to your ex wife.

User avatar
thomasl
Forum Regular
Forum Regular
Posts: 107
Joined: Sun Feb 04, 2018 10:26 am

Re: Computer is lagging when there is high I/O on ext. HDD

#6 Post by thomasl » Wed Nov 07, 2018 2:33 pm

clicktician wrote:
Wed Nov 07, 2018 1:07 pm
NTFS and exFat? Are you stuck with those?
Yes, indeed. Mostly NTFS, very rarely FAT32.
clicktician wrote:
Wed Nov 07, 2018 1:07 pm
I don't dual boot or have to deal with Windows much, so I got rid of NTFS and exFat after I discovered how much they were bogging down my processes.
I run 5 I/O tests on each of 8 filesystems whenever I think about upgrading to a kernel. NTFS is consistently the least performant FS on MX17. And write speeds are especially poor.
Hm... perhaps I should, just for the sake of it, format two of unused 32GB sticks as ext4 and simply see what happens when I copy ~32GB between them.

However, while this is certainly interesting from a technical POV it doesn't solve the problem as I have to keep those bigger sticks and HDDs as NTFS for the foreseeable future. (True even if I could get rid of Windows on my PCs... one set of disks holds music and photos, another videos and these are used with my TVs and various other "smart" devices which able to read NTFS but not ext4. :frown: )
Dual-boot MX17.1/64 frugal root persistence + Windows 7 on Lenovo Edge72 i5-3470S/12GB and Tosh R950 i5-3340M/8GB
“In foreign countries they fear baldness. They are so rich in foreign countries, they can afford to fear all kinds of silly things.”

clicktician
Forum Regular
Forum Regular
Posts: 192
Joined: Sat May 02, 2015 4:35 pm

Re: Computer is lagging when there is high I/O on ext. HDD

#7 Post by clicktician » Wed Nov 07, 2018 4:44 pm

thomasl wrote:
Wed Nov 07, 2018 2:33 pm
perhaps I should, just for the sake of it, format two of unused 32GB sticks as ext4 and simply see what happens when I copy ~32GB between them.
Copying 32GB of files onto a 32GB stick... did you mean 64GB stick? If you really want to run your sticks at 100% full, EXT4 (or any EXT FS or XFS) is probably not the best choice.
The reason is, these filesystems leave freespace between files in anticipation of resizing them later, and it spreads the files out over the partition. This "freespace fragmentation" allows for the easily adjustable extents -- hence the name EXTX. This helps control "file fragmentation" (the second type of frag, and that's the bad kind).

As the EXTX disk becomes full, tho, the FS will attempt to squeeze new file parts into the ever-shrinking spare extent space it left open adjacent to other files, which results in terrible fragged performance when reading and writing the last bits.
As a rule, I never fill an EXT file system past 80%, because performance suffers.

If you want to wipe the stick and then fill it up 100%, I would suggest JFS instead. In my testing, it is lightest on CPU usage, it's often faster than EXT type filesystems. But because it doesn't manage extents well, it gets "file fragmentation" with repeated deletes, adds and updates. But if you're deleting everything (or formatting) and then re-writing the whole stick, you don't have to worry about any file fragmentation. You're just writing all your files back-to-back which is how JFS should store them.
Son, someday all this will belong to your ex wife.

User avatar
thomasl
Forum Regular
Forum Regular
Posts: 107
Joined: Sun Feb 04, 2018 10:26 am

Re: Computer is lagging when there is high I/O on ext. HDD

#8 Post by thomasl » Thu Nov 08, 2018 11:14 am

@clicktician: I was/am thinking to do such a ~32GB copying exercise simply as a test as to whether ext4 on two USB sticks is indeed faster than the same setup but with NTFS. This is not and will never be a production issue since I have to stick with NTFS as explained earlier. However, it would certainly be interesting to see whether ext4 is indeed faster than NTFS in that scenario, so I will try it.
Dual-boot MX17.1/64 frugal root persistence + Windows 7 on Lenovo Edge72 i5-3470S/12GB and Tosh R950 i5-3340M/8GB
“In foreign countries they fear baldness. They are so rich in foreign countries, they can afford to fear all kinds of silly things.”

User avatar
postcd
Forum Novice
Forum  Novice
Posts: 51
Joined: Mon Oct 29, 2018 2:02 pm

Re: Computer is lagging when there is high I/O on ext. HDD

#9 Post by postcd » Thu Nov 08, 2018 1:56 pm

Sorry to bother you all. Apologize for my ignorance, but i really do not know how to discover root cause and fix it. I believe that high utilization of the external USB 2.0 HDD (respectively the virtual HDD - decrypted Veracrypt container) is not a reason to the MX Linux to be lagging while my system HDD, CPU and RAM has some room. I think this is problem in Linux. Linux should be able to do its job, no matter what slow external HDD does? The ext. HDD is only the bottleneck of the process that is using it. Why should MX menu (responsiveness) and Firefox responsiveness etc be affected by it?
OS: 4.15.4-1~mx17+1 (2018-02-23) x86_64 ; XFCE 4 ; MB: G31M ES2L rev. 2.0 ; GPU: AMD Radeon RX 560 ; full HW specs. as of Oct 30/2018

clicktician
Forum Regular
Forum Regular
Posts: 192
Joined: Sat May 02, 2015 4:35 pm

Re: Computer is lagging when there is high I/O on ext. HDD

#10 Post by clicktician » Thu Nov 08, 2018 3:23 pm

postcd wrote:
Thu Nov 08, 2018 1:56 pm
Why should MX menu (responsiveness) and Firefox responsiveness etc be affected by it?
Are your mouse and keyboard also USB devices? <I don't know> If so, they are sharing the same buss as your busy disk device.
That would every application interface that uses your mouse and keyboard appear to respond slowly to your human actions.
Son, someday all this will belong to your ex wife.

Post Reply

Return to “Bugs and Non-Package Requests Forum”