Welcome!

The kernel problem with recent updates has been solved. Find the solution here

Important information
-- Required MX 15/16 Repository Changes
-- Information on torrent hosting changes
-- Information on MX15/16 GPG Keys
-- Spectre and Meltdown vulnerabilities

News
-- Introducing our new Website
-- MX Linux on social media: here

Current releases
-- MX-18.3 Point Release release info here
-- Migration Information to MX-18 here
-- antiX-17.4.1 release info here

New users
-- Please read this first, and don't forget to add system and hardware information to posts!
-- Here are the Forum Rules

[(sort of) Solved] Nearly unusable system

Help for Current Versions of MX
Post Reply
User avatar
thomasl
Forum Regular
Forum Regular
Posts: 190
Joined: Sun Feb 04, 2018 10:26 am

[(sort of) Solved] Nearly unusable system

#1

Post by thomasl » Mon Aug 27, 2018 12:12 pm

I have a PC with 3rd gen core i5 CPU (4 real cores), 12GB RAM and a 240GB Samsung EVO850 SSD. This system is not the fastest under the sun but it's no slouch either and normally, everything is smooth and fluid and well-behaved.

However, I've just copied (with rsync -aW) a rather large directory of .jpg files (~50GB) from the SSD to a USB3.0 stick (it's recognised as USB3, I've checked that with lsusb). The stick has a nominal write speed of ~90MB/s but it seemed the copy wasn't much faster than 25MB/s.

However, the real problem is that during the copy process, the PC became almost unusable. Almost any browser activity (Palemoon) like loading a new page or even just adding a bookmark resulted in 30, 40, 50 seconds of complete standstill (I first thought the browser was dead but it was just totally incommunicado for nearly a minute). I checked in a terminal window with htop, none of the 4 cores had more than 5% load over any 10 second period, memory use was well below 1GB. In the same terminal a df command took more than 40 seconds to complete and sometimes the command line was dead as well for some 5 or 10 seconds.

This looks like a very bad case of a severe i/o bottleneck somewhere... but I can't see what I could do differently. I have done the same or very similar jobs with the exact same hardware in Windows 7/64 and while the system is clearly not as sprightly as under zero load it remains definitely on the usable side.

I do hope there's a better way to copy $BIG_DATA to a USB stick than that... because I'd have been better off to actually boot into Windows, do the copy there and then reboot into MX... ;)
Last edited by thomasl on Tue Sep 11, 2018 5:43 am, edited 1 time in total.
Dual-boot MX17.1/64 frugal root persistence + Windows 7 on Lenovo Edge72 i5-3470S/12GB and Tosh R950 i5-3340M/8GB
I used to think that good grammar is important. Now I know that good wine is importanter.

User avatar
dolphin_oracle
Forum Veteran
Forum Veteran
Posts: 11880
Joined: Sun Dec 16, 2007 1:17 pm

Re: Nearly unusable system

#2

Post by dolphin_oracle » Mon Aug 27, 2018 12:15 pm

I'm wondering if the tumblerd daemon is out of control (if its running). tumbler generates thumbnails for Xfce apps, but it can be problematic on very large directories of graphics.

That's a theory. You could try killing tumblerd (pkill tumblerd).
http://www.youtube.com/runwiththedolphin
lenovo ThinkPad T530 - MX-18.3
lenovo s21e - antiX-19 beta 2 (live-USB)
FYI: mx "test" repo is not the same thing as debian testing repo.

User avatar
GuiGuy
Forum Guide
Forum Guide
Posts: 1642
Joined: Sun Dec 16, 2007 6:29 pm

Re: Nearly unusable system

#3

Post by GuiGuy » Mon Aug 27, 2018 12:47 pm

Of course we we do not yet know whether tumbler(d) is the culprit here.
But, just in case, here is my tuppenceworth:-
I have found that MX is perfectly usable with tumbler completely removed.

User avatar
thomasl
Forum Regular
Forum Regular
Posts: 190
Joined: Sun Feb 04, 2018 10:26 am

Re: Nearly unusable system

#4

Post by thomasl » Mon Aug 27, 2018 12:55 pm

@dolphin_oracle: Nope, tumblerd isn't/wasn't running. And during the whole copy process the CPU load was virtually all the time below 5% on all 4 cores. htop itself had no trouble whatsoever updating its display. I assume that the browser and the other hangs were i/o-related: once a page was finally loaded, scrolling in the page was as fast as ever.

@GuiGuy: I am sure it is... and anyway, this nailing of thumbs (and similar such things) is something I disable if I can.
Dual-boot MX17.1/64 frugal root persistence + Windows 7 on Lenovo Edge72 i5-3470S/12GB and Tosh R950 i5-3340M/8GB
I used to think that good grammar is important. Now I know that good wine is importanter.

User avatar
ChrisUK
Forum Regular
Forum Regular
Posts: 279
Joined: Tue Dec 12, 2017 1:04 pm

Re: Nearly unusable system

#5

Post by ChrisUK » Mon Aug 27, 2018 1:10 pm

I know you can use iotop to get an idea what's happening, but I'd suggest you install dstat to get a more complete picture. Maybe try

Code: Select all

dstat -c --top-cpu -d --top-bio --top-latency
in the terminal before a copy operation.

Example output:
Screenshot.png
You do not have the required permissions to view the files attached to this post.
Chris

MX 18 - Manjaro

User avatar
Stevo
Forum Veteran
Forum Veteran
Posts: 19561
Joined: Fri Dec 15, 2006 8:07 pm

Re: Nearly unusable system

#6

Post by Stevo » Mon Aug 27, 2018 2:34 pm

It might also be related to how the kernel handles the copying, so some tests with the Liquorix kernel might have different results...or one of the backported Debian kernels.

clicktician
Forum Regular
Forum Regular
Posts: 227
Joined: Sat May 02, 2015 4:35 pm

Re: Nearly unusable system

#7

Post by clicktician » Mon Aug 27, 2018 2:43 pm

If you just want your system to be usable during the copy, try ultracopier (in the repositories - you can just apt-get it or synaptic).

It puts a disk icon on the lower part of the left panel. If you right-click on it, you can enqueue directories to be copied from a source to a destination in background.

Each copy task has options to limit the transfer rate in KB/s, among other things.
Son, someday all this will belong to your ex wife.

User avatar
thomasl
Forum Regular
Forum Regular
Posts: 190
Joined: Sun Feb 04, 2018 10:26 am

Re: Nearly unusable system

#8

Post by thomasl » Tue Aug 28, 2018 1:33 pm

@ChrisUK and @clicktician: Thanks for that, I will try these things next time I have a big copy job and report back.

@Stevo: I had the Liquorix kernel on a test install but in the quick tests I did I saw no difference so went back to stock. Also creating rootfs files needed a lot more time with Liquorix than with stock.

It seems this problem is somehow related to USB sticks (ie flash). USB HDDs (even slow ones) don't seem to have such a pronounced negative effect although the system still feels less responsive than the equivalent hardware under Windows. Sigh :snail:
Dual-boot MX17.1/64 frugal root persistence + Windows 7 on Lenovo Edge72 i5-3470S/12GB and Tosh R950 i5-3340M/8GB
I used to think that good grammar is important. Now I know that good wine is importanter.

User avatar
thomasl
Forum Regular
Forum Regular
Posts: 190
Joined: Sun Feb 04, 2018 10:26 am

Re: Nearly unusable system

#9

Post by thomasl » Tue Sep 11, 2018 5:43 am

Well... I've posted this very same question on superuser.com and did receive an answer that, if not completely solved my troubles, helped a lot. There are two main things to do:

1. Install the nocache utility (it's in the stable repo) and use it to disable rsync's caching of the files copied. (Apparently, there's a --drop-cache option in some patched rsync's but it's not in the one installed here.) This improves performance significantly.

2. Disable transparent huge pages by adding "transparent_hugepage=never" to the kernel boot options. Not so sure about that one but it may be good idea anyway.
Dual-boot MX17.1/64 frugal root persistence + Windows 7 on Lenovo Edge72 i5-3470S/12GB and Tosh R950 i5-3340M/8GB
I used to think that good grammar is important. Now I know that good wine is importanter.

Post Reply

Return to “MX Help”