MX-19 Beta 2.1 did not check antiX/linuxfs integrity, result: installed system had mystic problems

[attila123]

Hi, I installed MX-19 Beta 2.1 using a pendrive, but unfortunately the pendrive was faulty and it caused an unworkable system with several "random", unexplainable errors (such as some system file did not get installed, /usr/sbin/NetworkManager had "Exec format error", etc.).
When I downloaded the iso image, I checked the sha256 of the downloaded iso, it was OK.
I used dd to write the iso to the pendrive, but due to HW failure of the pendrive the antiX/linuxfs file was written corrupted to it.
Even though next to this file there is antiX/linuxfs.md5, but the installer did not used that to verify antiX/linuxfs! So this is an improvement request to verify antiX/linuxfs based on its checksum (and any other such files, if any, did not check). This could save some trouble for users in similar situations, plus it would help to avoid false random bugreports.

(How I found out that antiX/linuxfs was corrupted on the pendrive: to compare stuff I loop mounted the downloaded iso + plugged in the pendrive (which was automounted) and compared the two filesystems/directories with 'meld'. Soon meld found that antiX/linuxfs was different in the pendrive.)

(Then I did the installation from a healthy pendrive and it went well.)

[fehlix]

Hi, I installed MX-19 Beta 2.1 using a pendrive, but unfortunately the pendrive was faulty and it caused an unworkable system with several "random", unexplainable errors (such as some system file did not get installed, /usr/sbin/NetworkManager had "Exec format error", etc.).
When I downloaded the iso image, I checked the sha256 of the downloaded iso, it was OK.
I used dd to write the iso to the pendrive, but due to HW failure of the pendrive the antiX/linuxfs file was written corrupted to it.
Even though next to this file there is antiX/linuxfs.md5, but the installer did not used that to verify antiX/linuxfs! So this is an improvement request to verify antiX/linuxfs based on its checksum (and any other such files, if any, did not check). This could save some trouble for users in similar situations, plus it would help to avoid false random bugreports.

When you look on to on-screen log or the saved installer log at /var/log/mx-live-usb-maker.log
you will find something like this:

Code: Select all

live-usb-maker: check-usb-md5 >> check-usb-md5
check md5 for initrd.gz
check md5 for linuxfs
check md5 for vmlinuz
live-usb-maker: uuids >> uuids
live-usb-maker: install >> install

which indicates the md5-sum check on the copied files was done. So it appears rather a faulty USB-stick.
EDIT: On LiveUSB created not with Live-USB-Maker you would check integrity with bootoption "checkmd5"

[attila123]

@fehlix it cannot be assumed that people will use Live-USB-Maker. They may use dd (like I did) or use their other favourite graphical live iso writer of their favourite/current OS.
It can be a nice feature in such an application to verify the result, but the responsibility falls entirely on the installer to do this, and (in my opinion) unconditionally (by default).

Eg. any decent package manager is checking the integrity of the packages it is installing right before installation. I don't think it is a good solution to put this functionality aside in some "Customize Boot" menu. It's just way more important than that. Besides, I tried to run it, but it did not get to the point where it would actually check the integrity, because it first tried to find a partition with enough free space. This is (IMHO) an unnecessary tight coupling between checkmd5 and the actual installation. One may want to check the interity of the live usb on a computer which is full regarding partitions.

I think probable the best solution would be to at least have an optional, but well visible step during the installation steps to check the integrity. Whether it should be enabled or disabled is not so important: some people would run the check, some not.

[fehlix]

@fehlix it cannot be assumed that people will use Live-USB-Maker. They may use dd (like I did) or use their other favourite graphical live iso writer of their favourite/current OS.

Right, so I added this "On LiveUSB created not with Live-USB-Maker you would check integrity with bootoption "checkmd5"'. What you are proposing is like "check my system is not corrupt" before installing. Don't think that the task if the Installer is to check the LiveSystem for integrity. The integrity check of the main livesystem files can be made by the user as boot option, regardless how the LiveSystem was created.

[attila123]

In basically any other distro installer I do not need to worry about the integrity of the LiveSystem, because if there is a corruption in the USB medium, then either the installer will crash, or the package manager will catch and point out the error (unless a newer package is downloaded from the Internet, but let's stick to offline installs), and the situation is obvious. So I think most users got used to it and not checking the integrity of the installer in the USB medium.
The problem here is that if the user is not aware of the checkmd5 option then the installation can seemingly go through like everything is OK, and on the first boot he/she will experience very strange and confusing errors.
So yes, I propose that it would have a significant value to include an option (e.g. a checkbox) *during the normal installation wizard* to check the intergrity of the files like antiX/linuxfs, because the normal (and expected) "safety belt" of the package managers is not there during this installation.

[fehlix]

So yes, I propose that it would have a significant value to include an option (e.g. a checkbox) *during the normal installation wizard* to check the intergrity of the files like antiX/linuxfs, because the normal (and expected) "safety belt" of the package managers is not there during this installation.

.. undestand. OTOH, the live system runs on linuxfs etc, if they would be corrupt, the message you would get from such check would be that the system you are running might not be integer, which would include the message itself. So you can not rely on the check at all. In addition, you can install during live lot's of software before you install. The installer has no way to check those additional changes. So I still think checking integrity within the installer might not be the best place.

[m_pav]

attaila123 has a good point that needs to be addressed. I proposed checking hard disk drives S.M.A.R.T status prior to installation, and that has been going well to the best of my knowledge, but with the rise in SSD implementation, that might need to be revisited as it only checks spinning platter drives.

When I was running an IT shop and people wanted different distros, I recall seeing a clear option prior to boot, or install, can't recall which, to check the installation media. We already have the F4 option checkmd5 and that runs well, but it's not easily seen by the masses. Could that not at the very least be replicated as a persistent GRUB boot entry for when running live to bring it more into the limelight and worded so as to dumb it down to laymans language? An entry something like "Check media first and continue booting"

This may assist with those who insist on using those decrepit Sandisk thumb drives that cause so many issues.

[anticapitalista]

"Check media first and continue booting" will not fit in the live F4 menu. Where do you suggest it 'appears'? It will also slow down live boot?

[attila123]

Hi,

1. I start to understand the situation better. The integrity check needs to be performed at an early stage, before e.g. antiX/linuxfs gets mounted.
2. Also the content of the antiX/linuxfs may change (sorry I am not familiar with AntiX and its live feautres, maybe I should have started with that), if some applications are installed to it, right? Which would make antiX/linuxfs.md5 invalid? Or not?
3. I observed that the F4 thing you are talking about only appears in BIOS boot, the UEFI boot screen looks quite different (it is possible to see the difference in VirtualBox if you enable EFI (in VM Settings -> System there is a checkbox for that). In UEFI boot you don't have F4 and you need to select "MX-19beta2.1 X64 Customize Boot (text menus)" and after selecting langueage, console columns, timezone, you can select checkmd5 from a 15 items list. Then you need to select the persistence option. Finally md5 checking is done.
3a. So far everything is quite technical, but now the output is hidden from the user by default and need to press Alt + F1 to see the output, which is unnecessary in my opinion in this situation. If someone is willing to go through text based menus perhaps he/she is not intimidated from the text based output.
3b. I tried it with my faulty pendrive (which has antiX/linuxfs corrupted), so the screen "Non-Fatal Error" appeared prompting the user to continue, power off, or reboot. Well, I wonder what is fatal error then? Also, I think it is unnecessary to clear the screen to bring up this new screen. It is only about 7 lines, and I would love to see the details about the file checks.
4. As there is a distinct top-level entry for Memory Test (in both BIOS and UEFI mode), I think similarly it would be a useful top-level menu item sg. like "Check media" as m_pav is also suggesting.
5. It would be nice to somehow reconcile the BIOS and UEFI mode boot menu, taking the "best of the both worlds" for both boot modes. Otherwise e.g. you may be talking about BIOS boot, user sees the UEFI boot screen and there may be confusion.