Welcome!
Important information
-- Spectre and Meltdown vulnerabilities
-- Change in MX sources

News
-- MX Linux on social media: here
-- Mepis support still here

Current releases
-- MX-17.1 Final release info here
-- antiX-17 release info here

New users
-- Please read this first, and don't forget to add system and hardware information to posts!
-- Here are the Forum Rules

[SOLVED] Multi-line replacing in large text files.

Message
Author
User avatar
fehlix
Forum Guide
Forum Guide
Posts: 1734
Joined: Wed Apr 11, 2018 5:09 pm

Re: [SOLVED] Multi-line replacing in large text files.

#21 Post by fehlix » Fri Oct 12, 2018 5:39 am

JmaCWQ wrote:
Fri Oct 12, 2018 12:34 am
Multiple text files all have the single line MyOziMap in them, starting at the beginning of the line with nothing else on the line.
Multi-file processing I'd rather do with sed on the commandline instead with vim, like this example ( with echo to show )

Code: Select all

i=1; for f in $( ls MyMapFile*.map) ; do I=$((i++)); echo sed -i '/^MyOziMap$/{s/$/-'$(printf '%04d' $I)'/;q}' $f ; done 
sed -i /^MyOziMap$/{s/$/-0001/;q} MyMapFile-AA.map
sed -i /^MyOziMap$/{s/$/-0002/;q} MyMapFile-AB.map
sed -i /^MyOziMap$/{s/$/-0003/;q} MyMapFile-AC.map
sed -i /^MyOziMap$/{s/$/-0004/;q} MyMapFile-AD.map
sed -i /^MyOziMap$/{s/$/-0005/;q} MyMapFile-AE.map
sed -i /^MyOziMap$/{s/$/-0006/;q} MyMapFile-AF.map
sed -i /^MyOziMap$/{s/$/-0007/;q} MyMapFile-AG.map
sed -i /^MyOziMap$/{s/$/-0008/;q} MyMapFile-AH.map
sed -i /^MyOziMap$/{s/$/-0009/;q} MyMapFile-AI.map
sed -i /^MyOziMap$/{s/$/-0010/;q} MyMapFile-AJ.map
...
Gigabyte Z77M-D3H, Intel Xeon E3-1240 V2 (Quad core), 32GB RAM,
GeForce GTX 770, Samsung SSD 850 EVO 500GB, Seagate Barracuda 4TB

User avatar
JmaCWQ
Forum Novice
Forum  Novice
Posts: 50
Joined: Fri Sep 09, 2016 4:42 am

Re: [SOLVED] Multi-line replacing in large text files.

#22 Post by JmaCWQ » Fri Oct 12, 2018 8:03 am

baldyeti wrote:
Fri Oct 12, 2018 5:37 am
Please do not add to threads marked as "solved" but open a new one.
Ok no probs, was just trying to keep it all together as I've found it very very frustrating over the last few weeks trying to learn this stuff having to search hundreds of threads for each tiny little snippet of info.
I was thinking it might be far easier for someone else trying to learn similar things if all the info was in the one spot.
baldyeti wrote:
Fri Oct 12, 2018 5:37 am
It seems to me you're using the wrong tool.
You mention thousands of files, how long do you thing your vim line will get when you need to add a new ":n etc" block for every additional file/buffer ?
It may be possible to do using vimscript but I believe time spent on learning tried and true unix command-line utilities is better spent.
The line is already done, took me about 5 minutes to sort out, yes there is thousands of files but the maximum number I'll be doing at one time is 121.
In the last hour or so I've got it working successfully for that 16 files previously mentioned by changing the desired MyOziMap-1 to MyOziMap-001, MyOziMap-002 etc. etc..
Why it makes a difference I have no idea but it now works.
I'm now reading up on how ranges are applied with Vim in commands, as if I try with numbers of files over about 20 I get a No range allowed error, even though each command specifies :% which is supposed to be the file's full range if I understand things correctly.
baldyeti wrote:
Fri Oct 12, 2018 5:37 am
See sh, sed, awk...
You'd need to write a script but that is easier to maintain and get right than awkwardly long vim command lines.
Thanks for the replies and the advice, to be perfectly honest I'm really struggling to get as far as I have trying to learn Vim & am not sure I could learn something else I don't understand at present, why I decided to stick with Vim for the moment is it's starting to make sense to me how it's commands work while I've been using it...I understand why you've suggested what you have though.

This is a one-off project for me hopefully, I first started it about 5 years ago & have been doing bits n pieces as time allowed over that time.
Thanks to the use of Vim and the help from you guys over the last few weeks I'm now in the home straight, getting these lines in each file named differently instead of all the same is the last step before I can start compiling the finished product.

User avatar
JmaCWQ
Forum Novice
Forum  Novice
Posts: 50
Joined: Fri Sep 09, 2016 4:42 am

Re: [SOLVED] Multi-line replacing in large text files.

#23 Post by JmaCWQ » Fri Oct 12, 2018 8:16 am

fehlix wrote:
Fri Oct 12, 2018 5:39 am
JmaCWQ wrote:
Fri Oct 12, 2018 12:34 am
Multiple text files all have the single line MyOziMap in them, starting at the beginning of the line with nothing else on the line.
Multi-file processing I'd rather do with sed on the commandline instead with vim, like this example ( with echo to show )

Code: Select all

i=1; for f in $( ls MyMapFile*.map) ; do I=$((i++)); echo sed -i '/^MyOziMap$/{s/$/-'$(printf '%04d' $I)'/;q}' $f ; done 
sed -i /^MyOziMap$/{s/$/-0001/;q} MyMapFile-AA.map
sed -i /^MyOziMap$/{s/$/-0002/;q} MyMapFile-AB.map
sed -i /^MyOziMap$/{s/$/-0003/;q} MyMapFile-AC.map
sed -i /^MyOziMap$/{s/$/-0004/;q} MyMapFile-AD.map
sed -i /^MyOziMap$/{s/$/-0005/;q} MyMapFile-AE.map
sed -i /^MyOziMap$/{s/$/-0006/;q} MyMapFile-AF.map
sed -i /^MyOziMap$/{s/$/-0007/;q} MyMapFile-AG.map
sed -i /^MyOziMap$/{s/$/-0008/;q} MyMapFile-AH.map
sed -i /^MyOziMap$/{s/$/-0009/;q} MyMapFile-AI.map
sed -i /^MyOziMap$/{s/$/-0010/;q} MyMapFile-AJ.map
...
Thanks again fehlix, I will attempt to use your example as a guide if adding the line number for the range in my Vim command line fails 8)

User avatar
baldyeti
Forum Novice
Forum  Novice
Posts: 79
Joined: Sat Dec 05, 2009 4:37 pm

Re: [SOLVED] Multi-line replacing in large text files.

#24 Post by baldyeti » Fri Oct 12, 2018 9:31 am

JmaCWQ wrote:
Fri Oct 12, 2018 8:03 am
Thanks for the replies and the advice, to be perfectly honest I'm really struggling to get as far as I have trying to learn Vim & am not sure I could learn something else I don't understand at present, why I decided to stick with Vim for the moment is it's starting to make sense to me how it's commands work while I've been using it...I understand why you've suggested what you have though.
I know where you're at - learning vi is some rite of passage. I still remember my 1st day at that new job, I was super eager to learn about Unix which i had not used previously. They put me in front of a serial terminal and told me to get acquainted with the environment. When they came back to take me to lunch, I was so embarrassed to admit i did not even know how to close the bloody thing! 30-odd years forward, hitting the <esc> key has become 2d nature. I have used vi in some form on DOS, Windows, Unices, OS/2, BeOS and VMS. Works the same everywhere. Of course most of these platforms are irrelevant today but vim sure has kept vi alive and i am so grateful all I learnt after that lunch remains applicable today !

I think vi/vim can sort of be used as a streaming editor somewhat like sed if you invoke it as "ex" or use the -e argument to its command line, but have'nt used that much. I'd' probably resort to awk for this kind of things.

User avatar
JmaCWQ
Forum Novice
Forum  Novice
Posts: 50
Joined: Fri Sep 09, 2016 4:42 am

Re: [SOLVED] Multi-line replacing in large text files.

#25 Post by JmaCWQ » Fri Oct 12, 2018 11:25 am

baldyeti wrote:
Fri Oct 12, 2018 9:31 am
I know where you're at - learning vi is some rite of passage. I still remember my 1st day at that new job, I was super eager to learn about Unix which i had not used previously. They put me in front of a serial terminal and told me to get acquainted with the environment. When they came back to take me to lunch, I was so embarrassed to admit i did not even know how to close the bloody thing! 30-odd years forward, hitting the <esc> key has become 2d nature. I have used vi in some form on DOS, Windows, Unices, OS/2, BeOS and VMS. Works the same everywhere. Of course most of these platforms are irrelevant today but vim sure has kept vi alive and i am so grateful all I learnt after that lunch remains applicable today !

I think vi/vim can sort of be used as a streaming editor somewhat like sed if you invoke it as "ex" or use the -e argument to its command line, but have'nt used that much. I'd' probably resort to awk for this kind of things.
Got the job done with Vim, once I figured out what the problem was it only took a few minutes to change that one line in nearly 3200 files.
Problem was between the keyboard and chair as usual, lol, missed a couple of characters in the entry for the 21st file in the command, fixed that and all is now well.
It was good to see the final 'Press Enter or type a command to continue'.
I've probably done more towards the final outcome using Vim in the last few weeks than everything else combined in the last 5 years, and learnt something during the journey which is always a bonus.
Must be pretty powerful software if one understands enough to use it at it's full potential.

Edit: Thanks again to everyone who helped me achieve this :cool:

User avatar
baldyeti
Forum Novice
Forum  Novice
Posts: 79
Joined: Sat Dec 05, 2009 4:37 pm

Re: [SOLVED] Multi-line replacing in large text files.

#26 Post by baldyeti » Fri Oct 12, 2018 1:57 pm

JmaCWQn you're past the basics, but for reference the best vi tutorial I know of is the series of articles Walter Zintz once wrote for UnixWorld. Oddly enough it seems to have become hard to find. There is a copy here : http://www.hzdr.de/FWR/VB/TUTORIALS/

Vim can be overwhelming but even plain vi is powerful (and might be all you have anyway if using nvi or working occasionally under another unix)

User avatar
JmaCWQ
Forum Novice
Forum  Novice
Posts: 50
Joined: Fri Sep 09, 2016 4:42 am

Re: [SOLVED] Multi-line replacing in large text files.

#27 Post by JmaCWQ » Fri Oct 12, 2018 2:56 pm

baldyeti wrote:
Fri Oct 12, 2018 1:57 pm
There is a copy here : http://www.hzdr.de/FWR/VB/TUTORIALS/
Thanks for that, bookmarked for future reading :toast:

Post Reply

Return to “Software / Configuration”