Welcome!
Important information
-- Spectre and Meltdown vulnerabilities
-- Change in MX sources

News
-- MX Linux on social media: here
-- Mepis support still here

Current releases
-- MX-17.1 Final release info here
-- antiX-17 release info here

New users
-- Please read this first, and don't forget to add system and hardware information to posts!
-- Here are the Forum Rules

[SOLVED] Multi-line replacing in large text files.

Message
Author
User avatar
fehlix
Forum Guide
Forum Guide
Posts: 2438
Joined: Wed Apr 11, 2018 5:09 pm

Re: Multi-line replacing in large text files.

#11 Post by fehlix » Sun Sep 16, 2018 1:00 pm

JmaCWQ wrote:
Sun Sep 16, 2018 12:38 pm
Had a play with Vim and got it replacing the correct text after a couple of tries.

Code: Select all

:%s/^\[POLYGON\]\nType=0x1/[POLYGON]^MType=0x4b/
Was replacing all instances of Label=0x1, the 0x1c's, 0x1f's etc. instead of just the ones matching.
Needed a $ after Type=0x1.

Code: Select all

:%s/^\[POLYGON\]\nType=0x1$/[POLYGON]^MType=0x4b/
works correctly :cool:
u r :number1: cool!
JmaCWQ wrote:
Sun Sep 16, 2018 12:38 pm
Forgot to mention when using Gvim the SHIFT+; (full colon) allows for typing/pasting in commands down the bottom same as in Vim running in a Terminal.
Thanks. Good to know - rarely using Gvim ;=)
Gigabyte Z77M-D3H, Intel Xeon E3-1240 V2 (Quad core), 32GB RAM,
GeForce GTX 770, Samsung SSD 850 EVO 500GB, Seagate Barracuda 4TB

User avatar
JmaCWQ
Forum Novice
Forum  Novice
Posts: 68
Joined: Fri Sep 09, 2016 4:42 am

Re: [SOLVED] Multi-line replacing in large text files.

#12 Post by JmaCWQ » Mon Sep 24, 2018 5:25 am

Hit a minor stumbling block with Vim that I can't seem to get around as yet.
Just wondering if anyone knows what I'm doing wrong here, or if what I'm trying to do is not possible?

Still working on the same large text file, I'm trying to remove all instances of

Code: Select all

[POI]
Type=0x2800
Label=MAPCENTRE
Data0=(-18.00003,124.50003)
[END]
If I search in Vim using

Code: Select all

/[POI\]\nType=0x2800\nLabel=MAPCENTRE\nData0=(-\d\d.\d\d\d\d\d,\d\d\d.\d\d\d\d\d)\n[END\]
It finds every instance with the different co-ordinates (Data0= line) correctly, but if I try to use

Code: Select all

:%s/^\[POI\]\nType=0x2800\nLabel=MAPCENTRE\nData0=(-\d\d.\d\d\d\d\d,\d\d\d.\d\d\d\d\d)\n[END\]//
to replace it with nothing, or if I try

Code: Select all

:%s/^\[POI\]\nType=0x2800\nLabel=MAPCENTRE\nData0=(-\d\d.\d\d\d\d\d,\d\d\d.\d\d\d\d\d)\n[END\]/Label=0000001/
it fails with E486: Pattern not found.
No matter what I seem to try after [END\] it fails, even though it finds each instance correctly when searching but not replacing, so I'm thinking it's maybe something to do with the square brackets, or a missing new line char or something?

Ideally I'm wanting to remove the POIs labelled MAPCENTRE info from between other info sets, still leaving a single line space (required) between each.
So instead of say for example:

Code: Select all

[POI]
Type=0x5000
Label=Bore
Data0=(-22.07398,135.32165)
[END]

[POI]
Type=0x2800
Label=MAPCENTRE
Data0=(-23.00001,135.00000)
[END]

[POI]
Type=0xb00
Label=DULCIE RANGE
City=Y
CityIdx=3807
Data0=(-22.58617,135.69304)
[END]
It would end up as:

Code: Select all

[POI]
Type=0x5000
Label=Bore
Data0=(-22.07398,135.32165)
[END]

[POI]
Type=0xb00
Label=DULCIE RANGE
City=Y
CityIdx=3807
Data0=(-22.58617,135.69304)
[END]
Possibly I've already seen a solution somewhere but not understood it & after spending nearly a day trying & achieving nothing it's time to ask for a little more help.
Thanks.

User avatar
fehlix
Forum Guide
Forum Guide
Posts: 2438
Joined: Wed Apr 11, 2018 5:09 pm

Re: [SOLVED] Multi-line replacing in large text files.

#13 Post by fehlix » Mon Sep 24, 2018 5:34 am

JmaCWQ wrote:
Mon Sep 24, 2018 5:25 am
Still working on the same large text file, I'm trying to remove all instances of

Code: Select all

[POI]
Type=0x2800
Label=MAPCENTRE
Data0=(-18.00003,124.50003)
[END]
Could you re-phrase your condition for deleting this line-block, to make it clearer.
Gigabyte Z77M-D3H, Intel Xeon E3-1240 V2 (Quad core), 32GB RAM,
GeForce GTX 770, Samsung SSD 850 EVO 500GB, Seagate Barracuda 4TB

User avatar
baldyeti
Forum Regular
Forum Regular
Posts: 109
Joined: Sat Dec 05, 2009 4:37 pm

Re: [SOLVED] Multi-line replacing in large text files.

#14 Post by baldyeti » Mon Sep 24, 2018 5:43 am

your %s line is simply missing a backslash in front of the opening square bracket for [END]; the following works for me:

Code: Select all

:%s/^\[POI\]\nType=0x2800\nLabel=MAPCENTRE\nData0=(-\d\d.\d\d\d\d\d,\d\d\d.\d\d\d\d\d)\n\[END\]//
and to remove surrounding newlines include them as optional elements of your pattern (with the question mark qualifier)

Code: Select all

:%s/^\n\?\[POI\]\nType=0x2800\nLabel=MAPCENTRE\nData0=(-\d\d.\d\d\d\d\d,\d\d\d.\d\d\d\d\d)\n\[END\]\n\?//

User avatar
JmaCWQ
Forum Novice
Forum  Novice
Posts: 68
Joined: Fri Sep 09, 2016 4:42 am

Re: [SOLVED] Multi-line replacing in large text files.

#15 Post by JmaCWQ » Mon Sep 24, 2018 7:09 am

fehlix wrote:
Mon Sep 24, 2018 5:34 am
JmaCWQ wrote:
Mon Sep 24, 2018 5:25 am
Still working on the same large text file, I'm trying to remove all instances of

Code: Select all

[POI]
Type=0x2800
Label=MAPCENTRE
Data0=(-18.00003,124.50003)
[END]
Could you re-phrase your condition for deleting this line-block, to make it clearer.
When it's deleted from between the previous line-block and the following line block, there needs to be a single line space left between the remaining blocks, so that when removed the remaining looks like this:

Code: Select all

[POI]
Type=0x6616
Label=DELLS BLUFF
Data0=(-41.77139,146.73331)
[END]

[POI]
Type=0x6616
Label=PARADISE HILL
Data0=(-42.08905,147.05861)
[END]
Not this:

Code: Select all

[POI]
Type=0x6616
Label=DELLS BLUFF
Data0=(-41.77139,146.73331)
[END]
[POI]
Type=0x6616
Label=PARADISE HILL
Data0=(-42.08905,147.05861)
[END]
Without that space it can mess up the format of the .mp (polish format text) file.
The large file is a single map I created by combining around 600 individual map tiles into one.
Each individual map had a POI co-ordinate for the centre of it and these POI's were transferred to the combined map.
I want to remove them all before finalising as they are unnecessary.
The mapping software I'm using, GPSMapEdit from geopainting dot com, allows selecting by type all objects on a map for changing to another type or deleting etc., but it will only modify or delete all selected map objects of the same type, otherwise they have to be done individually, which is why I'm using Vim to edit this file.
In this instance removing only the Type=0x2800 POI's with the label of MAPCENTRE, leaving the differently labelled POI's with the same type alone.

User avatar
JmaCWQ
Forum Novice
Forum  Novice
Posts: 68
Joined: Fri Sep 09, 2016 4:42 am

Re: [SOLVED] Multi-line replacing in large text files.

#16 Post by JmaCWQ » Mon Sep 24, 2018 7:24 am

baldyeti wrote:
Mon Sep 24, 2018 5:43 am
your %s line is simply missing a backslash in front of the opening square bracket for [END]; the following works for me:

Code: Select all

:%s/^\[POI\]\nType=0x2800\nLabel=MAPCENTRE\nData0=(-\d\d.\d\d\d\d\d,\d\d\d.\d\d\d\d\d)\n\[END\]//
and to remove surrounding newlines include them as optional elements of your pattern (with the question mark qualifier)

Code: Select all

:%s/^\n\?\[POI\]\nType=0x2800\nLabel=MAPCENTRE\nData0=(-\d\d.\d\d\d\d\d,\d\d\d.\d\d\d\d\d)\n\[END\]\n\?//
Yes, the missing backslash was the culprit, I would probably never have figured that out.

Thank You.

User avatar
fehlix
Forum Guide
Forum Guide
Posts: 2438
Joined: Wed Apr 11, 2018 5:09 pm

Re: [SOLVED] Multi-line replacing in large text files.

#17 Post by fehlix » Mon Sep 24, 2018 2:16 pm

JmaCWQ wrote:
Mon Sep 24, 2018 7:09 am
fehlix wrote:
Mon Sep 24, 2018 5:34 am
Could you re-phrase your condition for deleting this line-block, to make it clearer.
In this instance removing only the Type=0x2800 POI's with the label of MAPCENTRE, leaving the differently labelled POI's with the same type alone.
Perhaps, this would then fit to your requirements:

Code: Select all

# vim multi-line block delete
#
:%s/^\n\?\[POI\]\nType=0x2800\nLabel=MAPCENTRE\n\_.\{-}\[END\]\n\?//
# with
#  \_.     any single character or end-of-line
#  \{-}    0 or more  as few as possible
Gigabyte Z77M-D3H, Intel Xeon E3-1240 V2 (Quad core), 32GB RAM,
GeForce GTX 770, Samsung SSD 850 EVO 500GB, Seagate Barracuda 4TB

User avatar
JmaCWQ
Forum Novice
Forum  Novice
Posts: 68
Joined: Fri Sep 09, 2016 4:42 am

Re: [SOLVED] Multi-line replacing in large text files.

#18 Post by JmaCWQ » Mon Sep 24, 2018 3:52 pm

Thank You fehlix, I will try that next time I'm working on that file.

User avatar
JmaCWQ
Forum Novice
Forum  Novice
Posts: 68
Joined: Fri Sep 09, 2016 4:42 am

Re: [SOLVED] Multi-line replacing in large text files.

#19 Post by JmaCWQ » Fri Oct 12, 2018 12:34 am

Something else I can't figure out with Vim & would be grateful for any assistance with.

Multiple text files all have the single line MyOziMap in them, starting at the beginning of the line with nothing else on the line.
I planned to change this when originally editing them but forgot, now I have several thousand I would like to change.
I'm trying to change MyOziMap to MyOziMap-1 for the first file, MyOziMap-2 for the second file, MyOziMap-3 for the third file, and so on.

I can almost get it to work but when it hits a number with multiples of 10 there seems to be something I'm missing.

If I do either:

Code: Select all

:set hidden | :%s/^MyOziMap/MyOziMap-1/ | :n | :%s/^MyOziMap/MyOziMap-2/ | :n | :%s/^MyOziMap/MyOziMap-3/ | :n | :%s/^MyOziMap/MyOziMap-4/ | :n | :%s/^MyOziMap/MyOziMap-5/ | :n | :%s/^MyOziMap/MyOziMap-6/ | :n | :%s/^MyOziMap/MyOziMap-7/ | :n | :%s/^MyOziMap/MyOziMap-8/ | :n | :%s/^MyOziMap/MyOziMap-9/ | :n | :%s/^MyOziMap/MyOziMap-10/ | :%s/^MyOziMap/MyOziMap-11/ | :n | :%s/^MyOziMap/MyOziMap-12/ | :n | :%s/^MyOziMap/MyOziMap-13/ | :n | :%s/^MyOziMap/MyOziMap-14/ | :n | :%s/^MyOziMap/MyOziMap-15/ | :n | :%s/^MyOziMap/MyOziMap-16/ | :wa
or:

Code: Select all

:set hidden | :%s/^MyOziMap/&-1/ | :n | :%s/^MyOziMap/&-2/ | :n | :%s/^MyOziMap/&-3/ | :n | :%s/^MyOziMap/&-4/ | :n | :%s/^MyOziMap/&-5/ | :n | :%s/^MyOziMap/&-6/ | :n | :%s/^MyOziMap/&-7/ | :n | :%s/^MyOziMap/&-8/ | :n | :%s/^MyOziMap/&-9/ | :n | :%s/^MyOziMap/&-10/ | :%s/^MyOziMap/&-11/ | :n | :%s/^MyOziMap/&-12/ | :n | :%s/^MyOziMap/&-13/ | :n | :%s/^MyOziMap/&-14/ | :n | :%s/^MyOziMap/&-15/ | :n | :%s/^MyOziMap/&-16/ | :wa
It appears it works fine when I check them, except instead of seeing MyOziMap-10 and then in the next file MyOziMap-11, I end up with MyOziMap-11-10 in the tenth file, then it skips MyOziMap-11 & goes straight to MyOziMap-12, and in the last file it makes no changes.
If there's more files than the above example at each multiple of 10 the same happens, leaving the same number of files unchanged at the end of the list.

I guess I've got the formatting of the command wrong somehow but I've searched and searched, probably not understanding the correct terms to search for and so far haven't found anything similar to the issue I see here.

Thanks

User avatar
baldyeti
Forum Regular
Forum Regular
Posts: 109
Joined: Sat Dec 05, 2009 4:37 pm

Re: [SOLVED] Multi-line replacing in large text files.

#20 Post by baldyeti » Fri Oct 12, 2018 5:37 am

Please do not add to threads marked as "solved" but open a new one.

It seems to me you're using the wrong tool.
You mention thousands of files, how long do you thing your vim line will get when you need to add a new ":n etc" block for every additional file/buffer ?
It may be possible to do using vimscript but I believe time spent on learning tried and true unix command-line utilities is better spent.

See sh, sed, awk...
You'd need to write a script but that is easier to maintain and get right than awkwardly long vim command lines.

Post Reply

Return to “Software / Configuration”