Web Hosting Talk







View Full Version : perl or shell list compare/extraction


Slidey
03-08-2005, 09:08 AM
hi

ive got 2 files, each with about 650k lines

the lines in each are in the form: id data

one file has the id, and a date. the other has the id and some other information about that id.

I need to make a new file which has an amalgamation of the original 2, ie id data1 data2

any ideas?

the id's are mostly the same in both, but not 100% so i cant rely on using that. I'd also like a list (separately is fine) of any that are in one file or not the other (but i can probably do this with awk and diff)

cheers in advance

folsom
03-08-2005, 09:40 AM
join should do the trick

join -1 1 -2 1 file1 file2

which just means join field 1 of file1 (-1 1) to field 1 of file2 (-2 1)

Slidey
03-08-2005, 09:48 AM
what happens if the key of field1 of file2 doesnt match the key of field2 ?

folsom
03-08-2005, 09:54 AM
gears% cat a.txt
a d
b e
c f
d foo
gears% cat b.txt
a 1
b 2
c 3
gears% join -1 1 -2 1 a.txt b.txt
a d 1
b e 2
c f 3

Notice that record d in file 1 does not show up in the join, but if I add -a 1 then


gears% join -1 1 -2 1 -a 1 a.txt b.txt
a d 1
b e 2
c f 3
d foo