The tool that comes to my mind with tasks like this is 'Awk.' I don't know exactly what the acronym stands-for but it's really useful for working with complex inputs a line at a time. For example, it's handy when all you've got is the output of a COBOL report (saved to a disk file) and you're supposed to extract some data from it.
Another tool might be 'grep' (global regular expression pooh-bah).
A surprisingly useful programming language for working with files like this is also, Python. If you can use Perl then be my guest. ;-)