Newbie needs help w/ creating database application
I work as a coop student and I need some serious help so I sincerely appreciate everyone's input.
Basically my manager has told me to create an application which takes text files, processes them and creates a report which the user can change tolerances on what columns or fields they want to view. For example, if 10000 files (each file represents company/client audit info) are input into the application for 1 year, the user should be able to filter and compare files from separte months based on column headings in the files such as "% population" and "field".
Background info on why this app needs to be created: 1000s of files containing client audit info is fed into our servers daily. These files contain several columns in regards to client financial info. Someone needs to go through each of these files and just check if there is any irregularities in the data. An example of the irregularities could include info in a column containing number of docs to go from 100 000 docs in 1 year to 10 docs the next.
The problem is that no one has the time to sit down and check irregularities in 1000 files. Thus, it would be great if there was an application which goes through each file, reports any irregularities AND have a good user interface where the auditor can compare stats for files based on filters such as column headings and monthly.
I'm new to database programming so I have the following questions:
1) What language should I use to make this app? We have access to XML, SQL, VB, Visual Studio
2) Any General Tips for making this app?
3) Did anyone do something similar to this before?
4) General logic behind this application?
5) Flowchart help
6) Any link to helpful resources for an app of this context