RMS files come in at least two flavors. Application managed (meaning you would need the COBOL copybook to understand the data), and RMS managed (meaning that the database engine understands at least enough about the data to present it without any application code at all).
If you have RMS managed files there are other options, but for the sake of this discussion I'll assume that you don't have that convenience.
Your first step is to get the data converted to flat VMS text files. These files need to have no application specific features (so you could open them in a text editor), and need to include ALL of the data in a form that is acceptable to every user involved in the transition. Beware of rounding issues and formatting details, since these are often sticking points in any system conversion.
Your next step is to transfer those flat text files to somewhere that your SQL Server can access them. This is usually just a simple FTP, but it can sometimes be much more complex than that.
Once you've got the text files where the SQL Server can access them, you need to import those text files "as is" into a staging area inside SQL Server. The DTS import wizard makes this almost trival. If you need more speed for large amounts of data (over about 300 Gb), you can take the time to learn to use BCP or the BULK INSERT command which will buy you better performance, but requires considerably more skill to use than the import wizard.
At that point, you'll have access to your raw data. Then you need to start to think about how you want to arrange your data for use by your application. Most of the time, the "flat file" approach that is needed for transferring data from one system to another is a poor choice for application usage. This is an area where you'll probably need the most help because PC programming is radically different than VMS programming, and the PC users are much more demanding than most VMS users too. Unfortunately, this stage is pure application design, so you'll need someone that understands a lot about your application to help you much with this stage.
Thanks for your useful info, I have them as RMS managed.
So far what you provided me is sufficient, but do you have any links that speak about that migration in case I have any further questions?
As a side note to this discussion, since you have RMS managed data, you may be able to find an ODBC driver that will allow SQL Server to read the RMS data directly. Check your VMS distribution or your IT support group to see if this is available, since it could make your job much faster and easier.