Results 1 to 7 of 7
  1. #1
    Join Date
    Jun 2005
    Location
    Denver, CO
    Posts
    100

    Unanswered: Import Multiple CSV Files to Multiple Tables

    I have about 2,000 CSV files that have "widget" pricing information in them from about 60 markets. Each line in the CSV represents a market, the name is indicated in column 2. Each CSV file also represents the widget price in each market for a single day. What I need to do is import all 2K CSV files into 60 different tables so that I have 60 tables (markets) with a line for each price on each day. Any suggestions on where to begin? I tired to make things easy by using the SQL Data import tool, but found that to be a non-starter. Many thanks in advance.
    BillS

  2. #2
    Join Date
    Nov 2004
    Location
    on the wrong server
    Posts
    8,835
    Provided Answers: 6
    SSIS packages that take the CSV filename in as a package variable.

    Use this old trick to get all of files in your directory...

    INSERT INTO mytable(folder)
    xp_cmdshell 'DIR C:\myfolder'

    update mytable set folder = RIGHT(folder,LEN(folder) - 39)

    And just spin that in a cursor to fire off your package and pass it in your file name.

    piece of cake.
    “If one brings so much courage to this world the world has to kill them or break them, so of course it kills them. The world breaks every one and afterward many are strong at the broken places. But those that will not break it kills. It kills the very good and the very gentle and the very brave impartially. If you are none of these you can be sure it will kill you too but there will be no special hurry.” Earnest Hemingway, A Farewell To Arms.

  3. #3
    Join Date
    Jun 2005
    Location
    Denver, CO
    Posts
    100
    When importing, does SSIS allow me to split a single CSV file across multiple tables? I have one line per CSV that corresponds to a table.
    BillS

  4. #4
    Join Date
    Nov 2004
    Location
    on the wrong server
    Posts
    8,835
    Provided Answers: 6
    Dump everything into a staging table and split out what you need with SQL.
    “If one brings so much courage to this world the world has to kill them or break them, so of course it kills them. The world breaks every one and afterward many are strong at the broken places. But those that will not break it kills. It kills the very good and the very gentle and the very brave impartially. If you are none of these you can be sure it will kill you too but there will be no special hurry.” Earnest Hemingway, A Farewell To Arms.

  5. #5
    Join Date
    Jun 2005
    Location
    Denver, CO
    Posts
    100
    I just now thought of that too! Thanks!

    I first combined all the CSVs into a single file and imported to a temporary table.
    BillS

  6. #6
    Join Date
    Jun 2004
    Location
    Long Island
    Posts
    696
    Wouldn't even bother with SSIS to load a csv file, look at OpenRowset function or BulkCopy, spares you the SSIS headache.

  7. #7
    Join Date
    Jun 2004
    Location
    Long Island
    Posts
    696
    Quote Originally Posted by Thrasymachus
    SSIS packages that take the CSV filename in as a package variable.

    Use this old trick to get all of files in your directory...

    INSERT INTO mytable(folder)
    xp_cmdshell 'DIR C:\myfolder'

    update mytable set folder = RIGHT(folder,LEN(folder) - 39)

    And just spin that in a cursor to fire off your package and pass it in your file name.

    piece of cake.

    If you just want the filename, and no path, use DIR /B C:\myfolder

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •