Results 1 to 9 of 9
  1. #1
    Join Date
    Aug 2005
    Posts
    20

    Unanswered: Increasing Speed

    I'm not sure if this is the right forum, but I have a general question about running/storing databases. I have been running a process with 60+ million records in one table and another 16 million in another table and it is taking forever to get everything imported in and run the appropriate queries. I've been doing this all on a desktop and I am anxious to learn of a more efficient, faster method of processing this amount of data.

    What solution should I pursue if I am doing this work a few times a year so that it doesn't take three full days of processing to reach an answer with the data?

    Thanks.

  2. #2
    Join Date
    Sep 2002
    Location
    UK
    Posts
    5,171
    Provided Answers: 1
    What DBMS are you using?

  3. #3
    Join Date
    Aug 2005
    Posts
    20
    MS SQL Server 2000

  4. #4
    Join Date
    Sep 2002
    Location
    UK
    Posts
    5,171
    Provided Answers: 1
    I would suggest that you re-post the question in the MS SQL Server forum.

  5. #5
    Join Date
    Apr 2002
    Location
    Toronto, Canada
    Posts
    20,002
    no. don't do that, we'll just move this thread over there

    rudy.ca | @rudydotca
    Buy my SitePoint book: Simply SQL

  6. #6
    Join Date
    Nov 2002
    Location
    Jersey
    Posts
    10,322
    well that's just clear as mud....why not tell us what/how/why you are doing this.
    Brett
    8-)

    It's a Great Day for America everybody!

    dbforums Yak CorralRadio 'Rita
    dbForums Member List
    I'm Good Once as I ever was

    The physical order of data in a database has no meaning.

  7. #7
    Join Date
    Jun 2003
    Location
    Ohio
    Posts
    12,592
    Provided Answers: 1
    What are you using to import the data?
    Are you loading it into staging tables?
    What processing is being done on the data?

    The big question: are there any logical loops in your process (such as a cursor or a WHILE statement) or is all you TSQL code set-based?
    If it's not practically useful, then it's practically useless.

    blindman
    www.chess.com: "sqlblindman"
    www.LobsterShot.blogspot.com

  8. #8
    Join Date
    Jan 2004
    Location
    In a large office with bad lighting
    Posts
    1,040
    Are you joining tables? Do you have columns that define relationships between the tables? Are you using indexes? Are you doing string manipulations?

    Read the sticky at the top of the forum, provide more information, and maybe we can help.

    Or don't and we can provide suggestions of another sort

    -- This is all just a Figment of my Imagination --

  9. #9
    Join Date
    Jun 2004
    Location
    Arizona, USA
    Posts
    1,848
    are there any triggers on the tables in question?
    Lou
    使大吃一惊
    "Lisa, in this house, we obey the laws of thermodynamics!" - Homer Simpson
    "I have my standards. They may be low, but I have them!" - Bette Middler
    "It's a book about a Spanish guy named Manual. You should read it." - Dilbert


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •