Results 1 to 8 of 8
  1. #1
    Join Date
    Sep 2011
    Location
    Australia
    Posts
    264
    Provided Answers: 2

    Unanswered: Import large XLS files (from Excel)

    Hello all
    Started new tread on this xls importing as becoming a drawn out saga.
    Today cleared my desktop computer, reformated, fresh copy of XP pro installed with basic software needed (AV. Jzip Fox Reader etc). Then installed every version of BD 9 - even V10 pro. Result just the same as before, slow as grass growing. Draged out old Dell laptop "D600 model" and did same thing (fresh install of every thing). Test with same XLS file and all imported i under 3 minutes. No idea why portables can achieve in a few minutes what is taking hours on desktop - especially as it was ok previously. FWIW Anti-virus same both desktop and netbook.
    I suppose can use laptop for all importing but would like to here from other members if they can try this out. Using freeware program "Spawner" to generate csv data for Excel input (semi-colon separator).
    As mentioned in another post the record counter does not update untill finished on portables but rolls over on desktop - WHY?
    Can't spend any more time on this, need to get some work done!
    Regards
    David

  2. #2
    Join Date
    Jan 2012
    Posts
    37
    Very interesting info

    I appreciate your input tamcind -I already posted on my original thread but I do also have a old Dell so maybe I need to wipe the cobwebs off

    Thanks again

  3. #3
    Join Date
    Sep 2011
    Location
    Australia
    Posts
    264
    Provided Answers: 2
    What I can not understand is why BD updates record count to my Dell desktop ( but impossibly slow) while an old Dell laptop counter stays fixed at last existing record until finished importing (and at a reasonable speed - a few minutes). Even allowing for different settings and hardware the difference makes no sense to me. Will pull out an old 1g Compaq desktop on Wednesday (P/holiday here in AU) and try it. Another idea I might try is to copy the BD installation folder form laptop to desktop - should be same (??) but not hard to do.
    UPDATE>
    Just downloaded V10 Ultimate trial and IT WORKS - like 2 minutes to import 65,000 records to my desktop. Kept repeating same import until records reached over 500,000 (yes 1/2 million @ 127 meg file size). Each import was consistently around 2 minutes even at 500,000 and yes the counter DOES NOT update until finished like on the laptops! Did several queries on 1 field (author) and returned a few hundred records in 2 seconds (first run seemed a bit slower). So far no crashes or error messages! Now if Brilliant Database can maintain stability, we are talking serious data here. Have fired up the "dreaded" optimization engine but might have to let run over-nite to test as it is still going as I type and it's after 10pm here.
    Last edited by tamcind; 04-22-13 at 09:07.

  4. #4
    Join Date
    Apr 2013
    Posts
    226
    So was that on the old Compaq or do you mean that now works like this on your normal desktop? Ultimate 10 doesn't import like that for me. Unless there has been a build update.

  5. #5
    Join Date
    Dec 2012
    Location
    Huntsville, AL
    Posts
    379
    Provided Answers: 2
    tamcind,
    This is really interesting news! Since you were able to successfully import such a huge volume of records, is it to correct to assume that the earlier 100,000 limit on records to a single folder no longer exists in v10?

    One question for you... is your file still stable if you close it and then try to re-open it?
    Last edited by fireant911; 04-22-13 at 17:54. Reason: added question concerning stability of large file
    Daryl G

  6. #6
    Join Date
    Sep 2011
    Location
    Australia
    Posts
    264
    Provided Answers: 2
    Hi Matty
    No this was on the original "problem" Dell desktop that did not like V9.42. As I mentioned last post I left on a optimization cycle over-nite and this morning the dialog says all ok with 500,000 records. Will need a lot more testing yet but does give me some hope. I thick users may have to accept a little delay in queries with a database this size (1/2 million records searched in under 5 second to return results - will test this with smaller result sets etc latter tonight). Also need to download the V10 pro, wp and server to cross check the results.
    Regards
    David
    Last edited by tamcind; 04-22-13 at 17:43.

  7. #7
    Join Date
    Apr 2013
    Posts
    226
    All a mystery. Import of 10,000 records takes about ten minutes for me with 9.42 Ultimate and shows the incrementing counter.

    EDIT: I've found that Optimization corrupting the database occurs when a version 9 database has been opened in version 10 and converted .
    Last edited by matty1965; 04-22-13 at 17:56.

  8. #8
    Join Date
    Sep 2011
    Location
    Australia
    Posts
    264
    Provided Answers: 2
    A bit more information. Import works same on Pro as Ultimate so far. There is a problem with display of records in the list above a form - at exactly 200,000 and from here on, the list area is blank but the form is still displaying data normally. If one of these blank list records is edited in form or list all the fields appear in the former blank list line ( odd but same in 2 databases). So at moment continue to experiment but don't trust with "good" data yet. Tried optimization on 200,000 records and did get update empty element protection error although data appears ok - with 200,000 records to check can't be sure. Keen to try out V10.1 with bug fixes (and I believe from earlier messages from support some new features).

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •