Results 1 to 14 of 14
  1. #1
    Join Date
    Dec 2012
    Location
    Huntsville, AL
    Posts
    379
    Provided Answers: 2

    Unanswered: Limits of Database Size

    I have a new project that will require a fairly large amount of records. I know that 'fairly large' is rather ambiguous but I am guessing that during normal operation, it will require around 70,000 records within a folder. In looking at the statistics listed in Brilliant database, it states "Maximum record number per folder: 100000". I have tried before when experimenting and once I got up to 32,000 records within a folder things froze up and became unstable (this was in V9.42). Has anyone successfully used, for example, 80,000 records within a folder and it worked properly?

    On a side note, James Tubbritt shared with me how he archived older information to reduce the size which works very well on another of my earlier projects - this project will not allow that because it is very data heavy.
    Daryl G

  2. #2
    Join Date
    Jul 2012
    Location
    Ireland
    Posts
    815
    Provided Answers: 17
    I've tested v10.5 up as far as 250,000 records in a single folder. I saw no stability issues whatsoever so I didn't see the point in generation higher numbers of records. 250,000 is far more than I'll ever need.

    If your end users will need to use a Search function, it will stop of their search result is found in over 1000 records.

    The search function can be very slow too depending on the complexity of your form.

    Regards
    James

  3. #3
    Join Date
    Dec 2012
    Location
    Huntsville, AL
    Posts
    379
    Provided Answers: 2
    James,
    Thanks! I will start the actual development of this project tomorrow. Given the sheer volume of records that will accumulate over a three year window (the length of the training activities that will be tracked), I was uncertain if Brilliant Database could support such an amount.

    I have a question regarding the search function you mentioned - is this feature under 'Folder' and then 'Find records...'?
    Daryl G

  4. #4
    Join Date
    Jul 2012
    Location
    Ireland
    Posts
    815
    Provided Answers: 17
    Given the sheer volume of records that will accumulate over a three year window (the length of the training activities that will be tracked), I was uncertain if Brilliant Database could support such an amount.
    Should be no problem in v10.5
    As with any database though, don't forget to plan an auto backup either into your database itself or at the server level.

    I have a question regarding the search function you mentioned - is this feature under 'Folder' and then 'Find records...'?
    Yes, that's the one. Same as the Action you can assign to a menu or button.

    Best to try totally avoid having to use this function for a database of the size your planning. If you need include a search engine into your database, best to make it a “user input” query as a query will run in seconds on a huge database.

    Regards
    James.

  5. #5
    Join Date
    Dec 2012
    Location
    Huntsville, AL
    Posts
    379
    Provided Answers: 2
    Thanks for your replies, James. I doubt that the search function will be needed by the end users; however, I could have certainly utilized that feature during the design and testing phase (I did not know that feature existed and I was manually 'searching' when required).

    After some previous 'not so pleasant' experiences with queries, I made a decision not to use any queries at all. I do everything via scripting and have removed all evidence of any queries. I would be hard pressed to ever even consider using them! Granted, my uncomfortableness with the use of queries may stem from my relative newness to Brilliant Database but the query setup and usage were too restrictive for what analyses I needed to obtain.
    Daryl G

  6. #6
    Join Date
    Dec 2012
    Location
    Huntsville, AL
    Posts
    379
    Provided Answers: 2
    James,
    This morning I started adding records to ensure that Brilliant Database can support the required volume... it failed. In fact, it failed miserably. I added 588 records at a time and was not able to even get up to 6,000 records in a single folder. I closed the database and attempted to re-open it and it will not open - it just hangs and shows a message in the lower left-hand corner of "98% complete. Please wait...". I had similar results when I tried doing this in V9.42 though I was hoping that this new version could support this higher volume.

    These results greatly differ from what you were able to do and it is substantially less than what Brilliant Database states as being the maximum. I definitely can not use Brilliant Database for my new project as it has to be able to hold 70,000 records in a folder. Do you or anyone else have any idea as to why I am experiencing such limits on the size of records allowed within a folder?
    Daryl G

  7. #7
    Join Date
    Jul 2012
    Location
    Ireland
    Posts
    815
    Provided Answers: 17
    Hi Daryl.
    Whatever the problem your experience is, it has nothing to do with how many records you are importing into a single folder. I use that function all the time and I'm importing vast amounts of data many times the size of what your doing.

    As a quick example, see attached HTML/ FLASH Animation where I do a test and import over 16,000 records, each record containing 8 long random values.

    Try download this, reboot and then try running your import.
    https://www.microsoft.com/en-us/down....aspx?id=10019

    This fixes some VB issues that brilliant database depends on.

    If that doesn't help, I'd really need to see your files as this all should work.

    Regards
    James
    Attached Files Attached Files

  8. #8
    Join Date
    Dec 2012
    Location
    Huntsville, AL
    Posts
    379
    Provided Answers: 2
    James,
    Unfortunately, it did not work. I will try to send you my file tomorrow. I would ABSOLUTELY love it if it did function because I would be able to do some great things... although it should, it does not.
    Daryl G

  9. #9
    Join Date
    Dec 2012
    Location
    Huntsville, AL
    Posts
    379
    Provided Answers: 2
    Quote Originally Posted by Tubbritt View Post
    If that doesn't help, I'd really need to see your files as this all should work.
    James,
    I sent you a PM.
    Daryl G
    Daryl G

  10. #10
    Join Date
    Jul 2012
    Location
    Ireland
    Posts
    815
    Provided Answers: 17
    Great, sent you a reply.

    Regards
    James

  11. #11
    Join Date
    Dec 2012
    Location
    Huntsville, AL
    Posts
    379
    Provided Answers: 2
    I want to nominate James Tubbritt as the 2014 Person of the Year. I had an issue with Brilliant Database in regards to the number of records in a folder. I put away my project because other (non-Brilliant Database) projects arrived on my desk. Just recently, I was contacted about a new project that is ideally suited for Brilliant Database. I revisited this earlier issue and was still plagued with insufficient amount of record storage. As of yesterday, I was limited to around 4,000 records. James examined my project and diagnosed the issue (I had improperly defined a variable... I had wrongly set up a single variable to hold ALL the records in a large folder) and explained what the solution was to rectify this issue. I made the change and it works PERFECTLY! This morning, I added 84,000 records and it still has room to grow.

    Now, I must go and tackle that newest project. James, prepare yourself for more questions!
    Daryl G

  12. #12
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    Provided Answers: 54
    I don't know that James Tubritt will help us achieve whirled peas , but he's one heck of a guy for issues with Brilliant Database, a great musician, and a really nice guy!

    James has done more to retain current users of BDB and get new ones than any other person or thing (at least as far as I know).

    -PatP
    In theory, theory and practice are identical. In practice, theory and practice are unrelated.

  13. #13
    Join Date
    Jul 2012
    Location
    Ireland
    Posts
    815
    Provided Answers: 17
    Thanks guys. I'm all embarrassed now.

    Kind regards
    James.

  14. #14
    Join Date
    Dec 2012
    Location
    Huntsville, AL
    Posts
    379
    Provided Answers: 2
    I tested my new, still-in-development database with greater than 102,000 records this morning and it worked like a charm! I will not need this volume but it is great to know that it is a possibility.
    Daryl G

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •