Results 1 to 10 of 10
  1. #1
    Join Date
    Jul 2005
    Posts
    63

    Unanswered: Storing pictures and documents

    What would be the ideal way to store hundreds of thousands of pictures and documents (word/excel) on a website - as disk files or inside the database as blob data? If storing in db, could someone give me a script to upload files into the db using PHP? And if possible, how to offer the users to download the files later to save onto thier machine.

    TIA

  2. #2
    Join Date
    Jul 2005
    Posts
    63

    Storing pictures and documents on a website

    What would be the ideal way to store hundreds of thousands of pictures and documents (word/excel) on a website - as disk files or inside the database as blob data? If storing in db, could someone give me a script to upload files into the MySQL using PHP? And if possible, how to offer the users to download the files later to save onto thier machine...

    TIA

  3. #3
    Join Date
    Nov 2004
    Location
    out on a limb
    Posts
    13,692
    Provided Answers: 59
    personally

    I'd store images outside the server as disk files with the URL to the image in the server

    there are arguments either way.
    storing outside the server raises some other issues... primarily security (the images are no longer under the direct control of the db and its security policies. you don't know if the image has changed (been edited after linking), or if its been deleted.
    the advantages.. well you aren't backing up redundant data, you can associate images in other locations.. you aren't limited to images on the server.. but any locations accessible via a URL.
    I'd rather be riding on the Tiger 800 or the Norton

  4. #4
    Join Date
    Nov 2004
    Location
    out on a limb
    Posts
    13,692
    Provided Answers: 59
    please don't duplicate post questions in different threads

    if you don't get an appropriate answer to your question after a while.. say several days then by all means ask it in another thread.
    I'd rather be riding on the Tiger 800 or the Norton

  5. #5
    Join Date
    Jul 2005
    Posts
    63
    Hi,

    How does the website perform with hundreds of thousands of images as disk files? These files are not the files used by the website - they are uploaded by the users as their property and they manage their files as they do on sites like Flickr and Shutterfly.

    TIA



    Quote Originally Posted by healdem
    personally

    I'd store images outside the server as disk files with the URL to the image in the server

    there are arguments either way.
    storing outside the server raises some other issues... primarily security (the images are no longer under the direct control of the db and its security policies. you don't know if the image has changed (been edited after linking), or if its been deleted.
    the advantages.. well you aren't backing up redundant data, you can associate images in other locations.. you aren't limited to images on the server.. but any locations accessible via a URL.

  6. #6
    Join Date
    Nov 2004
    Location
    out on a limb
    Posts
    13,692
    Provided Answers: 59
    there shouldnt be any significant problem, providing you design your directory structure appropriately so you dont blow the OS's file limits.

    if the images are being shared you may even get a performance benefit by storing them externally.. the images could be cashed locally reducing the page download times for repeat loads

    as to how flicker or who ever do it I dunno. Id be very surprised if they stored images within a DB.
    I'd rather be riding on the Tiger 800 or the Norton

  7. #7
    Join Date
    Jul 2005
    Posts
    63
    That makes sense. But again, I have this concern for a large site:

    Say we store all images in a directory called "user_images".
    We create a directory for each user with their username under this directory.
    We create a directory for each album they create (no problem because there may be a few albums for each user and a few pictures in each album).
    We have hundreds of thousands of users. That itself adds hundreds of thousands of user directories to the main "user_images" folder which would cross the file limits. We are talking about millions of users here.

    How do we organize and design the directory structure for the users under the user_images directory without crossing the OS file limits?

    If storing files externally is more efficient, what's the purpose of binary data type for columns in a table?

    TIA

  8. #8
    Join Date
    Mar 2007
    Location
    636f6d7075746572
    Posts
    770
    Firstly, what is your OS file limit? I don't believe there is a limitation to the number of directories/sub directories on a linux system (although feel free to correct me on this point). So there's shouldn't be a problem creating all those directories.

    In terms of efficiency, think about it this way. Each file on the system takes up disk space, it also consumes memory to serve it up (using Apache). Each file in a database, takes up system disk space, takes up memory overheard on loading from the databases, takes up memory before serving (using script calls) and then takes up memory getting served. You can guess which is more efficient

    As for using binary data types (clob/blob): i'm sure they have their uses, but in terms of serving for the web it's my belief that serving direct from disk will always be better. Of course you can't query a disk as well ...

  9. #9
    Join Date
    Nov 2004
    Location
    out on a limb
    Posts
    13,692
    Provided Answers: 59
    I don't think any modern OS has a significant limit in terms of no of directories.. but there are limits imposed by the OS in terms of number of files/directories per directory.

    so I'd agree that there shouldn't be a problem PROVIDING the design accommodates that and stores images & files in some structure so that an individual limit on a directory isn't blown. its a trade off between flatness of design dso that you don't have to navigate up and down tree structures, ease of locating similar documents or files and copying with hardware limitations. unlike a conventional app, where if there is a problem, your users are likely to tell you pretty soon, a web app relies on remote unknown users to tell you when there is a problem. In reality they may not, they may tell you some time after the event, if at all.. so its soemthign to pay close attention to the first time.

    I suspect that also any other web servers on the same physical server may get upset if you nuke the server, I'm not even certain how you would recover form a borked file system
    I'd rather be riding on the Tiger 800 or the Norton

  10. #10
    Join Date
    Mar 2007
    Location
    636f6d7075746572
    Posts
    770
    Here's a question, would you like people outside of your application to be able to access the photos/images? Consider applications like flickr/gettyimages that allow you to externally link to them. Could provide added SEO ranking...

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •