Results 1 to 11 of 11

Thread: An Inquiry

  1. #1
    Join Date
    Jan 2014
    Posts
    7

    An Inquiry

    I have ventured into uncharted waters. I have a questions and I hope you guys have an answer.

    First of all Hello dbforums community. I have a questions as to whether there is a way to copy databases from websites. First, the legality of it all, is it legal in all cases, in some or none.

    For example, I know that you can download the ticker information from yahoo finance and from what I have been told it is legal.

    But, my questions of the following: is there a way to download nutrition information from usda.gov website? I need databases of food contents for an app I am working on.

    Thank you for reading,
    UPOD

  2. #2
    Join Date
    Nov 2004
    Location
    out on a limb
    Posts
    13,692
    although the data may be int he public domain, the database itself isn't.

    if you need such data approach the owners of the website

    mind you short of writing a web scraping program I don't see how you will get access to the data without approaching the site owners
    I'd rather be riding on the Tiger 800 or the Norton

  3. #3
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    Normally I'd agree with healdem, but this is an area near and dear to my heart so I have a bit of experience that he might lack... See Nutrient Data : USDA National Nutrient Database for Standard Reference for details!

    -PatP
    In theory, theory and practice are identical. In practice, theory and practice are unrelated.

  4. #4
    Join Date
    Jan 2014
    Posts
    7

    Thank You

    Thank You Pat. This is VERY useful. Are there databases that are more specific? For example, like the nutrition content of Cheetos?

  5. #5
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    My favorite answer: "it depends"!

    There are two big hurdles, and many small ones to getting a simple, straight answer.

    The biggest problem is that most of this nutrition information is variable because the details depend on packaging and formulation of the product. Manufacturers want to make money more than anything else and to do that they need to sell product. This means that for many products there are subtle or not-so subtle variations in formula and packaging based on the targeted sales region.

    While the standard package of chips for the ADA is 1 oz (about 28g), the actual packages sold in stores varies between half and five times that amount! Chips that sell well using one formulation in New England sell better with a corn meal dusting in the south, and a wheat dusting in the Mid-West. This results in a 10x difference in serving size and substantial differences in carb and fiber for the same product marketed in different regions!

    There are similar differences in some restaurant chains. They adjust their serving size, recipe and prep based on customer preferences.

    There are some applications that have solved these problems. My personal favorite is My Fitness Pal. They solved the packaging issues by using the UPC codes to determine which package you have. They used some amazing proprietary algorithms that handle restaurant recipe differences (although they can't always get the prep differences). HR 1249 is a bill before the US Congress which will make this a LOT easier if the bill is passed!

    You have a couple of choices at this point. By far and away the easiest for both you and your users is probably to piggy-back on one of the existing tools. Because things change so quickly, I would find someone that does a job right and offer them a reasonable payment for use of their services! If you have enough people and resources to dedicate to the task (I'd guess that I could do this job pretty well for under $1,000,000 USD / year) and are determined to do it yourself, then I can send your contact information to two universities that are leaders in the field that would probably work with you on the project.

    -PatP
    In theory, theory and practice are identical. In practice, theory and practice are unrelated.

  6. #6
    Join Date
    Jan 2014
    Posts
    7

    Thank You

    WOW! I am just amazed regarding how much knowledge and value you bring to the table. Thank you very much for the detailed response.

    Couple of things. You mentioned UPC codes is there a database for UPC codes and the respective label somewhere for free or provided by government?

    Unfortunately this is a start up so we cant really pay anybody yet, I would appreciate if you do send our contact info to the universities:

    My name: Fedor
    Email: Fedor.Borovykh@easternventuresplus.com

    Also, I would love to talk to you some more and perhaps pick you brains regarding this area.

    Anyways, appreciate your help so far.

    take care,
    UPOD

  7. #7
    Join Date
    Jan 2014
    Posts
    7

    Thank You

    WOW! You bring so much knowledge and value to the table and I just want to thank you for the detailed responses.

    Couple of things:

    You mentioned UPC codes. Are there databases that link the code to the nutrition label on the back of the foods available that are either government owned or are somehow available for download?

    Also, I would appreciate it if you could send my contact information to the universities:

    Name: Fedor Borovykh
    Email: Fedor.Borovykh@EasternVenturesPlus.com

    Unfortunately this is a start up and are unable to pay anybody yet. However, I do appreciate you assistance so far and I hope that I can pick you brain a bit more regarding the subject.

    Thank you so far,
    Fedor B

  8. #8
    Join Date
    Jan 2014
    Posts
    7

    hello

    WOW! You bring so much knowledge and value to the table and I just want to thank you for the detailed responses.

    Couple of things:

    You mentioned UPC codes. Are there databases that link the code to the nutrition label on the back of the foods available that are either government owned or are somehow available for download?

    Also, I would appreciate it if you could send my contact information to the universities:

    Name: Fedor Borovykh
    Email: Fedor.Borovykh@EasternVenturesPlus.com

    Unfortunately this is a start up and are unable to pay anybody yet. However, I do appreciate you assistance so far and I hope that I can pick you brain a bit more regarding the subject.

    Thank you so far,
    Fedor B

  9. #9
    Join Date
    Jan 2014
    Posts
    7

    Hey

    The forum would not allow me to post for some reason.

    WOW! I just want to tell you how much knowledge and value you bring to the table and I wanted to thank you.

    Couple of Things:

    You mentioned UPC codes. Are there databases that that link the UPC codes to the food products that are either available through government or some public entity?

    Also, I would appreciate you do send my contact information to those universities you mentioned.

    Name: Fedor Borovykh
    Email: Fedor.Borovykh@EeasternVenturesPlus.com

    Unfortunately this is a startup and so we cannot pay anybody yet. However, I want to thank you for your help so far.

    Take care,
    Fedor B

  10. #10
    Join Date
    Jun 2014
    Posts
    1
    I think that the legality of downloading data from the Web is the very serious question. This process very often violates the rights of others. We should think twice before using serious information that has an owner.

  11. #11
    Join Date
    Feb 2004
    Location
    In front of the computer
    Posts
    15,579
    Kate:

    You have a very good point, but keep in mind that many of the sites with this kind of information are government funded meaning that the information is explicitly published for the use of the public or university hosted meaning that the information is covered by one of the GPL derivatives and also available to the public.

    When using information from the web you really need to consider the source. If it is privately gathered and maintained, then you ought to license it. If the information was gathered and maintained using public funding, then you as a member of the public have already paid for it so there is no obligation to pay a second time for it. I'm particularly sensitive to companies that "enhance" publicly funded data and then claim copyright on the whole... That to me smacks of intellectual theft.

    -PatP
    In theory, theory and practice are identical. In practice, theory and practice are unrelated.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •