Results 1 to 2 of 2
  1. #1
    Join Date
    Oct 2010
    Atlanta, GA

    Question Unanswered: What is the most frequently used word in a column of open text?

    Here is an interesting piece of data that I am trying to obtain. The company I work for sells products online. We have a table that stores the product descriptions that are visible to the users: "this lovely widget was handcrafted in widgetsville...". I am wondering if there is a way to search through this field for all products and see what are the most commonly used words across all descriptions and how many instances of this word are there. For example, of the 10,000 item descriptions, the word "widget" appears 15,000 times.

  2. #2
    Join Date
    Feb 2004
    In front of the computer
    Provided Answers: 54
    Too much depends on how you define "words" for this purpose. It might be possible to use Full Text Search, but I would:

    1. create a tokenizer (probably based on regular expressions), probably as an SSIS job.
    2. Put the tokens into a temp table
    3. Analyze the temp table to your heart's content.

    In theory, theory and practice are identical. In practice, theory and practice are unrelated.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts