12-07-15, 16:18 #1Registered User
Provided Answers: 2
- Join Date
- Oct 2010
- Atlanta, GA
Unanswered: What is the most frequently used word in a column of open text?
Here is an interesting piece of data that I am trying to obtain. The company I work for sells products online. We have a table that stores the product descriptions that are visible to the users: "this lovely widget was handcrafted in widgetsville...". I am wondering if there is a way to search through this field for all products and see what are the most commonly used words across all descriptions and how many instances of this word are there. For example, of the 10,000 item descriptions, the word "widget" appears 15,000 times.
12-07-15, 16:56 #2Resident Curmudgeon
Provided Answers: 54
- Join Date
- Feb 2004
- In front of the computer
Too much depends on how you define "words" for this purpose. It might be possible to use Full Text Search, but I would:
- create a tokenizer (probably based on regular expressions), probably as an SSIS job.
- Put the tokens into a temp table
- Analyze the temp table to your heart's content.
-PatPIn theory, theory and practice are identical. In practice, theory and practice are unrelated.