Extracting an array of keywords from a post...

Posted By: jsampsonPC | May 8th, 2008 @ 11:26 AM
page 1 of 1
Comments: 5 | Views: 526
#May 8th, 2008 @ 11:26 AM
jsampsonPC
jsampsonPC
SampsonBlog.com SampsonVideos.com
Anybody know of a method that will find keywords in a post quickly, and intelligently?

I'm working on a form right now that will be used for people to submit messages, comments, questions, etc. When they submit a comment, rather than send the email to me, I want to first extract keywords from their text, search those against a database of articles, suggest the articles to them, and then submit the question only after they click "None of these help". So basically, I'm forcing them to use the resources already in place before wasting my time.

Any suggestions on how I should find the REAL keywords in these posts? Obviously I'll strip out 'a', 'and', 'of', etc...but I'm still a little in the dark on the full operation.

Thanks in advance.
#May 8th, 2008 @ 11:37 AM
stevo_
stevo_
Human after all
Well, I imagine its a lot like tagging, you tokenize the content (depending on your scenario, you may want content type and culture aware word breakers)..

I guess then its up to you how you want to relate them..
#May 8th, 2008 @ 11:39 AM
jsampsonPC
jsampsonPC
SampsonBlog.com SampsonVideos.com
Here's a portion of a method I found online just now - rather simple. It works primarily off of a man-made list of common words ($words).

(Yes, this is PHP Smiley)

    foreach ($words as $value) {
        $common = false;
        if (strlen($value) > 3){
            foreach($commonWords as $commonWord){
                if ($commonWord == $value){
                    $common = true;
                }
                else{
                }
            }
            if($common != true){
                $keywords[] = $value;
            }
            else{
            }
        }
        else{
        }
    }
#May 8th, 2008 @ 11:40 AM
jsampsonPC
jsampsonPC
SampsonBlog.com SampsonVideos.com
stevo_ wrote:
Well, I imagine its a lot like tagging, you tokenize the content (depending on your scenario, you may want content type and culture aware word breakers)..

I guess then its up to you how you want to relate them..


Can you give me a simple example of what you mean?
#May 8th, 2008 @ 12:48 PM
jsampsonPC
jsampsonPC
SampsonBlog.com SampsonVideos.com
I do see a problem with the method I posted up a couple posts earlier. It only looks for single words, like "Architect, Engineer, Analyst", and not words like "Systems Engineer, Senior Analyst" etc.
#May 8th, 2008 @ 1:48 PM
grassBlade
grassBlade
jsampsonPC wrote:
I want to first extract keywords from their text, search those against a database of articles...Any suggestions on how I should find the REAL keywords in these posts? Obviously I'll strip out 'a', 'and', 'of', etc...but I'm still a little in the dark on the full operation.



Where & how are you implementing this (platform, language)? How is the db set up? How large is the table?

Dependent upon its size, you may want to create the keyword list from the article db and then search through the comments. 

if you want the user to do all the work, use javascript:

create an associative array of the keywords in javascript [ var keyList = {"Word1", "url4Word1", "title of Word1 article", "first phrase", "url4firstPhrase", "title of first phrase article",......} ]
split the user's comments into an array that you can then use to match against the keyList.   If found, add the url to another array that you can display in a div, as a recommendation.

page 1 of 1
Comments: 5 | Views: 526
Microsoft Communities