Channel 9 Forums - Tech Off - How do I get search engine results programatically? http://mschnlnine.vo.llnwd.net/d1/Dev/App_Themes/C9/images/feedimage.png Channel 9 Forums - Tech Off - How do I get search engine results programatically? http://channel9.msdn.com/Forums Channel 9 keeps you up to date with the latest news and behind the scenes info from Microsoft that developers love to keep up with. From LINQ to SilverLight – Watch videos and hear about all the cool technologies coming and the people behind them. http://channel9.msdn.com/Forums en Sat, 25 May 2013 02:29:05 GMT Sat, 25 May 2013 02:29:05 GMT Rev9 4 -4 -1 Tech Off - How do I get search engine results programatically? How do I get search engine results programatically?  Some search engines use AJAX to display content and so the URL does not change. This makes it difficult to get result content from a search engine programatically.

Most search engines do not use AJAX.  But two popular ones that I am interested in is YANDEX and DUCKDUCKGO.  Both of these search engines use AJAX to advance beyond the first page of results.  How do I find a way to programatically access these pages?

]]> http://channel9.msdn.com/Forums/TechOff/How-do-I-get-search-engine-results-programatically/3568eb6ee0294050b0eaa11300892094#3568eb6ee0294050b0eaa11300892094 Sat, 24 Nov 2012 08:19:15 GMT http://channel9.msdn.com/Forums/TechOff/How-do-I-get-search-engine-results-programatically/3568eb6ee0294050b0eaa11300892094#3568eb6ee0294050b0eaa11300892094 complete 4 http://channel9.msdn.com/Niners/complete/Discussions/RSS Tech Off - How do I get search engine results programatically? I do not use screenscraping.  I use the c# WebBrowser class and I get the HTML after I use this class to navagate to a URL.  I was able to use yandex by using a "&p=" to navagate to a particular page.

The duckduckgo search engine does not work this way, though.

I have found out how to make a specialized query for YANDEX in a modified URL.  I will ask DUCKDUCKGO for advice.

https://duckduckgo.com/api.html does not provide a solution.

Running this, http://api.duckduckgo.com/?q=stuff+to+search&format=xml , in a browser's address bar does not provide anything as http://duckduckgo.com/?q=stuff+to+search does.

This shouldn't be rocket science.  I am able to see the HTML code.  Shouldn't there be some sort of AJAX tag in the HTML that I can find to make use of?

]]> http://channel9.msdn.com/Forums/TechOff/How-do-I-get-search-engine-results-programatically/1ac8d199e64d4a70abf3a11300c959ec#1ac8d199e64d4a70abf3a11300c959ec Sat, 24 Nov 2012 12:13:05 GMT http://channel9.msdn.com/Forums/TechOff/How-do-I-get-search-engine-results-programatically/1ac8d199e64d4a70abf3a11300c959ec#1ac8d199e64d4a70abf3a11300c959ec complete 4 http://channel9.msdn.com/Niners/complete/Discussions/RSS Tech Off - How do I get search engine results programatically? From DuckDuckGo:

So is there then another API with all the Web links?

Since we use many underlying APIs in our full results (merged with our own indexes), we do not have rights at this time provide an API with all our Web links. We do, however, provide search box integration.

]]> http://channel9.msdn.com/Forums/TechOff/How-do-I-get-search-engine-results-programatically/f65c6ca0aa4042a899f7a11300d72324#f65c6ca0aa4042a899f7a11300d72324 Sat, 24 Nov 2012 13:03:17 GMT http://channel9.msdn.com/Forums/TechOff/How-do-I-get-search-engine-results-programatically/f65c6ca0aa4042a899f7a11300d72324#f65c6ca0aa4042a899f7a11300d72324 ZippyV 4 http://channel9.msdn.com/Niners/ZippyV/Discussions/RSS Tech Off - How do I get search engine results programatically? Yandex provides a XML api for you:

http://api.yandex.com/xml/

]]> http://channel9.msdn.com/Forums/TechOff/How-do-I-get-search-engine-results-programatically/e7936f1836ec41789064a11300d91078#e7936f1836ec41789064a11300d91078 Sat, 24 Nov 2012 13:10:18 GMT http://channel9.msdn.com/Forums/TechOff/How-do-I-get-search-engine-results-programatically/e7936f1836ec41789064a11300d91078#e7936f1836ec41789064a11300d91078 ZippyV 4 http://channel9.msdn.com/Niners/ZippyV/Discussions/RSS