Tech Off Thread

4 posts

Forum Read Only

This forum has been made read only by the site admins. No new threads or comments can be added.

How do I get search engine results programatically?

Back to Forum: Tech Off
  • User profile image

    How do I get search engine results programatically?  Some search engines use AJAX to display content and so the URL does not change. This makes it difficult to get result content from a search engine programatically.

    Most search engines do not use AJAX.  But two popular ones that I am interested in is YANDEX and DUCKDUCKGO.  Both of these search engines use AJAX to advance beyond the first page of results.  How do I find a way to programatically access these pages?

  • User profile image

    I do not use screenscraping.  I use the c# WebBrowser class and I get the HTML after I use this class to navagate to a URL.  I was able to use yandex by using a "&p=" to navagate to a particular page.

    The duckduckgo search engine does not work this way, though.

    I have found out how to make a specialized query for YANDEX in a modified URL.  I will ask DUCKDUCKGO for advice. does not provide a solution.

    Running this, , in a browser's address bar does not provide anything as does.

    This shouldn't be rocket science.  I am able to see the HTML code.  Shouldn't there be some sort of AJAX tag in the HTML that I can find to make use of?

  • User profile image

    From DuckDuckGo:

    So is there then another API with all the Web links?

    Since we use many underlying APIs in our full results (merged with our own indexes), we do not have rights at this time provide an API with all our Web links. We do, however, provide search box integration.

  • User profile image

    Yandex provides a XML api for you:

Conversation locked

This conversation has been locked by the site admins. No new comments can be made.