Web scraping through WebBrowser (.Net WinForm Control)


In this post, I am going to explain how it is easy to extract and change the the web-page content through Web-Browser control. The Web-Browser is a Microsoft windows form control and easy to use. This post will explain you how you can use this control for your need.

The Web-Browser control is used to display the web-page in your application. The Web-Browser class is a powerful class that give you leverage to manipulate the html code, interact with JavaScript, automate the web scrapping and many more.

You can find more about the Web-Browser from MSDN library. The following steps will guide you how to use the Web-Browser.

1.       The first thing that you should do is to create a windows application project.
2.       Add a Web-Browser and two button controls on web form.  
3.       Set the “Url” property of the control or you can do this by writing code using the below code snippet.
webBrowser.Navigate("https://www.google.com");
4.       When you run the application, you will see the screen look as per below screen print

5.       If you want to set anything to web-page control use the below code and even you can fire the control events as well. Here, I am going to set the value to the search box (for this example I am going to set my name “Mohd Azharuddin Ansari” to search box) and then I will fire “Click” event of search button programmatically. If everything will go as per plan then google will present me the result based on the search criteria.
Code (This code will go on first button click event)
                webBrowser.Document.GetElementById("q").SetAttribute("value", "Mohd Azharuddin Ansari");

            HtmlElement button = webBrowser.Document.GetElementById("btnK");

            button.InvokeMember("click");
Result
6.       Now if you need to extract this results on somewhere your code then you can do it using the below code
Code (This code will go on second button click event)
string searchResultText = "";
            HtmlElementCollection searchResult = webBrowser.Document.GetElementsByTagName("h3");

            foreach(HtmlElement he in searchResult)
            {
                searchResultText += he.InnerText + System.Environment.NewLine;
            }

            MessageBox.Show(searchResultText);
Result


Comments

Post a Comment

Popular posts from this blog

SSIS Merge Join - Both inputs of the transformation must contain at least one sorted column, and those columns must have matching metadata SSIS

Image
SSIS Merge Join Issue ( Both inputs of the transformation must contain at least one sorted column, and those columns must have matching metadata SSIS )   I was working on one of my client requirement where I need to split the flat file data based on the conditions and then add join that data with reference data (again a flat file). I used the MergeJoin task to achieve this. Unfortunately I ended up with this error. I tried most of the options available on the internet but none of them worked for me because my mistake was silly I set a property of “Sort” task in one not the other. That’s why MergeJoin was throwing “ Both inputs of the transformation must contain at least one sorted column, and those columns must have matching metadata SSIS ” error.     Lets first discuss about the solution of my problem Root Cause – The main problem was related to the Sort task. I selected the “Comparison Flags” option in one Sort task but not in other task because of that M...
Read more

jsGrid

Image
I was working one a Asp.Net MVC application where I was using the Web Grid to display the data based on the selected search criteria. It was working great but client asked me to use the jsGrid instead of Web Grid as it has many ready to use features like (Edit, Delete, Search, etc..).  Initially, jsGrid gave me very hard time to display the data in it but after few hit and trial I manage to display the data. After spending couple of hours on jsGrid, I started liking it. In this article, I am going to explain how to use the jsGrid to display the data and some other tactics in very easy steps. I am building the application in ASP.NET MVC. Step 1. Content delivery network (CDN) of jsGrid and jQuery libraries are as below https://cdnjs.cloudflare.com/ajax/libs/jsgrid/1.5.3/jsgrid.min.css https://cdnjs.cloudflare.com/ajax/libs/jsgrid/1.5.3/jsgrid.css https://cdnjs.cloudflare.com/ajax/libs/jsgrid/1.5.3/jsgrid-theme.min.css https://code.jquery.com/jquery.min.js ...
Read more

Add Item to SharePoint List with attachment using client object model

Add Item to SharePoint List with attachment using client object model If your requirement is to add the item to a SharePoint list with attachment using the client object model, this post will help you in this. You can achieve this by following the below steps.   Step 1 – Define some constant variables those will hold the values of SharePoint web service, List Id, etc. private const string siteUrl = "http://appsuat.portal.intranet.aegonuk.com/sites/genapps/CashFlowsCashPositions" ;         private const string spWebServiceUrl = "http://appsuat.portal.intranet.aegonuk.com/sites/genapps/CashFlowsCashPositions/_vti_bin/Lists.asmx" ;         private const string updateSOAPAction = "http://schemas.microsoft.com/sharepoint/soap/UpdateListItems" ;         private const string addAttachSOAPAction = "http://schemas.microsoft.com/sharepoint/soap/AddAtta...
Read more