script - web scraping post form c#




How do you programmatically fill in a form and 'POST' a web page? (4)

I had a situation where I needed to post free text from a html textarea programmatically and I had issues where I was getting <br /> in my param list i was building.

My solution was a replace of the br tags with linebreak characters and htmlencoding just to be safe.

Regex.Replace( HttpUtility.HtmlDecode( test ), "(<br.*?>)", "\r\n" ,RegexOptions.IgnoreCase);

Using C# and ASP.NET I want to programmatically fill in some values (4 text boxes) on a web page (form) and then 'POST' those values. How do I do this?

Edit: Clarification: There is a service (www.stopforumspam.com) where you can submit ip, username and email address on their 'add' page. I want to be able to create a link/button on my site's page that will fill in those values and submit the info without having to copy/paste them across and click the submit button.

Further clarification: How do automated spam bots fill out forms and click the submit button if they were written in C#?


View the source of the page and use the WebRequest class to do the posting. No need to drive IE. Just figure out what IE is sending to the server and replicate that. Using a tool like Fiddler will make it even easier.


You can see a sample of that here: http://en.csharp-online.net/HTTP_Post

Basically, the code will look something like this:

WebRequest req = WebRequest.Create("http://mysite/myform.aspx");
string postData = "item1=11111&item2=22222&Item3=33333";

byte[] send = Encoding.Default.GetBytes(postData);
req.Method = "POST";
req.ContentType = "application/x-www-form-urlencoded";
req.ContentLength = send.Length;

Stream sout = req.GetRequestStream();
sout.Write(send, 0, send.Length);
sout.Flush();
sout.Close();

WebResponse res = req.GetResponse();
StreamReader sr = new StreamReader(res.GetResponseStream());
string returnvalue = sr.ReadToEnd();

You can use the UploadValues method on WebClient - all it requires is passing a URL and a NameValueCollection. It is the easiest approach that I have found, and the MS documentation has a nice example:
http://msdn.microsoft.com/en-us/library/9w7b4fz7.aspx

Here is a simple version with some error handling:

var webClient = new WebClient();
Debug.Info("PostingForm: " + url);
try
{
     byte [] responseArray = webClient.UploadValues(url, nameValueCollection);
     return new Response(responseArray, (int) HttpStatusCode.OK);
}
catch (WebException e)
{
     var response = (HttpWebResponse)e.Response;
     byte[] responseBytes = IOUtil.StreamToBytes(response.GetResponseStream());
     return new Response(responseBytes, (int) response.StatusCode);
}  

The Response class is a simple wrapper for the response body and status code.





asp.net