scraping (1,042)


Mapping Stream data to data structures in C#

Mapping Stream data to data structures in C#

Is there a way of mapping data collected on a stream or array to a data structure or vice-versa? In C++ this would simply be a matter of casting a pointer to the stream as a data type I want to use(…


Options for HTML scraping?

Closed.This question needs to be more focused.It is not currently accepting answers. Want to improve this question? Update the question so it focuses on one problem only by…


.net - What is your preferred method of sending complex data over a web service?

It's 2008, and I'm still torn on this one. So I'm developing a web method that needs a complex type passed into it and returned from it. The two options I'm toying with are: Pass and return actual…


What HTML parsing libraries do you recommend in Java

I want to parse some HTML in order to find the values of some attributes/tags etc. What HTML parsers do you recommend? Any pros and cons?…


c# - How do you programmatically fill in a form and 'POST' a web page?

Using C# and ASP.NET I want to programmatically fill in some values(4 text boxes) on a web page(form) and then 'POST' those values. How do I do this? Edit: Clarification: There is a service(www.sto…


How to implement a web scraper in PHP?

What built-in PHP functions are useful for web scraping? What are some good resources(web or print) for getting up to speed on web scraping with PHP?…


What are the key considerations when creating a web crawler?

I just started thinking about creating/customizing a web crawler today, and know very little about web crawler/robot etiquette. A majority of the writings on etiquette I've found seem old and awkward…



HTML Scraping in Php

This question already has an answer here: How do you parse and process HTML/XML in PHP? 28 answers I've been doing some html scraping in PHP using regular…


python - How to generate urls in django

In Django's template language, you can use{% url[viewname][args] %} to generate a URL to a specific view with parameters. How can you programatically do the same in Python code? What I need is to…


When should one use a project reference opposed to a binary reference?

My company has a common code library which consists of many class libary projects along with supporting test projects. Each class library project outputs a single binary, e.g. Company.Common.Serializ…




web python beautifulsoup tutorial pages multiple from example through looping