Data Scraping | Web Scraping
I will build a Java application to scrape data from websites, online portals etc.
I will automate the entire process of scraping. The tasks involved are:
- Build Java application to scrape data.
- Optional: Create a Database to save the scraped data. (you can also get data in xml, JSON, excel formats).
- I will deploy the application on remote Linux/UNIX server and automate scraping process by setting on cron jobs and using shell scripts.
- Optional: Create a Dashboard application to monitor the progress of scrapers, manually trigger scrapers, view data, download data, generate reports etc.
I have built a Price Comparision Application, which scrapes 23 e-commerce websites, compares prices of products and generates a price comparision report.