PROJECT

Scraping Websites for Structured Contact Data

Custom software development, Website data scraping, Contact data extraction, Reusable scraper development, Data cleaning and validation, CSV and Excel export, JSON export, CRM-ready data

  • Industry: Market research
  • Croatia Location: Croatia
  • Delivery time: 10 weeks
  • Technologies: Python, JavaScript / TypeScript (Node.js)
3 months

This client spends time collecting contact and business details from company websites, public databases, and online directories. The software is a custom scraping websites solution that takes source URLs and selected fields. It extracts contact records, cleans and validates them, and returns organized files for the market research team. Reusable extraction makes recurring data collection less dependent on manual entry.

Summary

Problem

The client spends time collecting and checking contact data by hand.

Solution

The solution became a reusable scraper for extracting, cleaning, validating, and organizing specified website data.

Result

The software is made to reduce manual collection and rework in market research data work.

Client story

The client is a market research company from Croatia. It collects company names, email addresses, and phone numbers from company websites, public databases, and online directories.

Physical addresses and other business details are also part of the data.

The team prepares contact data for market analysis, business development, telecalling, and CRM import. Manual collection, repeated checks, and duplicate records cost time and create rework.

The request was simple: extract selected contact fields from defined sources, rerun the collection when needed, and deliver clean structured files.

The solution was shaped around a source setup screen, a run monitor, a data cleaning area, and export settings. The records could be reviewed before use.

The client asked for

  1. Extract contact fields from websites, databases, directories, and online sources
  2. Run a reusable scraper and receive its source code or notebook
  3. Clean, validate, organize, and deduplicate the extracted records
  4. Export structured data as CSV, Excel, JSON, or categorized workbooks

Why ready-made software did not fit

Ready-made data extraction tools cover parts of the task, but their descriptions do not define the client's exact fields, cleaning rules, and output files together. A custom solution was needed to combine repeatable scraping with validation, deduplication, organized workbooks, and reusable code in one process.

Project overview

The solution was designed around the client's real sources, fields, and rules. The first screen defines website URLs, contact fields, and recurring extraction settings before a run begins.

To handle collection from one or more sources, an extraction monitor was provided. It shows the run state, source URLs, extracted records, and loading or access notes in one place.

The records then move through cleaning and review. Duplicate matches, validation status, contact fields, and categories stay visible so the data manager can prepare a consistent contact list.

The final screen was designed for practical delivery. It exports organized records as CSV, Excel, or JSON and keeps setup instructions or process notes available with the file.

The scraper uses reusable code and can be tuned to the client's rules. Documentation and training materials support independent work with the system.

Key features delivered

  • Website and source list setup
  • Selected contact-field extraction
  • Reusable and rerunnable scraping
  • Extraction run monitoring
  • Email and phone number collection
  • Record validation and cleaning
  • Duplicate detection and deduplication
  • Contact data categorization
  • CSV, Excel, and JSON export

Can be added

+ Recurring Excel database updates + Email and domain verification + Keyword-based categorization + Date-based categorization + Source code walkthrough + Additional scraper methods

Who else can use this solution

These businesses have a similar task, and the same solution can be adapted to their sources, fields, and delivery rules.

A business services company

Task It collects company and contact details from public websites and directories.
How the solution helps The solution extracts selected fields, removes duplicate records, and prepares files for CRM import or business development.
Business services

A professional services company

Task It builds contact lists from company websites and online sources.
How the solution helps The solution collects email addresses, phone numbers, and business details in structured files.
Professional services

Technology stack

Python

Scraping and browser automation

Python supports web crawling, page parsing, and browser-based extraction.

JavaScript / TypeScript (Node.js)

Structured file processing

JavaScript and TypeScript support structured file processing.

Scrapy

Website crawling and scraping

Scrapy (a framework for crawling and scraping websites) collects records from defined sources.

BeautifulSoup

HTML page parsing

BeautifulSoup (a library that reads information from web pages) parses page content and finds specified fields.

selenium

Browser-based extraction

selenium (a tool that controls a web browser) supports extraction where loading behavior requires a browser.

csv-parse

CSV reading and parsing

csv-parse (a Node.js library for reading CSV files) reads structured CSV data.

maatwebsite/excel

Excel and CSV export

maatwebsite/excel (a Laravel package for Excel and CSV files) supports organized spreadsheet delivery.

jsonify

JSON data handling

jsonify (a JavaScript library for working with JSON) supports JSON records without changing global objects.

CSV, Excel, JSON

Structured data delivery

These formats carry organized contact records for analysis, CRM import, and other business uses.

Integrations

CRM systems

The solution exports organized contact records in formats suitable for CRM import.

Excel

The solution exports categorized contact data and organized workbooks as Excel files.

CSV files

The solution reads and writes comma-separated contact records for structured exchange.

JSON files

The solution delivers structured records as JSON for systems that use this format.

Email verification service

An email verification service can check email addresses and domains before delivery.

The result

Set precise sources and fields before collection

The first screen keeps the extraction request specific. Website URLs, source lists, and selected contact fields define what the scraper collects.

How it works

  1. The software stores the specified websites and source lists.
  2. The software keeps selected fields such as email, phone number, and postal address.
  3. The software stores recurring extraction settings for repeat runs.
  • Email addresses and phone numbers are available as separate contact fields.
  • Physical addresses and business details can be included in the selected fields.
Source and field setup

Source and field setup

Functions on this screen
Scrape website data Scrape email addresses Phone Number Extraction Extract physical addresses

See extraction progress and collected records

The extraction monitor brings source URLs, run status, and extracted records together. Access warnings and loading notes make source behavior visible during collection.

The team gets a clear view of the records returned from one or more websites before the cleaning stage.

How it works

  1. Scrapy (a framework for crawling and scraping websites) collects data from the listed sources.
  2. BeautifulSoup (a library that parses web pages) reads website content and extracts specified fields.
  3. The software shows the run state, source URLs, extracted data, and loading or access notes.
  • The extraction supports multiple pages and websites.
  • The scraper can be rerun with reusable rules.
Note: Collection rules must respect website behavior, platform terms, privacy laws, GDPR, and other applicable requirements.
Extraction run monitor

Extraction run monitor

Functions on this screen
Scrape website data Extract Data

Prepare cleaner contact records

The cleaning screen gives the data manager one place to review validation status, duplicate matches, and categories. Contact records stay organized before export.

Duplicate indicators help separate repeated entries from the usable contact list, while categories prepare the data for analysis or outreach.

How it works

  1. The software displays email addresses, phone numbers, and postal addresses in the contact table.
  2. The software compares records and displays duplicate matches and validation status.
  3. The software applies categories to organized contact records.
  • Records can be categorized by dates or keywords.
  • A data deduplication service can support duplicate prevention.
Clean and deduplicate records

Clean and deduplicate records

Functions on this screen
Organize contact data Duplicate Prevention Data deduplication service Clean Large Datasets Data Categorization Extract Contact Details

Deliver data in the format the team uses

The export screen turns reviewed records into a practical handoff for analysis or CRM import. CSV, Excel, and JSON keep the delivery flexible without changing the source fields.

Setup instructions and process notes stay linked to the delivery settings, helping the team understand how the extraction works.

How it works

  1. The software prepares organized contact records for CSV, Excel, or JSON output.
  2. The software creates a file summary for the selected format.
  3. The software provides links to setup instructions or process notes.
  • CSV files support structured exchange.
  • Excel files support categorized workbooks.
  • JSON files support system-to-system data use.
Export and delivery settings

Export and delivery settings

Functions on this screen
Export company contact lists Excel Export Tool Manage CSV data

What the solution gives

With this solution, the market research company can:

  • Collect specified contact and business fields from defined online sources
  • Rerun extraction rules for recurring data collection
  • Review validation status and duplicate matches before delivery
  • Organize records by categories such as dates or keywords
  • Export clean contact data for analysis and CRM import
  • Maintain reusable code, setup instructions, and process notes

Could ready-made systems do the job?

Before the design, the task was compared with ready-made products by their own descriptions.

What the client needs Web scraping APIs Zyte, Scrapfly, ScrapingBee Web scraping tools Octoparse, Oxylabs Data extraction software Data Extraction Custom solution
Extract contact fields from sources partly
Rerun scraper with source code — partly partly
Clean and deduplicate records — partly
Export structured contact files
Fits the client's own rules: stages, roles, reports — — —
All steps in one place — — —
No manual steps between programs partly — partly

By the public descriptions of the products and their category.

Ready-made products cover website access, crawling, extraction, and file delivery, while general data extraction software covers broader data retrieval.

A ready-made product is enough when the task needs only one of these parts. This client needed its own software because the required fields, cleaning rules, categories, reusable code, and exports had to work together in one place.

The products describe automated collection, but the client's review, validation, deduplication, and delivery rules still require a tailored process.

In the end

In the end, the solution was made to take manual work out of the path from public online sources to clean contact files.

FAQ

Thinking about building a similar system? Read the answers below to see how scraping, structured exports, and custom rules fit together.

Need something similar? Let's talk
Or browse more projects