Scrapy Jobs

PROJECT TITLE Web Scraping Developer for Global Legal & Regulatory Data Collection PROJECT OVERVIEW We are looking for a developer who can build an automated system to collect legal and regulatory documents from multiple global sources. The goal is to create a scalable automated pipeline that can gather legal data across multiple jurisdictions and regulatory domains. DATA COLLECTION SCOPE The system will collect information related to: - Medical law and healthcare regulation - Medical advertising regulation - Corporate formation and company governance laws - Investment regulation (stocks, cryptocurrency, real estate) - Tax law and administrative tax rulings - Beauty and cosmetic regulation - Medical and cosmetic manufacturing compliance - Import and export law - Customs and tariff...

Project Overview I am seeking an expert Magento 2 Developer with strong data extraction (scraping) capabilities to build out a comprehensive medical supply catalog. The previous developer started a sample set but is no longer on the project. I need a professional to scrape high-volume product data and perform a structured, complex import into my Magento 2 staging site. Scope of Work 1. Data Scraping & Extraction Source: Target website (medical supply industry) to be provided. Requirements: Full extraction of Product Titles, High-Res Images, Long/Short Descriptions, SKUs, and Technical Specifications. Variation Mapping: Correctly identify and link "Simple" products to their "Configurable" parents (e.g., mapping different sizes or packaging options to a single prod...

I need a robust yet easy-to-maintain web scraper that pulls player statistics from four different sports sites—a blend of official league pages, sports news outlets, and a couple of well-known fan forums. All scraped data should flow into a single database and surface through a lightweight web dashboard where I can search by player, season, and team, compare numbers side by side, and export results to CSV. My ideal flow looks like this: enter or schedule the URLs, run or auto-run the scraper, watch progress logs, and then immediately view fresh stats inside the dashboard—no command-line work once everything is deployed. If any source changes its HTML, the scraper should fail gracefully and flag the issue in the UI so I can react quickly. Tech stack is flexible; Python with Be...

I need a robust AI-driven scraper that reliably pulls product text, images, SKUs, and pricing from selected e-commerce websites and our own admin-controlled catalog portals. The script must handle pagination, dynamic content, and, where required, authenticated sessions without manual intervention. Core expectations • Extract and store: product titles/descriptions, all associated images, SKU codes, and current prices. • Export: structured CSV or JSON for data, separate folder (or S3 bucket) for images, with clear file naming that links each image back to its SKU. • Tech stack: Python with libraries such as Scrapy, Playwright/Selenium, BeautifulSoup, or a comparable approach—whatever you can prove is most efficient and resilient. Basic computer-vision or OCR hoo...

Need experienced dev to fix my claude-coded python playwright automation script. Note that I don't know any coding. So need your help to review and then fix. Details will be provided. Reply with "claude" so I know you've really read this very short job description.

I need a clean, well-structured scrape of vendor details from jiii.ng. The goal is straightforward: pull each vendor’s name and contact information and hand the results back to me in a single Excel workbook. Categories The focus is on the main marketplace sections. I will confirm the exact categories before you begin, but expect the usual high-traffic areas (Electronics, Real Estate, Automobiles) to be included. Data points • Vendor name • All available contact details (phone, email, WhatsApp, or any other channel the site exposes) Delivery You’ll give me: 1. The Excel file, one sheet per category, neatly labeled and deduplicated. 2. The script or notebook you used (Python with requests / BeautifulSoup, Scrapy, or a comparable tool is fine) so I c...

I need an automated service that visits eight real-estate platforms every day—, Zillow (FSBO only), , Hubzu, HomePath, HUDHomestore, InvestorLift, and Crexi—pulls both property details and price data, and drops the results into a single clean dataset. Each site has its own search criteria that I will supply (zip codes, status flags, auction dates, etc.). The scraper must respect those individual rules and still be easy for me to adjust later without having to rewrite code—think a clearly labeled config section or similarly simple mechanism. Key expectations • Runs on a schedule without manual intervention and retries gracefully if a site is temporarily unavailable. • Captures all standard listing fields (address, beds, baths, square footage, list/asking ...

I need every piece of publicly available data pulled from three separate websites and packaged so I can work with it offline. The moment the contract is awarded I’ll hand you the URLs; from there I expect you to spin up an automated solution—Python with Scrapy, BeautifulSoup, Selenium, or a comparable stack—that captures all records, follows pagination, and handles any lazy-loaded content. Time is tight: the full job must be wrapped up within 3 calendar days. My total budget is fixed at US $300, so please be sure your approach fits comfortably inside that limit. Deliverables • A clean, well-structured file for each site (CSV or Excel, whichever you prefer) containing every scraped field. • The runnable scripts or notebooks you used, clearly commented so I ...

We are building a high-accuracy OCR system that extracts structured data from scanned documents and images such as invoices, receipts, ID cards, and application forms. This is not just text recognition — the goal is to build a reliable pipeline that converts messy, real-world documents into clean, validated, structured JSON data ready for databases or ERP systems. -Objective Develop an intelligent OCR engine that: Extracts text from scanned PDFs and images (JPG, PNG) Detects document type automatically (invoice, ID card, form, etc.) Identifies key-value pairs (e.g., Name, Date, Total Amount, Invoice No) Handles noisy images (low resolution, skewed, shadows) Returns structured JSON output Achieves high accuracy (≥95% on test dataset) -Technical Scope - Image Preprocessin...

Long-Term Project Idea: AI-Powered Web Scraping + SaaS App for Lead Intelligence - Project Overview We are building a scalable system that: Scrapes structured & unstructured data from thousands of websites Extracts business contact information Enriches and validates data Provides a dashboard for clients Runs continuously as a subscription-based SaaS platform This is not a one-time scraping script — This is a long-term product development project. - Phase 1 – Smart Web Scraping Engine Objective: Build a scalable scraping infrastructure that can: Crawl 50,000+ websites Detect relevant pages (Contact, About, Team, Directory) Extract: Names Emails Phone numbers Social media Company info Handle: Dynamic websites (JavaScript-rendered) Rate limiting Anti-bot ...