Python Developer - Web Scraping & Automation (3 to 5 yrs)
Python Developer – Web Scraping & Automation (WFH) Experience: 3-5 Years Location: Remote Mode of Engagement: Full-time No of Positions: 3 Educational Qualifications: Bachelor’s degree in Computer Science / Information Technology Industry: IT / Software Development Notice Period: Immediate Joiners Preferred What We Are Looking For: Strong hands-on experience in Python for large-scale web scraping and automation. Expert in web scraping using Scrapy, BeautifulSoup, Selenium, Playwright, and Requests. Excellent understanding of HTML, CSS, DOM structure , and browser-rendered content. Experience in backend/API development using FastAPI, Django, or Flask . Ability to handle proxies, user-agent rotation, cookies, sessions, and anti-bot mechanisms. Strong debugging, error-handling, and optimization skills. Ability to independently deliver reliable and scalable data extraction solutions. Key Responsibilities: Develop, optimize, and maintain web scraping scripts and automation pipelines in Python. Extract data from static, dynamic, and JavaScript-heavy websites using browser automation. Manage pagination, authentication of workflows, cookies, sessions , and rendering challenges. Build and maintain REST APIs to support data access and processing. Implement end-to-end pipelines for data cleaning, transformation, and storage (CSV, JSON, DB). Work with SQL/NoSQL databases to store and manage scraped datasets. Use Docker for containerization and deploy scraping tools to cloud environments. Troubleshoot scraping failures, automate recovery, and ensure high data accuracy. Maintain logs, monitoring, and documentation for scraping workflows. Qualifications: Strong programming experience in Python . Proficiency with Scrapy, Beautiful Soup, Selenium, Playwright, and Requests . Understanding APIs, headers, cookies, sessions, and web protocols. Experience with backend frameworks: FastAPI / Django / Flask . Experience working with Docker and basic CI/CD workflows is good to have . Knowledge of relational and non-relational databases. Strong debugging, optimization, and troubleshooting abilities.
