Cross-Border Scraping Dispatch & Notification Hub
Centralized scraping task management with Microsoft Teams webhook orchestration
An internal cross-team automation hub linking international stakeholders (Japan) with technical scraping engineers (Indonesia), providing structured URL ingestion, automated Microsoft Teams alerts, and artifact delivery tracking.
Challenge & Bottlenecks
Scraping tasks requested by overseas teams were fragmented across emails and chat messages, leading to missed URLs, unclear requirements, and untracked deliverable delivery statuses.
Architecture & Strategy
Designed and deployed a centralized request queue system with structured input schemas, automated Microsoft Teams webhook notifications, task status lifecycle management, and secure result repository.
How The System Operates
Structured Request Ingestion
Japan-side requester submits target websites, extraction fields, and scheduling criteria into a structured form.
Instant Webhook Broadcast
System formats the request payload and triggers an immediate alert directly into the Indonesia engineering team's Microsoft Teams channel.
Task Claim & Execution
Engineers pick up the ticket, run targeted scraping scripts, and validate extracted dataset outputs.
Result Upload & Status Closure
Scraped datasets are uploaded back to the central portal, notifying the requester and archiving the task lifecycle.
Key Technical Highlights
- •Eliminated ad-hoc communication overhead between international business teams
- •Real-time Microsoft Teams webhook notifications with deep links to request details
- •Audited repository of completed scraping deliverables and historic schemas