Back to projects
Case Study 04

Competitive
Intelligence

Under NDA · client name not disclosed
Segment E-commerce · Retail
Type Automated competitive intelligence system
Stack Python · Playwright · PostgreSQL · Cron
Context

An e-commerce company needed to know what competitors were doing in real time. Prices, stock availability, new products, promotions — daily, automatically, without human intervention.

Manual monitoring of 15+ competitor websites took 3–4 hours daily. Results came late. Pricing decisions were based on gut feeling, not data. Goal: information advantage.

COMPETITOR-A.CZ COMPETITOR-B.CZ COMPETITOR-C.CZ + 12 MORE LAYERS INTELLIGENCE ENGINE {"product":"SKU-4821","price":2690,"stock":true} {"product":"SKU-4821","price":2990,"stock":true} {"product":"SKU-4821","price":3190,"stock":false} → LOWEST: €109 | YOU: €121 | Δ -300 PRICE DASHBOARD ⚠ COMPETITOR-B DROPPED PRICE BY 12% ✓ YOU HAVE LOWEST PRICE ON 4 SKUs △ 3 PRODUCTS OUT OF STOCK AT COMP. LAST SCAN: 06:00 CET · NEXT: 12:00
Situation

15+ competitors. Thousands of SKUs. Prices change daily.

A segment with aggressive price competition. Customers compare prices on aggregator platforms — a $2 difference decides the order.

Without real-time competitor data, reactions were delayed. By the time a competitor price drop was noticed, two days of sales were lost. During peak season, that meant tens of thousands in lost revenue.

What we built

Autonomous intelligence system. Nonstop monitoring. Automatic alerts.

Playwright-based scraping pipeline that every 6 hours crawls the entire catalog of competitor websites — prices, availability, promotions, new products. Data flows into PostgreSQL, historical trends are visualized in a dashboard.

Automatic alerting: competitor dropped a price? Product went out of stock? New SKU? The answer arrives within a minute. Not two days later.

System capabilities
Price monitoring
Real-time price collection from competitor e-shops. SKU-level comparison. Historical trends. Averages, medians, deviations.
Stock availability
Competitor inventory monitoring. Out-of-stock detection = opportunity to raise prices or increase PPC spend.
Alerting & notifications
Automatic notifications on price changes, stock outages, or new products. Slack, email, or custom webhook.
Historical trends
Price history going back months. Seasonal patterns, Black Friday reactions, pattern detection. Data for predicting competitor strategy.
Anti-detection
User agent rotation, proxy management, fingerprint randomization, rate limiting. System designed for long-term uninterrupted operation.
Custom dashboard
Data visualization in a clean internal dashboard. Filtering by category, brand, price range. Export to CSV/Excel.
Architecture
01
Crawl
Playwright headless browser. Cron schedule (4× daily). Proxy rotation. Anti-bot evasion. Parallel processing.
02
Parse
Structured data extraction. Prices, names, SKUs, availability. Normalization for cross-site comparison. Deduplication.
03
Store
PostgreSQL with time series. Historical data never deleted. Indexes optimized for range queries and aggregations.
04
Alert
Rule-based change detection. Threshold notifications. Slack webhook + email. Dashboard with live data and export.
Tech Stack
Scraping
Playwright
Headless Chromium for JavaScript-heavy e-shops. Full page rendering. Stealth mode with fingerprint randomization.
Language
Python
Async pipeline with asyncio. BeautifulSoup for HTML parsing. Custom normalization layers for cross-site matching.
Database
PostgreSQL
Time series of pricing data. Partitioned tables for fast queries. Full history retention for trend analysis.
Scheduling
Cron + Supervisor
4× daily automatic runs. Process monitoring. Auto-restart on failure. Logging and error tracking.
Alerting
Slack + Email
Webhook notifications to Slack channel. Email digest with daily overview. Custom rules for trigger conditions.
Deploy
Railway
Cron jobs, worker processes, database hosting. Zero-downtime deploys. Environment isolation.
Results
15+
competitors monitored
Non-stop
automatic scans per day
Cron pipeline
<1min
reaction to price change
Real-time alerting
0
hours of manual work
Fully automated

This project is covered by a non-disclosure agreement. We do not share the company name — but we're happy to discuss the project itself, architecture, and results in more detail.

If you're solving a similar problem — competitor monitoring, data collection automation, or pricing intelligence — we'd love to show you more.

Need competitive intelligence, a scraping pipeline, or automated monitoring?

Book a call