SENIOR DATA ENGINEER

Building Data Pipelines

Senior Data Engineer | 7+ Years | Israel

📍 Israel 🔗 LinkedIn 💻 GitHub

About

Senior Data Engineer with 7+ years building data pipelines and analytics platforms in Israeli fintech and eCommerce.

Specialised in high-volume ETL, regulatory data delivery, and operational analytics for banking environments. Reduced processing times by 50%+ across multiple engagements. Stack: BigQuery, PostgreSQL, MSSQL, Python, AWS, Informatica, Airflow. Currently integrating Claude API into production automation pipelines that run daily via GitHub Actions.

⚙️

Engineering

📊

Analytics

🚀

Automation

Projects

Enterprise Revenue SSOT & Semantic Layer

Multiple teams calculated the same KPIs with different results. 6-hour pipeline runs. Stakeholders didn't trust the data.

ARCHITECTURE

Source Tablestransactions · customers

→

Control-Morchestration

→

Semantic Layersingle calc engine

→

Dashboardsreliability + KPIs

WHAT I BUILT

Centralised semantic layer with a single standardised calculation engine
Control-M job chain replacing fragmented, unreliable scheduling
Automated data validation and integrity checks at every pipeline stage
Executive dashboards exposing data reliability alongside business metrics

58% faster (6h → 2.5h) 100% KPI consistency Company-wide SSOT 15% customer base in risk analytics

PostgreSQLControl-MTableauPython

View on GitHub ↗

Production-Ready PostgreSQL Development Environment

No safe place to develop and test complex database logic without risking production data or systems.

ARCHITECTURE

DDL Scriptsschemas · functions

→

Docker Containerisolated env

→

PostgreSQLmulti-schema + RBAC

→

pytest SuiteFK · views · functions

WHAT I BUILT

Multi-schema Docker environment matching production architecture exactly
Custom functions, stored procedures, and materialized views with CONCURRENTLY refresh
Role-based access control matching production security tiers
Automated FK integrity and view completeness tests via pytest

Zero production risk 2-min setup from scratch Automated FK + view validation

DockerPostgreSQLPythonBash

View on GitHub ↗

Multi-Source Cloud Analytics Platform

On-prem user behaviour data was siloed from external cloud datasets. No infrastructure to combine them efficiently — cross-source reports took hours manually.

ARCHITECTURE

PostgreSQL + S3on-prem + external

→

AWS Glue ETLtransform + catalog

→

AthenaParquet / serverless

→

BigQueryunified analytics

WHAT I BUILT

AWS Glue ETL jobs for automated extraction and transformation from multiple sources
Glue Data Catalog connecting PostgreSQL, S3, and Athena into a unified metadata layer
Partitioned Parquet storage reducing Athena scan costs by 60%+ vs CSV
BigQuery integration for cross-cloud analytics and unified reporting

40% infra cost reduction 5+ sources consolidated 92% faster queries (2-5 min → 5-15 sec) Same-day insights

AWS S3GlueAthenaBigQueryPython

View on GitHub ↗

Materialized Views with Control-M Scheduling

Report queries joining multiple large tables ran 10+ minutes during business hours, contending with production workloads and causing unpredictable delays.

ARCHITECTURE

Source Tablestransactions · risk

→

Materialized Viewpre-aggregated KPIs

→

Control-M18:30 job chain

→

BI Dashboardssub-second queries

WHAT I BUILT

Optimised materialized view pre-aggregating all KPIs at refresh time — not query time
Composite indexes built around actual dashboard filter patterns, not schema normalization
CONCURRENTLY refresh (requires unique index) keeping dashboards live during the refresh window
Control-M job chain gating refresh on upstream pipeline completion with SLA alerting

>95% query time reduction 10 min → sub-second Zero daytime prod load 99.5% refresh reliability

PostgreSQLControl-MSQL

View on GitHub ↗

Product Funnel Analytics & User Journey Optimization

High drop-off rates with no visibility into where users failed or why — UX issues and technical errors were indistinguishable, and problems were found hours after the fact.

ARCHITECTURE

Web / Mobile Eventspage views · clicks

→

BigQueryfunnel pipeline

→

Happy + Error Flowdual dashboards

→

Looker + Alertsself-service

WHAT I BUILT

BigQuery event processing pipeline with per-funnel-step tracking across web and mobile
Dual dashboards: happy flow (completions + engagement) vs error flow (failures + abandonment)
Cohort analysis framework for measuring product change impact over time
Automated anomaly alerting — conversion drops detected within 5 minutes, not 2 hours

+23% signup conversion 2h → 5 min incident detection 15% fewer support tickets 60% fewer analyst requests

BigQuerySQLLookerPython

View on GitHub ↗

Side Projects

Job Scraping Pipeline

Manual job search across 6 platforms daily, with no way to generate ATS-optimised CVs at scale or track which applications were sent.

ARCHITECTURE

6 Job BoardsLinkedIn · Indeed · 4 more

→

Selenium Scrapersscore + dedup

→

LLM ChainGemini → Groq → Claude

→

Email + OutreachResend · auto-dedup

WHAT I BUILT

Selenium scrapers for LinkedIn, Indeed, AllJobs, Drushim, Wellfound, and Glassdoor
Weighted scoring: skills 40%, title 20%, experience 20%, location 10%
LLM failover chain (Gemini → Groq → Cerebras → Anthropic) with 75% ATS threshold — below 75% is rejected and regenerated
Automated recruiter email discovery (Hunter.io → Snov.io → Apollo.io) with outreach dedup

6 boards scraped daily ATS-gated CV generation Fully automated outreach

PythonSeleniumPostgreSQLClaude APIGemini

View on GitHub ↗

Garmin Daily Brief

No way to get personalised daily workout recommendations based on actual recovery data from the watch — generic plans ignore HRV and sleep quality.

ARCHITECTURE

Garmin ConnectHRV · sleep · load

→

Claude APIclaude-sonnet-4-6

→

SQLite Historytraining context

→

Watch + EmailJSON push · Resend

WHAT I BUILT

garth library for Garmin Connect API authentication and data extraction
Claude interprets HRV, sleep score, and training load to set workout intensity
Structured workout JSON pushed directly to the Garmin watch via API
SQLite stores training history for longitudinal context in each prompt

Runs daily on GitHub Actions Recovery-based adaptation Zero manual input

PythonClaude APIGitHub ActionsSQLiteResend

View on GitHub ↗

Data Engineering Curriculum

No curriculum connecting Python fundamentals to the production data engineering patterns actually used in banking and eCommerce — most courses stop at toy examples.

ARCHITECTURE

Python Basicstypes · functions

→

SQL & Modelingwindows · indexes

→

Cloud StorageS3 · BigQuery

→

OrchestrationAirflow · Docker

WHAT I BUILT

Module-based progression from fundamentals to production pipeline patterns
Real-world scenarios drawn from fintech and eCommerce environments
Covers AWS, BigQuery, Airflow, Docker, and common orchestration patterns
Code exercises at each stage with worked solutions

Structured learning path Production-ready patterns Hands-on exercises

PythonSQLAWSAirflowDocker

View on GitHub ↗

Impact

58%

Faster pipeline processing (6h → 2.5h)

40%

Infrastructure cost reduction

23%

Conversion rate improvement

99.5%

Pipeline reliability uptime

95%+

Query time reduction (10 min → sub-second)

100+

KPIs standardised across all business units

Career

Dec 2024 — Present

Sr. BI Developer & Analytics Engineer

Finubit (Bank Leumi Digital Banking)

Production data pipelines with Control-M orchestration
Advanced PostgreSQL development — stored procedures, materialized views, query optimisation
Informatica source-to-target pipelines for Bank Leumi regulatory files
Grafana dashboards across all environments including external Bank Leumi QA dashboard
AI-assisted development workflows integrated into daily practice

PostgreSQL Control-M Informatica Grafana Python

Nov 2021 — Nov 2024

Sr. Data Analyst & BI Developer

Pepper Banking (Bank Leumi)

Owned MSSQL functions in a Control-M → Python dispatch → MSSQL pipeline; reduced ETL runtime from 6-8h to 4h
Took operational ownership after senior DE departure — monitored daily runs, diagnosed failures, re-executed
Enterprise SSOT architecture and semantic layer design — 100% KPI consistency across business units
Multi-cloud integration: AWS (S3, Glue, Athena) + GCP (BigQuery)
BigQuery + Looker dashboards for app-side funnel analytics; 23% conversion improvement

MSSQL Control-M BigQuery AWS Looker

May 2019 — Oct 2021

BI Team Lead

TCM eCommerce

Built QuickBooks → MSSQL → Tableau pipeline covering 50+ Amazon stores
Hired and led a 2-person BI team; introduced self-service BI and sprint planning
SSIS ETL; reduced reporting time from 2 days to 2 hours

MSSQL SSIS Tableau QuickBooks

Earlier