Data Integration Engineer

Key Qualifications MUST HAVE:

• Snowflake Data Engineering –

o Design and implement enterprise-grade data pipelines using Snowflake, including ingestion and transformation o Must be strong in both Core and Semantic aspects o Develop complex SQL transformations, stored procedures and Dynamic tables inside Snowflake to enable near real-time and batch processing o Implement Snowflake data sharing, data marketplace integrations o Engineer Snowpipe and Kafka-to-Snowflake streaming ingestion pipelines also handling high throughput event data at scale o Optimize Snowflake cluster performance – virtual warehouse sizing, query profiling, clustering keys o Architecture, design aspects, performance tuning, time travel, warehouse concepts - scaling, clustering, micro-partitioning o Experience with SnowSQL, Snowpipe

• Data Integration aspects –

o Design and maintain end-to-end ETL/ELT pipelines using Apache Airflow o Experience in building reusable parameterized data ingestion pipelines/frameworks is beneficial. o Thorough on data quality checks

• AI and Data Science –

o Integrate AI/LLMs with data pipelines via Python UDFs or API callouts – enabling text analytics, semantic search and GEN-AI augmented workflows o Experience with Python based frameworks – scikit learn, PyTorch, TensorFlow o Experience with NLP and text-mining techniques on unstructured data to identify actionable information o Time-series forecasting, anomaly detection and propensity modeling

• Experience with Data Visualization aspects
• Hands-on experience with writing Complex queries using – Joins, Self Joins, Views, Materialized Views, Cursor also Recursive, use of GROUP BY, PARTITION BY functions / SQL Performance tuning
• Hands-on experience with ETL and Dimensional Data Modelling – Slowly Changing Dimensions (SCD – Type 1, 2, 3)

o Good understanding of concepts like schema types, table types - fact-dimension etc. like how to design a dimension vs fact, design considerations factored etc.

• Proficiency in Python scripting/programming – using Pandas, PyParsing, Airflow.

o Pandas, Tableau server modules, Numpy, Datetime, Apache Airflow related modules, APIs o Data Pipeline automation o Strong Python programming skills

• Actively participating in discussions with business to understand requirements, perform thorough impact analysis and provide suitable solutions.

Key Words to search in Resume Snowflake, Advanced SQL, Dimensional Data Modelling (Slowly Changing Dimensions), Python, AI, Data Science, Data Visualization

Location: Sunnyvale, CA Salary range:$80,000-$140,000 a Year. #LI-AS3

Location Sunnyvale, CA Job Function TECHNOLOGY Role Engineer Job Id 412666 Desired Skills SQL | ETL Testing | SNOW Salary Range $80,000-$140,000 a year