Think Beyond The Label Jobs

Gemini Space Station seeks Senior Data Engineer in New York, NY.Duties: Develop and implement batch and real time data ingestion solutions, unified web solutions, and data visualization (business intelligence) solutions. Utilize structured spark streaming, auto loader capabilities of Databricks, and AWS services such as DMS, API gateway, kinesis stream, lambda functions and PySpark to achieve schematization, schema evolution, self-healing nature, and robust data availability. Develop orchestration and optimization of critical data pipelines using Apache Airflow and custom Python codes. Prototype, develop, and build data quality framework to achieve data profiling and validation and ensure data integrity, quality, and observability. Prepare ADR (Architecture Design Review) by analyzing and researching on the challenges of the existing components and tools and identifying a healthy alternative framework through an iterative process of prototyping, throughput elevations, performance improvements, health monitoring, log analysis, and mock implementations. Lead the architecture design review process to align on the proposed solution and ensure resilience, reliability, and robustness. Prepare and implement dimensional models that define tables, Spark SQL, ETL flows, and pipelines flows during Delta Lake implementations using Apache Spark to achieve optimal BI reporting. Mentor Data Engineers, Data Analysts and Data Scientists on data ingestion pipelines for data pipeline development, orchestration, and integration. Position based at headquarters and may be assigned to unanticipated worksites throughout the U.S. as determined by management. Telecommuting permitted.Minimum Requirements: Requires a Bachelor's degree in Computer Engineering, Electronic Engineering, or a related technical field (or foreign equivalent) plus 5 years of experience in a data engineering position, technical architect, or related occupation. Experience must include: schema design and dimensional data modeling; ETL design, implementation, and maintenance; developing data structures and algorithms; building real-time data solutions and processes; using data warehousing technologies and tools such as Python, SQL, Redshift, Snowflake.$205,000-$225,000 per year. Must also have authority to work permanently in the U.S. Applicants who are interested in this position may apply at jobpostingtoday.com Ref# 87123.

Minimum Salary: 205,000 Maximum Salary: 225,000 Salary Unit: Yearly