Note: The job is a remote job and is open to candidates in USA. Scalence L.L.C. is seeking a Lead Data Engineer to design, build, and optimize large-scale data solutions on AWS. This role combines hands-on technical expertise with leadership responsibilities, ensuring adherence to engineering best practices and mentoring team members.
Responsibilities
- Architect and maintain data pipelines using AWS native services (Glue, Kinesis, Lambda, S3, Redshift)
- Design and optimize data models on AWS Cloud leveraging Redshift, RDS, and S3
- Implement ETL/ELT workflows and PySpark jobs for data ingestion, transformation, and storage
Skills
- 5 years in designing and deploying big data applications and ETL jobs using PySpark APIs/SparkSQL
- Strong experience with AWS services across multiple domains including Kinesis, S3, RDS, Redshift, DynamoDB, Glue, EMR, Athena, SageMaker, Bedrock, EC2, Lambda, ECS, IAM, KMS, SSE
- Proficiency in SQL and relational databases (Oracle, SQL Server, Teradata); expert-level query tuning
- Hands-on experience with Python development, REST APIs (AWS API Gateway, Node.js), and CI/CD pipelines using GitHub
- Prior work experience at client or in client's Industry
- Applicants must be able to work directly for Artech on W2
- Experience with BI tools (QuickSight, Tableau)
- Knowledge of data governance, security, and compliance standards
- Familiarity with performance testing and observability tools for data pipelines
Company Overview