Note: The job is a remote job and is open to candidates in USA. IT Associates is looking to add a Lead Data Engineer / Architect to their team that will have full authority over data architecture decisions and the engineering roadmap. The role involves designing a Data Lakehouse, establishing data governance frameworks, and leading a team of engineers while ensuring alignment with business needs.
Responsibilities
- Design and own the end-to-end data architecture. Example: Databricks(or Snowflake, or Hybrid)-— Unity Catalog, Delta Lake, medallion layer structure (Bronze / Silver / Gold)
- Define data modeling standards: Example: star schema, dimensional modeling, and semantic layer design using dbt
- Establish data governance frameworks including lineage, cataloging, and access controls
- Own the technology selection process in partnership with the COO, consultants, and other stakeholders
- Ensure the architecture can support investment analytics, portfolio reporting, and operational workflows at scale
- Build and own the ingestion layer: Example: Fivetran/Airbyte connectors for custodians, Bloomberg, and internal systems
- Develop and maintain dbt transformation models, tests, and documentation
- Configure and manage pipeline orchestration using Airflow or Prefect
- Write clean, production-grade Python for data pipelines, custom connectors, and analytical tooling
- Build internal dashboards and data applications using for example Plotly Dash or Streamlit
- Integrate Claude Enterprise (Anthropic) into data workflows for AI-assisted analysis and automation
- Recruit, interview, onboard, and mentor direct reports — you define who joins this team
- Define scope, manage delivery, and sign off on all work from external consultants (pipeline specialists, viz specialists)
- Ensure all consultant engagements include knowledge transfer and documentation before close
- Build a team culture grounded in engineering excellence, documentation discipline, and continuous improvement
- Report directly to the COO with regular updates on build progress, risks, and resource decisions
- Translate complex technical decisions into clear business terms for investment and operations leadership
- Collaborate with portfolio managers, operations, and compliance teams to understand and prioritize data needs
- Own the data roadmap and communicate trade-offs transparently
Skills
- 7+ years in data engineering, data architecture, or a combined role
- Demonstrable experience designing and building a modern Data Lakehouse or cloud data warehouse from the ground up — not just maintaining one
- 3+ years working with cloud data platforms (Databricks or snowflake strongly preferred)
- Strong hands-on dbt experience: model design, testing, documentation, and semantic layer development
- Experience managing or mentoring engineers — you have grown people, not just pipelines
- Track record of working directly with senior business stakeholders and translating requirements into technical solutions
- Databricks, Snowflake: Delta Lake, Unity Catalog, Databricks SQL, cluster configuration
- Dbt Cloud: transformations, testing, documentation, multi-environment deployment
- Data ingestion: Fivetran, Airbyte, custom connector development
- Orchestration: Apache Airflow or Prefect (managed or self-hosted)
- Python: production-grade scripting, pipeline development, Plotly/Dash or Streamlit
- Data modeling: star schema, dimensional modeling, slowly changing dimensions
- SQL: expert-level; complex transformations, window functions, query optimization
- Git, CI/CD, and engineering best practices
- Financial services or asset management experience strongly preferred
- Familiarity with custodian data feeds, Bloomberg API, or investment data sources is a significant advantage
- Working knowledge of data governance, lineage, and quality frameworks
- Direct experience with Databricks Unity Catalog rollout
- Hands-on experience with data catalog tooling (Atlan, Monte Carlo, or similar)
- Familiarity with Claude Enterprise or Anthropic AI tools in a production context
- Experience with SOC 2 data environments and security-conscious engineering practices
- Exposure to portfolio management systems, order management systems (OMS), or similar investment platforms
- Experience standing up a data team from zero — first hire, first pipeline, first architecture decision
Benefits
- A range of medical, dental, vision, retirement, paid time off, and/or other benefits are available.
Company Overview