Responsibilities
- Design and develop scalable backend services integrating with external vendor systems via RESTful APIs
- Implement robust data ingestion services including pagination and incremental/time-window retrieval
- Handle API error handling, retries, and idempotent processing to ensure reliable integrations
- Build and maintain data pipelines for processing large volumes of transactional and event data
- Perform data transformation, validation, and enrichment for downstream systems
- Support batch and near real-time data processing workflows
- Develop and deploy applications using Microsoft Azure services such as Azure Functions, App Services, Storage, Key Vault, APIM, and VNET
- Design scalable, resilient, and highly available cloud-native solutions
- Implement observability practices including logging, monitoring, and alerting
- Ensure data accuracy, consistency, and reconciliation across multiple data sources
- Support auditability and traceability of data flows for regulatory requirements
- Collaborate with external vendors on API contracts and integration designs
- Deliver high-quality, maintainable, and reusable software solutions
- Contribute to engineering standards, best practices, and continuous improvement initiatives
- Support CI/CD pipelines, automated testing, and continuous delivery practices
Requirements
Core Technical Skills
- Strong proficiency in Python (preferred) or backend languages such as Java or Node.js
- Strong SQL proficiency including PostgreSQL, MySQL, or T-SQL
- Solid experience building backend services and API integrations
- Experience working with RESTful APIs and JSON payloads
- Knowledge of secure integration patterns including API keys, TLS, certificates, and PGP
Good understanding of:
- Pagination, filtering, and incremental data retrieval
Idempotency, retries, and distributed system failure handling - Experience with data visualisation tools such as Power BI, including connectivity and configuration
Data Engineering & Processing
- Experience building batch or streaming data pipelines
- Familiarity with data transformation, validation, and data modelling
- Experience handling large datasets and high-volume ingestion systems
- Understanding of data reconciliation and audit requirements is an advantage
Databases
- Experience with relational databases such as PostgreSQL or MS SQL
- Knowledge of query optimisation and large-scale data handling
Cloud & DevOps
- Experience with cloud platforms, preferably Microsoft Azure
- Experience setting up and managing CI/CD pipelines using GitLab CI/CD or similar tools
- Familiarity with Docker/containerisation is an advantage
Observability & Reliability
- Experience implementing logging, monitoring, metrics, and alerting solutions
- Understanding of system reliability and fault-tolerant design principles
Originally posted on