Job Description: Azure Data Factory & Power BI Developer
Position: Data Engineer
Experience Level: 7+ years
Location: [Remote/Bangalore]
Job Type: Full-Time
Job Summary:
The Azure Data Factory & Power BI Developer will be responsible for designing, developing, and managing data integration solutions using Azure Data Factory, as well as creating visually compelling dashboards and reports using Power BI. You will work closely with business stakeholders to understand their data needs, extract, transform, and load data from various sources, and transform data into insightful, actionable reports for decision-making.
Key Responsibilities:
- Azure Data Factory Development:
- Design, develop, and maintain scalable and efficient ETL pipelines using Azure Data Factory (ADF) for extracting, transforming, and loading data from multiple on-premises and cloud sources to Azure-based storage solutions.
- Create, configure, and manage linked services, datasets, pipelines, and triggers within Azure Data Factory.
- Automate and schedule data workflows using ADF to handle both real-time and batch data processing.
- Ensure data pipelines are optimized for performance and scalability, meeting operational SLAs.
- Power BI Development:
- Design and develop Power BI dashboards, reports, and data visualizations that align with business requirements and provide actionable insights.
- Implement complex DAX measures, Power Query transformations, and custom visuals for optimized reporting.
- Develop and maintain Power BI data models that support high-performance querying and reporting.
- Integrate Power BI with data from various sources, ensuring seamless and secure data flow from backend systems (Azure Data Lake, SQL Database, Data Warehouses, etc.).
- Data Modeling & Integration:
- Work closely with data engineers and architects to create data models that ensure data is structured for efficient querying and reporting.
- Design and implement data integration solutions that enable a seamless flow of data between source systems and Power BI reports.
- Collaboration & Requirements Gathering:
- Engage with business stakeholders to understand data needs, define reporting requirements, and translate them into technical solutions.
- Collaborate with data engineers to integrate new data sources and ensure data consistency and accuracy in reports.
- Performance Optimization:
- Tune both Azure Data Factory pipelines and Power BI reports for high performance, focusing on query optimization, resource utilization, and report loading times.
- Security & Governance:
- Implement security and data governance best practices to ensure sensitive data is protected and compliant with relevant regulations (e.g., GDPR, HIPAA).
- Ensure role-based access and secure data-sharing policies are implemented within Power BI and Azure Data Factory.
- Documentation & Support:
- Provide comprehensive documentation on ETL processes, data models, and Power BI report designs.
- Provide ongoing support for production environments, monitor data workflows, and troubleshoot performance or data-related issues.
Required Skills & Qualifications:
- Experience: At least 7 years of experience in data integration, ETL development, and data visualization, with a strong focus on Azure Data Factory and Power BI.
- Azure Data Factory Expertise: Proficient in Azure Data Factory for creating and managing data pipelines, and working with linked services, datasets, and triggers. Experience with Azure Data Lake, Blob Storage, and SQL-based solutions is a must.
- Power BI Development: Strong experience in Power BI dashboard/report creation, DAX, Power Query, data modeling, and advanced visualization techniques.
- SQL Skills: Advanced knowledge of SQL for querying and transforming data. Experience with database management systems such as SQL Server, Azure SQL Database, or other relational databases.
- ETL Process Knowledge: Strong understanding of ETL processes and data integration best practices for cloud-based data solutions.
- Data Modeling: Knowledge of data warehousing and dimensional data modeling concepts, as well as the ability to design efficient and scalable data models.
- Cloud Expertise: Experience with Azure cloud technologies, including Azure Data Lake, Azure SQL, Azure Synapse Analytics, and related Azure data services.
- Performance Optimization: Proven ability to optimize the performance of both Azure Data Factory pipelines and Power BI reports, ensuring that they meet performance and scalability standards.
- Problem-Solving: Strong analytical skills and the ability to troubleshoot complex data issues, identify root causes, and implement solutions quickly.
- Communication Skills: Excellent verbal and written communication skills with the ability to interact with stakeholders, both technical and non-technical.
- Agile Methodology: Experience working in Agile environments, using tools such as Jira, Azure DevOps, or similar.
Desirable Skills:
- Certification in Microsoft Azure Data Engineer or Power BI.
- Experience with Azure Synapse Analytics, Azure Databricks, and other advanced data engineering services.
- Knowledge of advanced data visualization techniques and custom visuals within Power BI.
- Familiarity with data governance frameworks and compliance standards (e.g., GDPR, HIPAA).
- Familiarity with version control systems such as Git.
Education:
- Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field. Master's degree or relevant certifications (e.g., Microsoft Azure Data Engineer) is a plus.
