Key skills Azure Data Factory (primary) , Azure Data bricks Spark (PySpark, SQL
Must-have skills
- Cloud certified in one of these categories
- Azure Data Engineer
- Azure Data Factory , Azure Data bricks Spark (PySpark or scala), SQL
- Experience in Azure ingestion from on-prem source, e.g. mainframe, SQL server, Oracle.
- Experience in Sqoop / Hadoop
- Microsoft Excel (for metadata files with requirements for ingestion)
- Any other certificate in Azure/AWS/GCP and data engineering hands-on experience in cloud
- Strong Programming skills with at least one of Python, Scala or Java
- Strong SQL skills ( T-SQL or PL-SQL)
- Data files movement via mailbox
- Source-code versioning/promotion tools, e.g. Git/Jenkins
- Orchestration tools, e.g. Autosys, Oozie
- Source-code versioning with Git.
Nice-to-have skills
- Experience working with mainframe files
- Experience in Agile environment, JIRA/Confluence tools.
DataBricks - Data Engineering
