Job Description
Job Summary
Looking for a 5-7 years Data Engineer with Python exp
Years of experience needed
5-7 years
Technical Skills
A Data Engineer with Python experience is responsible for designing, building, and maintaining the infrastructure and systems required for processing and analyzing large datasets. They focus on data pipelines, data integration, and data storage while leveraging Python for automation, data transformation, and machine learning workflows. They collaborate with data scientists, analysts, and other engineers to ensure data availability and accuracy for various use cases.
Key Responsibilities
- Data Pipeline Development and Maintenance
Build Data Pipelines: Design and develop scalable, reliable, and efficient data pipelines to extract, transform, and load (ETL/ELT) data from various sources into data warehouses or data lakes.
Maintain Data Workflows: Use Python and other tools to automate data ingestion and transformation processes, ensuring they run reliably and on schedule.
Optimize Pipelines: Continuously monitor and optimize pipelines for performance, scalability, and cost-efficiency, ensuring low latency and high throughput in data processing.
- Data Integration and Transformation
Data Integration: Integrate data from multiple structured and unstructured sources (e.g., databases, APIs, logs, third-party platforms) into unified datasets.
Data Transformation: Use Python to clean, normalize, and transform raw data into usable formats for downstream data analytics, reporting, or machine learning models.
Data Quality Management: Implement data validation rules, error handling, and monitoring to ensure data accuracy, consistency, and reliability across systems.
- Data Storage and Architecture
Data Warehousing: Design and maintain robust data storage solutions, including relational databases (e.g., PostgreSQL, MySQL) and cloud-based data warehouses (e.g., AWS Redshift, Google BigQuery, Snowflake).
Data Lake Management: Set up and manage data lakes (e.g., AWS S3, Azure Data Lake) for handling large volumes of unstructured and semi-structured data.
Database Optimization: Optimize database queries, indexing, and partitioning to ensure efficient data retrieval and processing.
- Automation and Scripting with Python
Automation: Use Python to automate repetitive data engineering tasks, such as data extraction, data cleaning, and system monitoring.
Custom Scripts: Develop Python scripts for various purposes, such as API integrations, data scraping, or automated s for monitoring data flows.
ETL Automation: Design and implement ETL automation frameworks using Python libraries like Pandas, PySpark, Airflow, or Luigi for scheduling and monitoring jobs. Certifications Needed:
NA
About Mphasis
Mphasis applies next-generation technology to help enterprises transform businesses globally. Customer centricity is foundational to Mphasis and is reflected in the Mphasis Front2Back Transformation approach. Front2Back uses the exponential power of cloud and cognitive to provide hyper-personalized (C=X2C2TM=1) digital experience to clients and their end customers. Mphasis Service Transformation approach helps shrink the core through the application of digital technologies across legacy environments within an enterprise, enabling businesses to stay ahead in a changing world. Mphasis core reference architectures and tools, speed and innovation with domain expertise and specialization are key to building strong relationships with marquee clients.
Equal Opportunity Employer
Mphasis is an equal opportunity/affirmative action employer. We provide equal employment opportunities to applicants and existing associates and evaluate qualified candidates without regard to race, gender, national origin, ancestry, age, color, religious creed, marital status, genetic information, sexual orientation, gender identity, gender expression, sex (including pregnancy, breast feeding and related medical conditions), mental or physical disability, medical conditions military and veteran status or any other status or condition protected by applicable federal, state, or local laws, governmental regulations and executive orders. View the EEO in the law poster , view the EEO in the law supplement . To view the pay transparency nondiscrimination provision please click and to view the E-Verify posting click .
Mphasis is committed to providing reasonable accommodations to individuals with disabilities. If you need a reasonable accommodation because of disability to search and apply for a career opportunity, please send an email to [Confidential Information] and let us know your contact information and the nature of your request.
