LiftmyCVLiftmyCV
IU

Python with Spark Developer

Infotel UKOn-siteChennai, IN
AI job summary

Infotel UK is seeking a Python with Spark Developer in Chennai, India. The successful candidate will be responsible for designing and developing data pipelines using Python, PySpark, and SQL. Key responsibilities include optimizing SQL queries, collaborating with data teams, maintaining CI/CD processes, and monitoring production workloads. The ideal candidate should have strong knowledge of Python, PySpark, SQL, and ETL processes, along with good interpersonal skills and the ability to work with new technologies. A Bachelor's degree or equivalent is required.

Job description

Responsibilities

Direct Responsibilities

  • Design & develop robust ingestion, transformation, and enrichment pipelines with Python, PySpark, and SQL.
  • Write and optimize complex SQL queries, analytical UDFs, and window functions for data aggregation and reporting.
  • Collaborate with CEFS data architects, data scientists, and business analysts to translate functional requirements into technical specifications.
  • Unit‑test, integrate‑test, and review code.
  • Maintain CI/CD pipelines (Git, Jenkins, Docker) for automated build, test, and deployment of jobs.
  • Monitor production workloads and troubleshoot performance bottlenecks, memory issues, and job failures.
  • Document data lineage, pipeline design, and operational run‑books in Confluence/SharePoint.
  • Keep up to date with latest technologies and trends and provide input, expertise and recommendations.

Contributing Responsibilities

  • Contribute towards innovation (e.g. AI/ML); suggest new technical practices for efficiency improvement.
  • Participate in Agile ceremonies (sprint planning, daily standups, retrospectives) and help groom the backlogs.
  • Mentor junior engineers and champion best practices in Python coding, Spark optimization, and data-engineering patterns.
  • Evaluate emerging technologies and deliver proof‑of‑concepts for CEFS.

Technical & Behavioral Competencies

  • Resourceful to quickly understand complexities involved and provide the way forward.
  • Good experience in technical analysis of n-tier applications with multiple integrations using object oriented, APIs & Microservices approaches.
  • Strong knowledge about design patterns and development principles.
  • Inclination and prior experience of working across SQL, Python and ETL.
  • Strong Hands-on experience - Python (NumPy, pandas, Python Frameworks, Restful APIs, MS-SQL or Oracle).
  • PySpark - DataFrames, Spark SQL, Structured Streaming, performance tuning (partitioning, caching, broadcast joins).
  • Advanced SQL – complex queries, stored procedures, query optimization.
  • Good Knowledge and experience to use Python packages such as Pandas, NumPy, etc. for cleaning up of Data, Data Wrangling, Analysis of Data and Visualization of Data.
  • Good experience in development and maintenance of code/scripts in both functional and technical specifications of all applications component, bug fixing and production support.
  • Good knowledge on Linux/Unix environment (basic commands, shell scripting, etc.), testing phases, documentation and new framework.
  • Some experience of working with build tools like Maven & DevOps tools like Bitbucket, Jenkins.
  • Knowledge of Agile, Scrum, DevOps.
  • Development experience in Data Engineering environment.
  • Ability & willingness to learn & work on diverse technologies (languages, frameworks, and tools).
  • Self-motivated, good interpersonal skills and inclination to constantly upgrade new technologies and frameworks.
  • Good communication and co-ordination skills.

Specific Qualifications

  • Good to have knowledge of front-end technologies preferably Flask.

Skills Referential (Required knowledge, skills and abilities)

Technical Skills:

  • Python
  • Pyspark
  • SQL
  • ETL

Behavioral Skills:

  • Ability to synthetize / simplify
  • Ability to collaborate / Teamwork
  • Attention to detail / rigor
  • Ability to deliver / Results driven

Education Level:

  • Bachelor’s degree or equivalent

Location:

  • Chennai

Job details

Published
June 12, 2026
Last checked
June 19, 2026
Job source
jworkable
Workplace
On-site · Chennai, IN
First seen
June 12, 2026
AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Infotel UK

Similar jobs from this company

None listed.

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.