Log In
Sign Up
Job Title: PySpark Engineer/Senior Data Engineer
Location: Canada
Travel/Relocation/Remote: Remote
Job Description:
Senior Data Engineer role to be part of our scrum teams and perform functional system development for applications for our Enterprise Data Lake initiative This is high visibility fast paced key initiative will integrate data across internal and external sources provide analytical insights and integrate with our critical systems Essential Responsibilities Participate in the agile development process Develop functional and technical specifications from business requirements for the commercial platform Ensure application quality and adherence to performance requirements Pair up with data engineers to develop cutting edge Analytic applications leveraging Big Data technologies Spark PySpark SQL Hive Big Data Hadoop Python Scripting Shell Scripting Help create project estimates and plans Represent engineering team in project meetings and solution discussions Participate code review process Work with team members to achieve business results in a fast paced and quickly changing environment Mentor and influence up and down the chain of command Perform other duties and or projects as assigned
Requirements
- Bachelor s degree in a quantitative field such as Engineering Computer Science Statistics Econometrics
- Experience in designing and driving changes in engineering practices throughout Train and Functional Data Engineering teams
- Experience in Hive Spark PySpark w JSON Big Data Hadoop HDFS shell scripting and Python scripting
- Experience in with industry leading scheduling tools like Autosys Control
- Strong business acumen Critical Thinking and Creativity Performance tuning experience
- Experience with DevOp tools like Github BitBucket Jenkins Cloud Formation TerraForm
- Willingness to learn new technologies quickly
- Superior oral and written communication skills as well as the willingness to collaborate across teams of internal and external technical staff business analysts software support and operations staff
- Strong business acumen including a broad understanding of business processes and practices
- Spark PySpark w JSON SQL Hive Big Data Hadoop Python Scripting Shell Scripting
Disclaimer
Capgemini is an Equal Opportunity Employer encouraging diversity in the workplace. All qualified applicants will receive consideration for employment without regard to race, national origin, gender identity/expression, age, religion, disability, sexual orientation, genetics, veteran status, marital status or any other characteristic protected by law.
This is a general description of the Duties, Responsibilities and Qualifications required for this position. Physical, mental, sensory or environmental demands may be referenced in an attempt to communicate the manner in which this position traditionally is performed. Whenever necessary to provide individuals with disabilities an equal employment opportunity, Capgemini will consider reasonable accommodations that might involve varying job requirements and/or changing the way this job is performed, provided that such accommodations do not pose an undue hardship.
Click the following link for more information on your rights as an Applicant – http://www.capgemini.com/resources/equal-employment-opportunity-is-the-law
Applicants for employment in the US must have valid work authorization that does not now and/or will not in the future require sponsorship of a visa for employment authorization in the US by Capgemini.
