Databricks Engineer Job at DataTeams, New York, NY

SU9TRnlZYm1jYTlPRXYvMUppTVBiRWlyYnc9PQ==
  • DataTeams
  • New York, NY

Job Description

This is a remote position.

About Us

DataTeams help Technology Companies find the most skill specific AI enabled Data Professionals.

What You Will Help Us With as a Senior Databricks Engineer: As a Databricks Engineer, you will lead our data engineering efforts, pushing the boundaries of data processing and performance optimization. Your primary focus will be to enhance ingestion speeds to meet substantial data demands, ensuring our client systems operate at peak efficiency. You will supervise and guide your project team, ensuring successful execution and delivery, while providing expert support across various projects, leveraging your deep expertise to drive success throughout the organisation.

Your role will involve addressing complex challenges and continuously seeking innovative solutions to improve data processing and performance. Collaboration and mentorship will be vital, as you work closely with cross-functional teams, sharing insights and best practices, and mentoring junior engineers to foster a culture of continuous learning and improvement.



Requirements

Experience & Skills We’re Looking For: - Extensive experience with Databricks for building and optimising data pipelines and workflows. - Strong proficiency in SQL, Python, Scala, or Java for data processing and analysis. - Deep understanding of cloud services (AWS, Azure, or GCP) and their data offerings. - Proven experience in designing and implementing efficient ETL processes and data ingestion pipelines. - Strong background in performance optimization and tuning of data processing systems. - Ability to analyse complex datasets and extract actionable insights. - Familiarity with advanced analytics techniques and data modelling concepts.

Requirements:

- Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.

- 4+ years of experience in data engineering, with extensive experience in Databricks, or certification as a Data Engineer Professional. 

- Exceptional leadership and communication skills, with the ability to convey information clearly and provide constructive feedback, including addressing mistakes.

Preferred Certifications:

- Additional certifications in data engineering or related fields.

- Experience with machine learning and AI workflows.

- Familiarity with data governance and security best practices (e.g., Unity Catalog).

- Experience with Cloudera Impala or Apache Kudu, demonstrating proficiency in managing and optimising distributed data storage and querying systems.

- Knowledge of Apache workflows, such as Apache Airflow or Apache NiFi, showcasing the ability to design, implement, and manage complex data pipelines.

- Familiarity with data visualisation and reporting tools (e.g., Power BI, Tableau).



Job Tags

Full time, Remote job,

Similar Jobs

Comfort Keepers of Delaware County, Montgomery County and Ch...

Now Hiring: Caregivers, Home Health Aides, and Certified Nursing Assistants (CNA) Job at Comfort Keepers of Delaware County, Montgomery County and Ch...

 ...Now Hiring: Caregivers, Home Health Aides, and Certified Nursing Assistants (CNA) Are you passionate about making a difference in the lives of others? Join Comfort Keepers, where your dedication as a caregiver, home health aide, or certified nursing assistant (CNA)... 

Eyetastic Services

Medical Assistant | No Experience Required - La Crosse, WI Job at Eyetastic Services

 ...optometric practice in La Crosse, WI. No experience is required as training will be provided....  ...preliminary eye exams to gathering patient medical histories, these multitasking maestros are the backbone of the practice. They assist the optometrist in performing diagnostic... 

Medtigo

Pediatric Emergency Medicine Physician BE/BC Pittsburgh, Pennsylvania Job at Medtigo

(Physician/MD qualifications required) Pediatric Urgent Care - Currently recruiting for a faculty position at the rank of Assistant or Associate Professor outside the tenure stream in the Division of Pediatric Emergency Medicine. The candidate must have completed a Pediatric... 

E & J Gallo Winery E & J Gallo Winery

Coastal Winemaking Internship Job at E & J Gallo Winery E & J Gallo Winery

 ...024 Applications will not be reviewed until the posting close date. Join GALLO, where we own and operate over ten premium wineries across California, including renowned brands like Louis M. Martini, William Hill, Apothic and more! We're seeking passionate individuals... 

Worldwide Flight Services

Airport Customer Service/ Ramp Agent Part-Time $15.50 Per Hour FSD Job at Worldwide Flight Services

 ...team of people is highly trained, highly skilled, and confident airport service professionals who are supported by local training and...  ...Join our Worldwide Flight Services family and contribute to the timely delivery of cargo shipment, luggage, business to customer delivery...