Databricks Data Engineer for a Leading Consulting Firm in Innovative Technology Solutions - BEON.tech

Company name will be shared upon confirmed interview

Databricks Data Engineer for a Leading Consulting Firm in Innovative Technology Solutions

Dallas, Texas, United States (REMOTE)

A strategic consulting and technology solutions firm that partners with organizations to drive digital transformation and business growth. They specialize in delivering customized solutions that integrate strategy, technology, and organizational development to help businesses optimize operations, enhance customer experiences, and stay ahead in a competitive market. Their expertise spans cloud computing, data analytics, software development, and enterprise architecture, empowering clients to navigate complex challenges and achieve long-term success. With a focus on continuous learning and leadership development, they foster a culture of innovation and collaboration to create sustainable value for their clients.

Salary up to:US$5,000

Timezone: GMT-5

Industries: Commercial & Professional Services

Website:sensitive

We are looking for a hands-on Data Engineer with strong Databricks experience to build, test, deploy, and support production data pipelines. The role focuses on building batch and incremental ingestion pipelines with Auto Loader and Delta Lake, developing production-quality Python and SQL notebooks and jobs, designing and maintaining Delta tables, and configuring Unity Catalog. The position also involves supporting CI/CD and environment promotion, monitoring and troubleshooting pipelines for performance, security, and cost, and creating technical documentation and knowledge-transfer materials.

Tech Stack

PPython

Databricks

5+ years in Data Engineering

Strong hands-on Databricks experience in production: Auto Loader, Delta Lake, Unity Catalog, and Workflows.

Experience building production-grade batch and incremental pipelines

Knowledge of Delta table design, schema enforcement, schema evolution, partitioning, and optimization

Strong troubleshooting, communication, and documentation skills

Strong Python and SQL skills for building production data pipelines

Every job requires English proficiency, being based in LATAM.

Bonus Skills

Experience with Git-based development and CI/CD for Databricks deployments

Strong understanding of checkpointing, idempotent processing, retries, reruns, and failure recovery

Experience tuning Spark jobs and Delta tables for performance and cost

Familiarity with Delta Live Tables or Lakeflow Declarative Pipelines

Interview process

Time to hire:2-3 weeks

BEON.TECH ASSESSMENT STAGE

Initial Interview

30 min

Technical Interview

1 hr

THE CLIENT STAGE

To be confirmed soon

TBD

We are committed to provide equal opportunities

BEON.tech connects the brightest Latin American talent with the most innovative and disruptive U.S. companies. You'll get access to a custom vetted pool of full-time, long-term, remote software jobs with compensation comparable to U.S.-based positions. To join BEON.tech is to be in a devs-first company, which means you are the priority when it comes to decision-making, client selection, and growth planning. Develop your career at the pace you deserve.

Trip to Headquarters in Buenos Aires

A US$ 1,000 welcome package to get you started with the right gear

USD compensation comparable to U.S.-based positions

English conversation club & workshops

Health insurance

Psychotherapy sessions

Flexible payments in crypto, wire transfer, Wise, PayPal, or Payoneer

Unlimited reskilling in Udemy, Education IT & O'Reilly

Internet service

AI Tools & Learning Program

Co-working spaces

Paid time off

Participate in weekly challenges and win quarterly prizes.Top performers earn US$750 (1st place) and US$350 (2nd place).

In this role, you'll be expected to take ownership of the following responsibilities as part of your day-to-day work:

Build batch and incremental ingestion pipelines with Databricks Auto Loader and Delta Lake

Develop production-quality Python and SQL notebooks, jobs, and reusable components

Design and maintain Delta tables

Configure Unity Catalog catalogs, schemas, external locations, storage credentials, and permissions

Work with cloud object storage and CSV, delimited text, and Parquet data

Support CI/CD and environment promotion using Git and Databricks Asset Bundles or comparable tooling

Monitor and troubleshoot pipeline execution, data issues, performance, security, and costs

Create technical documentation, runbooks, test evidence, and knowledge-transfer materials

Year founded: 2003

Company size: 251-1K

Company type: private

Key details to catch your eye

As a certified B Corp, this company is part of a global community committed to leveraging business as a force for good. They embrace a business model that harmonizes purpose with profit, ensuring a focus on inclusion, equity, and collaborative service to create a positive impact.

This consulting firm collaborates with leading companies to deliver innovative solutions, secure platforms, and technology stacks that seamlessly integrate with their ecosystem, including Salesforce, Google, Microsoft, and more.

Highlights

2 BEONers from Brazil.

Recognized by various organizations, including Glassdoor and Great Places to Work, for its commitment to employee satisfaction and workplace excellence.

The company has a track record of successful projects, such as scaling generative AI impact across data platforms and improving enterprise project delivery through better PMO collaboration, showcasing its expertise in delivering impactful solutions.

Culture

The team fosters a positive and collaborative culture, where open communication is encouraged.

Every member is ensured to be heard and appreciated, promoting a strong sense of inclusion.

Work-life balance is prioritized, allowing everyone to give their best while maintaining their well-being.

This supportive environment not only boosts productivity but also makes the workplace enjoyable and fulfilling for all involved.

Project's Core Tech Stack

Frontend

AAngular

RReact Native

SSwift

Ionic

Backend

..NET

MongoDB

DevOps

AWS S3

AWS EC2

Microsoft Azure DevOps

Engineering staff

A single team working on a project to streamline well-planning operations by consolidating various business processes into a centralized system.

Team structure: 1 Tech Lead / 2 Frontend Developer / 1 Backend Developer

Development methodology

Scrum - 2 weeks.

Task management is handled through Azure DevOps, which is used to organize tickets, track progress, and ensure smooth workflow throughout the project.

Deployment process

The project is still in its early stages and has not been deployed to production.

Manual deployments are currently used in the development environment.

The team is working on automating the deployment process.

QA process

The team incorporates unit tests and integration tests into their workflow.

Performance testing and UAT are performed manually.

Engineers are responsible for testing and quality assurance throughout development.

What I appreciate the most is the recognition of good work. I've worked in places where the only feedback we received was when something went wrong, so getting positive feedback motivates me to keep doing my best.

- Gabriel from Brazil

What drives me the most is the team culture and the fact that we constantly take on new challenges that help us grow and improve.

- Tulio from Brazil