About Data Factory

Mission Statement: “Be the main source of "CDE's" data.”

Data Factory lives and aggregates all the data from the CDE apps / projects. In the future the aim is for the data to power the GenAI. The practice has been built on Azure Synapse and is going to be migrated to DataBricks.

Business overview

Formerly known as Decision Analytics Platform (DAP) or Global Common Data Model (GCDM).

Data Studio is a cloud-based solution designed to empower users to build datasets for their projects in a swift, efficient, and standardized manner.

The platform provides an intuitive interface and a uniform data model structure, allowing users to leverage off-the-shelf analyses (known as "Blueprints") which include both business logic (DAX) and standardized analysis packs in Excel or Power BI.

Key features

• Efficient Data Processing: Handles large datasets in a cloud-based environment for faster speeds and efficient processing.
• Model Enhancement: Allows users to update mapping tables and add business logic (DAX) to enhance data models.
• Secure Insight Generation: Deploys enhanced models to the cloud for secure analysis using tools like Excel or Power BI.

Seniority: Senior with 6+ years of relevant experience

Qualifications

• Advanced Python knowledge and software engineering skills.
  
• Proven experience with Spark and SQL for data engineering and analysis.
• Extensive experience with Databricks for data ingestion, transformation and all its latest features.
• Familiarity with Delta Lake and data warehousing concepts.
• Hands-on experience with one or more cloud services (Azure - must, AWS, GCP).
• Experience with Azure Data Factory, Azure Cosmos DB, Azure functions, Event store, ADLS, Key Vault.
• Proficiency in RDBMS/NoSQL data stores and appropriate use cases.
  
• Experience with Data as Code; version control, small and regular commits, unit tests, CI/CD, packaging, familiarity with containerization tools such as Docker (must have) and Kubernetes (plus).
• Knowledge of reporting tools like Power BI is a plus.
• Experience with ML/AI is a plus.
• Solid understanding of the software development life cycle.
• Strong communication, interpersonal and presentation skills.

Good to know before considering this project:

- We expect top notch delivery work. Also we are looking for the right mindset:

- Acting as a professional (quality, behavior, work attitude)

- Detail oriented approach (dev testing, adhering to ways of working, …)

- Ownership and responsibility (being responsible for your area of development and being proud of what has been done)

- Being able to present yourself / the work you or your team delivered

- Proactivity (actively communicating when you need something or you’re blocked, following up, helping others, etc)

- The team needs a “can do” approach type of people and “team players”

- Taking in the negative feedback constructively and being able to constructively give feedback (stating problem, suggesting improvements, stressing benefits)

‍