DP-3028: Implement Generative AI engineering with Azure Databricks
Overview
Generative Artificial Intelligence (AI) engineering with Azure Databricks uses the platform's capabilities to explore, fine-tune, evaluate, and integrate advanced language models. By using Apache Spark's scalability and Azure Databricks' collaborative environment, you can design complex AI systems.
Who should attend
This course is best suited to Data Scientists
Prerequisites
Before attending this course, students should have familiarity with fundamental AI concepts and Azure Databricks.
What you'll learn
After completing this course, students will be able to:
- Describe Large Language Models (LLMs)
- Use LLMs for Natural Language Processing (NLP) tasks
- Set up a RAG workflow
- Improve model accuracy by reranking your search results
- Identify the need for multi-stage reasoning systems
- Implement multi-stage reasoning with libraries like LangChain, LlamaIndex, Haystack, and the DSPy framework
- Understand when to use fine-tuning
- Compare LLM and traditional ML evaluations
- Describe the responsible AI principles for implementation of language models
- Implement key security tooling for language models
Course content
- Describe Generative AI.
- Describe Large Language Models (LLMs).
- Identify key components of LLM applications.
- Use LLMs for Natural Language Processing (NLP) tasks.
- Lab: Explore language models
- Set up a RAG workflow.
- Prepare your data for RAG.
- Retrieve relevant documents with vector search.
- Improve model accuracy by reranking your search results.
- Lab: Set up RAG
- Identify the need for multi-stage reasoning systems.
- Describe a multi-stage reasoning workflow.
- Implement multi-stage reasoning with libraries like LangChain, LlamaIndex, Haystack, and the DSPy framework.
- Lab: Implement multi-stage reasoning with LangChain
- Understand when to use fine-tuning.
- Prepare your data for fine-tuning.
- Fine-tune an Azure OpenAI model.
- Lab: Fine-tune an Azure OpenAI model
- Compare LLM and traditional ML evaluations.
- Describe the relationship between LLM evaluation and evaluation of entire AI systems.
- Describe generic LLM evaluation metrics like accuracy, perplexity, and toxicity.
- Describe LLM-as-a-judge for evaluation.
- Lab: Evaluate an Azure OpenAI model
Common questions
When does DP-3028: Implement Generative AI engineering with Azure Databricks next run?
The next date is Mon, 14 Sept 2026, and there is 1 further date scheduled. Delivered ACE Training Wellington, ACE Training Auckland. Every date and its format is listed on this page.
How much does DP-3028: Implement Generative AI engineering with Azure Databricks cost?
From $895 + GST per person. Pricing can differ by date and location, so check the option you want before booking.
Do I need any prior experience?
Before attending this course, students should have familiarity with fundamental AI concepts and Azure Databricks.
Can I attend online, or does it have to be in a classroom?
This one runs in-class. If you'd rather it ran online or at your workplace, get in touch and we'll look at options.
Can you run this course for my team?
Yes. Any ACE course can be delivered in-house at your workplace and tailored to your team's tools and goals — quote course code DP3028 when you enquire.