Make a career shift and become a Data Engineer

If designing web services, writing CSS styles or preparing static analytics dashboards don't make you happy anymore and you think that the data engineering will be your new passion, it's the place for you.

Back in 2016 I was in one of these situations. After the years spent on web and web services development with Java, I decided to look around to find a new passionate domain. After a short hesitation between data science and data engineering I chose the latter one, and today I can say that it was the best choice I could make at that time!

Unfortunately, I spent almost 2 years at discovering data engineering concepts and figuring out how to use them. Meantime I worked on different data projects in streaming and batch, applied serverless approach, experienced different data architectures and read a lot. It took me a lot of time to figure the things out. You can follow my path or take a shortcut.

I used all my previous experiences to create a learning path based on the problems I faced, and the solutions and patterns I found to solve them. It will help you to assimilate basic data concepts faster and gain some hands-on experience before you write your first data pipelines as a data engineer. I'm saying hands-on because you'll not only get the learning material but also have to implement a data system on your own!

Curious about the learning path? See the description of the content below.

Your instructor

My name is Bartosz Konieczny and I am a data engineer working with software for 2009. I'm also an Apache Spark enthusiast, AWS Big Data certified user, blogger and speaker. I like to share and you can discover it on my waitingforcode.com blog or conferences like Spark+AI Summit 2019.

Check me on social media: LinkedIn Twitter Github Stack Overflow Facebook

Pricing plans

Watch 3 samples of Become a Data engineer course

Frequently asked questions

❓ What I will be capable of by the end of the course?

After 12 weeks you should be able to:

• understand scalability in data distributed system
• design data systems
• know the most important patterns of modern data architectures
• understand globally data concepts and use that to understand existing data frameworks and data stores
• write batch and streaming pipelines with modern data frameworks and data stores
• switch faster to cloud managed services

❓When does the course start and how long does it take?

The course is intended to start 4 times a year, March, June, September and November. You will need at least 12 weeks to finish it. Joining the course outside these dates won't be possible. If you missed the date, you can subscribe to mailing list and be alerted before the next opening.

❓How long do I have access to this course?

You get a lifelong access to the course, including all updates.

❓ Can I follow the course at my own pace?

Yes, you can organize your learning plan as you want. The course is divided into 12 Weeks for easier presentation of the content but it doesn't mean that you have to terminate it after 12 weeks.

❓How long every week takes?

Except Weeks 0 and 12, every week has between 70 and 138 minutes of the recorded lessons.

❓What do I need to follow the course?

Time and motivation. The content tries to cover as many data engineering parts as possible so it will require a strong motivation. And technically, if you want to make the homework exercises, you should be able to write some code (preferably in Python, Scala or Java), and execute Docker images on your computer.

❓What about the code snippets?

Most of the code snippets are written in Scala, Java and Python. However, they use very basic concepts of these languages, so even if you don't know any of them, you should be able to understand the examples.

❓What if I'm not satisfied with the course?

If for any reason the course doesn't satisfy you, we will issue a refund. The guarantee is valid for 21 days from the first week publication date.

❓Will I get an invoice?

Yes, just let me know on [email protected] For the EU users, I cannot include the VAT in my price, so don't have the EU VAT number neither.

❓Do you have a group offer?

YES! If you have a team of 3 people or more, contact me at [email protected] and you will get a 10% discount!

❓Can I pay for the course in installments?

It's too complicated logistically, I prefer one-time payment.

❓How can I communicate with you?

The content and all public communication will be in English but if something is unclear I can explain it in French or Polish in private. Just let me know 😀

❓How can I ask a question?

Please create a new topic on our user group. The idea of open questions and transparency is to share this extra knowledge with all the class.

❓How will I get an access to the course?

After the confirmation of your payment I will send you an e-mail with all details to access the learning platform.

❓What is the difference between a class and 1-on-1 call?

An 1-on-1 call is the call between you and me whereas a class call is the call with other students.

❓How to join live calls?

Every 3 weeks you will find there the link to the class Live Call.

❓When the class live will happen?

On Saturday at 5PM UTC. If it's too late for your timezone, and you're not alone in this situation, I will propose you another schedule on Saturday before noon (UTC).

❓How to join an 1-to-1 live call? You have to write me a message that you need one and we'll find a schedule that fits for both of us.

❓Can you add new content?

If you think that there is something missing in the schedule, please contact me at [email protected] and I will try to add this topic. You have lifetime access to the course, so you will see this and other updates.

❓What do I need to make the homework exercises?

You should be able to run Docker images and use an IDE. I will give all my examples on IntelliJ and PyCharm so having the same tools will facilitate the troubleshooting.

I will give the examples in Scala or Python but you can do the homework exercises in any language you want.

❓What is the format of the course?

Most of the time you will see me explaining concepts with a blackboard. From time to time I will hide myself and show you me screen to explain data concepts with some code or slides.

❓What I will code during the course?

You've just joined data engineering team at MyBlogAnalytics and you were asked to implement different data pipelines with Apache Spark, Apache Kafka, Apache Airflow and PostgreSQL, preferably with Python or Scala.

During your first week you will have to implement a data ingestion part, i.e. move the data from your consumers to the system.

Just after that you'll implement an analytical pipeline and make it visible to the end and not technical users.

By the end of your mission, you'll expose your data through an API to other technical departments of your company. You'll also collaborate with the data scientists of your team to create a Machine Learning pipeline.

❓Do I have to do all the homework exercises in order?

No. Even though the homework announcements are logically connected, they don't have to be made in order. For example, you can start with a homework of the Week 5 and go back to the Week 1 later.

❓What is the format of the homework exercises?

There are 3 tasks categories: conceptual work where I will ask you to define some architectures, coding exercises where I will ask you to implement one component of our system, and open questions where I will ask you to do some research before answering.

Every week's homework is composed of 2 parts called "main" and "bonus". To move forward you should at least terminate the "main" part and if you're in trouble, you can always ask on our user group or during a call for some help.

❓How you will review my homeworks?

Most of the time you will have to create a Pull Request on Github and send me the link with your solution. From time to time, I will give you a feedback on the forum but it's limited to the conceptual tasks, just to spread the knowledge. If you don't want to play the game, you can still submit me the homework as a Pull Request from your Github private project.

❓How can you help me if I don't know how to start the homework?

I will answer your message on our user group. For every coding homework I also prepared one video explaining a possible solution. You can use it as an inspiration.

❓Will I learn Apache Spark internals?

No, the course is not intended to deep delve into Apache Spark internals but rather to show how to use it in data projects. But I will also do my best to answer any question related to your homework or a concept described in the course.

❓Where I can find the answers to my question?

If I cannot provide a satisfactory answer during our call, I will put it on our user group, as well for 1-on-1 and class calls. All questions are anonymous, so on the user group you will only see the question and my answer.

❓How long I can access the user group?

If the user group is included in your plan (Community or Community+), you have the access to the user group as long as the course will be online.

❓Why the number of Community+ accesses is restricted?

To give the best experience for the students, at every opening only few access of that type are proposed.