Wojtek Mikołajczyk’s tech blog

Machine Learning books I recommend

2022-09-03T17:00:00+00:00

Some time ago I realized that I like to learn new things in a structured way.
That’s why I like learning new things from books.
There are tons of Machine Learning books available, I’m pretty sure that no one can read them all :)
That’s why it’s so important to choose the right ones.

The Hundred-Page Machine Learning Book - Andriy Burkov

This is great as the first book about Machine Learning.
It greatly introduces all key concepts and it’s very concise - it literally consists of a little bit more than 100 pages.
This book might also be useful if you need to quickly refresh some base knowledge about a particular topic.

Machine Learning Engineering - Andriy Burkov

This book introduces key concepts of working on a real Machine Learning project.
It describes that there is much more than Machine Learning algorithms and their mathematical background. It’s about all aspects of Machine Learning! :)
From starting a project, defining goals and metrics, through acquiring the data, data preprocessing, and feature engineering, to the model training and deployment on production.
The book gives a good overview of all these areas.

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow - Aurélien Géron

This is a Machine Learning “Bible”.
I loved this book. It gives a deep understanding of key Machine Learning algorithms and Neural Networks.
It also consists of hands-on exercises which sometimes are a little challenging, but all of them are provided the right answers or code snippets.
This book does exactly what the title says - it provides you required knowledge and gives you hands-on experience while completing the exercises.

Natural Language Processing with Transformers - Lewis Tunstall, Leandro von Werra, Thomas Wolf

This book shows how to use Hugging Face Transformers for Natural Language Processing.
It describes the required theory and gives practical examples of using the whole Hugging Face ecosystem to effectively deliver NLP models for different tasks.
The book addresses also topics related to large-scale training, like multi-GPU training and making models efficient using knowledge distillation or quantization.

I can greatly recommend the books mentioned above to Machine Learning enthusiasts.
The first two are more like an introduction to the topic, and the two latter are more specialized.

If you have any other Machine Learning books that you want to recommend - feel free to put them in the comments section below :)

How to start a machine learning project?

2022-08-28T17:00:00+00:00

Machine learning is getting more and more popular nowadays. Companies are creating or extending their R&D teams. Having a product that delivers some kind of AI solution is prestigious.

In terms of software development, building business web applications is a quite mature field. There are well-known practices and methodologies on how to plan, design, and implement a solution. But AI is quite a greenfield and when it comes to building AI solutions, it’s a big challenge how to do it well.

image: Rocket icons created by Freepik - Flaticon

Machine learning and in general AI are very fast-paced in terms of development. The SOTA (State Of The Art) models and solutions are changing rapidly, and new tools and libraries are being released.

Business people are often not aware of what is possible, and where are the limitations of AI solutions, they might be driven by AI hype from clickbait news. Data scientists might be excellent in terms of machine learning knowledge and statistics but they might lack project planning skills and they might not be able to deliver large-scale production-grade solutions. Machine learning engineers should be somewhere between software engineers, data engineers, DevOps, and data scientists to be able to support data scientists with developing well-crafted AI solutions for production - but they might be more focused on making an interesting project in terms of their development, and not optimally meeting business requirements. Project managers might have trouble with understand well project risks because it requires some prior machine learning knowledge.

The things described above might be some of the challenges, but there is one more, probably the biggest of them. Machine learning and AI have an experimental nature. It’s almost impossible to tell if the designed solutions will work in practice or not. It requires experiments, and usually a lot of iterations on the whole process of collecting the data, cleaning and preprocessing, and model training.

Luckily, those problems are common in this area and a lot more people have occurred them. There are some good practices on how to start a machine learning project properly, what are the most important points to define, and in general, how to approach developing an AI solution.

Machine Learning Project Checklist:

what is the goal?
what is the metric of success?
what is the minimum value of the metric of success to go live with the solution?
can it be solved with a non-machine learning solution?
what is a better way to solve this problem (machine learning or non-machine learning solution) and why?

assuming the decision is to go with the machine learning approach

what data will be used?
what is the cost of acquiring/labeling the data?
what will be a baseline model?
what are the risks of the project?
what are the unknowns?
can this problem be simplified and how?

It’s good to address those questions while starting a machine learning project. The business should discuss that with the development team and they should together come up with the answers to those questions. It’s worth spending more time on project planning than starting without a clear vision and some misunderstanding between business and an AI development team - to deliver in a few months solution that… cannot be used in production because it lacks some features or is not good enough.

The Machine Learning Project Checklist was created based on my own experience and on the Machine Learning Engineering by Andriy Burkov which I greatly recommend reading! :)

If you have any more ideas about starting a machine learning project, I greatly encourage you to put them in the comments section below :)

The mystery of the memory leak…

2020-11-30T09:00:00+00:00

Some time ago, I was developing some data processing script in Python. I was sure I did everything well - my script worked as intended, all unit tests passed and I even tested the script on small data, and it worked properly. But I encountered an interesting problem, the execution failed…

Everything seemed to be fine until… I ran it to process a HUGE (1 400 000 files, 50GB) volume of data. The script was massively processing big data but suddenly a container, in which the script was run, was killed due to Out Of Memory error. The script was wrapped into a Docker container and deployed on the Kubernetes with defined maximum usage of RAM limit. So, what actually happened?

In such cases, it’s always a good idea to check for CPU and memory usage over time graphs. I checked the graph of memory usage over time, and it looked like that.

I saw that memory usage constantly grows. It means that my script allocates more and more memory for objects, much faster than it frees some memory. This problem is called a MEMORY LEAK. But, what is the cause of that?…

I had to process a big volume of data, so to speed up processing I used multithreading in Python. The threads were my first suspect, so I started to analyze how I use them.

I needed to provide GCS (Google Cloud Storage) client object for each thread to be able to perform my data processing. Google Cloud Storage client isn’t thread-safe so each thread requires its own copy of that. During the script execution, a bunch of threads is working in a pool, performing input-output heavy operations in a loop. Probably the problem lies in a way of storing data in the threads. Let’s verify that!

I performed a code-review together with my friend and he helped me to find that this actually is a problem. Each thread created a seemingly harmless GCS Client object every time the operation is performed. We’ve found out that it’s not actually harmless - it was the root cause of the memory leak!

The solution was to use threading.local() in Python (docs.python.org/…#threading.local) - which allows storing data for each thread. After applying this fix - GCS client object was created only once for each thread. The memory leak problem is solved now! :)

Code with the Memory Leak issue

from google.cloud import storage

...

class SomeClass:
    def __init__(project_id):
        self.project_id = project_id

    @property
    def client(self):
        return storage.Client(project=self.project_id)

Code after applying fix - using threading.local()

from google.cloud import storage

...

class SomeClass:
    def __init__(project_id):
        self.project_id = project_id
        self.thread_local = threading.local()

    @property
    def client(self):
        if "gcs_client" not in self.thread_local.__dict__:
            self.thread_local.gcs_client = storage.Client(
                project=self.project_id)
        return self.thread_local.gcs_client

After applying the fix, the script was able to process all this data in around 5 hours, thanks to multi-threaded operations which sped up the whole thing a lot.

Key takeaways: