Enhancing AI Alignment with Direct Preference Optimization

Posted on Wed 14 February 2024 in deep learning • 3 min read

Aligning models to human values and desires is a formidable task, particularly when we're working with vast datasets that resist meticulous control. Concerns arise when language models spew out content that could be false or harmful. Luckily, advancements in AI research are opening pathways for safer and more reliable interactions …


Continue reading

Low-rank Adaptation

Posted on Mon 29 January 2024 in deep learning • 4 min read

Large language models, as the name already states, have a huge quantity of parameters and have been trained on large-scale datasets to obtain excellent generalization capabilities.

But if it's so good, why would people want to modify such models?
1. To have a specific behavior - Let's say you have a …


Continue reading

Wav2Vec2.0 paper notes

Posted on Sat 20 May 2023 in paper review • 3 min read

Wav2vec2.0, as the name already suggests, claims to receive a raw audio file in its waveform and convert it to a vector (encode it in the latent space). Therefore, many downstream tasks can be performed on top of this vector representation extracted by the model. Meta claims that using …


Continue reading

Reinforcement learning with human feedback

Posted on Fri 12 May 2023 in reinforcement learning • 5 min read

In basic terms, reinforcement learning is a technique that allows machine learning models to learn by tries and observations. It is beneficial whenever we need a more clear idea of what the optimal behavior for our model is - for ChatGPT for example. We don't have an optimal answer; instead a …


Continue reading

HuggingFace dataloading on remote storage data

Posted on Sat 06 May 2023 in code • 2 min read

HuggingFace is an open-source library that started to make transformers-based architectures more accessible to a lot of programmers through their APIs. Therefore, being familiar with their technologies it's a good idea for anyone who wants to be able to implement training and inference pipelines faster and still be able to …


Continue reading

Equal error rate and biometric systems

Posted on Fri 05 May 2023 in metrics • 4 min read

Biometrics systems are focused on obtaining the characteristics of an individual and verifying if it does match with someone in the database. These characteristics can be the fingerprint, the voice, and even our iris. These kinds of systems are everywhere nowadays, in our phones and even door locks, therefore some …


Continue reading

Digital Signal Processing Concepts

Posted on Sun 30 April 2023 in digital signal processing • 3 min read

In this blog post, I'll be diving into some concepts of digital signal processing (DSP). Most of them I had to learn before starting to work with models that take audio files as input and therefore, require some preprocessing steps.

The concepts I'll briefly discuss here are:
1. Fourier transforms …


Continue reading