Computers are great at math, but they don't understand human words out of the box. In modern Natural Language Processing (NLP), how do we represent words so that models can process them mathematically while capturing their semantic meanings and relationships?
Select an answer to reveal the explanation.
Short Explanation and Infographic
Here's the deal—a computer doesn't know what a "king" or a "queen" is—it just sees strings of characters. To do NLP, we have to turn those words into numbers. The old-school way was one-hot encoding, where every word got its own column. But if you had a vocabulary of 50,000 words, you ended up with massive, empty tables (vectors with 49,999 zeros and a single one). Worse, the computer had no idea that "king" and "queen" are related. Word embeddings solved this by mapping every word to a dense vector of numbers—usually around 300 values. In this vector space, words with similar meanings (like "king" and "queen", or "dog" and "puppy") end up clustered close together. It's like giving the computer a map of human language where similar concepts live in the same neighborhood. Trust me, it makes all the difference in NLP.
Full explanation below image
Full Explanation
In Natural Language Processing (NLP), word embedding is a representation learning technique where words from a vocabulary are mapped to vectors of real numbers in a continuous vector space.
Historically, simple representation techniques like one-hot encoding were used. In a one-hot representation, each word is represented as a sparse vector with a dimension equal to the size of the vocabulary. This approach has two major flaws: first, it suffers from extreme high-dimensionality and sparsity (the curse of dimensionality); second, it fails to capture semantic similarity, as the dot product between any two distinct one-hot vectors is always zero, meaning the computer treats "cat" and "kitten" as completely unrelated.
Word embeddings (such as Word2Vec, GloVe, or FastText) resolve these issues by representing words in a dense, lower-dimensional space (typically 100 to 300 dimensions). Because the vectors are learned based on the context in which words appear (the distributional hypothesis: words that occur in similar contexts tend to have similar meanings), semantically related words are positioned near each other in the vector space. This spatial arrangement allows the model to perform vector arithmetic that reflects semantic relationships, such as the classic example: Vector("King") - Vector("Man") + Vector("Woman") ≈ Vector("Queen").
Let's review the incorrect options: - Text-to-speech (TTS) engines (Option A) convert written text into spoken audio, which is an output generation task rather than an NLP input representation method. - Stripping punctuation and stop words (Option B) is a basic text preprocessing step, not a mathematical representation technique. - One-hot encoding (Option D) represents words as numbers, but it creates sparse, high-dimensional vectors that fail to capture semantic relationships.
Therefore, mapping words to dense, low-dimensional vectors in a continuous space (Option C) is the main purpose of word embeddings.