In the previous post, we built a simple linear regression model that maps one-hot vector to its label value. In this post, we build a model that reverse this operation.
Let's build a unit vector from scalar value.
$$ \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix} \rightarrow \begin{bmatrix} [1 & 0 & 0] \\ [0 & 1 & 0] \\ [0 & 0 & 1] \end{bmatrix} $$Building a new model in Deep Learning is not easy. Figuring out why a model is not acting as expected is difficult. It is important to know how what each component is capable of. Dense layer is one of the most fundamental component of Machine Learning. Here, we explore what it is capable of through series of simple and easy-to-interpret experiments.
Let's consider a model that maps one-hot vectors into indices.
$$ \begin{bmatrix} [1 & 0 & 0] \\ [0 & 1 & 0] \\ [0 & 0 & 1] \end{bmatrix} \rightarrow \begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix} $$