We prove a central limit theorem for the components of the largest eigenvectors of the adjacency matrix of a finite-dimensional random dot product graph whose true latent positions are unknown. We use the spectral embedding of the adjacency matrix to construct consistent estimates for the latent positions, and we show that the appropriately scaled differences between the estimated and true latent positions converge to a mixture of Gaussian random variables. We state several corollaries, including an alternate proof of a central limit theorem for the first eigenvector of the adjacency matrix of an Erdos-Rényi random graph.