The above quotation is to be found in Choses Vues, a posthumously published collection of heterogeneous writings by Victor Hugo. (Page 271 in the reprint—also here— from 1913.) It is placed under the header Faits Contemporains of the year 1847. The year is definitely wrong, because the famous physicist Arago died in 1853, and Hugo talks about him in the past tense. So the anecdote has been written down somewhere between 1853 and 1885, when Hugo died. The first volumes of Laplace's Traité de Mécanique Céleste appeared in 1799, when Laplace was also, for a few months, Napoleon's Minister of the Interior. If this particular meeting between Napoleon and Laplace really occurred, 1799 is the most likely year. It is registered by Hugo half a century later, and the anecdote is third-hand information anyhow (Hugo telling us what Arago told him Laplace had said). Nevertheless, it has a certain credibility. Hugo knew Arago very well, Arago and Laplace were simultaneously involved in the same scientific matters, and Napoleon was very familiar with his minister and his scientific work.
Curiously, Hugo's wrong year 1847 is also the year where, verifiably, the anecdote is registered by Augustus de Morgan. De Morgan was an active contributor to the London magazine Athenaeum, and therein (Budget of Paradoxes p.247) he reviewed the 1847 issue of The Reasoner, under the title Is there sufficient proof of the existence of God? In his review we find the following version of the same anecdote, longer but less precise than Hugo's. According to Hugo, Napoleon was "furious"(at least before he heard Laplace's answer), whereas De Morgan makes him "greatly amused".
Comment
There are multiple aspects to the concept of "God", but the Napoleon-Laplace controversy clearly deals with "God" as "Creator of the Universe". Comments below deal with this aspect only.
The "Creator" hypothesis (trivially) solves one question:
Q. Where does the universe come from?
A. The Creator created it.
On the other hand, it raises the following questions, all of them harder than the one above:
-Q. Where does the Creator come from?
The answer "He was there all the time" doesn't help, because "the universe was there all the time" is then a simpler answer to the original question, doing away with a redundant step.
-Q. Why would a perfect, eternal Creator disturb his perfect state by creating anything external, least of all the imperfect universe we live in?
That reason must have been an internal one, resulting from some internal evolution, but then an internally evolving universe is a simpler model.
-Q. Why is the universe such a lousy design (physically and, on the insignificant scale of human experience, morally)?
It is unimaginably huge, unimaginably empty and unimaginably complicated. Whatever purpose it might have could have been realized in a much simpler way. (Or not realized at all, which is still the simplest way.)
-Q. Why is the Creator unwilling or unable to communicate with his intelligent creatures in such a way that they can understand it?
Religions are plenty, and all abound in obscure texts, often said to contain a hidden extra layer, even harder to understand. Clearly, if there is a message, it's messy and difficult to make sense of. Why a perfect omnipotent being would amuse Himself emitting such noisy signals is beyond comprehension.
-Q. Why, if the Creator is concerned with making mankind happy (as religions, invariably dealing with an outdated man-centered miniature "universe", claim) did He need billions of years to generate, on some insignificant speck in some remote part of his universe, a highly imperfect and arguably unhappy Mankind, doomed to disappear with the solar system?
If the Creator is so concerned with the happiness of the human species on planet earth, He could use His unlimited powers to create a happy species right away, or (better still) create a universal state of "happiness" without the detour of billions of years of evolution in carbon chemistry, or (still the best) He could not create anything at all, remaining what He was in the first place: the perfect self-contained universe-filling state of happiness.