My favorite ones are as follow:

-The Colour out of Space
-At the Mountains of Madness
-The Alchemist(not technically in the Cthulhu Mythos, but still, written by Lovecraft)
-Dagon
-The Lurking Fear
-Herbert West: Reanimator
-The Doom that came to Sarnath
-The Shunned House


Also, be sure to read the works of the people who inspired Lovecraft, especially Algernon Blackwood and Edward Plunkett(he wrote under the pseudonym "Lord Dunsany")

My brother actually got me this big black book called "The Necromonicon" for Christmas last year, containing 'the best weird tales' of Lovecraft. But yeah all his stuff is public domain so you can download it wherever, don't need to buy.

But my personal favorite story is The Shadow Out of Time

What does it mean that it is public domain? I never heard of that expression.

Expired IP. See Wikipedia

Says something about current copyright laws that people haven't heard about public domain :(

Ah okay so it's some kind of abandonware but legal.

Yeah, and it applies to all forms of intellectual property, including books, games, movies, music, etc.

I recommend kind of separating the Dream Cycle stories (Like the White Ship, Dream Quest of Unknown Kadath, and some others), the Mythos Stories (Like Call of Cthulhu, Case of Charles Dexter Ward and In the Mountains of Madness etc) and the gothic horror titles (which constitute the rest). It can be confusing to read some of them if you don't know if it is mythos or not. Or not, it is a matter of taste really.

Apart from that, I recommend you read everything, they are interesting stories. The longish stories tend to be the best in my books: Case of C.D Ward, In the Mountains of Madness, Shadow over Innsmouth and so on.

Outside Lovecraft I recommend the King in Yellow by R.W. Chambers, also in public domain as far as I know. It is arguably mythos, since Hastur and the King in Yellow appear in Lovecraft's writing. Also a damn good book in it's own right.

Another good story that I don't think has been mentioned is The Outsider. It's not a mythos story so much as a stand-alone gothic horror thingy, but it's good.

Man, everyone's recommending Mountains of Madness, which I remember as being pretty boring. To everyone their own opinions, I guess. My personal recommendation would be his dream cycle; Shadow over Innsmouth as an example of his mythos stories; and 'Colors Out of Space' as a straight, non-mythos horror story (Wikipedia tells me it was also Lovecraft's personal favorite).

And following in Crowned's steps, I'll recommend a related non-Lovecraft story, "The Willows" by Algernon Blackwood; Lovecraft called it the best weird story every written, and he may well have been right.

I got this little thingy at home, it covers anything you might want from Lovecraft.
http://www.amazon.com/Necronomicon-Weird-Lovecraft-Commemorative-Edition/dp/0575081570/ref=sr_1_2?ie=UTF8&qid=1346768941&sr=8-2&keywords=necronomicon

"Night-Gaunts" (A great sonnet from "Fungi from Yuggoth")
"Dagon"
"The Statement of Randolph Carter"
"The Doom that Came to Sarnath"
"The Cats of Ulthar"
"The Nameless City"
"Herbert West--Reanimator"
"The Music of Erich Zann"
"The Lurking Fear"
"The Hound"
"The Rats in the Walls"
"Under the Pyramids"
"The Unnamable"
"In the Vault"
"The Outsider"
"The Horror at Red Hook"
"The Colour out of Space"
"Pickman's Model"
"The Call of Cthulhu"
"Cool Air"
"The Shunned House"
"The Silver Key"
"The Dunwich Horror"
"The Whisperer in Darkness"
"The Strange High House in the Mist"
"The Dreams in the Witch-House"
"From Beyond"
"Through the Gates of the Silver Key" (with E. Hoffman Price)
"At the Mountains of Madness"
"The Shadow over Innsmouth"
"The Shadow out of Time"
"The Haunter of the Dark"
"The Thing on the Doorstep"
"The Case of Charles Dexter Ward"
"The Dream-Quest of Unknown Kadath"
"To a Dreamer" (A final Lovecraft poem)
Afterword: A Gentleman of Providence by Stephen Jones

There are also many anthologies written by many talented fantasy authors set in the mythos.

Support my brother's play on crowdfunder, and he'll send you a 5 part Cthulhu Mythos story! :)

http://www.crowdfunder.co.uk/investment/drowning-rock-lovecraftian-horror-play-1118

(Seems as good an opportunity as any to plug this bad boy again...)

Personal favourites:

Shadow over Innsmouth

At the Mountains of Madness

The Whisperer in the Darkness

Herbert West - Reanimator

The Case of Charles Dexter Ward

The Shadow out of Time

You forgot "The Colour out of Space".

If you don't have it already, this is a good place to get the stories.

http://cthulhuchick.com/free-epub-complete-works-lovecraft-nook/

If I remember rightly it has his works in order of when they were written, which might not be a bad order to go by really.